Discuss mailing list

[Jeff Taylor <shdwdrgn AT sourpuss.net>]

    The API and the servers pages are run from a single VM machine that
    doesn't expect a lot of traffic.  The servers page sees what I
    consider 'typical' traffic, where people come in and check out the
    information, browse around a bit, or perform updates to their own
    server entries.  The API pages provide information that may
    periodically change.  For instance, the ACL lists may change at
    1-minute intervals but will generally see hours between any
    updates.  Something like the geoip page will probably not see
    changes for *days* at a time.
    
    Recently someone released what I imagine is a very poorly programmed
    script which was absolutely flooding the PHP code used to compile
    the information on the API pages, with queries as frequent as five
    times per second.  I mean really, what does this idiot hope to
    achieve with such frequent queries for the same information?  As a
    result, access to the servers page has been very slow or dropped out
    completely at times, so in October I added new code to cache the
    replies and rate-limit the queries and send a warning message if you
    exceeded a certain threshold.  Of course the caching greatly reduced
    the load on the script, and once they started receiving invalid
    information in the form of the warning message most of the bots
    completely dropped out.  Unfortunately there always has to be that
    one guy...
    
    Since at least October 4th, someone at 208.82.39.26 has been running
    a script with the following query:
/geoip/?ip=174.49.73.80&pct=95&list&res=4&nearest&noscript
    What this means is they are doing a lookup for servers nearest to
    the location of the IP at 174.49.73.80, and they apparently have no
    idea what they are requesting because 'noscript' and 'nearest'
    aren't even recognized parameters.
    
    This query has been coming consistently at a rate of more than once
    per second, which means they haven't received a single valid reply
    since I implemented the rate-limiting warning messages.  Because
    this person obviously isn't paying attention to what is happening,
    I've completely blocked this IP at the firewall, so they will have
    no access to either the API or servers pages.  Any such activity I
    see which is impacting access by opennic members will be considered
    abusive and will be shut down without notice.  For reference, the
    'normal' traffic I see for geoip lookups amounts to around five
    queries per *minute*, globally.  Any time a single IP exceeds the
    amount of queries that the rest of the world combined is performing,
    I will look at it with suspicion.
    
    For anyone wishing to actually implement a script to query the geoip
    information, please have some realistic expectations.  You could ask
    for updates once per hour and at reboots, and you would still have a
    reasonably accurate and usable list of nearby DNS servers to query
    from.  Of course as mentioned above some of the API pages will have
    more frequent updates.  The ACL list provides a timestamp to signal
    when you should grab a new copy, but checking that list once a
    minute is expected.
    
    Note that bandwidth is NOT an issue here!  Despite the abusive
    traffic, the combination of traffic for all of the services I run on
    my network is using less than 1/50th of my available bandwidth.  The
    problem lies in features like the geoip page which requires a lot of
    calculations to generate the information, and each IP address has to
    be resolved individually.  I could further optimize the caching and
    such, but I think the real problem lies in people who are using
    opennic services for nefarious purposes.  We already know someone
    has been using opennic and .bit domains to spread malware, and I've
    seen many suggestions over the years that opennic DNS servers are
    being used to perform lookups by spam bots.  I believe the recent
    large number of queries to the geoip page were being performed for a
    similar purpose (the agent string and specific query were identical
    for each, but coming from IPs all around the globe).  I'll continue
    to keep an eye on the traffic though and will only block access when
    there is a clear case to do so.