Discuss mailing list

[Jeff Taylor <shdwdrgn AT sourpuss.net>]

Since this issue seems to come up from time to time, I'd like to work on a way 
to implement automated whitelisting and possibly include some rules for 
automated blacklisting.  For anyone interested in contributing ideas on a 
reliable rule set, please consider the following...

-- Types of abuse:
We have seen a couple forms hit our servers.  The first comes from spambots.  
Since most ISP's would be suspicious of the amount of activity generated by 
spambots, the operators point their bots at random open DNS services.  We 
typically notice this traffic on our servers due to the substantial amount of 
queries (a typical user will pull 1000-2000 queries per day, where a spambot 
will request hundreds of thousands of lookups), or by unusual patterns in the 
queries from a particular IP (I saw one last night that was pulling thousands of 
domain names in alphabetical order).

The other type of abuse is blatant DDOS attacks.  The most common seen is an 
incoming packet on source-port 25345, querying isc.org, however I have also seen 
periodic attacks on other ports directed at other domains.  These attacks work 
by sending a small query (39 bytes) requesting the DNSSEC record for a domain 
(which will reply with a packet containing thousands of bytes).  It only takes 
10-20 queries per second of this type to completely saturate a typical home 
connection.

-- Whitelisting:
A generic way to implement this is simply adding an IP address to the whitelist 
when the user queries for an OpenNic domain.  To keep the list fresh, we could 
expire any address that does not request OpenNic sites for more than 4 weeks.  
Now this is a nice, simple method to implement, but very easy to exploit.  We 
could perhaps require clients to run a small program which performs a whitelist 
request via a signed key, but then it becomes a hassle for users to simply jump 
onto OpenNic.

-- Blacklisting:
Anything blacklisted would fall under the above concepts of abuse.  If we detect 
abusive traffic from an IP address, we would want to drop that traffic for a 
certain period of time.  I have a fairly reliable script capable of detecting 
the DDOS packets that we've seen in the past, however trying to automatically 
detect a spambot is much harder.  You cannot simply use the amount of traffic as 
an indication, because any business office using OpenNic would reasonably 
generate a much larger number of queries per day.  Patterns in queries that may 
seem obvious to the human eye would be undetectable to a script.  So how do you 
automate the detection?

----------

So the initial problem we have is in creating reliable rules that can be 
scripted, and will properly whitelist or blacklist IP addresses.  We need to 
allow legitimate access without inconveniencing the users, but still block 
enough of the abusive traffic that services are not affected.

Expanding further, it would be ideal if we could coordinate the whitelists and 
blacklists between all servers.  Having a common shared database would be 
affective in preventing abusers from skipping around between servers, but if 
would also give the software a much larger data pool to work with, and may help 
point out abusive traffic that would otherwise be impossible to detect (consider 
the case of a spambot using 10 different servers to perform its queries... a 
normal user connection would never show up on more than 2-3 servers).

Anyway, I hope this gives folks something to think about.  Any discussion on 
this concept would be appreciated, so that we may find ways of protecting our 
Tier-2 servers.  Remember - abusive traffic takes up bandwidth and slows down 
everyone's queries.