Discuss mailing list

[Al Beano <albino AT autistici.org>]

Piwik would be fine — it's all information that could otherwise be extracted 
from server logs anyway. 

Users who don't want it can enable DNT, use a blocking extension or disable 
JavaScript. 

On 5 October 2017 14:35:08 BST, Jonah Aragon <jonah AT triplebit.net> wrote:
>Maybe I don’t know enough about JavaScript but I have no idea why it
>would change the user-agent. I’d imagine it’d just send the browser’s
>default user-agent. 
>
>The homepage is hosted at Netlify which has worked great for us but
>doesn’t exactly give us server logs. I’d like to use Piwik Analytics
>for the homepage, wiki, and servers page, but based on reactions from
>some people here I suspect even Do Not Track respecting, IP
>obfuscating, first-party JavaScript hosted by us would still be looked
>down upon :(
>
>To be fair though we clearly have JavaScript on the page anyways, so
>maybe we’ll go for that route regardless. 
>
>Jonah
>
>Sent from my iPhone
>
>> On Oct 5, 2017, at 8:28 AM, Al Beano <albino AT autistici.org> wrote:
>> 
>> Don't need to use JS analytics, just feed the web server logs into
>some kind of script. there are plenty online. 
>> 
>> Doesn't the request on the website set a specific user-agent or other
>header anyway?
>> 
>> albino
>> 
>>> On 5 October 2017 14:26:04 BST, Jonah Aragon <jonah AT triplebit.net>
>wrote:
>>> Nobody here likes JavaScript or analytics for some reason
>(paranoia?)
>>> so there’s no good way for us to check that out. I sort of doubt
>just
>>> normal web traffic to this one page is causing that level of load
>but
>>> I’m not sure how many visitors we actually get. 
>>> 
>>> Jonah 
>>> 
>>> Sent from my iPhone
>>> 
>>>> On Oct 5, 2017, at 7:57 AM, Rouben <rouben AT rouben.net> wrote:
>>>> 
>>>> Hmmm... any correlation with the website? As mentioned on this
>>> thread, it invokes the API via client-side JavaScript.
>>>> 
>>>>> On Thu, Oct 5, 2017 at 08:53 Jeff Taylor <shdwdrgn AT sourpuss.net>
>>> wrote:
>>>>> Possibly, but no single IP is making requests quite that fast. 
>What
>>> they're doing is flooding me with requests from all over the place,
>and
>>> "they" seem to be a whole lot more active after I go to bed. 
>Yesterday
>>> I was seeing about 200 IPs per hour and around 50 queries per
>second. 
>>> As I wake up this morning I find over 750 IPs per hour and over 300
>>> queries per second.  And I'm pretty sure opennic didn't just happen
>to
>>> pick up that many new users overnight.
>>>>> 
>>>>> 
>>>>>> On 10/04/2017 08:27 PM, Theo B. wrote:
>>>>>> Would it be possible to have a stacking rate limit per IP? For
>>> example, if an IP requests the list 10 times in a second, they get a
>20
>>> second rate limit, and if they keep requesting it gets higher?
>>>>>> 
>>>>>> -Theo
>>>>>> 
>>>>>> 
>>>>>>> On Oct 4, 2017, 10:21 PM -0400, Jeff Taylor
>>> <shdwdrgn AT sourpuss.net>, wrote:
>>>>>>> Oh yeah, forgot about that part.
>>>>>>> 
>>>>>>>> On 10/04/2017 06:50 PM, Jonah Aragon wrote:
>>>>>>>> It’s client side (javascript) for obvious reasons, so the API
>key
>>> would have to be embedded in the code which would kind of defeat the
>>> point. The browser is making the request, not the opennic.org
>server. 
>>>>>>>> 
>>>>>>>> Jonah
>>>>>>>> 
>>>>>>>> Sent from my iPhone
>>>>>>>> 
>>>>>>>>> On Oct 4, 2017, at 7:06 PM, Rouben <rouben AT rouben.net> wrote:
>>>>>>>>> 
>>>>>>>>> Not necessarily.... each individual or application could be
>>> issued an API key to use; www.opennic.org included. This has to be
>done
>>> for some API calls already anyway, and is generally a good idea...
>>>>>>>>> 
>>>>>>>>>> On Wed, Oct 4, 2017 at 19:47 Jonah Aragon
><jonah AT triplebit.net>
>>> wrote:
>>>>>>>>>> That would be unfortunate, it’d break the nearest servers
>list
>>> on www.opennic.org. 
>>>>>>>>>> 
>>>>>>>>>> Jonah
>>>>>>>>>> 
>>>>>>>>>> Sent from my iPhone
>>>>>>>>>> 
>>>>>>>>>>> On Oct 4, 2017, at 5:39 PM, Rouben <rouben AT rouben.net>
>wrote:
>>>>>>>>>>> 
>>>>>>>>>>> On second thought, a more practical option would be to
>change
>>> the geoip API to require authentication, similar to the BIND ACL
>api.
>>> That way at least you can determine the identity of the abuser and
>>> contact them, asking to correct the problem.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Rouben
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, Oct 4, 2017 at 6:02 PM, Jeff Taylor
>>> <shdwdrgn AT sourpuss.net> wrote:
>>>>>>>>>>>> Yeah there's plenty of options, and I actually use fail2ban
>>> on some of my other VMs, but I generally haven't had any problems
>with
>>> the apache servers.  It's not enough of a problem to require drastic
>>> measures yet, and I certainly don't want to go crazy with it and
>block
>>> legitimate lookups by opennic members, but I'm sort of stumped as to
>>> the source of this flood.  As I mentioned, they all have the same
>>> signature so it must be some sort of script or bot, and it has some
>>> minimal intelligence to it because the flood stopped as soon as I
>>> started returning unexpected answers... I wonder what sort of
>results I
>>> might see if I compared the IPs making these queries with a list of
>IPs
>>> sending email spam to my servers?
>>>>>>>>>>>> 
>>>>>>>>>>>> Anyway the only real problem here is the number of queries.
>
>>> I set up the VM with very low resources expecting only an occasional
>>> request for an API or the servers page.  The actual bandwidth used
>>> didn't even put a dent in my connection and I don't have metered
>>> traffic.  I'll probably restart the VM tonight with more memory
>though
>>> just to handle the extra traffic and see how it does.  Fortunately
>this
>>> VM runs on my biggest machine so I can throw a lot more resources at
>it
>>> as                                         needed.
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>>> On 10/04/2017 03:48 PM, Rouben wrote:
>>>>>>>>>>>>> May I suggest using either
>>>>>>>>>>>>> https://httpd.apache.org/docs/trunk/mod/mod_ratelimit.html
>?
>>>>>>>>>>>>> you'd need to get Apache 2.4, though, looks like you're
>>> still on 2.2.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I'd also disable HTTP KeepAlive, since API calls by their
>>> nature are atomic, and clients generally have no business asking the
>>> server to keep the connection alive for a single question-answer
>>> transaction typical of APIs.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I'd add also a second layer using IPTables, similar to how
>>> the DoS is mitigated for OpenNIC DNS servers:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> -p udp -m hashlimit --hashlimit-srcmask 24
>--hashlimit-mode
>>> srcip --hashlimit-upto 30/m --hashlimit-burst 10 --hashlimit-name
>>> HTTPSTHROTTLE --dport 443 -j ACCEPT
>>>>>>>>>>>>> -p udp -m udp --dport 53 -j DROP
>>>>>>>>>>>>> Above rule adapted from
>>> https://wiki.opennic.org/opennic/tier2security
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Alternatively, perhaps fail2ban can automate the iptables
>>> banning/unbanning based on a more sophisticated detection rule:
>>>>>>>>>>>>>
>https://www.maketecheasier.com/fail2ban-protect-apache-ddos/
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I like layered security solutions... :) Apache can handle
>>> the low-frequency "reasonable" DoS, and iptables can handle the
>>> high-frequency heavy abuse that would be too much for Apache (or
>even
>>> Varnish) to tackle.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Rouben
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Wed, Oct 4, 2017 at 4:09 PM, Alex Nordlund
>>> <deep.alexander AT gmail.com> wrote:
>>>>>>>>>>>>>> Have you considered putting Varnish in front of it?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Best regards
>>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On 4 Oct 2017, at 20:12, Jeff Taylor
>>> <shdwdrgn AT sourpuss.net> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> You may have noticed some issues reaching either the API
>>> or servers page recently.  I've tracked down the problem to some
>>> extremely excessive calls to the geoip page
>>> (https://api.opennicproject.org/geoip/).
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> If you are the owner of 208.82.39.26... your script is
>>> doing lookups four times per second.  Just how often do you think
>the
>>> list of servers changes?  I blocked this IP completely for now,
>please
>>> fix your script and let me know if you want access again.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Of course this one user wasn't enough to bring the
>server
>>> to its knees, this problem was because of yet another script that
>seems
>>> to be getting shared around the globe.  There are two aspects of the
>>> query that lead me to believe there is a common script running here:
>>>>>>>>>>>>>>> "GET /geoip/?bare&pct=95 HTTP/1.1"
>>>>>>>>>>>>>>> "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0;
>Touch;
>>> rv:11.0) like Gecko"
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I'm seeing well over 2000 unique IP addresses making the
>>> same query up to once ever five seconds.                            
>  
>>> That translated to about 200 queries per second.  Now the geoip page
>is
>>> rather expensive in terms of resources, because it has to look up
>the
>>> user's IP and try to match it geographically to the list of Tier-2
>>> servers.  I wrote up some code this morning to cache the queries by
>IP
>>> address for 5 minutes before re-checking.  Now this made a huge
>>> difference but still wasn't enough.  I may have another bottleneck
>in
>>> my network that was causing problems even with the cached content so
>>> I'll be looking into that.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> In the meantime I've added a level of blocking for any
>>> server making queries faster than every 15 seconds.  This will
>return a
>>> message                                                   warning
>the
>>> requester that server information doesn't change that fast, and
>doesn't
>>> give the expected reply.  I'm hoping whoever set up this script will
>>> see broken results and get it fixed.  At the moment this 15-second  
>  
>>> warning message is accounting for about 25% of all the queries. 
>I'll
>>> keep working on it, but just wanted to let folks know WHY in case
>>> anyone happens to see the warning message.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> --------
>>>>>>>>>>>>>>> You are a member of the OpenNIC Discuss list.
>>>>>>>>>>>>>>> You may unsubscribe by emailing
>>> discuss-unsubscribe AT lists.opennicproject.org
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> --------
>>>>>>>>>>>>>> You are a member of the OpenNIC Discuss list.
>>>>>>>>>>>>>> You may unsubscribe by emailing
>>> discuss-unsubscribe AT lists.opennicproject.org
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> --------
>>>>>>>>>>>>> You are a member of the OpenNIC Discuss list.  
>>>>>>>>>>>>> You may unsubscribe by emailing
>>> discuss-unsubscribe AT lists.opennicproject.org
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> --------
>>>>>>>>>>>> You are a member of the OpenNIC Discuss list.
>>>>>>>>>>>> You may unsubscribe by emailing
>>> discuss-unsubscribe AT lists.opennicproject.org
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> --------
>>>>>>>>>>> You are a member of the OpenNIC Discuss list.
>>>>>>>>>>> You may unsubscribe by emailing
>>> discuss-unsubscribe AT lists.opennicproject.org
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> --------
>>>>>>>>>> You are a member of the OpenNIC Discuss list.
>>>>>>>>>> You may unsubscribe by emailing
>>> discuss-unsubscribe AT lists.opennicproject.org
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> 
>>>>>>>>> Rouben
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --------
>>>>>>>>> You are a member of the OpenNIC Discuss list.
>>>>>>>>> You may unsubscribe by emailing
>>> discuss-unsubscribe AT lists.opennicproject.org
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --------
>>>>>>>> You are a member of the OpenNIC Discuss list.  
>>>>>>>> You may unsubscribe by emailing
>>> discuss-unsubscribe AT lists.opennicproject.org
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --------
>>>>>>> You are a member of the OpenNIC Discuss list.
>>>>>>> You may unsubscribe by emailing
>>> discuss-unsubscribe AT lists.opennicproject.org
>>>>>> 
>>>>>> 
>>>>>> --------
>>>>>> You are a member of the OpenNIC Discuss list. 
>>>>>> You may unsubscribe by emailing
>>> discuss-unsubscribe AT lists.opennicproject.org
>>>>> 
>>>>> 
>>>>> 
>>>>> --------
>>>>> You are a member of the OpenNIC Discuss list.
>>>>> You may unsubscribe by emailing
>>> discuss-unsubscribe AT lists.opennicproject.org
>>>> 
>>>> -- 
>>>> 
>>>> Rouben
>>>> 
>>>> 
>>>> --------
>>>> You are a member of the OpenNIC Discuss list. 
>>>> You may unsubscribe by emailing
>>> discuss-unsubscribe AT lists.opennicproject.org
>>> 
>>> 
>>>
>------------------------------------------------------------------------
>>> 
>>> 
>>> 
>>> --------
>>> You are a member of the OpenNIC Discuss list. 
>>> You may unsubscribe by emailing
>>> discuss-unsubscribe AT lists.opennicproject.org
>> 
>> 
>> 
>> --------
>> You are a member of the OpenNIC Discuss list. 
>> You may unsubscribe by emailing
>discuss-unsubscribe AT lists.opennicproject.org
>
>
>------------------------------------------------------------------------
>
>
>
>--------
>You are a member of the OpenNIC Discuss list. 
>You may unsubscribe by emailing
>discuss-unsubscribe AT lists.opennicproject.org