Skip to Content.
Sympa Menu

discuss - Re: [opennic-discuss] Looking for logs (just the domains) for censorship research

discuss AT lists.opennicproject.org

Subject: Discuss mailing list

List archive

Re: [opennic-discuss] Looking for logs (just the domains) for censorship research


Chronological Thread 
  • From: Alejandro Bonet <albogoal AT gmail.com>
  • To: discuss AT lists.opennicproject.org
  • Subject: Re: [opennic-discuss] Looking for logs (just the domains) for censorship research
  • Date: Tue, 4 Mar 2014 13:45:04 +0100

Hi again Frank:

Here you have another 2M7 domains list built asking google with dictionary
using wget and getting the domains from the google html responses:

http://185.16.40.143/estatico.babel7.com/alargador/dominios2.7z

(until google ban the queries...)

Alejandro Bonet
albogoal AT gmail.com


2014-03-04 13:31 GMT+01:00, Alejandro Bonet <albogoal AT gmail.com>:
> Hi Frank:
>
> Here you have a list of about 900.000 different domains queried by the
> people
> to my server since August 2013:
>
> http://185.16.40.143/estatico.babel7.com/alargador/dominios.7z
>
> This dont include in-addr.arpa reverse lookup domains, and it is
> alphabetically sorted.
>
> Rememeber these are QUERIES (not answers), and thus some of them could
> don't exist...
>
>
> Alejandro Bonet
> albogoal AT gmail.com
>
> PD: Which country government in west is trying to censor internet?
>
>
>
>
> 2014-03-03 21:08 GMT+01:00, Guillaume Parent <gparent AT gparent.org>:
>> Thanks for the explanations, it's likely that I'll contact you a bit
>> later
>> off list. Your work is appreciated, and do not take my refusal to hand
>> out
>> data as something I have against your project :) I hope others will
>> assist
>> you.
>>
>> -gp
>>
>>
>> On Mon, Mar 3, 2014 at 3:02 PM, Frank Minder <frminder AT yahoo.com> wrote:
>>
>>> Calum got it right, thanks for the explanation!
>>>
>>> I understand and appreciate that DNS operators are aware of their
>>> responsibility and I completely agree that you shouldn't share your
>>> complete log files with a random stranger who asks for it on a
>>> mailinglist.
>>> But I believe some people on this list are interested in the OpenNIC
>>> project because they dislike the idea of censorship on the Internet
>>> (like
>>> me) and would like to use their resources to help researching a secret
>>> government Internet censorship blacklist and debunk the mostly
>>> ridiculous
>>> implementations of those blacklists. I already found domains that are
>>> not
>>> registered anymore since several years, domains that used to host a porn
>>> website but now belong to startups that have no way to be successful in
>>> that country because of the censorship, several obvious typos and so on.
>>> I am explicitly not asking for the full logs with IPs, timestamps etc.,
>>> just the plain domains. Alexa provides a download of the top million
>>> domains according to their research (
>>> http://s3.amazonaws.com/alexa-static/top-1m.csv.zip), Quantcast as well
>>> (
>>> https://ak.quantcast.com/quantcast-top-million.zip). Hell, its not even
>>> difficult to get a *complete* list of all .com/net/org/info/biz etc.
>>> domains (
>>> http://www.leandomainsearch.com/blog/16-how-to-get-access-to-the-official-verisign--com-zone-file)
>>> -- I am asking for something very similar, just with subdomains and of
>>> course other TLDs as well. I don't see a big security/privacy issue
>>> here.
>>>
>>> If you would like to help but don't want to share your domainlist I can
>>> send you the hashes and show you how to use your local domainlist to try
>>> to
>>> recover the plaintext domains from the hashes. This would mean much more
>>> work for you and won't help me with the new hashes on the next update of
>>> the blacklist but it would still be appreciated very much.
>>> Just contact me off list.
>>>
>>>
>>> On Monday, March 3, 2014 7:50 PM, Guillaume Parent
>>> <gparent AT gparent.org>
>>> wrote:
>>> Yeah, if the idea is to identify individual domains rather than try and
>>> assess what percentage would be blocked then I'm afraid that personally
>>> I
>>> can't help you.
>>>
>>> -gp
>>>
>>>
>>> On Mon, Mar 3, 2014 at 2:45 PM, Hunter 9999 <mail AT hunter-9999.de> wrote:
>>>
>>> > Am 03.03.2014 um 19:19 schrieb Calum McAlinden
>>> > <calum AT mcalinden.me.uk>:
>>> >> On 3 March 2014 16:26, Guillaume Parent <gparent AT gparent.org> wrote:
>>> >> What you should do is tell us how to generate hashed lists so that we
>>> never
>>> >> have to give you logs in the first place.
>>> >>
>>> >> I assume most operators willing to give you this data would only do
>>> >> it
>>> with
>>> >> a strong one way hash pre-applied (such as SHA512)
>>> >
>>> > From what I understand, the individual wants to find out which
>>> > particular domains are on the list of hashes provided to the ISPs by
>>> > whichever government. He needs large lists of non-hashed domains to
>>> > hash and compare with the blocked lists; a matching hash will indicate
>>> > that particular domain being blocked by the government. Is this
>>> > correct? Sorry, I am unable to help as I do not operate any public DNS
>>> > servers.
>>>
>>> This is what I understand, too.
>>>
>>>
>>> --------
>>> You are a member of the OpenNIC Discuss list.
>>> You may unsubscribe by emailing
>>> discuss-unsubscribe AT lists.opennicproject.org
>>>
>>>
>>>
>>>
>>>
>>> --------
>>> You are a member of the OpenNIC Discuss list.
>>> You may unsubscribe by emailing
>>> discuss-unsubscribe AT lists.opennicproject.org
>>>
>>>
>>>
>>>
>>>
>>> --------
>>> You are a member of the OpenNIC Discuss list.
>>> You may unsubscribe by emailing
>>> discuss-unsubscribe AT lists.opennicproject.org
>>>
>>>
>>
>



Archive powered by MHonArc 2.6.19.

Top of Page