Skip to Content.
Sympa Menu

discuss - Re: [opennic-discuss] seed for initial crawl...

discuss AT lists.opennicproject.org

Subject: Discuss mailing list

List archive

Re: [opennic-discuss] seed for initial crawl...


Chronological Thread 
  • From: "JP Blankert (thuis & PC based)" <jpblankert AT zonnet.nl>
  • To: discuss AT lists.opennicproject.org
  • Subject: Re: [opennic-discuss] seed for initial crawl...
  • Date: Thu, 26 May 2011 13:21:04 +0200
  • List-archive: <http://lists.darkdna.net/pipermail/discuss>
  • List-id: <discuss.lists.opennicproject.org>

What program rule has to be left out of a 'normal search engine' in order to exclude non-IANA?

I would have guessed: use opnennic (.geek etc.) sites as seed sites, they refer to other opennic AND IANA anyway

BR,

Philippe

On 26-5-2011 10:41, Rene Paulokat wrote:
On Thu, 26 May 2011 16:54:14 +1000
Julian DeMarchi <julian AT jdcomputers.com.au> wrote:

any ideas / hints?

how is grep.geek/search.geek initiating its data?
I will share Jeff's blackmagic for indexing OpenNIC space. You AXFR the
TLD zone from the master server and use some script to grab all the
domains out of that.

>From there, you have some domains to work with.
oaky - this is what i thought.
made some prototyping yesterday - and noticed again that there is a vast majority of either down or not responding IN A's in the opennic-tlds. 

tested quick and dirty:  http://paste.null/view.php?id=21

thanks anyway.

lg
rene


Jeff - Can you fill in the blanks, share your script maybe?

--julian
_______________________________________________
discuss mailing list
discuss AT lists.opennicproject.org
http://lists.darkdna.net/mailman/listinfo/discuss
_______________________________________________
discuss mailing list
discuss AT lists.opennicproject.org
http://lists.darkdna.net/mailman/listinfo/discuss


Geen virus gevonden in het binnenkomende-bericht.
Gecontroleerd door AVG - www.avg.com 
Versie: 9.0.901 / Virusdatabase: 271.1.1/3660 - datum van uitgifte: 05/25/11 20:09:00





Archive powered by MHonArc 2.6.19.

Top of Page