-----Original Message-----
From: James Mills [mailto:pr******@shortcircuit.net.au]
Sent: Sunday, October 26, 2008 5:26 PM
To: sonich
Cc:
py*********@python.org
Subject: Re: Web crawler on python
On Mon, Oct 27, 2008 at 6:54 AM, sonich <So************@gmail.comwrote:
I need simple web crawler,
I found Ruya, but it's seems not currently maintained.
Does anybody know good web crawler on python or with python interface?
Simple, but it works. Extend it all you like.
http://hg.softcircuit.com.au/index.w...330d047ff663/e
xamples/spider.py
$ spider.py --help
Usage: spider.py [options] <url>
Options:
--version show program's version number and exit
-h, --help show this help message and exit
-q, --quiet Enable quiet mode
-l, --links Get links for specified url only
-d DEPTH, --depth=DEPTH
Maximum depth to traverse
cheers
James
--
--
-- "Problems are solved by method"