Need to crawl website

[Danny]

I am trying to crawl my site to get a list of all the links. I am using the
regular
expressions to get the href tags from the pages and reading the source pages
using
xmlhttp module.

Is there an efficient way to loop through the links?
I am looping through the links and avoiding the duplicate links, but it is
taking over 2 hours to crawl my site!!
What am i doing wrong? What is making it sooooo slow.

Thanks again

[WindAndWaves]

there is a great program out there called Xenu.exe
http://home.snafu.de/tilman/xenulink.html

I have tried it and really liked it. One of the best programs out there.

Does that help???
---
Please immediately let us know (by phone or return email) if (a) this email
contains a virus
(b) you are not the intended recipient
(c) you consider this email to be spam.
We have done our utmost to make sure that
none of the above are applicable. THANK YOU
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.691 / Virus Database: 452 - Release Date: 26/05/2004
---
Please immediately let us know (by phone or return email) if (a) this email
contains a virus
(b) you are not the intended recipient
(c) you consider this email to be spam.
We have done our utmost to make sure that
none of the above are applicable. THANK YOU
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.691 / Virus Database: 452 - Release Date: 26/05/2004