472,960 Members | 1,804 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,960 software developers and data experts.

Crawl a site

Is it possible to crawl a site using ASP & XML HTTP?

I know you can hit one link, but how can you go through each link in a
page and validate that it returns 200?
Jul 19 '05 #1
1 1555
Dave wrote:
Is it possible to crawl a site using ASP & XML HTTP?

I know you can hit one link, but how can you go through each link in a
page and validate that it returns 200?


Using PerlScript and the Robot module.

http://search.cpan.org/~awrigley/WWW...b/WWW/Robot.pm

Eric
Jul 19 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Steve Mauldin | last post by:
Can anyone recommend a good tool to crawl asp pages and make note of which Pages, Images, Includes, Etc. are used on a web site and which pages are not used? I have inherited a website that is...
1
by: Danny | last post by:
I am trying to crawl my site to get a list of links. I am using the regular expressions to get the href tags from the pages and reading the links using xmlhttp module. is there an efficient way...
0
by: Marlo Brandon | last post by:
Hiya , I want to create /find an ASP robot that will look at all the pages on a site (my site that is), and return every word, along with the URL, so I can put them all in my database. OK maybe...
1
by: Danny | last post by:
I am trying to crawl my site to get a list of all the links. I am using the regular expressions to get the href tags from the pages and reading the source pages using xmlhttp module. Is there...
2
by: mahesr | last post by:
Frds, how to crawl a specific word in whole website using php.need which page it has Example: want to crawl a 144 items found,on somepage.html. Pls if u have code pls mail email address...
2
by: John | last post by:
Hello everybody, I'm working on a Web site that reads a database file and diplays the contents in a datagrid my question is how can google index my page since in order to do that it needs to...
12
by: Snoopy33 | last post by:
I have a FE / BE setup on my database. One of the main reasons that I did this was so that I could edit the front end and upload it without causing problems with day to day activities. I have...
7
by: =?Utf-8?B?SklNLkgu?= | last post by:
How to get search engines crawl data I have a web application that uses user controls and pulls data directly from database and shows it to users in the internet. So there is not html that has the...
0
by: vinothPHP | last post by:
Hi Everyone, I want to make a crawler in php that will crawl site search results by images, links. when i am running the sample script in my localhost i am getting this error,...
0
by: lllomh | last post by:
Define the method first this.state = { buttonBackgroundColor: 'green', isBlinking: false, // A new status is added to identify whether the button is blinking or not } autoStart=()=>{
2
by: DJRhino | last post by:
Was curious if anyone else was having this same issue or not.... I was just Up/Down graded to windows 11 and now my access combo boxes are not acting right. With win 10 I could start typing...
4
NeoPa
by: NeoPa | last post by:
Hello everyone. I find myself stuck trying to find the VBA way to get Access to create a PDF of the currently-selected (and open) object (Form or Report). I know it can be done by selecting :...
3
NeoPa
by: NeoPa | last post by:
Introduction For this article I'll be using a very simple database which has Form (clsForm) & Report (clsReport) classes that simply handle making the calling Form invisible until the Form, or all...
1
by: Teri B | last post by:
Hi, I have created a sub-form Roles. In my course form the user selects the roles assigned to the course. 0ne-to-many. One course many roles. Then I created a report based on the Course form and...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 1 Nov 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM) Please note that the UK and Europe revert to winter time on...
3
by: nia12 | last post by:
Hi there, I am very new to Access so apologies if any of this is obvious/not clear. I am creating a data collection tool for health care employees to complete. It consists of a number of...
0
NeoPa
by: NeoPa | last post by:
Introduction For this article I'll be focusing on the Report (clsReport) class. This simply handles making the calling Form invisible until all of the Reports opened by it have been closed, when it...
0
isladogs
by: isladogs | last post by:
The next online meeting of the Access Europe User Group will be on Wednesday 6 Dec 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, Mike...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.