473,473 Members | 2,138 Online
Bytes | Software Development & Data Engineering Community
Create Post

Home Posts Topics Members FAQ

spider a website

How can I go about sending a spider to a website and retrieving data? I want
to sell product for a company that claims the cannot query their database.
Ridiculous! Anyway, is there a way that I can automate someway of collecting
data from their site? I would need UPC codes, descriptions etc.

thanks
Jul 19 '05 #1
1 1620
I wouldn't even consider ASP for a 'spider' - unless you mean just
hitting ONE page and getting back the data from it. A spider generally
runs for a long long time, something which ASP is NOT good at.

hitting a page once is easy, though - there's an article in the ASP
section of my site on it using either XMLHTTP or ASPHTTP

________________________________________
Atrax. MVP, IIS
http://rtfm.atrax.co.uk/

newsflash : Atrax.Richedit 1.0 now released.
http://rtfm.atrax.co.uk/infinitemonk...trax.RichEdit/

*** Sent via Developersdex http://www.developersdex.com ***
Don't just participate in USENET...get rewarded for it!
Jul 19 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Kyle Mizell | last post by:
I am looking for a script that I can use to spider a website, and then pull the images... I know how to do it for a single page, but, I would like to be able to do this for the entire site. Any...
3
by: Thomas Lindgaard | last post by:
Hello I'm a newcomer to the world of Python trying to write a web spider. I downloaded the skeleton from http://starship.python.net/crew/aahz/OSCON2001/ThreadPoolSpider.py Some of the...
3
by: martijn | last post by:
H! I thought I was ready with my own spider... But then there was a bug, or in other words a missing part in my code. I forget that people do this in website html: <a...
6
by: Erik Steffl | last post by:
I am trying to create a simple custom web spider using mozilla and javascript, the basic functionality is to open a website and then manipulate it using DOM (possibly opening links etc.). it...
0
by: dtsearch | last post by:
New release expands-through a .NET Spider API, to Linux, and to OpenOffice-dtSearch's ability to index over a terabyte of text in a single index, with indexed search time typically less than a...
7
by: baroque Chou | last post by:
anyone know how google spiders access web site, how dose they manage to get the href information? do they have special access right or something? any help is appreciated
3
by: Tony Lance | last post by:
Big Bertha Thing spider Cosmic Ray Series Possible Real World System Constructs http://web.onetel.com/~tonylance/spider.html Access page JPG 11K Image Astrophysics net ring Access site...
2
by: =?Utf-8?B?Q2hhcnRz?= | last post by:
I have been writing C# programs to spider yellow page to get list of restaurant name, address to the database. When I encounter button or hyperlink, I don’t know how to use the program to click...
0
by: kishorealla | last post by:
Hello I need to create a web bot/crawler/spider that would go into different web sites and collect data for us and store in a database. The crawler needs to 'READ' the options on a website (either...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
1
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
1
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...
0
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The...
0
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated ...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.