473,326 Members | 2,255 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

Ping Aaron - [Rel: Spiders]

Aaron,
Apologies for the ping but, your the only one I can think of offhand that
will know what I'm looking for.

Basically, to cut a long story short, I run an sURL service (surl.co.uk)
that allows one to convert long URL's to short etc, and am looking to have a
search facility on it that will allow one to locate an sURL for a specific
subject.

Now, the problem I'm running into is, I'd like to have some sort of spider
that will go to each site in the database, and grab the title, keywords, and
description from the meta tags, As I've never written a spider in my life,
and have been unsuccessful in locating info on how to go about doing so, I
was hoping you could point me in the right direction?

--

Regards

Steven Burn
Ur I.T. Mate Group
www.it-mate.co.uk

Keeping it FREE!


Jul 19 '05 #1
2 1283
One way is to use ServerXMLHTTP (see http://www.aspfaq.com/2173). Once you
have the response text back, you can parse it for <title> and </title>, etc.

--
http://www.aspfaq.com/
(Reverse address to reply.)


"Steven Burn" <pv*@noyb.com> wrote in message
news:u5**************@tk2msftngp13.phx.gbl...
Aaron,
Apologies for the ping but, your the only one I can think of offhand that
will know what I'm looking for.

Basically, to cut a long story short, I run an sURL service (surl.co.uk)
that allows one to convert long URL's to short etc, and am looking to have a search facility on it that will allow one to locate an sURL for a specific
subject.

Now, the problem I'm running into is, I'd like to have some sort of spider
that will go to each site in the database, and grab the title, keywords, and description from the meta tags, As I've never written a spider in my life,
and have been unsuccessful in locating info on how to go about doing so, I
was hoping you could point me in the right direction?

--

Regards

Steven Burn
Ur I.T. Mate Group
www.it-mate.co.uk

Keeping it FREE!

Jul 19 '05 #2
Cheers Aaron..... I had a feeling that was going to be the response (I've
got a page on my site that uses it (checks whether a site is online or not),
but it's terribly slow for some reason)

--

Regards

Steven Burn
Ur I.T. Mate Group
www.it-mate.co.uk

Keeping it FREE!
"Aaron [SQL Server MVP]" <te*****@dnartreb.noraa> wrote in message
news:e6**************@TK2MSFTNGP09.phx.gbl...
One way is to use ServerXMLHTTP (see http://www.aspfaq.com/2173). Once you have the response text back, you can parse it for <title> and </title>, etc.
--
http://www.aspfaq.com/
(Reverse address to reply.)


"Steven Burn" <pv*@noyb.com> wrote in message
news:u5**************@tk2msftngp13.phx.gbl...
Aaron,
Apologies for the ping but, your the only one I can think of offhand that will know what I'm looking for.

Basically, to cut a long story short, I run an sURL service (surl.co.uk)
that allows one to convert long URL's to short etc, and am looking to have
a
search facility on it that will allow one to locate an sURL for a

specific subject.

Now, the problem I'm running into is, I'd like to have some sort of spider that will go to each site in the database, and grab the title, keywords,

and
description from the meta tags, As I've never written a spider in my life, and have been unsuccessful in locating info on how to go about doing so, I was hoping you could point me in the right direction?

--

Regards

Steven Burn
Ur I.T. Mate Group
www.it-mate.co.uk

Keeping it FREE!


Jul 19 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

16
by: Robbie | last post by:
Hi All This is a belter that my little brain can't handle. Basically I have 1 SQL table that contains the following fields: Stock Code Stock Desc Reference Transaction Date
2
by: David Lozzi | last post by:
I need to determine from ASP if the database server is up and running, and if it isn't ping another database server. I posted this before about a different client, and I was told to request a file...
3
by: RTL | last post by:
Hello all, Actually, we're letting several of our domain names go back into the field. I believe there must be some industry way to use some HTML element/tag to get the Spiders to start updating...
4
by: Tony | last post by:
Hi everyone, Does anyone have any idea whether the "modern" email spiders are capable to understand javascript. If one makes a search with google one finds out that it is widely believed that...
26
by: Harrie | last post by:
Hi, After Brian mentioned the use for <link rel=..> for navigational purposes in another thread, I've been looking into it and found that HTML 3.2 has two other recognized link types than HTML...
10
by: moondaddy | last post by:
I'm writing an ecommerce app in asp.net/vb.net and need to make the pages searchable and crawlable by spiders, particularly Google's. As far as I know, if my pages's contents are mostly populated...
5
by: Deepak | last post by:
I am programing a ping application which pings various centers . I used timer loop and it pings one by one. Now when i finish pinging one center it should wait for the ping_completed function to...
6
by: ewolfman | last post by:
Hi, Recently we've hired professional SEO services to help up promote our website. They claim that pages which contain ASP.NET's Gridview with paging will not be scanned by the different spiders...
6
by: Dave Marden | last post by:
I currently use this routine in vbscript to ping computers and get the status of ping to determine whether to try to backup a machine, I am trying to change it to work with vb2003.net I am...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.