By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
428,684 Members | 1,439 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 428,684 IT Pros & Developers. It's quick & easy.

Web Crawling Spidering Question

P: n/a
My boss just gave me a project to create a spider that crawls a series of
web pages and harvests certain pieces of data they contain and store that
data in a database. Of course he needs all this done sooner than may be
realistic.

Can anyone point me to some samples, links or articles that might help me
significantly jump start my efforts on this? Thanks.
Jun 1 '07 #1
Share this Question
Share on Google+
3 Replies


P: n/a
Rusty,

Take a look at the HttpWebRequest and HttpWebResponse classes. These
will help you get the contents of the page.

As for placing the data in a database, if you are using Sql Server, take
a look at the System.Data.SqlClient namespace, as they will help you with
getting the data into the database.
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com

"Rusty Hill" <Ru********@hotmail.comwrote in message
news:Oh**************@TK2MSFTNGP03.phx.gbl...
My boss just gave me a project to create a spider that crawls a series of
web pages and harvests certain pieces of data they contain and store that
data in a database. Of course he needs all this done sooner than may be
realistic.

Can anyone point me to some samples, links or articles that might help me
significantly jump start my efforts on this? Thanks.


Jun 1 '07 #2

P: n/a
Have a look for searcharoo on codeproject.com.

--
Best regards,
Dave Colliver.
http://www.AshfieldFOCUS.com
~~
http://www.FOCUSPortals.com - Local franchises available
"Rusty Hill" <Ru********@hotmail.comwrote in message
news:Oh**************@TK2MSFTNGP03.phx.gbl...
My boss just gave me a project to create a spider that crawls a series of
web pages and harvests certain pieces of data they contain and store that
data in a database. Of course he needs all this done sooner than may be
realistic.

Can anyone point me to some samples, links or articles that might help me
significantly jump start my efforts on this? Thanks.


Jun 1 '07 #3

P: n/a
u will need httpwebrequest and hhtpwebresponse class in system.net
namespace and also an html parser to parse what you want.i have an
project about this.
Jun 3 '07 #4

This discussion thread is closed

Replies have been disabled for this discussion.