By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,364 Members | 1,278 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,364 IT Pros & Developers. It's quick & easy.

Screen Scraping C#

P: n/a
I've seen a lot about screen scraping with .NET, mostly in I have
been able to convert most of it over, but it is still just very basic stuff.

Can someone help direct me toward some good info / samples on the following:

I want to be able to do 3 things:
1) Set up a module in IBUYSPY Portal (like in the right or left pane) that
simply just scrapes headlines from certain sites.
2) Allow users to set up scraping of certain sites that they want -- I
realize this is VERY hard to do, but the following I thought might help:
3) Possibly have a worker process that would download the source of a site
(regardless of the language it's written in) and save it as an HTML stream
(I have this part down -- there are tons of examples out there), then parse
out the content wanted. However, as content changes on a regular basis on
some sites, including structure, is there a way to automate this? I know I
couldn't automate the parsing, but I thought maybe an ADMIN function that
simply downloads sites and a) checks to see if structure has changed, if
not, then no problem; or b) if a site structure has changed, then dump the
HTML stream in a file and send off a notification to the ADMIN to go and
look at it to set up proper parsing.

Any insight would be much appreciated.
Nov 17 '05 #1
Share this question for a faster answer!
Share on Google+

This discussion thread is closed

Replies have been disabled for this discussion.