Mini web crawler

[=?Utf-8?B?ZWxhZGxh?=]

Hi,
I have a table in my DB with a list of URLs.
What I want to do is go to each URL and get a div tag with an ID
"ArticalTitle" and get the inner HTML of that div.

What would be the best way to do this? (I`m talking about the HTML Parsing
part, I got the DB stuff covered)

Thanks!

[=?Utf-8?B?UGV0ZXIgQnJvbWJlcmcgW0MjIE1WUF0=?=]

Take a look at Simon Mourier's "HtmlAgilityPack". This provides XML DOM -
type XPATH access to the object model of a web page.
Peter

--
Site: http://www.eggheadcafe.com
UnBlog: http://petesbloggerama.blogspot.com
Short urls & more: http://ittyurl.net


"eladla" wrote:

Hi,
I have a table in my DB with a list of URLs.
What I want to do is go to each URL and get a div tag with an ID
"ArticalTitle" and get the inner HTML of that div.

What would be the best way to do this? (I`m talking about the HTML Parsing
part, I got the DB stuff covered)

Thanks!

[Kevin Spencer]

Use a Regular Expression:

(?i)(?s)id="ArticleTitle"[^>]*>([^<]*)

The InnerHtml will be in Groups[1]

--
HTH,

Kevin Spencer
Microsoft MVP

Printing Components, Email Components,
FTP Client Classes, Enhanced Data Controls, much more.
DSI PrintManager, Miradyne Component Libraries:
http://www.miradyne.net

"eladla" <el****@newsgroups.nospamwrote in message
news:EF**********************************@microsof t.com...

Hi,
I have a table in my DB with a list of URLs.
What I want to do is go to each URL and get a div tag with an ID
"ArticalTitle" and get the inner HTML of that div.

What would be the best way to do this? (I`m talking about the HTML Parsing
part, I got the DB stuff covered)

Thanks!