How to harness information of the web

Hello Everyone,

I am not sure if I posting this to the right forum or not. So, I apologize before hand if this is not the forum for posting this question.

I want a script that would run weather in Windows or Linux that will gather the information of a website and save it in a file (maybe to a .xls file). I want to use a program because I want to search more than 1000s of pages. However, the good thing is that each page is identical to the other. For example, the program browses: http://www.xxxxxx.yyy/1.html and grabs a variable's content from inside and saves it to a file. Next it goes to: http://www.xxxxxx.yyy/2.html and does the same thing.

Can v.b. do this?

Thanks

re: How to harness information of the web

vb . net can, try ' googling ' for ' webclient ' and ' regexes ' .

I have not done much with either of these but I believe that you download the page using the webclient and use a regex to scan through the code to find the information that you are looking for

there will be other ways of doing this but for people to advise you might need to explain what type of data and webpage you are looking at