LWP - Slurp the links of a webpage?

[cobra2341]

Hi again. I used LWP to download a html page. That page has over 140 links on it. What I am trying to do is get Perl to go through the main page and go to the links and download there info too. Thanks

[numberwhun]

Well, it depends, if you are just looking to download the HTML page itself, then you may want to check out the LWP::Simple package, instead of plain old LWP.

Why don't you post the code you are trying and describe what your goal(s) are and we can help you from there.

Regards,

Jeff

[cobra2341]

Ok.
I Alreadly have the page downloaded. It is this
If you look on that site, it has prior years, like 2005. When you click that link it goes to another page that has a list of players. When you click on a player it has all there info. What I want perl to do is download that page for all the players for all teh years.
thanks

[KevinADC]

Herding cats Jeff, good luck ;)

[numberwhun]

Ok.
I Alreadly have the page downloaded. It is this
If you look on that site, it has prior years, like 2005. When you click that link it goes to another page that has a list of players. When you click on a player it has all there info. What I want perl to do is download that page for all the players for all teh years.
thanks

Why is it things are never as simple as when you first explain them. I am no LWP expert. I have used it to download a page, parse out some info and use that elsewhere. If I were posed with this situation, I would download the page, as you have done, then parse out the subsequent links that you want (possibly loading them into an array), and then using LWP's get function once again to cycle through the array, downloading each link. Basically, you will have copies of the HTML. From there, you can use those pages to display the information any way you like.

Unless someone knows of a better way to do this, you are going to be writing a fair amount of code for this.

Regards,

Jeff

***Meeeooowww***

[cobra2341]

Sorry Jeff, but I don't know how to do what you said. I tryied reading the LWP page, but I try again. Thanks

[KevinADC]

look into WWW::Mechanize, which I think can be "told" to follow links. But I have little practical experience using the module so I will not be able to help with code.

[cobra2341]

I was at Perl Monks and have learned a little on Mech. Could anyone here give me some more info