Help with WWW::Mechanize Module using Authentication

[numberwhun]

I am having an issue with understanding something in the WWW::Mechanize module. I have a website which I want to download a whole plethora of pdf files from. It is a site that I have paid to access and it is perfectly legal for me to download them, but there are FAR too many files to download by hand so I want to automate the process. The problem is, is that the site has a login page (see http://stampalbums.com/worldwide_list.asp).

I was looking at the page for How to get WWW::Mechanize to handle authentication, but it is a little vague on somethings.

I took the code that they have:

   use MIME::Base64;
 
    my $agent = WWW::Mechanize->new();
    my @args = (
        Authorization => "Basic " .
            MIME::Base64::encode( USER . ':' . PASS )
    );
 
    $agent->credentials( ADDRESS, REALM, USER, PASS );
    $agent->get( URL, @args );
 

and tried to set it up. I created variables for USER, PASS, and URL. My questions is, what should ADDRESS and REALM be? Is ADDRESS the base url? (ie: stampalbums.com ) and the realm the rest of the address starting with the "/" after the base url? There is no explanation on either the link above or the LWP::UserAgent page, which is what is used with the 4 option version above.

Please, any assistance you can provide will be GREATLY appreciated. No, I am not going to say "URGENT" or "ASAP" as that is my issue, but I am just trying to get this done before I have to pay for another year. :-) You know how it is. If I can't get it working that quick, its only $20 / year.

Anywho, thanks in advance for any and all help you can give. I really appreciate it!!!

Regards,

Jeff

[numberwhun]

Ok, I was actually able to figure this one out. It took a fair amount of research and some creative thinking, but I did it. The code below will log into the site and download ALL of the pdf files on the page. For my own security, I have removed my login and password from the code. (I am sure you understand why).

use strict;
use warnings;
use WWW::Mechanize;
use File::Basename;
 
my $url = "http://stampalbums.com/worldwide_list.asp";
my $tempfile = "temp.txt";
 
my $mech = WWW::Mechanize->new;
$mech->get('http://stampalbums.com/worldwide_list.asp');
$mech->submit_form(with_fields => {
    USERNAME         => '*****',
    PASSWORD        => '*****'
});
 
#$mech->get($url, ':content_file' => $tempfile);
$mech->get($url);
 
#Get all the tarbulls
my @links = $mech->find_all_links(url_regex => qr/\.pdf/);
my @urls = map { $_->[0] } @links;
 
print("Found ", scalar @urls, " files to download\n");
 
for my $url1 (@urls)
{
    my $filename = basename($url1);
    print("$filename\n");
    $mech->get($url1, ':content_file'=>$filename);
}
 

Regards,

Jeff