HTTP file download

[mohanprasadgutta]

Hello,
I have written this code to make a file local before start working on it.
But when i execute it is not copying the file to local. I am getting 401 error.
could you please let me know whats wrong with this code.
thaks in advance.

#!/usr/bin/perl -w
use LWP::Simple;
 
my $filename = "http://sample.com/secure/sample.xls";
 
MakeFileLocal($filename);
 
sub MakeFileLocal {
  my $filename = shift;
 
  chomp($filename);
 
  $filename =~ m/.*\.(.*)$/;
  my $suffix = $1;
  print "Suffix : $suffix\n";
  my $newname = "d:/test/mohan/temp.$suffix";
 
  unlink("$newname");
 
  if ($filename =~ m/^http/i) {
    print "Name matches starts with http://\n";
    my $retval = getstore($filename, $newname);
    die "Could not store it!!\n" unless is_success($retval);
  } else {
    my $cmd = qq~copy "$filename" "$newname"~;
    print "$cmd\n";
    my @a = `$cmd`;
    }
 
  if (-e "$newname") {
    return $newname;
  }
 
  return 0;
  }
 

[KevinADC]

401 is document not found error. Are you sure the URI to the remote file is correct?

http://sample.com/secure/sample.xls

[mohanprasadgutta]

401 is document not found error. Are you sure the URI to the remote file is correct?

http://sample.com/secure/sample.xls

hello Kevin,
Document is available at the same url.
when i use the same url in browser it is asking me to save or open it through file download alert message box.

[KevinADC]

Hard to say, is there a firewall or proxy server maybe causing the error?

[mohanprasadgutta]

the server is a microsoft biztalk server.
So i am not sure about whether any firewall behind it.
As it is a corporate machine i believe definitely there will be firewall.
If a firewall is available does it block such type of requests.
I dont have much knowledge on firewalls working.
thanks in advance.

[KevinADC]

I don't know enough to be able to help you with firewall problems. I just tested your code and it works for me:

#!/usr/bin/perl -w
use LWP::Simple;
my $filename = "http://www.e-pixs.com/smilies/smiley10.gif";
MakeFileLocal($filename);
sub MakeFileLocal {
  my $filename = shift;
  chomp($filename);
  $filename =~ m/.*\.(.*)$/;
  my $suffix = $1;
  print "Suffix : $suffix\n";
  my $newname = "c:/perl_test/smiley10.$suffix";
#  unlink("$newname");
  if ($filename =~ m/^http/i) {
    print "Name matches starts with http://\n";
    my $retval = getstore($filename, $newname);
    die "Could not store it!!\n" unless is_success($retval);
  } else {
    my $cmd = qq~copy "$filename" "$newname"~;
    print "$cmd\n";
    my @a = `$cmd`;
    }
  if (-e "$newname") {
    return $newname;
  }
  return 0;
  }
 

the script got the gif from the site and stored it on my server.

[mohanprasadgutta]

I don't know enough to be able to help you with firewall problems. I just tested your code and it works for me:

#!/usr/bin/perl -w
use LWP::Simple;
my $filename = "http://www.e-pixs.com/smilies/smiley10.gif";
MakeFileLocal($filename);
sub MakeFileLocal {
  my $filename = shift;
  chomp($filename);
  $filename =~ m/.*\.(.*)$/;
  my $suffix = $1;
  print "Suffix : $suffix\n";
  my $newname = "c:/perl_test/smiley10.$suffix";
#  unlink("$newname");
  if ($filename =~ m/^http/i) {
    print "Name matches starts with http://\n";
    my $retval = getstore($filename, $newname);
    die "Could not store it!!\n" unless is_success($retval);
  } else {
    my $cmd = qq~copy "$filename" "$newname"~;
    print "$cmd\n";
    my @a = `$cmd`;
    }
  if (-e "$newname") {
    return $newname;
  }
  return 0;
  }
 

the script got the gif from the site and stored it on my server.

Hello Kevin,
Thank you very for your response.
From this discussion i got to know nothing wrong in the code i have written.
The only problem is may be due to firewall or proxy server.
I will discuss this thing further with that server administrator.

[KevinADC]

OK, you're welcome, good luck.

Kevin

[mohanprasadgutta]

OK, you're welcome, good luck.

Kevin

Hello,
As i am getting 401 error, i tried to understand the reason behind this error.
i found it will come if we don't have access to the particular links

[HTML]<a href="http://en.wikipedia.org/wiki/HTTP_401"> 401 error </a>[/HTML]

So later i modified my code so that it will send authentication details as well.
and when i requested my administrator he was telling i am having read access to the directory from where i am going to download the file.
so my new script is as follows.

#!/usr/bin/perl 
use LWP::UserAgent;
#my $url = 'http://si-airbag-www.de.bosch.com/cpc/Default.aspx'; 
my $url = 'http://boschbiz.us.bosch.com/sites/projcentral/1810/Secured%20Engineering/60.Software/A.SW_Project_Management/00.%20Schedule/DC10_60_A_00_SW_Schedule_Full.mpp'; 
my $ua = new LWP::UserAgent(keep_alive=>1);
$ua->no_proxy;
#$ua->agent('MSIE/6.0');
$ua->credentials('sample.com:80','sites','mohan' => 'mohan123');
$ua->timeout(15);
my $request = HTTP::Request->new('GET');
$request->url($url); 
my $response = $ua->request($request);  
my $code = $response->code; 
print "Return Code ====>\n $code\n";
my $headers = $response->headers_as_string;
#If the Error is 401.2 then, 
#The authentication methods that were tried are either disabled, or you are attempting to use NTLM through a proxy server.
print "Headers ======>\n $headers\n";
# HTML body:
my $body =  $response->content;
print "================\n$body====================\n";

i am getting the following output after executing the script.

Return Code ====>
401
Headers ======>
Date: Mon, 10 Nov 2008 04:53:08 GMT
Server: Microsoft-IIS/6.0
WWW-Authenticate: NTLM
Content-Length: 1656
Content-Type: text/html
Content-Type: text/html; charset=Windows-1252
Client-Date: Mon, 10 Nov 2008 04:53:08 GMT
Client-Peer: 10.24.250.27:80
Client-Response-Num: 1
Client-Warning: Unsupported authentication scheme 'ntlm'
MicrosoftSharePointTeamServices: 12.0.0.6219
Set-Cookie: RBNA_Boschbiz_07=R731053916; path=/
Title: You are not authorized to view this page
X-Powered-By: ASP.NET

================
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">

<HTML><HEAD><TITLE>You are not authorized to view this page</TITLE>

<META HTTP-EQUIV="Content-Type" Content="text/html; charset=Windows-1252">

<STYLE type="text/css">

BODY { font: 8pt/12pt verdana }

H1 { font: 13pt/15pt verdana }

H2 { font: 8pt/12pt verdana }

A:link { color: red }

A:visited { color: maroon }

</STYLE>

</HEAD><BODY><TABLE width=500 border=0 cellspacing=10><TR><TD>



<h1>You are not authorized to view this page</h1>

You do not have permission to view this directory or page using the credentials that you supplied because your Web browser is sending a WWW-Authenticate header field that the Web server is not configured to accept.

<hr>

<p>Please try the following:</p>

<ul>

<li>Contact the Web site administrator if you believe you should be able to view this directory or page.</li>

<li>Click the <a href="javascript:location.reload()">Refresh</a> button to try again with different credentials.</li>

</ul>

<h2>HTTP Error 401.2 - Unauthorized: Access is denied due to server configuration.<br>Internet Information Services (IIS)</h2>

<hr>

<p>Technical Information (for support personnel)</p>

<ul>

<li>Go to <a href="http://go.microsoft.com/fwlink/?linkid=8180">Microsoft Product Support Services</a> and perform a title search for the words <b>HTTP</b> and <b>401</b>.</li>

<li>Open <b>IIS Help</b>, which is accessible in IIS Manager (inetmgr),

and search for topics titled <b>About Security</b>, <b>Authentication</b>, and <b>About Custom Error Messages</b>.</li>

</ul>



</TD></TR></TABLE></BODY></HTML>
====================

and anybody could let me know whether the new code is correct, i mean the way i am sending the authentication details.

And one more thing i want to know is how can we make sure the http request we are making is going through proxy?

thanks a lot in advance.

Best Regards,
Mohan