469,909 Members | 1,693 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,909 developers. It's quick & easy.

LWP with proxy problem

When running the sample code below without a proxy the GET returns the
expected data. When run with the $proxy uncommented the GET returns
the content of the login page for the site being accessed. The site
in the code is valid for ease of testing. I also am including the LWP
debug info for each attempt.

Can anyone explain this behavior and what can I do to correct it?

Thanks in advance

Trace without proxy:

LWP::UserAgent::new: ()
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET
https://squid.servebeer.com/getservi...ing&format=csv
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::collect: read 28 bytes
LWP::UserAgent::request: Simple response: OK

Home
PE
Trace with proxy:

LWP::UserAgent::new: ()
LWP::UserAgent::proxy: https http://148.245.207.85:8080
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET
https://squid.servebeer.com/getservi...ing&format=csv
LWP::UserAgent::_need_proxy: Proxied to http://148.245.207.85:8080
LWP::Protocol::http::request: ()
LWP::Protocol::collect: read 236 bytes
LWP::Protocol::collect: read 594 bytes
LWP::Protocol::collect: read 416 bytes
LWP::Protocol::collect: read 450 bytes
LWP::Protocol::collect: read 1017 bytes
LWP::Protocol::collect: read 443 bytes
LWP::Protocol::collect: read 643 bytes
LWP::UserAgent::request: Simple response: OK

<html lang="en">

<!-- Start Head -->
<head>
<title>

Error

</title>
<script language="JavaScript">
....

Here is the sample code:
#!/usr/bin/perl -w

use LWP::UserAgent;
use HTTP::Request;
use HTTP::Response;
use Crypt::SSLeay;

LWP::Debug::level('+');

$url = "https://squid.servebeer.com/getservices.do?user=Guest&password=JustLooking&for mat=csv";
#$proxy="http://xxx.xxx.xxx.xxx:8080";

$ua = LWP::UserAgent->new();

if (defined $proxy)
{
$ENV{HTTPS_PROXY} = $proxy;

# initialize from environment variables
$ua->env_proxy;
}

$req = HTTP::Request->new(GET => $url);
$response = $ua->request($req);
if ($response->is_error())
{
printf " %s\n", $response->status_line;
}

else
{
$content = $response->content();
print $content;
}

exit;
Jul 19 '05 #1
1 12140
Note: you'll reach more people at comp.lang.perl.misc. This newsgroup
does not officially exist, and so is not universally propogated.

LWP::UserAgent sends the wrong request for HTTPS connections by proxy.
In the docs for Crypt::SSLeay, you'll find this:
http://search.cpan.org/~chamas/Crypt...#PROXY_SUPPORT
LWP::UserAgent has its own methods of proxying which may work for you
and is likely incompatible with Crypt::SSLeay proxy support. To use
LWP::UserAgent proxy support, try something like:

my $ua = new LWP::UserAgent;
$ua->proxy([qw( https http )], "$proxy_ip:$proxy_port");

At the time of this writing, libwww v5.6 seems to proxy https requests
fine with an Apache mod_proxy server. It sends a line like:

GET https://www.nodeworks.com HTTP/1.1

to the proxy server, which is not the CONNECT request that some
proxies would expect, so this may not work with other proxy servers
than mod_proxy. The CONNECT method is used by Crypt::SSLeay's internal
proxy support.
Crypt::SSLeay Proxy Support

For native Crypt::SSLeay proxy support of https requests, you need to
set an environment variable HTTPS_PROXY to your proxy server & port,
as in:

# PROXY SUPPORT
$ENV{HTTPS_PROXY} = 'http://proxy_hostname_or_ip:port';
$ENV{HTTPS_PROXY} = '127.0.0.1:8080';

Use of the HTTPS_PROXY environment variable in this way is similar to
LWP::UserAgent->env_proxy() usage, but calling that method will likely
override or break the Crypt::SSLeay support, so do not mix the two.
</excerpt>

This is the code I use, *after* the UserAgent has been created:

# LWP proxying does not work for HTTPS
# This must cannot be set at the time $a is created
$ENV{HTTPS_PROXY} = $proxy;
Jul 19 '05 #2

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

reply views Thread by Robert | last post: by
1 post views Thread by Lentdave67t | last post: by
reply views Thread by Owen Jenkins | last post: by
reply views Thread by Jimmy Carlsson | last post: by
3 posts views Thread by Ohad Young | last post: by
reply views Thread by Chad Z. Hower aka Kudzu | last post: by
1 post views Thread by =?Utf-8?B?U3VzaGlTZWFu?= | last post: by
1 post views Thread by Waqarahmed | last post: by
reply views Thread by Salome Sato | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.