469,270 Members | 1,731 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,270 developers. It's quick & easy.

Perl LWP content request behind firewall and proxy server

Hi,
I am trying to fetch a web content using LWP and HTTP modules behind
corp firewall. I set proxy server as shown in my internet explorer
connection setup.
When i run the script, i got successful response, but the return
content is not the webpage i requested at all. It shows something
like:
-------------------------------------------------
function FindProxyForURL(url,host) {
me=myIpAddress();
if (me == "127.0.0.1") {return "DIRECT";}
if (host == "127.0.0.1") {return "DIRECT";}
if (host == "localhost") {return "DIRECT";}
if (isPlainHostName(host)) {return "DIRECT";}
..
..
..
---------------------------------------------------

Below is my sample script:
require LWP::UserAgent;
require HTTP::Request;

my $ua = LWP::UserAgent->new;
$ua->timeout(10);
$ua->proxy(['http', 'ftp'], 'http://autoproxy:9090');

my $request = HTTP::Request->new(POST => 'http://www.google.com');
my $response = $ua->request($request);
if ($response->is_success) {
print $response->content; # or whatever
}
else {
die $response->status_line;
}

Anyone can help me on this? will appreciate.

by,
demon hunter
Jul 19 '05 #1
3 11514
demonhunter wrote:
I am trying to fetch a web content using LWP and HTTP modules
behind corp firewall.
<snip>
my $request = HTTP::Request->new(POST => 'http://www.google.com');

------------------------------------^^^^

A POST request to get a web page?

--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl
Jul 19 '05 #2
Gunnar Hjalmarsson <no*****@gunnar.cc> wrote in message news:<yU***************@newsb.telia.net>...
demonhunter wrote:
I am trying to fetch a web content using LWP and HTTP modules
behind corp firewall.


<snip>
my $request = HTTP::Request->new(POST => 'http://www.google.com');

------------------------------------^^^^

A POST request to get a web page?


Oh, that suppose to be a GET, i did some changes here and there when
it doesnt work in GET. And it gave me the same
"FindProxyForURL(url,host)" content. Any idea why?
Jul 19 '05 #3
de**********@hotmail.com (demonhunter) wrote in message news:<b3**************************@posting.google. com>...
I am trying to fetch a web content using LWP and HTTP modules behind
corp firewall. I set proxy server as shown in my internet explorer
connection setup. $ua->proxy(['http', 'ftp'], 'http://autoproxy:9090'); When i run the script, i got successful response, but the return
content is not the webpage i requested at all. It shows something
like:
-------------------------------------------------
function FindProxyForURL(url,host) {
me=myIpAddress();
if (me == "127.0.0.1") {return "DIRECT";}
if (host == "127.0.0.1") {return "DIRECT";}
if (host == "localhost") {return "DIRECT";}
if (isPlainHostName(host)) {return "DIRECT";}
.
.
.
---------------------------------------------------


Go back and look at the internet explorer connection setup again.

Where did you see 'http://autoproxy:9090'?

I suspect it was not the address of the proxy. I suspect was the URL
of the proxy autoconfiguration script. I further suppose that tje
HTTP server at autoproxy:9090 is very simplistic and just returns the
PAC in response to all GET requests.

AFAIK LWP doesn't understand PACs (because it would require a full
JavaScript subsystem). (Actually there are people working on
JavaScript for LWP so one day soon there may be PAC support).

To figure out the address of the proxy examine the PAC yourself and
look for a line that says:

return "PROXY host:port";

This newsgroup does not exist (see FAQ). Please do not start threads
here.
Jul 19 '05 #4

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

5 posts views Thread by Bill Hauver | last post: by
1 post views Thread by CARIGAR | last post: by
reply views Thread by zhoujie | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.