Python equivalent of LWP and HTTP in Perl

[Dekaritae]

I have a script that I've written in Perl that retrieves files generated
from a template. It works decently enough, but I'd like to rewrite it in
Python (Perl was just a detour; it was originally Sed).

Was wondering what the closest thing to what I'm using now in Perl
(LWP::UserAgent and HTTP::Request:C ommon) is in Python. The main bit of
my code is fairly simple.

my($ua) = LWP::UserAgent->new;
my $req = GET "$uri";
$req->header(Refer er => "$referer") ;
$ua->proxy('http' , 'http://localhost:8080/'); # Proxomitron
$response = $ua->request($req );
$respcode = $response->code;

[Irmen de Jong]

Dekaritae wrote:

I have a script that I've written in Perl that retrieves files generated
from a template. It works decently enough, but I'd like to rewrite it in
Python (Perl was just a detour; it was originally Sed). my($ua) = LWP::UserAgent->new;
my $req = GET "$uri";
$req->header(Refer er => "$referer") ;
$ua->proxy('http' , 'http://localhost:8080/'); # Proxomitron
$response = $ua->request($req );
$respcode = $response->code;

import urllib
proxies = {'http': 'http://localhost:8080/'}
response = urllib.urlopen( "some_url", proxies=proxies ).read()

--Irmen

[Erik Johnson]

"Irmen de Jong" wrote

my($ua) = LWP::UserAgent->new;
my $req = GET "$uri";
$req->header(Refer er => "$referer") ;
$ua->proxy('http' , 'http://localhost:8080/'); # Proxomitron
$response = $ua->request($req );
$respcode = $response->code;

import urllib
proxies = {'http': 'http://localhost:8080/'}
response = urllib.urlopen( "some_url", proxies=proxies ).read()

.... which is short and clean, but ignores the Referer header.
I have worked with httplib some, but not urllib. Maybe urllib can do this,
but may I suggest checking out httplib:
http://docs.python.org/lib/httplib-examples.html

-ej

[David M. Cooke]

"Erik Johnson" <sp**@nospam.or g> writes:

"Irmen de Jong" wrote

> my($ua) = LWP::UserAgent->new;
> my $req = GET "$uri";
> $req->header(Refer er => "$referer") ;
> $ua->proxy('http' , 'http://localhost:8080/'); # Proxomitron
> $response = $ua->request($req );
> $respcode = $response->code;

import urllib
proxies = {'http': 'http://localhost:8080/'}
response = urllib.urlopen( "some_url", proxies=proxies ).read()

... which is short and clean, but ignores the Referer header.
I have worked with httplib some, but not urllib. Maybe urllib can do this,
but may I suggest checking out httplib:
http://docs.python.org/lib/httplib-examples.html

Try urllib2:

import urllib2

proxy_support = urllib2.ProxyHa ndler({"http" : "http://localhost:8080/"})
opener = urllib2.build_o pener(proxy_sup port)
urllib2.install _opener(opener)

req = urllib2.Request ("some_url")
req.add_header( "Referer", referer)
response = urllib2.urlopen (req)
There's fancier handlers too, if you need digest authentication, for
example. If you have the environment variable 'http_proxy' set to your
proxy, you don't need to do the install_opener jazz.

--
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)phy sics(dot)mcmast er(dot)ca

[Michael Foord]

Dekaritae <ma**@mdi.ca> wrote in message news:<vA******* ******@fe61.use netserver.com>. ..

I have a script that I've written in Perl that retrieves files generated
from a template. It works decently enough, but I'd like to rewrite it in
Python (Perl was just a detour; it was originally Sed).

Was wondering what the closest thing to what I'm using now in Perl
(LWP::UserAgent and HTTP::Request:C ommon) is in Python. The main bit of
my code is fairly simple.

my($ua) = LWP::UserAgent->new;
my $req = GET "$uri";
$req->header(Refer er => "$referer") ;
$ua->proxy('http' , 'http://localhost:8080/'); # Proxomitron
$response = $ua->request($req );
$respcode = $response->code;

urllib2 is definitely the way forward for adding proxy support and
request headers. Very easy. If you want cookie handling you'll need to
look at ClientCookie which does similar things to the perl library you
mention.

Regards,

Fuzzy
http://www.voidspace.org.uk/atlantib...thonutils.html