470,841 Members | 1,096 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 470,841 developers. It's quick & easy.

Urllib2 / add_password method

I'm working on learning how to use urllib2 to use a proxy server. I've
looked through the postings on this group, and it's been helpful. I
have not, however, found complete documentation on the add_password()
functions. Here's what I've got so far:

#========================================
import urllib2
url = 'http://some.server/form.php'

proxy_url = 'http://202.62.252.3:8080' # A publicly available anonymous
proxy server

proxy_handler = urllib2.ProxyHandler( {'http': proxy_url } )

proxy_auth_handler = urllib2.HTTPBasicAuthHandler()
proxy_auth_handler.add_password('realm', 'host', 'username',
'password')

opener = urllib2.build_opener(proxy_handler, proxy_auth_handler)

urllib2.install_opener(opener)

try:
handler = urllib2.urlopen(url)
except urllib2.URLError:
print "whoops!"
else:
print handler.read()
#=======================================

It works, but I don't really know what I'm doing with the
proxy_auth_handler part. Specifying username and password make sense,
but I haven't found documentation on what 'realm' and 'host' are for.
Shouldn't username & password be sufficient?

Thanks,
--Steve (mr************@hotmail.com)

Apr 28 '06 #1
2 6077
"mrstephengross" <mr************@hotmail.com> writes:
I'm working on learning how to use urllib2 to use a proxy server. I've
looked through the postings on this group, and it's been helpful. I
have not, however, found complete documentation on the add_password()
functions. Here's what I've got so far:
Don't use a password manager with proxy auth in 2.4, it doesn't work:

http://python.org/sf/1470846

[...] It works, but I don't really know what I'm doing with the
proxy_auth_handler part. Specifying username and password make sense,
but I haven't found documentation on what 'realm' and 'host' are for.
Shouldn't username & password be sufficient?


Both proxy and auth handling in urllib2 are buggy in 2.4, but passing
the username and password in the "userinfo" component of the URL works
(didn't read your code, hope nothing important in there):

import urllib2

proxy_handler = urllib2.ProxyHandler({
"http": "http://john:pa******@proxy.example.com:3128"})
opener = urllib2.build_opener(proxy_handler)

response = opener.open(url)
However, ISTR specifying a port number as I do above (the ":3128" bit)
is not supported in 2.4 (it does in current SVN, so will in 2.5).
Hmm, I notice just now that urllib2's digest auth support breaks
urllib2's handler scheme. Must file another bug...
John

Apr 29 '06 #2
On Sat, 29 Apr 2006, John J. Lee wrote:
"mrstephengross" <mr************@hotmail.com> writes:
I'm working on learning how to use urllib2 to use a proxy server. I've
looked through the postings on this group, and it's been helpful. I
have not, however, found complete documentation on the add_password()
functions. Here's what I've got so far:


Don't use a password manager with proxy auth in 2.4, it doesn't work:

http://python.org/sf/1470846

[...]
It works, but I don't really know what I'm doing with the
proxy_auth_handler part. Specifying username and password make sense,
but I haven't found documentation on what 'realm' and 'host' are for.
Shouldn't username & password be sufficient?


Oops, didn't read your question.

I'm surprised if it worked at all given that bug (maybe you're using an
older Python, and I forget all the details of the bug).

Anyway, to answer your question: yes, username & password *should* be
sufficient, but the password manager classes in urllib2 at present aren't
very friendly like that. You can use HTTPPasswordMgrWithDefaultRealm so
that at least you can forget about the realm part. (The "realm" is what
you see in the title bar of the pop-up that web browsers display. See RFC
2617 for more details.)

I'm adding friendlier proxy/auth support to package mechanize ATM (and
finding/fixing bugs as I go), so maybe some of that will find its way into
Python 2.6.
John

Apr 29 '06 #3

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

2 posts views Thread by Andre Bocchini | last post: by
4 posts views Thread by bmiras | last post: by
1 post views Thread by Fuzzyman | last post: by
reply views Thread by Neil.Jin | last post: by
1 post views Thread by Ray Slakinski | last post: by
6 posts views Thread by Alejandro Dubrovsky | last post: by
3 posts views Thread by m.banaouas | last post: by
1 post views Thread by Alessandro Fachin | last post: by
1 post views Thread by Rob Wolfe | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.