urllib2 : https and proxy

[cp.finances.gouv]

Hello all,

I'm facing a strange behavior of urllib2 trying to access gmail
account behind a proxy (Squid).

The following works perfectly :

wget --save-cookies cookies --keep-session-cookies --post-
data="Em*************************@gmail.com&Passwd =cpfinances" --no-
check-certificate -O - https://www.google.com/accounts/ServiceLoginBoxAuth
While the following urllib2 code fail with an error 501 :

In [18]: import urllib2

In [19]: r=urllib2.urlopen("https://www.google.com/accounts/
ServiceLoginBoxAuth",

"Em*************************@gmail.com&Passwd=cpfi nances")
---------------------------------------------------------------------------
urllib2.HTTPError Traceback (most
recent call last)

/home/maric/<ipython console>

/home/maric/urllib2.py in urlopen(url, data)
128 if _opener is None:
129 _opener = build_opener()
--130 return _opener.open(url, data)

....

/home/maric/urllib2.py in http_error_default(self, req, fp, code, msg,
hdrs)
478 class HTTPDefaultErrorHandler(BaseHandler):
479 def http_error_default(self, req, fp, code, msg, hdrs):
--480 raise HTTPError(req.get_full_url(), code, msg, hdrs,
fp)
481
482 class HTTPRedirectHandler(BaseHandler):

HTTPError: HTTP Error 501: Not Implemented
I understood it's a response from the proxy about incompatible
protocols and queries but this seem rather strange in this case. Could
there be a bug in the ProxyHandler class ? Any idea about what could
cause this error on the proxy ?

[O.R.Senthil Kumaran]

* Maric Michaud <ma***@aristote.info[2007-06-28 23:38:39]:

Hmmm, i've tried something similar but I get the same result.

For the code example I gave, I've put correctly the http_proxy and
https_proxy environment variables, the default ProxyHandler get them right
as this is the proxy itself which returns the error. I checked this with
wireshark, the proxy said that there is incompatibilities in some request
methods, like POST, with some protocols, Gopher is the example it gave.
That doesn't make much sense to me as the wget command I tried is intended
to do exactly the same thing (POST over https and retrieve some cookies from
the response), and go through with no problems.

Maric, in the ActiveState urllib2 HOWTO which seems to apply for Python 2.5, I
came across a point which mentions urllib2 does not support PROXY for HTTPS.
Your case seems the same, I would suggest you to look up the Python Tracker
for any already open issues, or if not kindly log one yourself.
I shall if I can help further, working on urllib2 is a project am doing as
part of Summer of Code 2007.

Thanks,

--
O.R.Senthil Kumaran
http://uthcode.sarovar.org