472,984 Members | 1,870 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,984 software developers and data experts.

urllib2 : https and proxy

Hello all,

I'm facing a strange behavior of urllib2 trying to access gmail
account behind a proxy (Squid).

The following works perfectly :

wget --save-cookies cookies --keep-session-cookies --post-
data="Em*************************@gmail.com&Passwd =cpfinances" --no-
check-certificate -O - https://www.google.com/accounts/ServiceLoginBoxAuth
While the following urllib2 code fail with an error 501 :

In [18]: import urllib2

In [19]: r=urllib2.urlopen("https://www.google.com/accounts/

"Em*************************@gmail.com&Passwd=cpfi nances")
urllib2.HTTPError Traceback (most
recent call last)

/home/maric/<ipython console>

/home/maric/urllib2.py in urlopen(url, data)
128 if _opener is None:
129 _opener = build_opener()
--130 return _opener.open(url, data)


/home/maric/urllib2.py in http_error_default(self, req, fp, code, msg,
478 class HTTPDefaultErrorHandler(BaseHandler):
479 def http_error_default(self, req, fp, code, msg, hdrs):
--480 raise HTTPError(req.get_full_url(), code, msg, hdrs,
482 class HTTPRedirectHandler(BaseHandler):

HTTPError: HTTP Error 501: Not Implemented
I understood it's a response from the proxy about incompatible
protocols and queries but this seem rather strange in this case. Could
there be a bug in the ProxyHandler class ? Any idea about what could
cause this error on the proxy ?

Jun 28 '07 #1
1 3850
* Maric Michaud <ma***@aristote.info[2007-06-28 23:38:39]:
Hmmm, i've tried something similar but I get the same result.

For the code example I gave, I've put correctly the http_proxy and
https_proxy environment variables, the default ProxyHandler get them right
as this is the proxy itself which returns the error. I checked this with
wireshark, the proxy said that there is incompatibilities in some request
methods, like POST, with some protocols, Gopher is the example it gave.
That doesn't make much sense to me as the wget command I tried is intended
to do exactly the same thing (POST over https and retrieve some cookies from
the response), and go through with no problems.
Maric, in the ActiveState urllib2 HOWTO which seems to apply for Python 2.5, I
came across a point which mentions urllib2 does not support PROXY for HTTPS.
Your case seems the same, I would suggest you to look up the Python Tracker
for any already open issues, or if not kindly log one yourself.
I shall if I can help further, working on urllib2 is a project am doing as
part of Summer of Code 2007.


O.R.Senthil Kumaran
Jul 2 '07 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

by: O. Koch | last post by:
Until now, i know that ftplib doesn't support proxies and that i have to use urllib2. But i don't know how to use the urllib2 correct. I found some examples, but i don't understand them. Is...
by: news.easynet.be | last post by:
Hello, I would like to access an HTTPS site via a proxy The following code is working for HTTP://www.hotmail.com but not for HTTPS I have try with other sites without success l_proxy_info = {...
by: Benjamin Schollnick | last post by:
Folks, With Windows XP, and Python v2.41 I am running into a problem.... The following code gives me an unknown protocol error.... And I am not sure how to resolve it... I have a API...
by: Bloke | last post by:
I want to connect to a https server through an authenticating proxy like this: import urllib2 proxy_info = { 'user' : 'me', 'pass' : 'password', 'host' : 'mycompany.com.au', 'port' : 8008...
by: Fuzzyman | last post by:
Hello all (well, some of you I guess), Is it true that you can't configure a proxy handler for urllib2 to fetch https URLs through a proxy ? I know that you pass a dictionary, keyed by...
by: Ray Slakinski | last post by:
Hello, I have defined a function to set an opener for urllib2, this opener defines any proxy and http authentication that is required. If the proxy has authencation itself and requests an...
by: itay_k | last post by:
Hi, I am running the following simple code (just open connection to some https page with proxy): proxy= '666.179.227.666:80' proxy=urllib2.ProxyHandler({"https":'https://'+proxy}) opener =...
by: Ali.Sabil | last post by:
hello all, I just maybe hit a bug in both urllib and urllib2, actually urllib doesn't support proxy authentication, and if you setup the http_proxy env var to...
by: Astan Chee | last post by:
Hi again, According to https://demo.launchpad.net/products/python/+bug/56872 or more specifically, the example of its working code:...
by: DJRhino | last post by:
Was curious if anyone else was having this same issue or not.... I was just Up/Down graded to windows 11 and now my access combo boxes are not acting right. With win 10 I could start typing...
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 4 Oct 2023 starting at 18:00 UK time (6PM UTC+1) and finishing at about 19:15 (7.15PM) The start time is equivalent to 19:00 (7PM) in Central...
by: tracyyun | last post by:
Hello everyone, I have a question and would like some advice on network connectivity. I have one computer connected to my router via WiFi, but I have two other computers that I want to be able to...
by: giovanniandrean | last post by:
The energy model is structured as follows and uses excel sheets to give input data: 1-Utility.py contains all the functions needed to calculate the variables and other minor things (mentions...
by: Teri B | last post by:
Hi, I have created a sub-form Roles. In my course form the user selects the roles assigned to the course. 0ne-to-many. One course many roles. Then I created a report based on the Course form and...
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 1 Nov 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM) Please note that the UK and Europe revert to winter time on...
by: nia12 | last post by:
Hi there, I am very new to Access so apologies if any of this is obvious/not clear. I am creating a data collection tool for health care employees to complete. It consists of a number of...
by: isladogs | last post by:
The next online meeting of the Access Europe User Group will be on Wednesday 6 Dec 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, Mike...
by: GKJR | last post by:
Does anyone have a recommendation to build a standalone application to replace an Access database? I have my bookkeeping software I developed in Access that I would like to make available to other...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.