472,993 Members | 2,732 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,993 software developers and data experts.

how to force HTTP 1.1 when using urllib2?

When I request a URL using urllib2, it appears that urllib2 always
makes the request using HTTP 1.0, and not HTTP 1.1. I'm trying to use
the "If-None-Match"/"ETag" HTTP headers to conserve bandwidth, but if
I'm not mistaken, these are HTTP 1.1 headers, so I can't reasonably
expect a web server to respond correctly to my requests. (In my
limited testing, it looks like some servers respond correctly with an
HTTP 304 status, while some respond with an HTTP 200 and a full
response body.)

My specific issue notwithstanding, is there any way to force HTTP 1.1
to be used? Or am I doing something wrong?

I've condensed my code into the following example, which produces
similar results on two different setups, Python 2.3.2 on Windows and
Python 2.2.1 on Debian Linux. With this particular URL, the server
responds to my HTTP 1.0 request with an HTTP 1.1 response and an HTTP
304 status code, which suits my purposes, but I'd feel more comfortable
if my outgoing response also declared itself to be an HTTP 1.1 request.
Example code:

import httplib
httplib.HTTPConnection.debuglevel = 1
import urllib2

url = 'http://www.mozilla.org/images/firefox-oneoh-top.png'
etag = '"788054-2d18-3b21a80"'

request = urllib2.Request(url)
request.add_header('If-None-Match', etag)
opener = urllib2.build_opener()
response = opener.open(request)
Example output:

connect: (www.mozilla.org, 80)
send: 'GET /images/firefox-oneoh-top.png HTTP/1.0\r\nHost:
www.mozilla.org\r\nUser-agent: Python-urllib/2.0a1\r\nIf-none-match:
reply: 'HTTP/1.1 304 Not Modified\r\n'
header: Date: Tue, 21 Dec 2004 21:56:27 GMT
header: Server: Apache/2.0.46 (Red Hat)
header: Connection: close
header: ETag: "788054-2d18-3b21a80"
Traceback (most recent call last):
File "urllib2-test.py", line 11, in ?
response = opener.open(request)
File "D:\Python\lib\urllib2.py", line 333, in open
'_open', req)
File "D:\Python\lib\urllib2.py", line 313, in _call_chain
result = func(*args)
File "D:\Python\lib\urllib2.py", line 849, in http_open
return self.do_open(httplib.HTTP, req)
File "D:\Python\lib\urllib2.py", line 843, in do_open
return self.parent.error('http', req, fp, code, msg, hdrs)
File "D:\Python\lib\urllib2.py", line 359, in error
return self._call_chain(*args)
File "D:\Python\lib\urllib2.py", line 313, in _call_chain
result = func(*args)
File "D:\Python\lib\urllib2.py", line 419, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 304: Not Modified

Jul 18 '05 #1
0 2599

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

by: Matthew Wilson | last post by:
I am writing a script to check on my router's external IP address. My ISP refreshes my IP very often and I use dyndns for the hostname for my computer. My Netgear mr814 router has a webserver that...
by: OvErboRed | last post by:
Hi, I'm trying to determine whether a given URL exists. I'm new to Python but I think that urllib is the tool for the job. However, if I give it a non-existent file, it simply returns the 404 page....
by: Michael Foord | last post by:
#!/usr/bin/python -u # 15-09-04 # v1.0.0 # auth_example.py # A simple script manually demonstrating basic authentication. # Copyright Michael Foord # Free to use, modify and relicense. #...
by: John Reese | last post by:
Hello there. I've run into some missing functionality with HTTP Digest authentication in the 2.3 library and I was wondering if I'm just missing something. Missing functionality the first:...
by: john | last post by:
I'm trying to access the XML version of my Tivo now playing list with python. It uses auth digest HTTP authentication. I could really use some help! I'm able to get this page using curl: curl...
by: Alimah | last post by:
My objective is to log onto a wiki account (specifically wikipedia) using the http proxies provided by them ( - The operating system is Windows XP/Windows Server 2003....
by: Baptiste Lepilleur | last post by:
I activated httplib debug, and when trace are printed, a UnicodeError exception is thrown. I have already set sys.stdout to use utf-8 encoding (this removed the exception when *I* was printing...
by: Astan Chee | last post by:
Hi, Im trying to implement the logic from http://www.hypothetic.org/docs/msn/general/http_connections.php to a simple python code using urllib2 and some parts of urllib. Im behind a http proxy...
by: silk.odyssey | last post by:
I am getting the following error trying to download an html page using urllib2. urllib2.HTTPError: HTTP Error 204: NoContent The url is of this type: ...
by: lllomh | last post by:
Define the method first this.state = { buttonBackgroundColor: 'green', isBlinking: false, // A new status is added to identify whether the button is blinking or not } autoStart=()=>{
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 4 Oct 2023 starting at 18:00 UK time (6PM UTC+1) and finishing at about 19:15 (7.15PM) The start time is equivalent to 19:00 (7PM) in Central...
by: Aliciasmith | last post by:
In an age dominated by smartphones, having a mobile app for your business is no longer an option; it's a necessity. Whether you're a startup or an established enterprise, finding the right mobile app...
by: giovanniandrean | last post by:
The energy model is structured as follows and uses excel sheets to give input data: 1-Utility.py contains all the functions needed to calculate the variables and other minor things (mentions...
by: NeoPa | last post by:
Hello everyone. I find myself stuck trying to find the VBA way to get Access to create a PDF of the currently-selected (and open) object (Form or Report). I know it can be done by selecting :...
by: Teri B | last post by:
Hi, I have created a sub-form Roles. In my course form the user selects the roles assigned to the course. 0ne-to-many. One course many roles. Then I created a report based on the Course form and...
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 1 Nov 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM) Please note that the UK and Europe revert to winter time on...
by: isladogs | last post by:
The next online meeting of the Access Europe User Group will be on Wednesday 6 Dec 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, Mike...
by: GKJR | last post by:
Does anyone have a recommendation to build a standalone application to replace an Access database? I have my bookkeeping software I developed in Access that I would like to make available to other...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.