473,385 Members | 1,553 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

how to force HTTP 1.1 when using urllib2?

When I request a URL using urllib2, it appears that urllib2 always
makes the request using HTTP 1.0, and not HTTP 1.1. I'm trying to use
the "If-None-Match"/"ETag" HTTP headers to conserve bandwidth, but if
I'm not mistaken, these are HTTP 1.1 headers, so I can't reasonably
expect a web server to respond correctly to my requests. (In my
limited testing, it looks like some servers respond correctly with an
HTTP 304 status, while some respond with an HTTP 200 and a full
response body.)

My specific issue notwithstanding, is there any way to force HTTP 1.1
to be used? Or am I doing something wrong?

I've condensed my code into the following example, which produces
similar results on two different setups, Python 2.3.2 on Windows and
Python 2.2.1 on Debian Linux. With this particular URL, the server
responds to my HTTP 1.0 request with an HTTP 1.1 response and an HTTP
304 status code, which suits my purposes, but I'd feel more comfortable
if my outgoing response also declared itself to be an HTTP 1.1 request.
Example code:

import httplib
httplib.HTTPConnection.debuglevel = 1
import urllib2

url = 'http://www.mozilla.org/images/firefox-oneoh-top.png'
etag = '"788054-2d18-3b21a80"'

request = urllib2.Request(url)
request.add_header('If-None-Match', etag)
opener = urllib2.build_opener()
response = opener.open(request)
Example output:

connect: (www.mozilla.org, 80)
send: 'GET /images/firefox-oneoh-top.png HTTP/1.0\r\nHost:
www.mozilla.org\r\nUser-agent: Python-urllib/2.0a1\r\nIf-none-match:
"788054-2d18-3b21a80"\r\n\r\n'
reply: 'HTTP/1.1 304 Not Modified\r\n'
header: Date: Tue, 21 Dec 2004 21:56:27 GMT
header: Server: Apache/2.0.46 (Red Hat)
header: Connection: close
header: ETag: "788054-2d18-3b21a80"
Traceback (most recent call last):
File "urllib2-test.py", line 11, in ?
response = opener.open(request)
File "D:\Python\lib\urllib2.py", line 333, in open
'_open', req)
File "D:\Python\lib\urllib2.py", line 313, in _call_chain
result = func(*args)
File "D:\Python\lib\urllib2.py", line 849, in http_open
return self.do_open(httplib.HTTP, req)
File "D:\Python\lib\urllib2.py", line 843, in do_open
return self.parent.error('http', req, fp, code, msg, hdrs)
File "D:\Python\lib\urllib2.py", line 359, in error
return self._call_chain(*args)
File "D:\Python\lib\urllib2.py", line 313, in _call_chain
result = func(*args)
File "D:\Python\lib\urllib2.py", line 419, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 304: Not Modified

Jul 18 '05 #1
0 2622

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Matthew Wilson | last post by:
I am writing a script to check on my router's external IP address. My ISP refreshes my IP very often and I use dyndns for the hostname for my computer. My Netgear mr814 router has a webserver that...
2
by: OvErboRed | last post by:
Hi, I'm trying to determine whether a given URL exists. I'm new to Python but I think that urllib is the tool for the job. However, if I give it a non-existent file, it simply returns the 404 page....
7
by: Michael Foord | last post by:
#!/usr/bin/python -u # 15-09-04 # v1.0.0 # auth_example.py # A simple script manually demonstrating basic authentication. # Copyright Michael Foord # Free to use, modify and relicense. #...
3
by: John Reese | last post by:
Hello there. I've run into some missing functionality with HTTP Digest authentication in the 2.3 library and I was wondering if I'm just missing something. Missing functionality the first:...
2
by: john | last post by:
I'm trying to access the XML version of my Tivo now playing list with python. It uses auth digest HTTP authentication. I could really use some help! I'm able to get this page using curl: curl...
0
by: Alimah | last post by:
My objective is to log onto a wiki account (specifically wikipedia) using the http proxies provided by them (145.97.39.130 - 145.97.39.140:80). The operating system is Windows XP/Windows Server 2003....
0
by: Baptiste Lepilleur | last post by:
I activated httplib debug, and when trace are printed, a UnicodeError exception is thrown. I have already set sys.stdout to use utf-8 encoding (this removed the exception when *I* was printing...
0
by: Astan Chee | last post by:
Hi, Im trying to implement the logic from http://www.hypothetic.org/docs/msn/general/http_connections.php to a simple python code using urllib2 and some parts of urllib. Im behind a http proxy...
2
by: silk.odyssey | last post by:
I am getting the following error trying to download an html page using urllib2. urllib2.HTTPError: HTTP Error 204: NoContent The url is of this type: ...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.