468,288 Members | 1,961 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 468,288 developers. It's quick & easy.

httplib problems -- bug, or am I missing something?

I'm building an interface to Amazon's S3, using httplib. It uses a
single object for multiple transactions. What's happening is this:

HTTP PUT /unitest-temp-1161039691 HTTP/1.1
HTTP Date: Mon, 16 Oct 2006 23:01:32 GMT
HTTP Authorization: AWS <<cough>>:KiTWRuq/6aay0bI2J5DkE2TAWD0=
HTTP (end headers)
HTTP < HTTP/1.1 200 OK
HTTP < content-length: 0
HTTP < x-amz-id-2: 40uQn0OCpTiFcX+LqjMuzG6NnufdUk/..
HTTP < server: AmazonS3
HTTP < x-amz-request-id: FF504E8FD1B86F8C
HTTP < location: /unitest-temp-1161039691
HTTP < date: Mon, 16 Oct 2006 23:01:33 GMT
HTTPConnection.__state before response.read: Idle
HTTPConnection.__response: closed? False length: 0
reading response
HTTPConnection.__state after response.read: Idle
HTTPConnection.__response: closed? False length: 0

..later in the same connection..

HTTPConnection.__state before putrequest: Idle
HTTPConnection.__response: closed? False length: 0
HTTP DELETE /unitest-temp-1161039691 HTTP/1.1
HTTP Date: Mon, 16 Oct 2006 23:01:33 GMT
HTTP Authorization: AWS <<cough>>:a5OizuLNwwV7eBUhha0B6rEJ+CQ=
HTTP (end headers)
HTTPConnection.__state before getresponse: Request-sent
HTTPConnection.__response: closed? False length: 0
File "/usr/lib64/python2.4/httplib.py", line 856, in getresponse
raise ResponseNotReady()

If the first request does not precede it, the second request is fine.
To avoid excessive memory use, I'm calling request.read(16384)
repeatedly, instead of just calling request.read(). This seems to be
key to the problem -- if I omit the 'amt' argument to read(), then the
last line of the first request reads

HTTPConnection.__response: closed? True length: 0

and the later call to getresponse() doesn't raise ResponseNotReady.

Looking at the source for httplib.HTTPResponse.read, self.close() gets
called in the latter (working) case, but not in the former
(non-working). It would seem sensible to add 'if self.length == 0:
self.close()' to the end of that function (and, in fact, this change makes the
whole thing work), but this comment makes me hesitant:

# we do not use _safe_read() here because this may be a .will_close
# connection, and the user is reading more bytes than will be provided
# (for example, reading in 1k chunks)

What's going on here? Is this a bug I should report, or am I missing
something about how one should use httplib?

Thanks for any assistance.

Dustin
Oct 19 '06 #1
0 1177

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

2 posts views Thread by scummer | last post: by
1 post views Thread by Brian Beck | last post: by
4 posts views Thread by michaelparkin | last post: by
reply views Thread by Michael Ekstrand | last post: by
reply views Thread by philip20060308 | last post: by
6 posts views Thread by Haakon Riiser | last post: by
4 posts views Thread by Patrick Altman | last post: by
3 posts views Thread by rhXX | last post: by
2 posts views Thread by MrBee | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.