I am attempting to use a HEAD request against Amazon S3 to check
whether a file exists or not and if it does parse the md5 hash from
the ETag in the response to verify the contents of the file so as to
save on bandwidth of uploading files when it is not necessary.
If the file exist, the HEAD works as expected and I get valid headers
back that I can parse and pull the ETag out of the dictionary using
getheader('ETag ')[1:-1] (using the slice to trim off the double-quotes
in the string.
The problem lies when I attempt to send a HEAD request when no file
exists. As expected, a 404 Not Found response is sent back from
Amazon however, my test scripts seem to hang. I run python with
trace.py and it hangs here:
--- modulename: httplib, funcname: _read_chunked
httplib.py(536) : assert self.chunked != _UNKNOWN
httplib.py(537) : chunk_left = self.chunk_left
httplib.py(538) : value = ''
httplib.py(542) : while True:
httplib.py(543) : if chunk_left is None:
httplib.py(544) : line = self.fp.readlin e()
--- modulename: socket, funcname: readline
socket.py(321): data = self._rbuf
socket.py(322): if size < 0:
socket.py(324): if self._rbufsize <= 1:
socket.py(326): assert data == ""
socket.py(327): buffers = []
socket.py(328): recv = self._sock.recv
socket.py(329): while data != "\n":
socket.py(330): data = recv(1)
It eventually completes with an exception here:
File "C:\Python25\li b\httplib.py", line 509, in read
return self._read_chun ked(amt)
File "C:\Python25\li b\httplib.py", line 548, in _read_chunked
chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: ''
For reference, ethereal captured the following request and response:
HEAD <REMOVEDHTTP/1.1
Host: s3.amazonaws.co m
Accept-Encoding: identity
Date: Tue, 13 Mar 2007 02:54:12 GMT
Authorization: AWS <REMOVED>
HTTP/1.1 404 Not Found
x-amz-request-id: E20B4C0D0C48B2E F
x-amz-id-2: <REMOVED>
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Tue, 13 Mar 2007 02:54:16 GMT
Server: AmazonS3
Am I doing something wrong? Is this a known issue? I am an
experienced developer, but pretty new to Python and dynamic languages
in general.
Thanks,
Patrick
4  2321 
En Tue, 13 Mar 2007 00:07:55 -0300, Patrick Altman <pa*****@gmail. com>
escribió:
I am attempting to use a HEAD request against Amazon S3 to check
whether a file exists or not and if it does parse the md5 hash from
the ETag in the response to verify the contents of the file so as to
save on bandwidth of uploading files when it is not necessary.
The problem lies when I attempt to send a HEAD request when no file
exists. As expected, a 404 Not Found response is sent back from
Amazon however, my test scripts seem to hang. I run python with
trace.py and it hangs here:
Yes, it's a known problem. See this message with a self-response: http://mail.python.org/pipermail/pyt...ch/375087.html
--
Gabriel Genellina
On Mar 13, 3:16 pm, "Gabriel Genellina" <gagsl-...@yahoo.com.a r>
wrote:
En Tue, 13 Mar 2007 10:38:24 -0300, Patrick Altman <palt...@gmail. com>
escribió:
Yes, it's a known problem. See this message with a
self-response:http://mail.python.org/pipermail/pyt...ch/375087.html
Are there plans to include this fix in the standard Python libraries
or must I make the modifications myself (I'm running Python 2.5)?
Submit a bug report, if not already done.http://sourceforge.net/tracker/?group_id=5470
--
Gabriel Genellina
Bug already exists at: https://sourceforge.net/tracker/inde...70&atid=105470
In the meantime, I implemented a work around for my specific case in
the Amazon S3 library in that I implemented a head() method but am
actually just requesting a GET operation with a very small byte
range. This is essentially yielding all the same header data that I
need (md5 hash in the ETag if the file exists, 404 Not Found if it
doesn't). This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: Glauco |
last post by:
I'm using a library based on httplib.
Recently i've done a conversion for use of https with a key and
certificate file. This goes perfectly :-) .
Now, the problem is passing throw an http_proxy.
It seems because httplib (based on socket.py) try to autenticate on the
proxy.
I'm only with this problem ? it's impossible :-|
|
by: Milos Prudek |
last post by:
How can I set httplib timeout for httplib.request() ?
httplib.request timeouts after 3:10 with "socket" timeout error message.
I found socket.settimeout() and I believe there is a way to add
".settimeout()" function to httplib without hacking httplib source.
Is it possible?
--
|
by: Terry Kerr |
last post by:
Hi,
I have an app that makes a https POST to a remote server that I have no
control over. The app runs fine in python 2.1.3 with socket.ssl
compiled with openssl-0.9.6, however it will not run in python 2.3.3
compiled with openssl-0.9.7d. The script below demonstrates the problem.
============================================
import httplib
path = "/cmaonline.nsf/ePayForm?OpenForm"
|
by: michaelparkin |
last post by:
Hi,
Sorry to post what might seem like a trivial problem here, but its
driving me mad!
I have a simple https client that uses httplib to post data to a web
server.
When I post over http & https using curl the data is recieved by the
web server with no problems.
|
by: coloradowebdev |
last post by:
i am working on basically a proxy server that handles requests via remoting
from clients and executes transactions against a third-party server via TCP.
the remoting site works like a champ. my problem is executing the
transactions against the remote server and returning the response to the
remoting client. i can open the socket fine and, if i am executing one
transaction at a time, everything works great. it's when my proxy server...
| | |
by: philip20060308 |
last post by:
Hi all,
Has anyone ever seen Python 2.4.1's httplib choke when reading chunked
content? I'm using it via urrlib2, and I ran into a particular server
that returns something that httplib doesn't expect. Specifically, in
the code below where the error occurs, line == ''.
Python 2.4.1 (#2, Oct 12 2005, 01:36:32)
20050518] on freebsd6
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib2
|
by: Mitch.Garnaat |
last post by:
Hi -
I'm writing some Python code to interact with Amazon's S3 service. One
feature of S3 is that it will allow you to use the HTTP HEAD request to
retrieve metadata about an S3 object without actually retrieving the
content of the object. In trying to use this feature, I'm running into
what I think is a bug in httplib.py.
If the resource I perform the HEAD request on exists within S3 then
everything is fine. If, however, the...
|
by: Dustin J. Mitchell |
last post by:
I'm building an interface to Amazon's S3, using httplib. It uses a
single object for multiple transactions. What's happening is this:
HTTP PUT /unitest-temp-1161039691 HTTP/1.1
HTTP Date: Mon, 16 Oct 2006 23:01:32 GMT
HTTP Authorization: AWS <<cough>>:KiTWRuq/6aay0bI2J5DkE2TAWD0=
HTTP (end headers)
HTTP < HTTP/1.1 200 OK
HTTP < content-length: 0
HTTP < x-amz-id-2: 40uQn0OCpTiFcX+LqjMuzG6NnufdUk/..
|
by: rhXX |
last post by:
hi all,
i'm using this tutorial example
import httplib
h = httplib.HTTP("www.python.org")
h.putrequest('GET','/index.html')
h.putheader('User-Agent','Lame Tutorial Code')
h.putheader('Accept','text/html')
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
| | |
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| | |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |
