In the course of my current project, I've had to deal with connecting
to an HTTP server that isn't fully compliant with the HTTP requirements
for chunked encodings. Rather than sending the end-of-data sentinel
(0-length chunk), it just closes the connection (without even sending
the CRLF at the end of the data).
Because of this, using httplib would always throw nasty errors and not
give me any useful data. Therefore, I've had to modify the httplib.py
code to do something reasonable when the server just closes the
connection.
So, my questions are (my changes follow, against Python 2.3):
- Did I patch the right place to do Something Reasonable in this case
of server non-compliance?
- Is there a better way to handle this case that may be more robust? Or
handle more similar cases?
- Is there anything special I should do (besides obviously diff-ing
against CVS) before submitting a patch for this to SourceForge? (it
seems to me that being tolerant of bad servers is something that would
be of general interest.)
Thanks,
Michael
---8<------- BEGIN CONTEXT DIFF ------------
*** /usr/lib/python2.3/httplib.py 2005-05-04 02:08:57.000000 000 -0500
--- httplib.py 2005-08-05 10:33:08.000000 000 -0500
***************
*** 1,5 ****
--- 1,7 ----
"""HTTP/1.1 client library
+ Copyright (c) 2001 Python Software Foundation; All Rights Reserved
+
<intro stuff goes here>
<other stuff, too>
***************
*** 64,69 ****
--- 66,75 ----
Unread-response _CS_IDLE <response_class >
Req-started-unread-response _CS_REQ_STARTED <response_class >
Req-sent-unread-response _CS_REQ_SENT <response_class >
+
+ Modified 2005-07-20 by Michael Ekstrand <me******@iasta te.edu> to deal
+ gracefully wtih non-compliant systems which just terminate the connection
+ rather than sending the end-of-data chunk in chunked HTTP responses.
"""
import errno
***************
*** 442,448 ****
amt -= chunk_left
# we read the whole chunk, get another
! self._safe_read (2) # toss the CRLF at the end of the chunk
chunk_left = None
# read and discard trailer up to the CRLF terminator
--- 448,460 ----
amt -= chunk_left
# we read the whole chunk, get another
! try:
! self._safe_read (2) # toss the CRLF at the end of the chunk
! except IncompleteRead:
! # The server just closed on us, without providing appropriate
! # end-of-data things.
! self.close()
! return value
chunk_left = None
# read and discard trailer up to the CRLF terminator
---8<--------- END CONTEXT DIFF ------------- 0 1088 This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Rich |
last post by:
Hi all,
I'm trying to write an app to monitor an IIS server, by connecting and
getting a page via httplib. the problem seems to be when IIS (or ASP) dies
httplib does not always return and sometimes I'm left with an app that
hangs. what is the best way to handle this ?
thanks,
rich.
|
by: Shagshag |
last post by:
hello,
how can i set multiple cookies like :
Set-Cookie: cookiename1=value1; expires=Tue, 31-Mar-09 11:17:43 GMT;
path=/; domain=www.domain.com
Set-Cookie: cookiename2=value2; expires=Tue, 31-Mar-09 11:17:43 GMT;
path=/; domain=www.domain.com
to a httplib.HTTPConnection ?
|
by: alastair |
last post by:
Hi,
I'm attempting to test out some functionality of the Apache http
server. What I'd like to do is send a file to the server - eg. a text
file or binary file (I will be testing gzipped transfers eventually
....).
At the moment I can test out sending a set of parameters to the
server, and using mod_python, I have a python script which displays
these values.
|
by: Brian Beck |
last post by:
Hi.
I'm having some problems with code based directly on the following
httplib documentation code:
http://www.zvon.org/other/python/doc21/lib/httplib-examples.html
I've included the code and traceback at the end of this post.
The odd thing is, using DEPRECATED FUNCTIONS to perform the same
function works fine!
|
by: Robert |
last post by:
did you solve this problem? It seems to be still present here with
py2.3.5.
Robert
--
From: Manish Jethani <manish.j@gmx.net>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b)
Gecko/20031205 Thunderbird/0.4
X-Accept-Language: en-us, en
| |
by: Patrick Altman |
last post by:
I am attempting to use a HEAD request against Amazon S3 to check
whether a file exists or not and if it does parse the md5 hash from
the ETag in the response to verify the contents of the file so as to
save on bandwidth of uploading files when it is not necessary.
If the file exist, the HEAD works as expected and I get valid headers
back that I can parse and pull the ETag out of the dictionary using
getheader('ETag') (using the slice to...
|
by: rhXX |
last post by:
hi all,
i'm using this tutorial example
import httplib
h = httplib.HTTP("www.python.org")
h.putrequest('GET','/index.html')
h.putheader('User-Agent','Lame Tutorial Code')
h.putheader('Accept','text/html')
|
by: Henrik Lied |
last post by:
Hi there!
I'm using a recipe found on ASPN to upload some data to an
external server.
The request fails, and all I get in response is "No backend servers
available".
So I'm wondering: Is this a known exception-message in httplib, or
could it be an error in the requested external resource?
|
by: reacocard |
last post by:
Hi, I'm writing a download manager in python, and httplib is being
very slow when pulling from localhost or even other servers on the
local network. I'm getting about 10MB in 14s with httplib, while wget
hits 80MB in less than 3s. You can find the code I made to benchmark
this here: http://pastebin.ca/973486 (noslor is mapped to my IP in /
etc/hosts)
Does anyone have any idea what might be causing this, and how I can
fix it? I'm using...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
| |
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
| |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |