I have the following problem with HTTPResponse:
import httplib #, select
....
connection = httplib.HTTPConnection(host)
connection.connect()
connection.request('GET', url)
response = connection.getresponse()
# print response.status, response.reason gives '200 OK'
# ready = select.select([response],[],[], 5.0) # no fileno() method
# signal.alarm(5) # not on Windows
# ...
data=response.read() # this sometimes blocks
connection.close()
Sometimes the read() call blocks forever for no obvious reason
(response.status is OK); it even cannot be interrupted from the keyboard
(on Windows).
I would like to defend against this by throwing an exception when the
read() lasts too long. But I cannot use select.select(), because
HTTPResponse has no fileno() method. Neither can I use signal.alarm(),
as it is for Unixes only.
Is there any other way to break read() or make it non-blocking?
Regards,
Marcin 4 3393
Marcin Ciura wrote: I have the following problem with HTTPResponse:
import httplib #, select ... connection = httplib.HTTPConnection(host) connection.connect() connection.request('GET', url) response = connection.getresponse() # print response.status, response.reason gives '200 OK' # ready = select.select([response],[],[], 5.0) # no fileno() method # signal.alarm(5) # not on Windows # ... data=response.read() # this sometimes blocks connection.close()
Sometimes the read() call blocks forever for no obvious reason (response.status is OK); it even cannot be interrupted from the keyboard (on Windows).
I would like to defend against this by throwing an exception when the read() lasts too long. But I cannot use select.select(), because HTTPResponse has no fileno() method. Neither can I use signal.alarm(), as it is for Unixes only.
Is there any other way to break read() or make it non-blocking?
I'm using timeoutsocket.py which allows you to globally specify a
timeout for all sockets. Used signal.alarm before, which caused more
problems than it solved...
Marcin Ciura <ci*********@zeus.polsl.gliwice.pl> pisze: Sometimes the read() call blocks forever for no obvious reason (response.status is OK); it even cannot be interrupted from the keyboard (on Windows).
I would like to defend against this by throwing an exception when the read() lasts too long. But I cannot use select.select(), because HTTPResponse has no fileno() method. Neither can I use signal.alarm(), as it is for Unixes only.
Is there any other way to break read() or make it non-blocking?
You can try to write your own HTTP client module, that uses
asyncore/asynchat. You may find a minimal implementation in my project's
CVS (URL in signature). It is based on code found in "EffNews Part 1:
Fetching RSS Files" tutorial ( http://www.effbot.org/zone/effnews-1.htm).
--
Jarek Zgoda http://jpa.berlios.de/
Marcin Ciura <ci*********@zeus.polsl.gliwice.pl> writes: I would like to defend against this by throwing an exception when the read() lasts too long. But I cannot use select.select(), because HTTPResponse has no fileno() method. Neither can I use signal.alarm(), as it is for Unixes only.
Is there any other way to break read() or make it non-blocking?
I think I'd reach down into the response object and get the fileno.
From httplib.py:
class HTTPResponse:
# ...
def __init__(self, sock, debuglevel=0, strict=0):
self.fp = sock.makefile('rb', 0)
self.debuglevel = debuglevel
self.strict = strict
so maybe you can get the fileno from response.fp.
Jarek Zgoda <jz****@gazeta.usun.pl> writes:
[...] You can try to write your own HTTP client module, that uses asyncore/asynchat. You may find a minimal implementation in my project's CVS (URL in signature). It is based on code found in "EffNews Part 1: Fetching RSS Files" tutorial (http://www.effbot.org/zone/effnews-1.htm).
There used to be one called asynchttp.py. There was also an
asyncurl.py. I think both are likely still around somewhere, but not
maintained.
John This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: hitectahir |
last post by:
Hi,
I have been using a remote data server named Clarens which runs on top
of the Apache web server and communicates through xmlrpc on Redhat
Linux 7.3. It has been working fine for about a...
|
by: Carl-Johan Kjellander |
last post by:
I have a problem. I want to use httplib to a site that has both IPv4 and
IPv6 connectivity, hence it has both A and AAAA records.
Is there a way to force httplib to use IPv6 only, or to at least...
|
by: Glauco |
last post by:
I'm using a library based on httplib.
Recently i've done a conversion for use of https with a key and
certificate file. This goes perfectly :-) .
Now, the problem is passing throw an http_proxy....
|
by: scummer |
last post by:
Hi,
I am having a problem with the httplib HTTPConnection object. While I
can easily send requests that don't have any payload (ie. "get"), I
encounter issues if I want to post xml data. If you...
|
by: Milos Prudek |
last post by:
How can I set httplib timeout for httplib.request() ?
httplib.request timeouts after 3:10 with "socket" timeout error message.
I found socket.settimeout() and I believe there is a way to add...
|
by: alastair |
last post by:
Hi,
I'm attempting to test out some functionality of the Apache http
server. What I'd like to do is send a file to the server - eg. a text
file or binary file (I will be testing gzipped...
|
by: michaelparkin |
last post by:
Hi,
Sorry to post what might seem like a trivial problem here, but its
driving me mad!
I have a simple https client that uses httplib to post data to a web
server.
When I post over http &...
|
by: spamsink42 |
last post by:
this code
h=httplib.HTTPConnection('euronext.com')
h.request('GET',
'http://www.euronext.com/home/0,3766,1732,00.html')
fails with this message
File "httplib.py", line 532, in connect...
|
by: Haakon Riiser |
last post by:
After a long debugging session while scripting my webmail,
I believe I have traced the problem to the way httplib sends
POST requests.
I have compared tcpdump listings from Python 2.4.3 and...
|
by: Phoe6 |
last post by:
Hi,
The following piece of code works properly when my proxy password
contains characters etc. But when my proxy password
contained something like '|\/|' , the httplib incorrectly indentified
it...
|
by: CloudSolutions |
last post by:
Introduction:
For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
|
by: ryjfgjl |
last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
|
by: taylorcarr |
last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: ryjfgjl |
last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
| |