By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,621 Members | 1,101 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,621 IT Pros & Developers. It's quick & easy.

urrlib2 multithreading error

P: n/a
Hi,

I'm using urllib2 to retrieve some data usign http in a multithreaded
application.
Here's a piece of code:
req = urllib2.Request(url, txdata, txheaders)
opener = urllib2.build_opener()
opener.addheaders = [('User-agent', user_agent)]
request = opener.open(req)
data = request.read(1024)

I'm trying to read only the first 1024 bytes to retrieve http headers
(if is html then I will retrieve the entire page).
When I use it on a single thread everything goes ok, when I create
multiple threads the execution halts and the program terminates, just
before the last line (when I execute the request.read(.) ). Obviously I
tried to catch the exception but it doesn't work, the interpreter exits
without any exception or message.
How can I solve this?

lv

Jan 16 '07 #1
Share this Question
Share on Google+
2 Replies


P: n/a
At Tuesday 16/1/2007 11:41, vi******@gmail.com wrote:
>When I use it on a single thread everything goes ok, when I create
multiple threads the execution halts and the program terminates, just
before the last line (when I execute the request.read(.) ). Obviously I
tried to catch the exception but it doesn't work, the interpreter exits
without any exception or message.
Ouch... Can you reduce your program to the minimum code that fails,
and post it?
--
Gabriel Genellina
Softlab SRL


__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas

Jan 17 '07 #2

P: n/a
vi******@gmail.com wrote:

I'm using urllib2 to retrieve some data usign http in a multithreaded
application.
Here's a piece of code:
req = urllib2.Request(url, txdata, txheaders)
opener = urllib2.build_opener()
opener.addheaders = [('User-agent', user_agent)]
request = opener.open(req)
data = request.read(1024)

I'm trying to read only the first 1024 bytes to retrieve http headers
(if is html then I will retrieve the entire page).
Why so much bother? You just can create the Request, open it, and ask
for the headers:
>>req = urllib2.Request("http://www.google.com.ar")
u = urllib2.urlopen(req)
u.headers["content-type"]
'text/html'
>>>
Take into account that you can add the headers where you put
"txheaders", it's not necessary to use "addheaders".

And see that I'm not reading the page at all, urllib2.urlopen just
retrieves the headers...

Regards,

--
.. Facundo
..
Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/
Jan 17 '07 #3

This discussion thread is closed

Replies have been disabled for this discussion.