468,310 Members | 1,433 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 468,310 developers. It's quick & easy.

urllib2 request blocks

I just tried this:
import urllib2
urllib2.urlopen( "https://passenger.ssc.com/~dmarti/contrib-faq/" )


It sits forever. Loading that URL in a normal browser gets me a popup asking
about server certificate authenticity, which presumably is what urllib2 is
blocking at.

How can I prepare my code for this eventuality and ensure it doesn't block?
Jul 18 '05 #1
2 2370
Derek Fountain wrote:
I just tried this:

import urllib2
urllib2.urlopen( "https://passenger.ssc.com/~dmarti/contrib-faq/" )

It sits forever. Loading that URL in a normal browser gets me a popup asking
about server certificate authenticity, which presumably is what urllib2 is
blocking at.

How can I prepare my code for this eventuality and ensure it doesn't block?


You can try to write the URL like that :

"https://username:pa******@passenger.ssc.com/~dmarti/contrib-faq/"
Jul 18 '05 #2
Derek Fountain <de*****@example.com> writes:
I just tried this:
import urllib2
urllib2.urlopen( "https://passenger.ssc.com/~dmarti/contrib-faq/" )

It sits forever. Loading that URL in a normal browser gets me a popup asking
about server certificate authenticity, which presumably is what urllib2 is
blocking at.


That script doesn't hang for me, using Python 2.3. I get a 200
response, with content the same as I see in Mozilla (Linux journal
author's guide and FAQ). The socket module does not verify the
authenticity of servers, so neither does urllib or urllib2.

How can I prepare my code for this eventuality and ensure it doesn't block?


Depends on why it's blocking.

sockets in 2.3 now support a timeout (that parameter is not fed
through to the httplib, urllib or urllib2 public interfaces, but you
can still set the default socket timeout in your application code).
If it's DNS that blocking, though, that won't help you. I think
proxies like squid do DNS lookups in a separate process (I guess you
can use threads too).

Alternatively, you can start messing with things like asyncore or
Twisted, or use threads (I don't recommend the latter :-/).
John
Jul 18 '05 #3

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

4 posts views Thread by bmiras | last post: by
1 post views Thread by Doug Gray | last post: by
2 posts views Thread by John F Dutcher | last post: by
1 post views Thread by Doug Farrell | last post: by
reply views Thread by jacob c. | last post: by
4 posts views Thread by Monty | last post: by
7 posts views Thread by itay_k | last post: by
2 posts views Thread by Joshua Kugler | last post: by
reply views Thread by NPC403 | last post: by
reply views Thread by Teichintx | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.