By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
443,923 Members | 1,443 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 443,923 IT Pros & Developers. It's quick & easy.

urllib2 request blocks

P: n/a
I just tried this:
import urllib2
urllib2.urlopen( "https://passenger.ssc.com/~dmarti/contrib-faq/" )


It sits forever. Loading that URL in a normal browser gets me a popup asking
about server certificate authenticity, which presumably is what urllib2 is
blocking at.

How can I prepare my code for this eventuality and ensure it doesn't block?
Jul 18 '05 #1
Share this Question
Share on Google+
2 Replies


P: n/a
Derek Fountain wrote:
I just tried this:

import urllib2
urllib2.urlopen( "https://passenger.ssc.com/~dmarti/contrib-faq/" )

It sits forever. Loading that URL in a normal browser gets me a popup asking
about server certificate authenticity, which presumably is what urllib2 is
blocking at.

How can I prepare my code for this eventuality and ensure it doesn't block?


You can try to write the URL like that :

"https://username:pa******@passenger.ssc.com/~dmarti/contrib-faq/"
Jul 18 '05 #2

P: n/a
Derek Fountain <de*****@example.com> writes:
I just tried this:
import urllib2
urllib2.urlopen( "https://passenger.ssc.com/~dmarti/contrib-faq/" )

It sits forever. Loading that URL in a normal browser gets me a popup asking
about server certificate authenticity, which presumably is what urllib2 is
blocking at.


That script doesn't hang for me, using Python 2.3. I get a 200
response, with content the same as I see in Mozilla (Linux journal
author's guide and FAQ). The socket module does not verify the
authenticity of servers, so neither does urllib or urllib2.

How can I prepare my code for this eventuality and ensure it doesn't block?


Depends on why it's blocking.

sockets in 2.3 now support a timeout (that parameter is not fed
through to the httplib, urllib or urllib2 public interfaces, but you
can still set the default socket timeout in your application code).
If it's DNS that blocking, though, that won't help you. I think
proxies like squid do DNS lookups in a separate process (I guess you
can use threads too).

Alternatively, you can start messing with things like asyncore or
Twisted, or use threads (I don't recommend the latter :-/).
John
Jul 18 '05 #3

This discussion thread is closed

Replies have been disabled for this discussion.