471,123 Members | 810 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,123 software developers and data experts.

urllib2 request blocks

I just tried this:
import urllib2
urllib2.urlopen( "https://passenger.ssc.com/~dmarti/contrib-faq/" )


It sits forever. Loading that URL in a normal browser gets me a popup asking
about server certificate authenticity, which presumably is what urllib2 is
blocking at.

How can I prepare my code for this eventuality and ensure it doesn't block?
Jul 18 '05 #1
2 2459
Derek Fountain wrote:
I just tried this:

import urllib2
urllib2.urlopen( "https://passenger.ssc.com/~dmarti/contrib-faq/" )

It sits forever. Loading that URL in a normal browser gets me a popup asking
about server certificate authenticity, which presumably is what urllib2 is
blocking at.

How can I prepare my code for this eventuality and ensure it doesn't block?


You can try to write the URL like that :

"https://username:pa******@passenger.ssc.com/~dmarti/contrib-faq/"
Jul 18 '05 #2
Derek Fountain <de*****@example.com> writes:
I just tried this:
import urllib2
urllib2.urlopen( "https://passenger.ssc.com/~dmarti/contrib-faq/" )

It sits forever. Loading that URL in a normal browser gets me a popup asking
about server certificate authenticity, which presumably is what urllib2 is
blocking at.


That script doesn't hang for me, using Python 2.3. I get a 200
response, with content the same as I see in Mozilla (Linux journal
author's guide and FAQ). The socket module does not verify the
authenticity of servers, so neither does urllib or urllib2.

How can I prepare my code for this eventuality and ensure it doesn't block?


Depends on why it's blocking.

sockets in 2.3 now support a timeout (that parameter is not fed
through to the httplib, urllib or urllib2 public interfaces, but you
can still set the default socket timeout in your application code).
If it's DNS that blocking, though, that won't help you. I think
proxies like squid do DNS lookups in a separate process (I guess you
can use threads too).

Alternatively, you can start messing with things like asyncore or
Twisted, or use threads (I don't recommend the latter :-/).
John
Jul 18 '05 #3

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

4 posts views Thread by bmiras | last post: by
1 post views Thread by Doug Gray | last post: by
2 posts views Thread by John F Dutcher | last post: by
1 post views Thread by Doug Farrell | last post: by
reply views Thread by jacob c. | last post: by
4 posts views Thread by Monty | last post: by
7 posts views Thread by itay_k | last post: by
2 posts views Thread by Joshua Kugler | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.