472,119 Members | 1,691 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,119 software developers and data experts.

urllib timeout hole - long timeout if site doesn't send headers.

urllib has a "hole" in its timeout protection.

Using "socket.setdefaulttimeout" will make urllib time out if a
site doesn't open a TCP connection in the indicated time. But if the site
opens the TCP connection and never sends HTTP headers, it takes about
20 minutes for the read in urllib's "open" to time out.

There are some web servers that produce this behavior, and
many seem to be associated with British universities and nonprofits.
With these, requesting "http://example.com" opens a TCP connection
on which nothing is ever sent, while "http://www.example.com"
yields a proper web page.

Even Firefox doesn't time this out properly. Try "http://soton.ac.uk"
in Firefox, and be prepared for a long wait.

There was some active work in the urllib timeout area last summer.
What happened to that?

John Nagle
Jan 3 '08 #1
0 1995

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

1 post views Thread by Chris Lyon | last post: by
11 posts views Thread by Pater Maximus | last post: by
1 post views Thread by Timothy Wu | last post: by
3 posts views Thread by Chris Tavares | last post: by
8 posts views Thread by Gabriel Zachmann | last post: by
4 posts views Thread by kgrafals | last post: by
5 posts views Thread by supercooper | last post: by
5 posts views Thread by John Nagle | last post: by
reply views Thread by leo001 | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.