467,864 Members | 1,900 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 467,864 developers. It's quick & easy.

urllib leaves sockets open?

Hi all. I'm currently tracking down a problem in a little script[1] I have,
and I was hoping that those more experienced than myself could weigh in.

The script's job is to grab the status page off a DLink home router. This is
a really simple job: I just use urllib.urlopen() to grab the status page.
The router uses HTTP Basic authentication, so I've subclassed FancyURLOpener
to supply the credentials.

This all worked fine with an older router, but with the newer model there's
a long delay between sending the authentication information and actually
getting the response back. When just going in via a brower, there is no such
delay.

I did a little work with a tracing proxy, and I noticed something
interesting. urllib first makes an HTTP request without authentication
information. This gets back an HTTP 401 error code, as expected. urllib then
opens a second socket, and sends the Authentication header, again just as
expected.

Here's what I noticed: The socket for the first request that failed is still
connected. It looks like what's happening is that the router's only allowing
a single HTTP connection at a time. As a result, the second, authenticated
request, doesn't get it's response until there's some kind of timeout and
the first socket disconnects.

Is this normal behavior for urllib? Is there a way to force that initial
socket closed earlier? Is there something else I need to do?

Thanks for any insight,

-Chris

[1] The script in question is:

router_address = "xxx"
router_port = 80
router_user = "user"
router_password = "password"

class DI604Opener( urllib.FancyURLopener ):
def prompt_user_passwd( self, host, realm ):
return ( router_user, router_password )

urllib._urlopener = DI604Opener()

#
# Kick off the process when run from the command line
#
if __name__ == "__main__":
status_page = urllib.urlopen( "http://%s:%s/status.htm" % ( router_address,
router_port ) )
print status_page.read()

Aug 21 '05 #1
  • viewed: 1981
Share:
3 Replies
"Chris Tavares" <cc*@tavaresstudios.com> writes:
Is this normal behavior for urllib? Is there a way to force that initial
socket closed earlier? Is there something else I need to do?


I'd say open a sourceforge bug. There may be a way around it with the
fancy opener methods of urllib2, but it's a bug if regular urllib
opens a second socket without closing the first one. For http 1.1
it should be able to use just one socket anyway.
Aug 21 '05 #2
"Paul Rubin" <http://ph****@NOSPAM.invalid> wrote in message
news:7x************@ruckus.brouhaha.com...
"Chris Tavares" <cc*@tavaresstudios.com> writes:
Is this normal behavior for urllib? Is there a way to force that initial
socket closed earlier? Is there something else I need to do?


I'd say open a sourceforge bug. There may be a way around it with the
fancy opener methods of urllib2, but it's a bug if regular urllib
opens a second socket without closing the first one. For http 1.1
it should be able to use just one socket anyway.


Thanks, I'll do some poking around in urllib first and see if I can narrow
it down.

Is there a way to do HTTP 1.1 with urllib? The docs say 0.9 and 1.0 only.

Thanks,

-Chris
Aug 21 '05 #3
"Chris Tavares" <cc*@tavaresstudios.com> writes:
Is there a way to do HTTP 1.1 with urllib? The docs say 0.9 and 1.0 only.


I'm not sure. Try urllib2, but I'm still not sure.
Aug 21 '05 #4

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

2 posts views Thread by Andreas Dahl | last post: by
3 posts views Thread by Haim Ashkenazi | last post: by
1 post views Thread by John Hunter | last post: by
reply views Thread by Pieter Edelman | last post: by
11 posts views Thread by George Sakkis | last post: by
11 posts views Thread by Stefan Palme | last post: by
5 posts views Thread by supercooper | last post: by
5 posts views Thread by John Nagle | last post: by
2 posts views Thread by ShashiGowda | last post: by
reply views Thread by jack112 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.