urllib2 http authorization question

[Matthew Wilson]

I am writing a script to check on my router's external IP address. My
ISP refreshes my IP very often and I use dyndns for the hostname for my
computer. My Netgear mr814 router has a webserver that uses HTTP basic
authorization.

My script uses urllib2 to connect to the router and read the html page
with the current external IP address. This is the function I wrote to
lookup the router's external IP:

def mr814(router_user, router_password):
"Looks up the IP from the router and returns it as a string."
req = urllib2.Request('http://192.168.0.1/sysstatus.html')
req.add_header("USER-AGENT",
"Mozilla/4.76 [en] (X11; U; Linux 2.4.1-0.1.9 i586)")
req.add_header("AUTHORIZATION", string.strip("Basic " +
base64.encodestring("%s:%s" % (router_user, router_password))))
IP = re.compile('(\d{1,3}\.){3}\d{1,3}')
for line in urllib2.urlopen(req):
if IP.search(line):
ip = line.strip()
return ip

The first time I run this script, I always get a urllib2.HTTPError. But
then If I ran the exact same script again afterward, I get access.

lookup.mr814('username', 'password') Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "lookup.py", line 13, in mr814
for line in urllib2.urlopen(req):
File "/usr/lib/python2.3/urllib2.py", line 136, in urlopen
return _opener.open(url, data)
File "/usr/lib/python2.3/urllib2.py", line 333, in open
'_open', req)
File "/usr/lib/python2.3/urllib2.py", line 313, in _call_chain
result = func(*args)
File "/usr/lib/python2.3/urllib2.py", line 849, in http_open
return self.do_open(httplib.HTTP, req)
File "/usr/lib/python2.3/urllib2.py", line 843, in do_open
return self.parent.error('http', req, fp, code, msg, hdrs)
File "/usr/lib/python2.3/urllib2.py", line 359, in error
return self._call_chain(*args)
File "/usr/lib/python2.3/urllib2.py", line 313, in _call_chain
result = func(*args)
File "/usr/lib/python2.3/urllib2.py", line 419, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 401: Unauthorized lookup.mr814('admin', '54wins') '66.72.200.255'

This is baffling me. Does the webserver ignore the AUTHORIZATION header
on the first request? Should I do something like this:
try:

.... lookup.mr814('admin', '54wins')
.... except urllib2.HTTPError: #I only want to catch 401 here, but how?
.... print "I got back the 401 error. Trying again..."
.... lookup.mr814('admin', '54wins')
.... except:
.... print "something else went wrong."

If that is recommended, how do I distinguish between 401 errors, vs
other errors?

Thanks for the help.

[Troels Therkelsen]

In article <sl*****************@overlook.homelinux.net>, Matthew Wilson wrote:

[snip]

lookup.mr814('username', 'password') Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "lookup.py", line 13, in mr814
for line in urllib2.urlopen(req):
File "/usr/lib/python2.3/urllib2.py", line 136, in urlopen
return _opener.open(url, data)
File "/usr/lib/python2.3/urllib2.py", line 333, in open
'_open', req)
File "/usr/lib/python2.3/urllib2.py", line 313, in _call_chain
result = func(*args)
File "/usr/lib/python2.3/urllib2.py", line 849, in http_open
return self.do_open(httplib.HTTP, req)
File "/usr/lib/python2.3/urllib2.py", line 843, in do_open
return self.parent.error('http', req, fp, code, msg, hdrs)
File "/usr/lib/python2.3/urllib2.py", line 359, in error
return self._call_chain(*args)
File "/usr/lib/python2.3/urllib2.py", line 313, in _call_chain
result = func(*args)
File "/usr/lib/python2.3/urllib2.py", line 419, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 401: Unauthorized lookup.mr814('admin', '54wins') '66.72.200.255'
To me, but maybe this is a copy-n-paste error, it looks like you aren't calling
lookup.mr814() with exactly the same args:

lookup.mr814('username', 'password')
versus
lookup.mr814('admin', '54wins')

?

This is baffling me. Does the webserver ignore the AUTHORIZATION header
on the first request? Should I do something like this:
try:

... lookup.mr814('admin', '54wins')
... except urllib2.HTTPError: #I only want to catch 401 here, but how?
... print "I got back the 401 error. Trying again..."
... lookup.mr814('admin', '54wins')
... except:
... print "something else went wrong."

If that is recommended, how do I distinguish between 401 errors, vs
other errors?

I don't know if it's recommended, but take a look at the source for the
urllib2.HTTPError class, at the definition of __str__ (which is used by the
traceback system):

def __str__(self):
return 'HTTP Error %s: %s' % (self.code, self.msg)

In other words, the actual HTTP error code is in self.code... you could check
for that something like this:

try:
lookup.mr814('admin', '54wins')
except urllib2.HTTPError, err:
if err.code != 401:
raise

Thanks for the help.

I hope the above helps,

/Troels Therkelsen