infinite loop with httplib?

[Karl Witmann]

Anyone have experience with httplib getting stuck reading the first line
of a file?

Most urls work fine (of course), but I tried the following:
http://216.117.129.109/robots.txt

Both Firefox and IE return:
fopen: No such file or directory
httpd: could not open document config file /etc/httpd/conf/httpd.conf
A misconfigured server...

But for my test script:

import urllib2
f = urllib2.urlopen('http://216.117.129.109') print f.readline() fopen: No such file or directory print f.readline() fopen: No such file or directory print f.readline() fopen: No such file or directory
print f.read() Traceback (most recent call last):
File "<interactive input>", line 1, in ?
File "C:\Python23\lib\httplib.py", line 1156, in read
assert not self._line_consumed and self._line_left
AssertionError

Worse, the robotparser uses this code and thus ALL available memory: import robotparser
rp = robotparser.RobotFileParser()
rp.set_url('http://216.117.129.109')
rp.read()

a few minutes later, the kernel kills the process....

Any ideas?

I'd be fine if this returned an exception rather than a kernel panic.