469,328 Members | 1,274 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,328 developers. It's quick & easy.

urllib problem (maybe bugs?)

Hi,

I'm trying to fill the form on page
http://www.cbs.dtu.dk/services/TMHMM/ using urllib.

There are two peculiarities. First of all, I am filling in incorrect
key/value pairs in the parameters on purpose because that's the only
way I can get it to work.. For "version" I am suppose to leave it
unchecked, having value of empty string. And for name "outform" I am
suppose to assign it a value of "-short". Instead, I left out
"outform" all together and fill in "-short" for version. I discovered
the method my accident.

After I've done that it works fine for small SEQ values. Then, when I
try to send large amount of data (1.4MB), it fails miserably with
AttributeError exception.

I highly suspect the two problems I have are the result of some bugs
in the urllib module. Any suggestions?

This is my code:

================================================== ==========
fd = open('secretory0_1.txt', 'r')
txt = fd.read()
fd.close()

params = urllib.urlencode({'SEQ': txt, 'configfile':
'/usr/opt/www/pub/CBS/services/TMHMM-2.0/TMHMM2.cf', 'version':
'-short'})
f = urllib.urlopen("http://www.cbs.dtu.dk/cgi-bin/nph-webface", params)
data = f.read()

start = data.find('follow <a href="h') + 16
end = data.find('">This link</a>')
secondurl = data[start:end]

f = urllib.urlopen(secondurl)
print f.read()
================================================== ==========
The value pairs I am suppose to fill in are:

SEQ => some sequence here
configfile => '/usr/opt/www/pub/CBS/services/TMHMM-2.0/TMHMM2.cf'
version => ''
outform => '-short'
The exception I get when sending secretory0_1.txt is:

C:\Documents and Settings\thw\*>python testhttp.py
Traceback (most recent call last):
File "testhttp.py", line 11, in ?
f = urllib.urlopen("http://www.cbs.dtu.dk/cgi-bin/nph-webface", params)
File "C:\Python24\lib\urllib.py", line 79, in urlopen
return opener.open(url, data)
File "C:\Python24\lib\urllib.py", line 182, in open
return getattr(self, name)(url, data)
File "C:\Python24\lib\urllib.py", line 307, in open_http
return self.http_error(url, fp, errcode, errmsg, headers, data)
File "C:\Python24\lib\urllib.py", line 322, in http_error
return self.http_error_default(url, fp, errcode, errmsg, headers)
File "C:\Python24\lib\urllib.py", line 550, in http_error_default
return addinfourl(fp, headers, "http:" + url)
File "C:\Python24\lib\urllib.py", line 836, in __init__
addbase.__init__(self, fp)
File "C:\Python24\lib\urllib.py", line 786, in __init__
self.read = self.fp.read
AttributeError: 'NoneType' object has no attribute 'read'

Timothy
Jul 18 '05 #1
1 1871
Timothy Wu <2h*****@gmail.com> wrote:

I'm trying to fill the form on page
http://www.cbs.dtu.dk/services/TMHMM/ using urllib.

There are two peculiarities. First of all, I am filling in incorrect
key/value pairs in the parameters on purpose because that's the only
way I can get it to work.. For "version" I am suppose to leave it
unchecked, having value of empty string. And for name "outform" I am
suppose to assign it a value of "-short". Instead, I left out
"outform" all together and fill in "-short" for version. I discovered
the method my accident.

After I've done that it works fine for small SEQ values. Then, when I
try to send large amount of data (1.4MB), it fails miserably with
AttributeError exception.

I highly suspect the two problems I have are the result of some bugs
in the urllib module. Any suggestions?


The <form> on that page wants multipart/form-data encoding, where each
parameter is embedded in a separate MIME section. That is common with very
large data, and is _required_ when the form includes a file upload. urllib
doesn't do that. It always sends plain old
application/x-www-form-urlencoded data.

I think you're going to have to roll your own sender.
--
- Tim Roberts, ti**@probo.com
Providenza & Boekelheide, Inc.
Jul 18 '05 #2

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

4 posts views Thread by Gary Feldman | last post: by
3 posts views Thread by Michael | last post: by
3 posts views Thread by Haim Ashkenazi | last post: by
7 posts views Thread by Stuart McGraw | last post: by
reply views Thread by Ali.Sabil | last post: by
4 posts views Thread by John Nagle | last post: by
2 posts views Thread by shirish | last post: by
5 posts views Thread by John Nagle | last post: by
reply views Thread by listenups61195 | last post: by
reply views Thread by Purva khokhar | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.