how to use the form submit..

[linksterman]

i was trying to figure out the form.click() function and dont have a clue on how to implement it. can someone please help me? say by sending the search form in google (or something like that)?

[bartonc]

i was trying to figure out the form.click() function and dont have a clue on how to implement it. can someone please help me? say by sending the search form in google (or something like that)?

You post lacks specificity. There are many way/modules that you might be using.
Please be specific with your question and tell us (or better, post some code) what tools that you are using. Thanks.

[linksterman]

well urllib2 + ClientForm

from urllib2 import urlopen
from ClientForm import ParseResponse
 
response = urlopen("http://google.com")
forms = ParseResponse(response)
form = forms[0]
print form
form["q"] = "dogs"
 
# form.click() returns a urllib2.Request object
# (see HTMLForm.click.__doc__ if you don't have urllib2)
print urlopen(form.click()).read()

that is what im trying to work with. i keep getting an error 403 though.. i think that is because it doesnt save cookies. someone on these forums said something about a module that sends and recieves cookies invisibly and automatically, but i searched and found nothing.

[bartonc]

well urllib2 + ClientForm

from urllib2 import urlopen
from ClientForm import ParseResponse
 
response = urlopen("http://google.com")
forms = ParseResponse(response)
form = forms[0]
print form
form["q"] = "dogs"
 
# form.click() returns a urllib2.Request object
# (see HTMLForm.click.__doc__ if you don't have urllib2)
print urlopen(form.click()).read()

that is what im trying to work with. i keep getting an error 403 though.. i think that is because it doesnt save cookies. someone on these forums said something about a module that sends and recieves cookies invisibly and automatically, but i searched and found nothing.

That's funny: the 2.5 docs have documentation of the Cookie Module and the cookielib Module. I'd love to be able to help you further, but I'm not a web guy.

[linksterman]

That's funny: the 2.5 docs have documentation of the Cookie Module and the cookielib Module. I'd love to be able to help you further, but I'm not a web guy.

oh well i didnt know which to use because someone said that the module laid invisibly over urllib2 to automatically deal with the cookies. i think that cookielib does that... i couldnt get cookielib, all i could dig up was cookie. and i guess it would be good for you to know that i am working with 2.4 port.

but now i dont think that that is the problem since google search doesnt require cookies (i think). I tried out cookielib, and still get the 403 Forbidden error. maybe to do with the headers... well i got it to work with yahoo, but i sort of need this to work too.

[scaph01]

Linksterman,
I think I have figured out the problem. I am by no means a python developer, actually downloaded the installer today and started on the same thing you were working on, and ran into your same problem. Any way, what I found was the User-agent header needs to be set, once set google is ok with the search request. I would like to do a wire trace to compare HTTP streams but at least it works.

from urllib2 import urlopen
from ClientForm import ParseResponse

response = urlopen("http://google.com")
forms = ParseResponse(response)
form = forms[0]
print form
form["q"] = "dogs"

# ....updated code below this line....
request2 = form.click() # urllib2.Request object
request2.add_header('User-agent', 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; InfoPath.1; .NET CLR 1.1.4322)')
response2 = urllib2.urlopen(request2)


print "HTTP Code:"
print response2.code
print response2.info() # headers


Please note, the top portion of the inital HTTP GET was slightly different so the code may not work as exact, please use as a guide.

Phil