471,339 Members | 1,158 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,339 software developers and data experts.

how to use the form submit..

i was trying to figure out the form.click() function and dont have a clue on how to implement it. can someone please help me? say by sending the search form in google (or something like that)?
Jul 20 '07 #1
5 3156
bartonc
6,596 Expert 4TB
i was trying to figure out the form.click() function and dont have a clue on how to implement it. can someone please help me? say by sending the search form in google (or something like that)?
You post lacks specificity. There are many way/modules that you might be using.
Please be specific with your question and tell us (or better, post some code) what tools that you are using. Thanks.
Jul 20 '07 #2
well urllib2 + ClientForm


Expand|Select|Wrap|Line Numbers
  1. from urllib2 import urlopen
  2. from ClientForm import ParseResponse
  3.  
  4. response = urlopen("http://google.com")
  5. forms = ParseResponse(response)
  6. form = forms[0]
  7. print form
  8. form["q"] = "dogs"
  9.  
  10. # form.click() returns a urllib2.Request object
  11. # (see HTMLForm.click.__doc__ if you don't have urllib2)
  12. print urlopen(form.click()).read()

that is what im trying to work with. i keep getting an error 403 though.. i think that is because it doesnt save cookies. someone on these forums said something about a module that sends and recieves cookies invisibly and automatically, but i searched and found nothing.
Jul 21 '07 #3
bartonc
6,596 Expert 4TB
well urllib2 + ClientForm


Expand|Select|Wrap|Line Numbers
  1. from urllib2 import urlopen
  2. from ClientForm import ParseResponse
  3.  
  4. response = urlopen("http://google.com")
  5. forms = ParseResponse(response)
  6. form = forms[0]
  7. print form
  8. form["q"] = "dogs"
  9.  
  10. # form.click() returns a urllib2.Request object
  11. # (see HTMLForm.click.__doc__ if you don't have urllib2)
  12. print urlopen(form.click()).read()

that is what im trying to work with. i keep getting an error 403 though.. i think that is because it doesnt save cookies. someone on these forums said something about a module that sends and recieves cookies invisibly and automatically, but i searched and found nothing.
That's funny: the 2.5 docs have documentation of the Cookie Module and the cookielib Module. I'd love to be able to help you further, but I'm not a web guy.
Jul 21 '07 #4
That's funny: the 2.5 docs have documentation of the Cookie Module and the cookielib Module. I'd love to be able to help you further, but I'm not a web guy.
oh well i didnt know which to use because someone said that the module laid invisibly over urllib2 to automatically deal with the cookies. i think that cookielib does that... i couldnt get cookielib, all i could dig up was cookie. and i guess it would be good for you to know that i am working with 2.4 port.

but now i dont think that that is the problem since google search doesnt require cookies (i think). I tried out cookielib, and still get the 403 Forbidden error. maybe to do with the headers... well i got it to work with yahoo, but i sort of need this to work too.
Jul 22 '07 #5
Linksterman,
I think I have figured out the problem. I am by no means a python developer, actually downloaded the installer today and started on the same thing you were working on, and ran into your same problem. Any way, what I found was the User-agent header needs to be set, once set google is ok with the search request. I would like to do a wire trace to compare HTTP streams but at least it works.

from urllib2 import urlopen
from ClientForm import ParseResponse

response = urlopen("http://google.com")
forms = ParseResponse(response)
form = forms[0]
print form
form["q"] = "dogs"

# ....updated code below this line....
request2 = form.click() # urllib2.Request object
request2.add_header('User-agent', 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; InfoPath.1; .NET CLR 1.1.4322)')
response2 = urllib2.urlopen(request2)


print "HTTP Code:"
print response2.code
print response2.info() # headers


Please note, the top portion of the inital HTTP GET was slightly different so the code may not work as exact, please use as a guide.

Phil
Aug 1 '07 #6

Post your reply

Sign in to post your reply or Sign up for a free account.

Similar topics

2 posts views Thread by Terence Parker | last post: by
6 posts views Thread by CJM | last post: by
4 posts views Thread by Stuart Perryman | last post: by
5 posts views Thread by rjames.clarke | last post: by
5 posts views Thread by Navillus | last post: by
reply views Thread by rosydwin | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.