By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
443,400 Members | 903 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 443,400 IT Pros & Developers. It's quick & easy.

How to I automatcially fill out a 'textarea' in a webpage in python?

P: 32
Hi,

I have been trying to do this for a few days now ... I'd appreciate any help or insights ...

Basically I want to enter text into a web 'textarea' automatically ... here is what I
thought I should have ...

import urllib
import urllib2
import cookielib

url2 = "someUrl"
url = "someOtherUrl"

#Create empty cookie jar.
cj = cookielib.LWPCookieJar()
#Install cookie handler for urllib2.
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(c j))
urllib2.install_opener(opener)


request = urllib2.Request(url, None)
f = urllib2.urlopen(request)
f.close()

# this is just to log into the site first ...
data = urllib.urlencode({"username": "xxxxx", "password" : "somePassword"})
request = urllib2.Request(url, data)
f = urllib2.urlopen(request)

html = f.read()
#print html
#print data
f.close()


request2 = urllib2.Request(url2, None)
t = urllib2.urlopen(request2)
t.close()
user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
headers = { 'User-Agent' : user_agent }
#Parse the html here (html contains the page markup).
data2 = urllib.urlencode({"message" : "rows5>blah", "recipients[0]" : "xxxxxxx"})
#print data2
request2 = urllib2.Request(url2, data2, headers)
t = urllib2.urlopen(request2)
html2 = t.read()
print html2
t.close()


but I'm guessing that 'textarea' doesn't have a value attribute .. it looks like the text just goes between the two tags <textarea></textarea>

So how do I go about submitting predetermined text into the textarea? Do I have to parse the page and replace the whole 'textarea' section including my inputted text?

Hope you can help me,
Mar 11 '08 #1
Share this Question
Share on Google+
4 Replies


jlm699
100+
P: 314
Expand|Select|Wrap|Line Numbers
  1. import urllib
  2. import urllib2
  3. import cookielib
  4.  
  5. url2 = "someUrl"
  6. url = "someOtherUrl"
  7.  
  8. #Create empty cookie jar.
  9. cj = cookielib.LWPCookieJar()
  10. #Install cookie handler for urllib2.
  11. opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
  12. urllib2.install_opener(opener)
  13.  
  14.  
  15. request = urllib2.Request(url, None)
  16. f = urllib2.urlopen(request)
  17. f.close()
  18.  
  19. # this is just to log into the site first ...
  20. data = urllib.urlencode({"username": "xxxxx", "password" : "somePassword"})
  21. request = urllib2.Request(url, data)
  22. f = urllib2.urlopen(request)
  23.  
  24. html = f.read()
  25. #print html
  26. #print data
  27. f.close()
  28.  
  29.  
  30. request2 = urllib2.Request(url2, None)
  31. t = urllib2.urlopen(request2)
  32. t.close()
  33. user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
  34. headers = { 'User-Agent' : user_agent }
  35. #Parse the html here (html contains the page markup).
  36. data2 = urllib.urlencode({"message" : "rows5>blah", "recipients[0]" : "xxxxxxx"})
  37. #print data2
  38. request2 = urllib2.Request(url2, data2, headers)
  39. t = urllib2.urlopen(request2)
  40. html2 = t.read()
  41. print html2
  42. t.close()
but I'm guessing that 'textarea' doesn't have a value attribute .. it looks like the text just goes between the two tags <textarea></textarea>
What is the textarea that you speak of?
Mar 11 '08 #2

P: 32
What is the textarea that you speak of?
Sorry I should have said ... the textarea i'm speaking about is an area
in a web-form that will accept alot of text (depending on the limits the
web-master has set). The html code for this is :

Expand|Select|Wrap|Line Numbers
  1. <div class="form-row">
  2.     <div class="form-col-a"><label for="message">Message </label></div>
  3.     <div class="form-col-b"><textarea style="OVERFLOW: auto"
  4.         wrap="SOFT" name="message" tabindex="1" id="message" cols="25"
  5.         rows="5"></textarea></div>
  6.  
Usually the textarea 'value' goes just before the "</textarea>" part ... but I want to
fill this in automatically ... usually you can do this for a text input field using 'urllib.urlencode' and 'urllib2.Request' but I don't think this will work for a textarea and I was wondering if anyone else has done this?

It seems like it should be pretty easy to do but I am stumped!

Thanks
Mar 11 '08 #3

P: 43
A link to a Python program that claims to do this. I haven't tried it but have wondered how well it works. http://wwwsearch.sourceforge.net/ClientForm/ Please post back on whether it works or not.
Mar 13 '08 #4

P: 32
A link to a Python program that claims to do this. I haven't tried it but have wondered how well it works. http://wwwsearch.sourceforge.net/ClientForm/ Please post back on whether it works or not.
Hi,

Thanks for the reply ... I finally got this working ... thanks to ClientForm ... this
is a great module ... very easy to use.

Basically, I wanted to log into a website (using password and username) navigate to a page and enter details into a form and submit it. I wanted to do this so I could send sms text messages from the command line (without a browser open).

Here is the code that I wrote to do the task ...
any comments/suggestions are welcome :

You run it from command line using : python progname.py "your message" number

Expand|Select|Wrap|Line Numbers
  1. #!/scisoft/bin/python2.4
  2.  
  3. import time
  4. import urllib
  5. import urllib2
  6. import cookielib
  7. import ClientForm
  8. from HTMLParser import HTMLParser
  9. import sre
  10.  
  11. #----------------- START WEBSITE URLS  ---------------------------#
  12.  
  13. url = "www.someURL.com/login"
  14. url2 = "www.someURL.com/webtext/index.jsp"
  15. url3 = "www.someURL.com/myv/messaging/webtext/Process.shtml"
  16.  
  17.  
  18. #----------------- START USER VARIABLES --------------------------#
  19.  
  20. etag = "None"
  21. user_agent = "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT"
  22. protocol = 'https://'
  23. username = YOUR_USERNAME
  24. password = YOUR_PASSWORD
  25. maxMesgLen = 160
  26.  
  27. #--------------------------- END ---------------------------------#
  28.  
  29.  
  30. def sendSMS(textmessage,*number):
  31.         """
  32.  
  33.         """
  34.  
  35.         print "Sending ........"
  36.  
  37.         lenMessage = countCharacters(textmessage)
  38.         # Check to see if message is less than 160 characters
  39.         overFlowText = lenMessage - maxMesgLen
  40.         if lenMessage <= maxMesgLen:
  41.  
  42.  
  43.                 #Create empty cookie jar.
  44.                 cj = cookielib.LWPCookieJar()
  45.  
  46.                 opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
  47.                 urllib2.install_opener(opener)
  48.  
  49.                 request1 = urllib2.Request(protocol+url, None)
  50.                 t1 = opener.open(request1)
  51.                 forms = ClientForm.ParseResponse(t1, backwards_compat=False)
  52.                 t1.close()
  53.  
  54.                 form = forms[1]
  55.                 form["username"] = username
  56.                 form["password"] = password
  57.                 # Check the keep me logged on checkbox
  58.                 form.find_control("keeplogon").items[0].selected = True
  59.                 #print form
  60.  
  61.                 request11 = form.click()
  62.                 request11.add_header("User-Agent", user_agent)
  63.                 response11 = opener.open(request11)
  64.  
  65.  
  66.  
  67.                 request2 = urllib2.Request(protocol+url2, None)
  68.                 t = opener.open(request2)
  69.                 #print t.geturl()
  70.                 #print t.info()
  71.                 #print "This is the code for the webtext page: ",t.code
  72.                 forms = ClientForm.ParseResponse(t, backwards_compat=False)
  73.                 t.close()
  74.  
  75.  
  76.                 form = forms[1]  
  77.                 form["message"]  = textmessage
  78.  
  79.                 numberList = list(*number)
  80.                 noArgs = len(numberList)
  81.                 for i,recip in enumerate(numberList):
  82.                         form["recipients[%d]" % i] = recip
  83.  
  84.  
  85.                 request3 = form.click()
  86.  
  87.                 request3.add_header('User-Agent', user_agent)
  88.                 request3.add_header('Referer', protocol+url2)
  89.                 request3.add_header('Charset', "utf-8")
  90.                 request3.add_header('If-None-Match', etag)
  91.                 request3.add_header('Content-type', "text/html")
  92.  
  93.                 #This is vital for the message to send correctly!!!
  94.                 time.sleep(2)
  95.  
  96.                 response2 = opener.open(request3)
  97.  
  98.                 # Was your message sent successfully???
  99.                 success = response2.geturl()
  100.                 recipList = []
  101.                 if success == protocol+url3 :
  102.                         for recip in numberList:
  103.                                 if recip != "":
  104.                                         recipList.append(recip)
  105.                         print "Your message was sent successfully to : ", recipList
  106.                 else :
  107.                         print "Your message was _NOT_ sent!!!!!!"
  108.                 #print response2.info()  # headers
  109.                 #print 'This is the code :', response2.code
  110.                 ##print response2.read()
  111.                 response2.close()
  112.  
  113.  
  114.  
  115.                 # Get number of remaining Messages ... left
  116.                 reqMesgRem = urllib2.Request(protocol+url2, None)
  117.                 req = urllib2.urlopen(reqMesgRem)
  118.                 dataMesgRem = req.read()
  119.                 # find the number of remaining text messages left and the renewal date
  120.                 matches = sre.findall('<span class="msg-total">(.*?)</span>', dataMesgRem)
  121.                 print " "
  122.                 print "# Messages Remaining -----> ", matches[0]
  123.                 print "Renewal Date is      -----> ", matches[1]
  124.                 req.close()
  125.  
  126.         else :
  127.                 print "Message too long by", overFlowText,"!"
  128.  
  129.  
  130. def countCharacters(message):
  131.         """
  132.         Counts the number of characters in the message
  133.         """
  134.  
  135.         charCount= []
  136.         for char in message:
  137.                 charCount.append(char)
  138.         messageLength = len(charCount)
  139.         return messageLength
  140.  
  141.  
  142.  
  143. if __name__ == "__main__":
  144.         import sys
  145.         if len(sys.argv) < 8:
  146.                 sendSMS(sys.argv[1],sys.argv[2:])
  147.         else :
  148.                 print "ERROR : You have entered too many numbers!!!"
  149.  
Mar 20 '08 #5

Post your reply

Sign in to post your reply or Sign up for a free account.