473,404 Members | 2,114 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,404 software developers and data experts.

Using mechanize in python to navigate a website

25
Hi all....

I'm using pyscripter 1.7.2, on a Win XP Prof 2002 - service pack 2 machine to script a website. I have no problem logging into the site, loading and an xml file, and retrieving the confirmation transaction code xml file, but when I go to the reports page and try to grab a report, I run into a problem. The "Get Report" button code within the web page is the following:

Expand|Select|Wrap|Line Numbers
  1. <input type="button" name="change" value="Get Report" onclick="getReport()" class="Button">
Now I can't do a br.submit() on this button, as mechanize doesn't recognize it as "clickable". The "getReport()" function is javascript and is coded as follows:

Expand|Select|Wrap|Line Numbers
  1. function getReport(){
  2.         var elm = getEventElm();
  3.         var form = getParent(elm,"form");
  4.         form.action = 'demandBids.do?method=report';
  5.         form.target = "_top";
  6.         form.submit();
  7.         form.action = 'demandBids.do?method=view';
  8.         form.target = "main";        
  9.     }
When in the web page itself, and you click the "Get Report" button, it calls the javascript function "getReport()" and changes the action of the form itself from the following:

<action="/emkt/demand/demandBids.do?method=view">

to the following:

<action="/emkt/demand/demandBids.do?method=report">

and brings up a browser "file download" window. I would usually just do a
Expand|Select|Wrap|Line Numbers
  1.  response = br.submit().read() 
to get the xml report file and then write it to a file. I can put in the link to the report directly thru IE7 itself and it will bring up the "file download" window, but if I try to do a br.open() directly on the reports link and try to read it....it returns an empty xml file.

Does anyone have any input on how I can handle this situation??? Is there a way to invoke the javascript from python itself?? Is there a way to change the "input type" of the button to make mechanize see it as clickable??? Should I try to change the action of the form itself??? Any input on this problem would be greatly appreciated. Thank you.
Jan 28 '08 #1
2 20925
Silgd1
25
I figured it out. I had to do the following:

Expand|Select|Wrap|Line Numbers
  1.  
  2. br.form.action="the wanted action"
  3.  
  4.  
Jan 30 '08 #2
thanks, that was exactly what i was looking for :)
Jul 13 '08 #3

Sign in to post your reply or Sign up for a free account.

Similar topics

15
by: Larry Asher | last post by:
Hi all. I'm a bit of a novice in this arena so please forgive if this question reflects that. I am trying to grab the html from a website and display it within another webpage (once I get this to...
0
by: bruce | last post by:
hi... update to an ongoing issue i've been having regarding html/Browser and selecting forms. i've created a basic test app, and created a stripped down page of html. the html has a single...
12
by: John | last post by:
I have to write a spyder for a webpage that uses html + javascript. I had it written using mechanize but the authors of the webpage now use a lot of javascript. Mechanize can no longer do the job....
1
numberwhun
by: numberwhun | last post by:
I am having an issue with understanding something in the WWW::Mechanize module. I have a website which I want to download a whole plethora of pdf files from. It is a site that I have paid to access...
0
by: bruce | last post by:
i'm getting the following error: mechanize._response.httperror_seek_wrapper: HTTP Error 500: i'm running python 5.1 and mechanize 0.1.7b I have no idea as to what I have to...
1
by: tedpottel | last post by:
Hi, I am trying to install the mechanize lib so I can use python to do webbrowseing. First I set up easy_install When I ran the script, it download the files ok, then I got these error...
2
by: Rex | last post by:
Hello, I am working on an academic research project where I need to log in to a website (www.lexis.com) over HTTPS and execute a bunch of queries to gather a data set. I just discovered the...
0
by: trihaitran | last post by:
I am trying to write a web scraper and am having trouble accessing pages that require authentication. I am attempting to utilise the mechanize library, but am having difficulties. The site I am...
1
by: tedpottel | last post by:
Hi, I can read the home page using the mechanize lib. Is there a way to load in web pages using filename.html instad of servername/ filename.html. Lots of time the links just have the file...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.