473,382 Members | 1,332 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,382 software developers and data experts.

Using mechanize in python to navigate a website

25
Hi all....

I'm using pyscripter 1.7.2, on a Win XP Prof 2002 - service pack 2 machine to script a website. I have no problem logging into the site, loading and an xml file, and retrieving the confirmation transaction code xml file, but when I go to the reports page and try to grab a report, I run into a problem. The "Get Report" button code within the web page is the following:

Expand|Select|Wrap|Line Numbers
  1. <input type="button" name="change" value="Get Report" onclick="getReport()" class="Button">
Now I can't do a br.submit() on this button, as mechanize doesn't recognize it as "clickable". The "getReport()" function is javascript and is coded as follows:

Expand|Select|Wrap|Line Numbers
  1. function getReport(){
  2.         var elm = getEventElm();
  3.         var form = getParent(elm,"form");
  4.         form.action = 'demandBids.do?method=report';
  5.         form.target = "_top";
  6.         form.submit();
  7.         form.action = 'demandBids.do?method=view';
  8.         form.target = "main";        
  9.     }
When in the web page itself, and you click the "Get Report" button, it calls the javascript function "getReport()" and changes the action of the form itself from the following:

<action="/emkt/demand/demandBids.do?method=view">

to the following:

<action="/emkt/demand/demandBids.do?method=report">

and brings up a browser "file download" window. I would usually just do a
Expand|Select|Wrap|Line Numbers
  1.  response = br.submit().read() 
to get the xml report file and then write it to a file. I can put in the link to the report directly thru IE7 itself and it will bring up the "file download" window, but if I try to do a br.open() directly on the reports link and try to read it....it returns an empty xml file.

Does anyone have any input on how I can handle this situation??? Is there a way to invoke the javascript from python itself?? Is there a way to change the "input type" of the button to make mechanize see it as clickable??? Should I try to change the action of the form itself??? Any input on this problem would be greatly appreciated. Thank you.
Jan 28 '08 #1
2 20919
Silgd1
25
I figured it out. I had to do the following:

Expand|Select|Wrap|Line Numbers
  1.  
  2. br.form.action="the wanted action"
  3.  
  4.  
Jan 30 '08 #2
thanks, that was exactly what i was looking for :)
Jul 13 '08 #3

Sign in to post your reply or Sign up for a free account.

Similar topics

15
by: Larry Asher | last post by:
Hi all. I'm a bit of a novice in this arena so please forgive if this question reflects that. I am trying to grab the html from a website and display it within another webpage (once I get this to...
0
by: bruce | last post by:
hi... update to an ongoing issue i've been having regarding html/Browser and selecting forms. i've created a basic test app, and created a stripped down page of html. the html has a single...
12
by: John | last post by:
I have to write a spyder for a webpage that uses html + javascript. I had it written using mechanize but the authors of the webpage now use a lot of javascript. Mechanize can no longer do the job....
1
numberwhun
by: numberwhun | last post by:
I am having an issue with understanding something in the WWW::Mechanize module. I have a website which I want to download a whole plethora of pdf files from. It is a site that I have paid to access...
0
by: bruce | last post by:
i'm getting the following error: mechanize._response.httperror_seek_wrapper: HTTP Error 500: i'm running python 5.1 and mechanize 0.1.7b I have no idea as to what I have to...
1
by: tedpottel | last post by:
Hi, I am trying to install the mechanize lib so I can use python to do webbrowseing. First I set up easy_install When I ran the script, it download the files ok, then I got these error...
2
by: Rex | last post by:
Hello, I am working on an academic research project where I need to log in to a website (www.lexis.com) over HTTPS and execute a bunch of queries to gather a data set. I just discovered the...
0
by: trihaitran | last post by:
I am trying to write a web scraper and am having trouble accessing pages that require authentication. I am attempting to utilise the mechanize library, but am having difficulties. The site I am...
1
by: tedpottel | last post by:
Hi, I can read the home page using the mechanize lib. Is there a way to load in web pages using filename.html instad of servername/ filename.html. Lots of time the links just have the file...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.