473,371 Members | 1,508 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,371 software developers and data experts.

Pulling data from a .asps site

There's a government website which shows public data for banks. We'd
like to pull the data down programmatically but the data is "hidden"
behind .aspx...

Is there anyway in Python to hook in directly to a browser (firefox or
IE) to do the following...

1) Fill the search criteria
2) Press the "Search" button
3) Press another button (the CSV button) on the resulting page
4) Then grab the data out of the notepad file that pops up

If this is a wild good chase, let me know... (or if there's a better
way besides Python... I may have to explore writing a firefox plug-in
or something)...
Nov 27 '07 #1
4 1658
---------- Forwarded message ----------
From: Todd O'Bryan <to********@gmail.com>
Date: Nov 27, 2007 1:48 PM
Subject: Re: Pulling data from a .asps site
To: ha*******@gmail.com
Check out Selenium Remote Control! It's very easy to use.
On Nov 27, 2007 1:37 PM, <ha*******@gmail.comwrote:
There's a government website which shows public data for banks. We'd
like to pull the data down programmatically but the data is "hidden"
behind .aspx...

Is there anyway in Python to hook in directly to a browser (firefox or
IE) to do the following...

1) Fill the search criteria
2) Press the "Search" button
3) Press another button (the CSV button) on the resulting page
4) Then grab the data out of the notepad file that pops up

If this is a wild good chase, let me know... (or if there's a better
way besides Python... I may have to explore writing a firefox plug-in
or something)...
--
http://mail.python.org/mailman/listinfo/python-list
Nov 27 '07 #2
On Tue, 27 Nov 2007 20:37:19 +0200, <ha*******@gmail.comwrote:
>
There's a government website which shows public data for banks. We'd
like to pull the data down programmatically but the data is "hidden"
behind .aspx...

Is there anyway in Python to hook in directly to a browser (firefox or
IE) to do the following...

1) Fill the search criteria
2) Press the "Search" button
3) Press another button (the CSV button) on the resulting page
4) Then grab the data out of the notepad file that pops up

If this is a wild good chase, let me know... (or if there's a better
way besides Python... I may have to explore writing a firefox plug-in
or something)...
Well, Python supports moving the mouse pointer so it's should be quite
simple to write such a program.
Nov 27 '07 #3
ha*******@gmail.com schrieb:
There's a government website which shows public data for banks. We'd
like to pull the data down programmatically but the data is "hidden"
behind .aspx...

Is there anyway in Python to hook in directly to a browser (firefox or
IE) to do the following...

1) Fill the search criteria
2) Press the "Search" button
3) Press another button (the CSV button) on the resulting page
4) Then grab the data out of the notepad file that pops up

If this is a wild good chase, let me know... (or if there's a better
way besides Python... I may have to explore writing a firefox plug-in
or something)...
There needs no browser to be involved - unless there is AJAX in the
game, and even then - after alle, it's all HTTP-requests.

Whichfor there are several options to work with - one popular is

http://wwwsearch.sourceforge.net/mechanize/

Diez
Nov 27 '07 #4
On 27/11/2007, ha*******@gmail.com <ha*******@gmail.comwrote:
There's a government website which shows public data for banks. We'd
like to pull the data down programmatically but the data is "hidden"
behind .aspx...

Is there anyway in Python to hook in directly to a browser (firefox or
IE) to do the following...

1) Fill the search criteria
2) Press the "Search" button
3) Press another button (the CSV button) on the resulting page
4) Then grab the data out of the notepad file that pops up

If this is a wild good chase, let me know... (or if there's a better
way besides Python... I may have to explore writing a firefox plug-in
or something)...
Amongst others, IEC will allow you to open an IE browser window,
navigate around sites (including pushing buttons), and retrieve text.

http://www.mayukhbose.com/python/IEC/index.php

HTH :)
Nov 28 '07 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Rob Rutherford | last post by:
I have a PC running Win XP Home and want to learn how to write ASPs. In order to test them I think I need a web server installed on my PC. Since I don't believe IIS can be run under Win XP Home I...
5
by: middletree | last post by:
The site, which is almost finished, is at http://site234.webhost4life.com/crgmdal7/ If you click most of the links in the gold nav bar, they will take you to various places within the site. But...
1
by: BJS | last post by:
Hi there. I have been successfully running ASP on a W2K IIS5.0 server using Anon Access enabled, but would like to be able to restrict access to a particular folder containing some ASP content...
1
by: jpopper | last post by:
Hi, I am currently supporting a website that is a mix of asp and aspx pages. In one scenario I redirect the user from an asp page, to an aspx page, and then back to the asp page. When I return...
10
by: Jeff | last post by:
Hey gang. i have a code that i will list. when varM = 8 or 16, the script works fine, and pulls the top 8 or top 16, but if it =32 or 64, it is only pulling the top 17 records from the DB. db is...
5
by: Stewart | last post by:
Hi there, I have an Access database, which contains the details of company staff and services. The plan is to extract data from this database onto our forthcoming Intranet (no inserting,...
7
by: colleen1980 | last post by:
Hi: When i run the same code with minor changes in VB it works fine but when i run in ASP it runs but it not pulling any information from the web site. Needs help Thanks, Anna. ASP CODE ...
3
by: BARTHA CLIFTON | last post by:
HELLO I AM WONDERING IF ANYONE CAN HELP ME PUT THE ASPS IN THE TRANLOADAS I AM NOT VERY FAMILIAR WITH THIS METHOD PLEASE SOMEONE HELP ME THANK YOU~~~~~~~~!!!!!!!!!!!!!!!!!!
3
by: BobRoyAce | last post by:
I am using Visual Studio 2008 w/ VB.NET. For the database, I am using SQL Server 2005, which is running on a dedicated server box. I am creating a WinForms application for a client. It is run...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.