473,320 Members | 1,612 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Web Scraping Question

I'm trying to get some information from a web site where you normally enter
a value, and click a button. I'd like to do this automatically.

I'm using the WebClient class like this:
client.Headers.Add("Content-Type", "application/x-www-form-urlencoded")

byteArray = Encoding.ASCII.GetBytes(sLinkScript)

responseArray = client.UploadData(URL, byteArray)

For example, the California State Contractor's Licens board at

http://www2.cslb.ca.gov/CSLB_LIBRARY...se+request.asp

I'm setting the script to:

"LicNum=789168&EditForm=Yes&CheckLicense.x=81&Chec kLicense.y=13"
(I got this part from an example, and I don't fully understand what it's
supposed to do.
There is a text box named "LicNum" and a button named "CheckLicense" on the
web page...)

It fills in the license number, but just sits there on the web site without
looking up the contracor's license information.

What am I doing wrong? How do I "click" the button?

Thanks,
---Selden McCabe

Nov 18 '05 #1
1 1114
"Selden McCabe" <se*****@msn.com> wrote in news:ugDuNDJ#DHA.1052
@TK2MSFTNGP12.phx.gbl:
It fills in the license number, but just sits there on the web site without
looking up the contracor's license information.

What am I doing wrong? How do I "click" the button?


Dont use upload -thats the wrong method. You have to POST the data (or in
some case GET it)
--
Chad Z. Hower (a.k.a. Kudzu) - http://www.hower.org/Kudzu/
"Programming is an art form that fights back"
Nov 18 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: David Jones | last post by:
Hi, I'm interested in learning about web scraping/site scraping using Python. Does anybody know of some online resources or have any modules that are available to help out. O'Reilly published an...
4
by: Roland Hall | last post by:
Am I correct in assuming screen scraping is just the response text sent to the browser? If so, would that mean that this could not be screen scraped? function moi() { var tag = '<a href='; var...
1
by: mustafa | last post by:
anyone know some good reliable html scraping (with python) tutorials. i have looked around and found a few. one uses urllib2 and beautifull soap modules for scraping and parsing...
2
by: Selden McCabe | last post by:
I've been working on a web scraping program, and have the basics down. But I don't understand the parameters. Normally, you go to a URL (say a reverse yellow pages directory), and enter some...
3
by: Jim Giblin | last post by:
I need to scrape specific information from another website, specifically the prices of precious metals from several different vendors. While I will credit the vendors as the data source, I do not...
1
by: niv | last post by:
Hello, I would like to screen scrape certain parts of a webpage...how can I do this in asp.net For instance.... a stockticker thats embeded on a webpage.. I dont want the entire page.. I...
2
by: Victor | last post by:
I'm doing screen scraping by retrieving data from one site and entering into another site. I have a problem with logging into the site. User name and password field contain 'name' property, and...
4
by: jeffbg123 | last post by:
Hey, I am trying to make a bot for a flash game using python. However I am having some trouble with a screen scraping strategy. Is there an accepted way to compare a full screenshot with the...
3
by: bruce | last post by:
Hi... got a short test app that i'm playing with. the goal is to get data off the page in question. basically, i should be able to get a list of "tr" nodes, and then to iterate/parse them....
1
by: bruce | last post by:
Hi Paul... Thanks for the reply. Came to the same conclusion a few minutes before I saw your email. Another question: tr=d.xpath(foo) gets me an array of nodes.
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.