By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
446,218 Members | 1,257 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 446,218 IT Pros & Developers. It's quick & easy.

Screen Scrape POST Form Results Pages

P: 17
Hello. I want to scrape pages from a site that generates pages from form inputs using this web app . However, the URL of the results page (the page I want to scrape) is masked and is always the same. It looks something like this for every form input combination:

http://www.scrapedsite.com/the/results/page.htm

Is it even possible to scrape the individual results pages? I posted this same question in the PHP forum, because I wanted to scrape the site using PHP. However, the site I want to scrape is written in ASP, so I thought it might be useful to ask here.

Also, since the results are generated based on multiple forms (select from a drop down and submit to get to the next drop down), it appears that sessions variables are being used. Any thoughts on how this might complicate things? Thanks.
Aug 30 '07 #1
Share this Question
Share on Google+
1 Reply


markrawlingson
Expert 100+
P: 346
The result page will use Request.Form to obtain the values posted to it through the form. Since you're not actually submitting the form, those values will not exist - and therefore you aren't going to get your desired result. Even more so to the point is that if they are indeed using session level variables, those won't be set either.

It sounds like you're trying to get a scrape a post result without actually posting anything.
Aug 30 '07 #2

Post your reply

Sign in to post your reply or Sign up for a free account.