Screen Scrape POST Form Results Pages

[nbomike]

Hello. I want to scrape pages from a site that generates pages from form inputs using this web app . However, the URL of the results page (the page I want to scrape) is masked and is always the same. It looks something like this for every form input combination:

http://www.scrapedsite.com/the/results/page.htm

Is it even possible to scrape the individual results pages? I posted this same question in the PHP forum, because I wanted to scrape the site using PHP. However, the site I want to scrape is written in ASP, so I thought it might be useful to ask here.

Also, since the results are generated based on multiple forms (select from a drop down and submit to get to the next drop down), it appears that sessions variables are being used. Any thoughts on how this might complicate things? Thanks.

[markrawlingson]

The result page will use Request.Form to obtain the values posted to it through the form. Since you're not actually submitting the form, those values will not exist - and therefore you aren't going to get your desired result. Even more so to the point is that if they are indeed using session level variables, those won't be set either.

It sounds like you're trying to get a scrape a post result without actually posting anything.