Connecting Tech Pros Worldwide Help | Site Map

Retrieve files based on links in a html which is generated by a cgi script

Newbie
 
Join Date: Aug 2009
Posts: 1
#1: Aug 14 '09
This question may be a little complicated, at least for me, since I am fairly new to python. So I know a webpage that has two drop-down selection boxes. and a 'search' button. When I choose the parameters inside the two drop down boxes and click search I get a webpage that displays the results. In the result page there are a bunch of url links. What I want to do is to automate the process of choosing different combination of search parameters and download the files that are linked to the urls in the final search result page.

Hope I explained my question.
YarrOfDoom's Avatar
Expert
 
Join Date: Aug 2007
Location: Belgium
Posts: 1,118
#2: Aug 20 '09

re: Retrieve files based on links in a html which is generated by a cgi script


You can use urlib.request.urlopen() to get the HTML-source of the results page. From there on you can get the links and use urlopen() again to download the files and save them to a file on your computer (don't forget to use binary mode when saving the files, unless they're text-files, then it doesn't matter).
Reply

Tags
download; cgi