Web-Forms

[Mathias Waack]

Hi,

I need to access some information from a web site which are only accessible
through a form. Thus for each bucket of data you have to fill out the form,
submit it and wait for an answer. Very easy - if you don't have to check
some hundred times. Of course this site requires cookies, it is not
directly accessible by URL and so on. All that nice stuff used to make a
web site more professional;)

But now the question: how can this be solved by using Python?

Mathias

[Simon Brunning]

On 7/21/05, Mathias Waack <M.*****@gmx.de> wrote:

I need to access some information from a web site which are only accessible
through a form. Thus for each bucket of data you have to fill out the form,
submit it and wait for an answer. Very easy - if you don't have to check
some hundred times. Of course this site requires cookies, it is not
directly accessible by URL and so on. All that nice stuff used to make a
web site more professional;)

But now the question: how can this be solved by using Python?

Mechanize?

<http://wwwsearch.sourceforge.net/mechanize/>

--
Cheers,
Simon B,
si***@brunningonline.net,
http://www.brunningonline.net/simon/blog/

[Mike Meyer]

Mathias Waack <M.*****@gmx.de> writes:

Hi,

I need to access some information from a web site which are only accessible
through a form. Thus for each bucket of data you have to fill out the form,
submit it and wait for an answer. Very easy - if you don't have to check
some hundred times. Of course this site requires cookies, it is not
directly accessible by URL and so on. All that nice stuff used to make a
web site more professional;)

But now the question: how can this be solved by using Python?

urllib2 (in the standard library) for reading the data from the
website. BeautifulSoup <URL:
http://www.crummy.com/software/BeautifulSoup/> for extracting data
from the pages.

<mike

--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Fuzzyman]

Mike Meyer wrote:

Mathias Waack <M.*****@gmx.de> writes:

Hi,

I need to access some information from a web site which are only accessible
through a form. Thus for each bucket of data you have to fill out the form,
submit it and wait for an answer. Very easy - if you don't have to check
some hundred times. Of course this site requires cookies, it is not
directly accessible by URL and so on. All that nice stuff used to make a
web site more professional;)

But now the question: how can this be solved by using Python?
urllib2 (in the standard library) for reading the data from the
website. BeautifulSoup <URL:
http://www.crummy.com/software/BeautifulSoup/> for extracting data
from the pages.

Adding ClientForm for automatically handling the forms and
ClientCookie/cookielib for automatic cookie handling.

Best Regards,

Fuzzy
http://www.voidspace.org.uk/python
<mike

--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Michele Simionato]

Use twill:http://www.idyll.org/~t/www-tools/twill.html

Michele Simionato

[Mathias Waack]

Michele Simionato wrote:

Use twill:http://www.idyll.org/~t/www-tools/twill.html

I'll do so. Twill is great, anyway thanks for all other responses too.

Mathias