470,619 Members | 1,528 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 470,619 developers. It's quick & easy.

using httplib for authentication

Hello All,

I want to use python to download files from sites where authentication is
required. The page appears to send a form with the login and pass by post. I
would like to log in and keep this session open within python and
download a number of files automatically. Could anyone kindly point me in the
right direction? I have discovered httplib and read the documentation and
looked at the examples, but they don't seem helpful for this.

James

--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/
Aug 3 '05 #1
2 2472

James Stroud wrote:
Hello All,

I want to use python to download files from sites where authentication is
required. The page appears to send a form with the login and pass by post. I
would like to log in and keep this session open within python and
download a number of files automatically. Could anyone kindly point me in the
right direction? I have discovered httplib and read the documentation and
looked at the examples, but they don't seem helpful for this.

Hello James,

httplib is built on top of the socket module - but is still a library
meant for doing fairly low level http operations. For fetching URLs,
urllib2 is probably the way to go. It handles GET and POST with ease.

If you think this will do the job then you could try reading the
urllib2 tutorial at :

http://www.voidspace.org.uk/python/articles.shtml#http

You might wnat to look at other extension modules too - ClientCookie
for handling cookies, ClientForm for automatically filling in the
forms, BeautifulSoup for parsing HTML pages....

All the Best,

Fuzzy
http://www.voidspace.org.uk/python
James

--
James Stroud
UCLA-DOE Institute for Genomics and Proteomics
Box 951570
Los Angeles, CA 90095

http://www.jamesstroud.com/


Aug 3 '05 #2
Thank you Fuzzy, I will look into these things. Maybe the site is setting a
cookie, as you have suggested. I have never delved into the ways of http
except to configure apache and write some very bare-bones web pages, so I
have to say that some very obvious things do not occur to me.

James

On Wednesday 03 August 2005 08:41 am, Fuzzyman wrote:
Hello James,

httplib is built on top of the socket module - but is still a library
meant for doing fairly low level http operations. For fetching URLs,
urllib2 is probably the way to go. It handles GET and POST with ease.

If you think this will do the job then you could try reading the
urllib2 tutorial at :

http://www.voidspace.org.uk/python/articles.shtml#http

You might wnat to look at other extension modules too - ClientCookie
for handling cookies, ClientForm for automatically filling in the
forms, BeautifulSoup for parsing HTML pages....

All the Best,

Fuzzy
http://www.voidspace.org.uk/python

Aug 3 '05 #3

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

7 posts views Thread by luigipaioro | last post: by
4 posts views Thread by Roger Binns | last post: by
1 post views Thread by Brian Beck | last post: by
3 posts views Thread by John Reese | last post: by
reply views Thread by Pieter Edelman | last post: by
reply views Thread by Robert | last post: by
1 post views Thread by Michele Simionato | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.