473,396 Members | 2,115 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,396 software developers and data experts.

[2.5] Script to POST to web page with cookies?

Hello

I need to write a script to automate fetching data from a web site:
1. using the POST method, log on, with login/password saved as cookies
2. download page and extract relevent information using regexes
3. log off
4. wait for a random number of minutes, and GOTO 1

I'm a bit confused with how to get POST and cookies in the same
script:
- urllib vs urllib2 vs httplib?
- "ClientCookie.urlopen("www") behaves identically to urllib2.urlopen,
except that it deals with cookies automatically"

Has anyone some working code that I could borrow (er... steal) to do
this?

Thank you.
Jul 20 '07 #1
2 1816
Hello Gille,
I need to write a script to automate fetching data from a web site:
1. using the POST method, log on, with login/password saved as cookies
2. download page and extract relevent information using regexes
3. log off
4. wait for a random number of minutes, and GOTO 1
...
http://wwwsearch.sourceforge.net/mechanize/

HTH,
--
Miki <mi*********@gmail.com>
http://pythonwise.blogspot.com

Jul 20 '07 #2
On Jul 19, 7:43 pm, Gilles Ganault <nos...@nospam.comwrote:
Hello

I need to write a script to automate fetching data from a web site:
1. using the POST method, log on, with login/password saved as cookies
2. download page and extract relevent information using regexes
3. log off
4. wait for a random number of minutes, and GOTO 1

I'm a bit confused with how to get POST and cookies in the same
script:
- urllib vs urllib2 vs httplib?
- "ClientCookie.urlopen("www") behaves identically to urllib2.urlopen,
except that it deals with cookies automatically"

Has anyone some working code that I could borrow (er... steal) to do
this?

Thank you.
Also, see here:

http://www.voidspace.org.uk/python/a...ookielib.shtml

Jul 20 '07 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Wayne Smith | last post by:
Applies to: Microsoft FrontPage 2000, Microsoft Access 2000, IIS 5.0 Operating System: Microsoft Windows 2000 Professional I am trying to protect a portion of a web site by allowing users to...
4
by: James Johnson | last post by:
Dear C#Dex, I am trying to automate a POST to a web page that clicks a button. I have been able to hit a target web page and run the web page. However, the button on the page does not click. ...
0
by: WIWA | last post by:
Hi, I want to login to a password protected website and fetch the content of the page behind. I have based my code on http://weblogs.asp.net/jdennany/archive/2005/04/23/403971.aspx. When I use...
1
by: JohnathanKong | last post by:
Hello, I hope this is the right place to post, but anyways, here goes. I current have a forum, snitz, in place, but am having an IE specific problem. Basically what happens is, I have a page that...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.