Hello
I need to write a script to automate fetching data from a web site:
1. using the POST method, log on, with login/password saved as cookies
2. download page and extract relevent information using regexes
3. log off
4. wait for a random number of minutes, and GOTO 1
I'm a bit confused with how to get POST and cookies in the same
script:
- urllib vs urllib2 vs httplib?
- "ClientCookie.urlopen("www") behaves identically to urllib2.urlopen,
except that it deals with cookies automatically"
Has anyone some working code that I could borrow (er... steal) to do
this?
Thank you.
2  1815 
Hello Gille,
I need to write a script to automate fetching data from a web site:
1. using the POST method, log on, with login/password saved as cookies
2. download page and extract relevent information using regexes
3. log off
4. wait for a random number of minutes, and GOTO 1
...
http://wwwsearch.sourceforge.net/mechanize/
HTH,
--
Miki <mi*********@gmail.com> http://pythonwise.blogspot.com
On Jul 19, 7:43 pm, Gilles Ganault <nos...@nospam.comwrote:
Hello
I need to write a script to automate fetching data from a web site:
1. using the POST method, log on, with login/password saved as cookies
2. download page and extract relevent information using regexes
3. log off
4. wait for a random number of minutes, and GOTO 1
I'm a bit confused with how to get POST and cookies in the same
script:
- urllib vs urllib2 vs httplib?
- "ClientCookie.urlopen("www") behaves identically to urllib2.urlopen,
except that it deals with cookies automatically"
Has anyone some working code that I could borrow (er... steal) to do
this?
Thank you.
Also, see here: http://www.voidspace.org.uk/python/a...ookielib.shtml This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Wayne Smith |
last post by:
Applies to: Microsoft FrontPage 2000, Microsoft Access 2000, IIS 5.0
Operating System: Microsoft Windows 2000 Professional
I am trying to protect a portion of a web site by allowing users to...
|
by: James Johnson |
last post by:
Dear C#Dex,
I am trying to automate a POST to a web page that clicks a button. I
have been able to hit a target web page and run the web page. However,
the button on the page does not click. ...
|
by: WIWA |
last post by:
Hi,
I want to login to a password protected website and fetch the content of the
page behind. I have based my code on
http://weblogs.asp.net/jdennany/archive/2005/04/23/403971.aspx. When I use...
|
by: JohnathanKong |
last post by:
Hello, I hope this is the right place to post, but anyways, here goes. I current have a forum, snitz, in place, but am having an IE specific problem. Basically what happens is, I have a page that...
|
by: ryjfgjl |
last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
|
by: ryjfgjl |
last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
| |
