469,890 Members | 2,154 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,890 developers. It's quick & easy.

how to browse using urllib2 and cookeilib the correct way

Hi ,

I am using python2.4 "urllib2" and "cookelib".
In line "5" below i provide my credentials to
login into a web site.During the first attempt i "fail",
judging from the output of line "6".
I try again and the second time i succeed,judging
from the output of line "8".

Now using the "twill" module (http://www.idyll.org/~t/www-tools/twill/)
it suceeds in the first attempt(lines "10" to "19").

can anybody explain, in the first case why i need to do two attempts.
1) >>>import urllib2,urllib,cookielib
2) >>>cj = cookielib.CookieJar()
3) >>>opener = urllib2.build_opener( urllib2.HTTPCookieProcessor(cj))
4) >>>data = urllib.urlencode( { "username" : "user" , "password" :
"****" } )
5) >>>fp = opener.open(
"https://rhn.redhat.com/rhn/LoginSubmit.do",data )
6) >>>fp.url

'https://rhn.redhat.com/rhn/ReLogin.do?url_bounce=/network/index.pxt'
7) >>>fp = opener.open(
"https://rhn.redhat.com/rhn/LoginSubmit.do",data )
8) >>>fp.url
'https://rhn.redhat.com/network/index.pxt'

#twill module

10) >>>from twill.commands import *
11) >>>go("https://rhn.redhat.com")
12) ==> at https://rhn.redhat.com
13) 'https://rhn.redhat.com'
14) >>>fv("2","username","eipl")
15) >>>fv("2","password","ensim1234")
16) >>>submit()
17) Note: submit is using submit button: name="None", value=" ¬ Sign
In¬ "
18) >>>url('https://rhn.redhat.com/network/index.pxt')
19) 'https://rhn.redhat.com/network/index.pxt'

Thanks
Jitu

Apr 26 '06 #1
3 1367
jn***@ensim.com wrote:
can anybody explain, in the first case why i need to do two attempts.


I would guess it's because redhat requires your browser to submit a session
cookie with the login form. In the urllib2 example, the first request you
make tries to submit login form data directly. Since it's your first hit
on their site, you don't have a cookie yet. People browsing interactively
would at least load the login page first before submitting it.

Your twill example takes care of this by requesting a page before trying to
login.

That would be my guess.
Apr 26 '06 #2
ok , got it . Thanks

Apr 26 '06 #3
Edward Elliott <no****@127.0.0.1> writes:
jn***@ensim.com wrote:
can anybody explain, in the first case why i need to do two attempts.


I would guess it's because redhat requires your browser to submit a session
cookie with the login form. In the urllib2 example, the first request you
make tries to submit login form data directly. Since it's your first hit
on their site, you don't have a cookie yet. People browsing interactively
would at least load the login page first before submitting it.

Your twill example takes care of this by requesting a page before trying to
login.

That would be my guess.


Uh, yeah you're right actually. Forget what I said about Refresh...
John

Apr 26 '06 #4

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

4 posts views Thread by O. Koch | last post: by
2 posts views Thread by Andre Bocchini | last post: by
4 posts views Thread by bmiras | last post: by
reply views Thread by Lee, Dustin | last post: by
reply views Thread by jacob c. | last post: by
4 posts views Thread by Fuzzyman | last post: by
reply views Thread by Pieter Edelman | last post: by
1 post views Thread by Waqarahmed | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.