By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,602 Members | 1,507 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,602 IT Pros & Developers. It's quick & easy.

using openurl to log into Yahoo services

P: n/a
Hello all,

I'm trying to write a script to log into Yahoo! (mail, groups, etc),
but when I pass the full URL with all form elements via Python, I get
a resutling 400 - Bad Request page. When I just plop the full URL
into a browser, it works perfectly. Is there something I need to
specify in python to submit the correct info (headers, user agent,
etc) to get the response I want? Here is the code I have that returns
the Bad Request:

import urllib, win32gui, win32clipboard, win32con, os, getpass, re
yid = raw_input("Yahoo! ID: ")
pw = getpass.getpass(prompt = 'Yahoo password: ')
url =
"https://login.yahoo.com/config/login?.tries=1&.src=ygrp&.intl=us&.v=0&.challenge= U4VY1YGqdPf8z3SaVccJdhV63YCw&.chkP=Y&.done=http://groups.yahoo.com&login="+yid+"&passwd="+pw+"&.pers istent=y&.save=Sign
In"
temp = urllib.urlopen(url)
grp_list_source = temp.read()

Any thoughts or suggestions? Thanks.


Nov 22 '05 #1
Share this Question
Share on Google+
4 Replies


P: n/a
On Tue, 15 Nov 2005 16:08:06 -0000,
"joe_public34" <jo**********@yahoo.com> wrote:
Hello all,
I'm trying to write a script to log into Yahoo! (mail, groups, etc),
but when I pass the full URL with all form elements via Python, I get
a resutling 400 - Bad Request page. When I just plop the full URL
into a browser, it works perfectly. Is there something I need to
specify in python to submit the correct info (headers, user agent,
etc) to get the response I want? Here is the code I have that returns
the Bad Request: import urllib, win32gui, win32clipboard, win32con, os, getpass, re
yid = raw_input("Yahoo! ID: ")
pw = getpass.getpass(prompt = 'Yahoo password: ')
url =
"https://login.yahoo.com/config/login?.tries=1&.src=ygrp&.intl=us&.v=0&.challenge= U4VY1YGqdPf8z3SaVccJdhV63YCw&.chkP=Y&.done=http://groups.yahoo.com&login="+yid+"&passwd="+pw+"&.pers istent=y&.save=Sign
In"
temp = urllib.urlopen(url)
grp_list_source = temp.read() Any thoughts or suggestions? Thanks.


It is possible that the "challenge" field is created by yahoo's server
based on the request that caused the server to serve the login page;
perhaps it contains a hash of the user agent that sent the request.
When you reply to that challenge at a much later time, or with a
different user agent, or with something else that's different from the
original request, yahoo's server thinks you're trying to break into
yahoo.

Regards,
Dan

--
Dan Sommers
<http://www.tombstonezero.net/dan/>
Nov 22 '05 #2

P: n/a
On Tue, 15 Nov 2005 16:08:06 -0000,
"joe_public34" <jo**********@yahoo.com> wrote:
Hello all,
I'm trying to write a script to log into Yahoo! (mail, groups, etc),
but when I pass the full URL with all form elements via Python, I get
a resutling 400 - Bad Request page. When I just plop the full URL
into a browser, it works perfectly. Is there something I need to
specify in python to submit the correct info (headers, user agent,
etc) to get the response I want? Here is the code I have that returns
the Bad Request: import urllib, win32gui, win32clipboard, win32con, os, getpass, re
yid = raw_input("Yahoo! ID: ")
pw = getpass.getpass(prompt = 'Yahoo password: ')
url =
"https://login.yahoo.com/config/login?.tries=1&.src=ygrp&.intl=us&.v=0&.challenge= U4VY1YGqdPf8z3SaVccJdhV63YCw&.chkP=Y&.done=http://groups.yahoo.com&login="+yid+"&passwd="+pw+"&.pers istent=y&.save=Sign
In"
temp = urllib.urlopen(url)
grp_list_source = temp.read() Any thoughts or suggestions? Thanks.


It is possible that the "challenge" field is created by yahoo's server
based on the request that caused the server to serve the login page;
perhaps it contains a hash of the user agent that sent the request.
When you reply to that challenge at a much later time, or with a
different user agent, or with something else that's different from the
original request, yahoo's server thinks you're trying to break into
yahoo.

Regards,
Dan

--
Dan Sommers
<http://www.tombstonezero.net/dan/>
Nov 22 '05 #3

P: n/a
"joe_public34" <jo**********@yahoo.com> writes:
Hello all,

I'm trying to write a script to log into Yahoo! (mail, groups, etc),


I don't have an answer to your problem but I *do* have a script that
logs into yahoo. It's ugly, but it works. I'm refactoring it - if I
ever get back to it.

I've appended the crucial code, but haven't tried running this
excerpt. If you have problems or questions, feel free to ask.

<mike
myYahooID = 'Your Id Here'

from urllib2 import build_opener, HTTPCookieProcessor
from urllib import urlencode
from BeautifulSoup import BeautifulSoup
from sys import argv, exit

urlopen = build_opener(HTTPCookieProcessor()).open

def login(password):
"""Get past the multiple login forms that Yahoo uses."""

print "Logging in to Yahoo groups."

passage = BeautifulSoup(urlopen('http://groups.yahoo.com').read())
passage.done()
i = 0
while passage and i < 5:
page = passage
passage = handle_login(passage, password)
i = i + 1

if i >= 5:
handle_bogus(page, 'To many login attempts') # Never returns

while 1:
metas = page.fetch('meta', {'http-equiv': 'Refresh'})
if not metas:
return page
# Sigh. It's redirect page that didn't get handled by the library.
page = meta_redirect(metas[0]['content'])

return page
def handle_login(soup, password):
"""Parse a page for login information, and hand back the result of logging in.

Returns None if there is no login form."""

# Get the login page, scrape the form, and log in!
forms = soup.fetch('form', {'name': 'login_form'})
if not forms:
return None
form = forms[0]

postdata = []
for intag in form.fetchNext('input'):
if intag['type'] == 'hidden':
postdata.append((intag['name'], intag['value']))
elif intag['type'] == 'password':
postdata.append((intag['name'], password))
elif intag['type'] == 'checkbox':
postdata.append((intag['name'], 'n'))
elif intag['type'] == 'text' and intag['name'] == 'login':
postdata.append(('login', myYahooID))
elif intag['type'] == 'submit':
if intag.has_key(['name']):
postdata.append((intag['name'], intag['value']))
else:
postdata.append(('submit', intag['value']))
else:
print "Login form had unrecognized input tag:", str(intag)

out = BeautifulSoup(urlopen(form['action'], urlencode(postdata)).read())
out.done()
return out

def meta_redirect(value):
"""Handle a http-equiv redirect, since the library isn't reliable."""

for spam in value.split(';'):
stuff = spam.strip().split('=')
if stuff and stuff[0] == 'url':
out = BeautifulSoup(urlopen(stuff[1]).read())
out.done()
return out
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Nov 22 '05 #4

P: n/a
"joe_public34" <jo**********@yahoo.com> writes:
Hello all,

I'm trying to write a script to log into Yahoo! (mail, groups, etc),


I don't have an answer to your problem but I *do* have a script that
logs into yahoo. It's ugly, but it works. I'm refactoring it - if I
ever get back to it.

I've appended the crucial code, but haven't tried running this
excerpt. If you have problems or questions, feel free to ask.

<mike
myYahooID = 'Your Id Here'

from urllib2 import build_opener, HTTPCookieProcessor
from urllib import urlencode
from BeautifulSoup import BeautifulSoup
from sys import argv, exit

urlopen = build_opener(HTTPCookieProcessor()).open

def login(password):
"""Get past the multiple login forms that Yahoo uses."""

print "Logging in to Yahoo groups."

passage = BeautifulSoup(urlopen('http://groups.yahoo.com').read())
passage.done()
i = 0
while passage and i < 5:
page = passage
passage = handle_login(passage, password)
i = i + 1

if i >= 5:
handle_bogus(page, 'To many login attempts') # Never returns

while 1:
metas = page.fetch('meta', {'http-equiv': 'Refresh'})
if not metas:
return page
# Sigh. It's redirect page that didn't get handled by the library.
page = meta_redirect(metas[0]['content'])

return page
def handle_login(soup, password):
"""Parse a page for login information, and hand back the result of logging in.

Returns None if there is no login form."""

# Get the login page, scrape the form, and log in!
forms = soup.fetch('form', {'name': 'login_form'})
if not forms:
return None
form = forms[0]

postdata = []
for intag in form.fetchNext('input'):
if intag['type'] == 'hidden':
postdata.append((intag['name'], intag['value']))
elif intag['type'] == 'password':
postdata.append((intag['name'], password))
elif intag['type'] == 'checkbox':
postdata.append((intag['name'], 'n'))
elif intag['type'] == 'text' and intag['name'] == 'login':
postdata.append(('login', myYahooID))
elif intag['type'] == 'submit':
if intag.has_key(['name']):
postdata.append((intag['name'], intag['value']))
else:
postdata.append(('submit', intag['value']))
else:
print "Login form had unrecognized input tag:", str(intag)

out = BeautifulSoup(urlopen(form['action'], urlencode(postdata)).read())
out.done()
return out

def meta_redirect(value):
"""Handle a http-equiv redirect, since the library isn't reliable."""

for spam in value.split(';'):
stuff = spam.strip().split('=')
if stuff and stuff[0] == 'url':
out = BeautifulSoup(urlopen(stuff[1]).read())
out.done()
return out
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Nov 22 '05 #5

This discussion thread is closed

Replies have been disabled for this discussion.