473,325 Members | 2,785 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,325 software developers and data experts.

using openurl to log into Yahoo services

Hello all,

I'm trying to write a script to log into Yahoo! (mail, groups, etc),
but when I pass the full URL with all form elements via Python, I get
a resutling 400 - Bad Request page. When I just plop the full URL
into a browser, it works perfectly. Is there something I need to
specify in python to submit the correct info (headers, user agent,
etc) to get the response I want? Here is the code I have that returns
the Bad Request:

import urllib, win32gui, win32clipboard, win32con, os, getpass, re
yid = raw_input("Yahoo! ID: ")
pw = getpass.getpass(prompt = 'Yahoo password: ')
url =
"https://login.yahoo.com/config/login?.tries=1&.src=ygrp&.intl=us&.v=0&.challenge= U4VY1YGqdPf8z3SaVccJdhV63YCw&.chkP=Y&.done=http://groups.yahoo.com&login="+yid+"&passwd="+pw+"&.pers istent=y&.save=Sign
In"
temp = urllib.urlopen(url)
grp_list_source = temp.read()

Any thoughts or suggestions? Thanks.


Nov 22 '05 #1
4 3648
On Tue, 15 Nov 2005 16:08:06 -0000,
"joe_public34" <jo**********@yahoo.com> wrote:
Hello all,
I'm trying to write a script to log into Yahoo! (mail, groups, etc),
but when I pass the full URL with all form elements via Python, I get
a resutling 400 - Bad Request page. When I just plop the full URL
into a browser, it works perfectly. Is there something I need to
specify in python to submit the correct info (headers, user agent,
etc) to get the response I want? Here is the code I have that returns
the Bad Request: import urllib, win32gui, win32clipboard, win32con, os, getpass, re
yid = raw_input("Yahoo! ID: ")
pw = getpass.getpass(prompt = 'Yahoo password: ')
url =
"https://login.yahoo.com/config/login?.tries=1&.src=ygrp&.intl=us&.v=0&.challenge= U4VY1YGqdPf8z3SaVccJdhV63YCw&.chkP=Y&.done=http://groups.yahoo.com&login="+yid+"&passwd="+pw+"&.pers istent=y&.save=Sign
In"
temp = urllib.urlopen(url)
grp_list_source = temp.read() Any thoughts or suggestions? Thanks.


It is possible that the "challenge" field is created by yahoo's server
based on the request that caused the server to serve the login page;
perhaps it contains a hash of the user agent that sent the request.
When you reply to that challenge at a much later time, or with a
different user agent, or with something else that's different from the
original request, yahoo's server thinks you're trying to break into
yahoo.

Regards,
Dan

--
Dan Sommers
<http://www.tombstonezero.net/dan/>
Nov 22 '05 #2
On Tue, 15 Nov 2005 16:08:06 -0000,
"joe_public34" <jo**********@yahoo.com> wrote:
Hello all,
I'm trying to write a script to log into Yahoo! (mail, groups, etc),
but when I pass the full URL with all form elements via Python, I get
a resutling 400 - Bad Request page. When I just plop the full URL
into a browser, it works perfectly. Is there something I need to
specify in python to submit the correct info (headers, user agent,
etc) to get the response I want? Here is the code I have that returns
the Bad Request: import urllib, win32gui, win32clipboard, win32con, os, getpass, re
yid = raw_input("Yahoo! ID: ")
pw = getpass.getpass(prompt = 'Yahoo password: ')
url =
"https://login.yahoo.com/config/login?.tries=1&.src=ygrp&.intl=us&.v=0&.challenge= U4VY1YGqdPf8z3SaVccJdhV63YCw&.chkP=Y&.done=http://groups.yahoo.com&login="+yid+"&passwd="+pw+"&.pers istent=y&.save=Sign
In"
temp = urllib.urlopen(url)
grp_list_source = temp.read() Any thoughts or suggestions? Thanks.


It is possible that the "challenge" field is created by yahoo's server
based on the request that caused the server to serve the login page;
perhaps it contains a hash of the user agent that sent the request.
When you reply to that challenge at a much later time, or with a
different user agent, or with something else that's different from the
original request, yahoo's server thinks you're trying to break into
yahoo.

Regards,
Dan

--
Dan Sommers
<http://www.tombstonezero.net/dan/>
Nov 22 '05 #3
"joe_public34" <jo**********@yahoo.com> writes:
Hello all,

I'm trying to write a script to log into Yahoo! (mail, groups, etc),


I don't have an answer to your problem but I *do* have a script that
logs into yahoo. It's ugly, but it works. I'm refactoring it - if I
ever get back to it.

I've appended the crucial code, but haven't tried running this
excerpt. If you have problems or questions, feel free to ask.

<mike
myYahooID = 'Your Id Here'

from urllib2 import build_opener, HTTPCookieProcessor
from urllib import urlencode
from BeautifulSoup import BeautifulSoup
from sys import argv, exit

urlopen = build_opener(HTTPCookieProcessor()).open

def login(password):
"""Get past the multiple login forms that Yahoo uses."""

print "Logging in to Yahoo groups."

passage = BeautifulSoup(urlopen('http://groups.yahoo.com').read())
passage.done()
i = 0
while passage and i < 5:
page = passage
passage = handle_login(passage, password)
i = i + 1

if i >= 5:
handle_bogus(page, 'To many login attempts') # Never returns

while 1:
metas = page.fetch('meta', {'http-equiv': 'Refresh'})
if not metas:
return page
# Sigh. It's redirect page that didn't get handled by the library.
page = meta_redirect(metas[0]['content'])

return page
def handle_login(soup, password):
"""Parse a page for login information, and hand back the result of logging in.

Returns None if there is no login form."""

# Get the login page, scrape the form, and log in!
forms = soup.fetch('form', {'name': 'login_form'})
if not forms:
return None
form = forms[0]

postdata = []
for intag in form.fetchNext('input'):
if intag['type'] == 'hidden':
postdata.append((intag['name'], intag['value']))
elif intag['type'] == 'password':
postdata.append((intag['name'], password))
elif intag['type'] == 'checkbox':
postdata.append((intag['name'], 'n'))
elif intag['type'] == 'text' and intag['name'] == 'login':
postdata.append(('login', myYahooID))
elif intag['type'] == 'submit':
if intag.has_key(['name']):
postdata.append((intag['name'], intag['value']))
else:
postdata.append(('submit', intag['value']))
else:
print "Login form had unrecognized input tag:", str(intag)

out = BeautifulSoup(urlopen(form['action'], urlencode(postdata)).read())
out.done()
return out

def meta_redirect(value):
"""Handle a http-equiv redirect, since the library isn't reliable."""

for spam in value.split(';'):
stuff = spam.strip().split('=')
if stuff and stuff[0] == 'url':
out = BeautifulSoup(urlopen(stuff[1]).read())
out.done()
return out
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Nov 22 '05 #4
"joe_public34" <jo**********@yahoo.com> writes:
Hello all,

I'm trying to write a script to log into Yahoo! (mail, groups, etc),


I don't have an answer to your problem but I *do* have a script that
logs into yahoo. It's ugly, but it works. I'm refactoring it - if I
ever get back to it.

I've appended the crucial code, but haven't tried running this
excerpt. If you have problems or questions, feel free to ask.

<mike
myYahooID = 'Your Id Here'

from urllib2 import build_opener, HTTPCookieProcessor
from urllib import urlencode
from BeautifulSoup import BeautifulSoup
from sys import argv, exit

urlopen = build_opener(HTTPCookieProcessor()).open

def login(password):
"""Get past the multiple login forms that Yahoo uses."""

print "Logging in to Yahoo groups."

passage = BeautifulSoup(urlopen('http://groups.yahoo.com').read())
passage.done()
i = 0
while passage and i < 5:
page = passage
passage = handle_login(passage, password)
i = i + 1

if i >= 5:
handle_bogus(page, 'To many login attempts') # Never returns

while 1:
metas = page.fetch('meta', {'http-equiv': 'Refresh'})
if not metas:
return page
# Sigh. It's redirect page that didn't get handled by the library.
page = meta_redirect(metas[0]['content'])

return page
def handle_login(soup, password):
"""Parse a page for login information, and hand back the result of logging in.

Returns None if there is no login form."""

# Get the login page, scrape the form, and log in!
forms = soup.fetch('form', {'name': 'login_form'})
if not forms:
return None
form = forms[0]

postdata = []
for intag in form.fetchNext('input'):
if intag['type'] == 'hidden':
postdata.append((intag['name'], intag['value']))
elif intag['type'] == 'password':
postdata.append((intag['name'], password))
elif intag['type'] == 'checkbox':
postdata.append((intag['name'], 'n'))
elif intag['type'] == 'text' and intag['name'] == 'login':
postdata.append(('login', myYahooID))
elif intag['type'] == 'submit':
if intag.has_key(['name']):
postdata.append((intag['name'], intag['value']))
else:
postdata.append(('submit', intag['value']))
else:
print "Login form had unrecognized input tag:", str(intag)

out = BeautifulSoup(urlopen(form['action'], urlencode(postdata)).read())
out.done()
return out

def meta_redirect(value):
"""Handle a http-equiv redirect, since the library isn't reliable."""

for spam in value.split(';'):
stuff = spam.strip().split('=')
if stuff and stuff[0] == 'url':
out = BeautifulSoup(urlopen(stuff[1]).read())
out.done()
return out
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
Nov 22 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

10
by: asj | last post by:
BIG news from the web services front. Amazon will use web services to tie all its vendors together. The company implementing the system will be using Java/C++ (migrating to all-java later). ...
18
by: smileplzz | last post by:
r there any files which we can download from this group. can we only ask questions in this group? no files stored like it is there in yahoo groups. i think it should be there in group services....
3
by: Matt D | last post by:
In my web service project I've imported an assembly (that I also have the source to) that contains structs that are serializable. I've created some web service methods that return and take as...
3
by: JL | last post by:
I have a VB.NET desktop program that reads/writes data to a server using a Java-based Web Service. This web service, in identical formats, is located on several servers with each server being a...
0
by: joe_public34 | last post by:
Hello all, I'm trying to write a script to log into Yahoo! (mail, groups, etc), but when I pass the full URL with all form elements via Python, I get a resutling 400 - Bad Request page. When I...
3
by: Jim Lewis | last post by:
I have read several things that state accessing a Web Service through a Query String should work. However, when I try to execute http://localhost/webservice1/service1.asmx/HelloWorld I get the...
4
by: archana | last post by:
Hi all, I am new to web services. I am using stateserver to stored data which i set using session. I changed mode in web.config file to StateServer. But didn't change tcipip address. I kept...
8
by: howa | last post by:
Hello, Is it possible to take a screen shot of the browser using PHP? Something similar to Alexa Web Site Screen Dump... thx
8
by: Mo | last post by:
Hi, I can not find a decent example showing how to consume a asp.net 2.0 web service using classic ASP. Does any body have an example I could use? Thanks
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.