473,323 Members | 1,570 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,323 software developers and data experts.

Logging into and parsing a website using Perl

Hi,

I'm trying to create a perl script that will log into a website (the login
form uses POST), navigate to several pages, and append the (html) content parsed
from those pages to a seperate log file. I'm not very familiar with this aspect
of perl, and have been having some trouble in the POSTing of the form data,
while using cookies to log in.

Visting the site automatically redirects you to a login page. Once you fill
out the login form and click the submit button, you are redirected to the main
site index. The login form uses cookies to establish identity.

I've searched through several google resources, and have built upon what
I've read. I *believe* I am now storing the cookies I come across when loading
the page (using a cookie jar), however, this doesn't seem to be allowing me to
log on to the secure areas of the site. I suspect this is because I am not
properly POSTing the login form data, or otherwise not 'following' the redirect
to the secure area of the site.

At this point in time, I am just trying to get my script to login to the
page (using my appropriate credentials), and then display the site index.

If someone could please offer me some insight and direction into using the
appropriate modules, or else point out any flaws in my code (included below).

------START CODE------

#!/usr/bin/perl -w

use LWP::Simple;
use LWP::UserAgent;

use HTML::TokeParser;
use HTML::Parser;

use HTTP::Request::Common;
use HTTP::Cookies;

use POSIX;

#----Variables----#
$t_url='http://www.memberplushq.com/pe/register/include/processlogin.jsp';
# This is where the log in form is located - once logged in, you are
redirected to the secured content (below).
$s_url='http://www.memberplushq.com/pe/index.jsp';
# This is where the secured content is located. If you aren't logged in, you
are redirected to the above.

$login='My_username';
$password='My Password';
$submit_value='Login';
#----/Variables----#

#----User Agent Config----#
$ua = LWP::UserAgent->new;
$ua->cookie_jar(HTTP::Cookies->new(file => "cookies.txt", autosave => 1,
ignore_discard => 1));
#----/User Agent Config----#

#----Really posting my buttons----#
$content = $ua->request(POST $t_url , [ login_name => $login , password =>
$password , loginSubmit => $submit_value ] );
$ua->request(POST $t_url , [ login_name => $login , password => $password ,
loginSubmit => $submit_value ] );
#----/Really pressing my buttons----#

#----Completing----#
print "$content";
#----Completing----#

------END CODE------

As you can tell, I am *trying* to get through to the secure site. However, this
is proving to be somewhat interesting.

I would appreciate any guidance you can offer,

AntWerp
Jul 19 '05 #1
0 11497

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Ravi Shankar | last post by:
Hi all, I have an enterprise application. I am using Apache Log4J for the logging purposes. WHen a request is received by the application, it goes throug servlets and many classes, and the...
1
by: Scott | last post by:
I am new to perl, and have not found any good examples of parsing to help me out. I have a text file that I am reading into an array that has to be parsed out and put into another file. I have not...
23
by: Rotem | last post by:
Hi, while working on something in my current project I have made several improvements to the logging package in Python, two of them are worth mentioning: 1. addition of a logging record field...
3
by: Chris Smith | last post by:
Hola, pythonisas: The documentation for the logging module is good, but a bit obscure. In particular, there seems to be a lot of action at a distance. The fact that getLogger() can actually be a...
8
by: Jean-Marie Vaneskahian | last post by:
Reading - Parsing Records From An LDAP LDIF File In .Net? I am in need of a .Net class that will allow for the parsing of a LDAP LDIF file. An LDIF file is the standard format for representing...
0
by: supern | last post by:
this is my perl script saved as login.pl #!c:/perl/bin/perl.exe $basedir="c:/program files/apache software foundation/apache2.2/cgi-bin"; $datafile="regstr.txt"; $name=$in{'login'};...
6
by: Jeff | last post by:
ASP.NET 2.0 In my webproject I want to implement some kind of functionality which logs the vistits to the website. I want it to log what pages the user requested, when he arrived at the website...
1
by: worlman385 | last post by:
I need to parse the following HTML page and extract TV listing data using VC++ http://tvlistings.zap2it.com/tvlistings/ZCGrid.do any good way to extract the data? is easy for VC++ to call...
4
by: alag20 | last post by:
Hi Guys, Sorry for duplicate posting as this Question refers to both c# and perl cgi script on the net, so please help. Here is the original post...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.