473,548 Members | 2,744 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

webcrawle with LWP: how to make redirection work?

S
Hi,

I am making a webcrawler and it needs to go through the login on the
front page. I use LWP:UserAgent and HTML::Form to basically post the
username and password to the site. Once past the login, I should be
redirected to a new page if I use a browser. However, in my Perl
script, the response I got is "HTTP/1.1 302 (Found) Object moved". I
think that means my code works for the login but fails to have a
sucessful redirection? How to actually go to the desired redirected
page? Thank you.
Jul 19 '05 #1
1 2220
S <S@w.com> wrote in news:pa******** *************** ****@w.com:
Hi,

I am making a webcrawler and it needs to go through the login on the
front page. I use LWP:UserAgent and HTML::Form to basically post the
username and password to the site. Once past the login, I should be
redirected to a new page if I use a browser. However, in my Perl
script, the response I got is "HTTP/1.1 302 (Found) Object moved". I
think that means my code works for the login but fails to have a
sucessful redirection? How to actually go to the desired redirected
page? Thank you.


Hello, Quakey
Jul 19 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
12636
by: Woogie | last post by:
When running the sample code below without a proxy the GET returns the expected data. When run with the $proxy uncommented the GET returns the content of the login page for the site being accessed. The site in the code is valid for ease of testing. I also am including the LWP debug info for each attempt. Can anyone explain this behavior...
1
6801
by: Hal Vaughan | last post by:
I'm trying to access a site with data that needs to be paged through, one page at a time. It won't allow back buttons and you have to use the menu links to get through. In other words, it is run through CGI (ASP, I think) and must be able to keep track of sessions. I have a simple program I've written using Perl and LWP. I tried a totally...
0
5561
by: mdh | last post by:
I have an application written for mod_perl and Apache that needs to be able to send some XML queries to a WebMethods server for access to corporate systems. I have been attempting to use LWP to POST my XML query to the target WebMethods server and find I only get 401s (authorization errors) from the WebMethods server when it is configured...
0
2632
by: Travis | last post by:
This is an example of LWP that I found on one of the perl docs pages. use LWP 5.64; this line is causing an external error. Does that mean I don't have LWP on my server or is something else wrong? Let me know if this works for you. ############# use strict; use warnings; use LWP 5.64;
3
12012
by: suchindra | last post by:
The following is my very simple script : </script> #!/usr/bin/perl use strict; use LWP::UserAgent; my $url = "<My Web Server IP : PORT Number>"; my $browser = LWP::UserAgent->new(keep_alive => 5, timeout => 10, agent => 'unimportant',);
0
2541
by: sharif | last post by:
Anyone could help me out for n=my code ......I have written following code ,Here i m able to get and post the form successfuly..but after posting im not gettng proper response content... #!usr/bin/perl -w use strict; use LWP; use LWP::Simple; use LWP::UserAgent; use HTML::Form; use Switch; use HTTP::Cookies;
3
2107
by: pplers | last post by:
it's an example. I want a script to print the real url behind a lix.in one so i tried these: use LWP::UserAgent; use LWP::UserAgent; my $ua = new LWP::UserAgent; my $response = $ua->post('http://lix.in/53fe67',{ tiny => '53fe67', submit => 'continue',}); my $con = $response->content;
4
4224
by: kkamath | last post by:
Hi, I use LWP::UserAgent to automate windows login process. I set the credentials using $ua->credentials("host:port","realm","id","password"); but it fetches only the current page requested. It fails for subsequent requests and pops up the windows form again to supply the credentials. I have created a cookie jar too but this doesnt help as the...
3
3324
by: cketcham | last post by:
All, I have the following code: use 5.010; use LWP::UserAgent; use URI::URL; use LWP::Debug qw(+); my $browser = LWP::UserAgent->new();
0
7711
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
0
7954
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
1
7467
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
7805
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
0
6039
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5367
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
3478
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1932
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1054
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.