469,632 Members | 1,666 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,632 developers. It's quick & easy.

Not getting cookies in LWP

I'm trying to access a site with data that needs to be paged through, one
page at a time. It won't allow back buttons and you have to use the menu
links to get through. In other words, it is run through CGI (ASP, I think)
and must be able to keep track of sessions.

I have a simple program I've written using Perl and LWP. I tried a totally
innocuous site (TVLand.com) that I found had cookies. When I read in 1
page from this site, I got a cookie and it showed up in lwpcookies.txt.
Whenever I try this other site (which requires a password and account, so I
can't list it here, according to registration agreement -- and no, it isn't
pr0n!), I find that there is a line in the header to set a cookie in both
web pages. The only difference is that the one where the cookie isn't
storied doesn't list a domain name in the cookie line. Here's the header
lines with the cookie info (1st is TVLand, 2nd is private site):

Set-Cookie: JSESSIONID=M1LHYKX2DVXRMCQBAFML3UQ; domain=.tvland.com; path=/
Set-Cookie: JSESSIONID=0000CSPVV3Q5TXU2BUPIIDEWOCY:ulnfn1uq;Pa th=/

my program (listed below) prints out that I have a cookie. (Whenever I try
the site with Galeon, I can read a cookie for the site, too), but no cookie
shows up in lwpcookie.txt (the cookie file) at all.

Am I doing something wrong? I need to be sure that the cookie is persistant
in my program, but it never shows up in the cookie file. Is there a reason
for that, or am I doing something wrong? What can I do to make sure the
cookie from the 2nd site is stored for later -- and also read back when
needed by other pages?

Thanks!

Hal
--------------------------------------------------------------------
Program listing:

use LWP::UserAgent;
use HTTP::Cookies;

our $domain = "tvland.com";
our $locmenu = "schedule";
our $ua = LWP::UserAgent->new;
$ua->agent("Mozilla/4.0");
$ua->agent("MSIE/6.0");
$ua->cookie_jar(HTTP::Cookies->new(file =>"lwpcookies.txt", autosave =>
1));
$url = "HTTP://".$domain."/".$locmenu;
print "Url: $url\n";
$req = HTTP::Request->new(GET => $url);
$req->content_type("application/x-www-form-urlencoded");
$req->header('Accept' => 'text/html');
$res = $ua->request($req);
print "Cookie: ".$res->status_line."\n";
$page = $res->as_string;
# print "Page: $page\n";
Jul 19 '05 #1
1 6579
Hal Vaughan wrote:
I'm trying to access a site with data that needs to be paged through, one
page at a time. It won't allow back buttons and you have to use the menu
links to get through. In other words, it is run through CGI (ASP, I think)
and must be able to keep track of sessions.

I have a simple program I've written using Perl and LWP. I tried a totally
innocuous site (TVLand.com) that I found had cookies. When I read in 1
page from this site, I got a cookie and it showed up in lwpcookies.txt.
Whenever I try this other site (which requires a password and account, so I
can't list it here, according to registration agreement -- and no, it isn't
pr0n!), I find that there is a line in the header to set a cookie in both
web pages. The only difference is that the one where the cookie isn't
storied doesn't list a domain name in the cookie line. Here's the header
lines with the cookie info (1st is TVLand, 2nd is private site):

Set-Cookie: JSESSIONID=M1LHYKX2DVXRMCQBAFML3UQ; domain=.tvland.com; path=/
Set-Cookie: JSESSIONID=0000CSPVV3Q5TXU2BUPIIDEWOCY:ulnfn1uq;Pa th=/

my program (listed below) prints out that I have a cookie. (Whenever I try
the site with Galeon, I can read a cookie for the site, too), but no cookie
shows up in lwpcookie.txt (the cookie file) at all.

Am I doing something wrong? I need to be sure that the cookie is persistant
in my program, but it never shows up in the cookie file. Is there a reason
for that, or am I doing something wrong? What can I do to make sure the
cookie from the 2nd site is stored for later -- and also read back when
needed by other pages?

Thanks!

Hal
--------------------------------------------------------------------
Program listing:

use LWP::UserAgent;
use HTTP::Cookies;

our $domain = "tvland.com";
our $locmenu = "schedule";
our $ua = LWP::UserAgent->new;
$ua->agent("Mozilla/4.0");
$ua->agent("MSIE/6.0");
$ua->cookie_jar(HTTP::Cookies->new(file =>"lwpcookies.txt", autosave =>
1));
$url = "HTTP://".$domain."/".$locmenu;
print "Url: $url\n";
$req = HTTP::Request->new(GET => $url);
$req->content_type("application/x-www-form-urlencoded");
$req->header('Accept' => 'text/html');
$res = $ua->request($req);
print "Cookie: ".$res->status_line."\n";
$page = $res->as_string;
# print "Page: $page\n";


$ua = LWP::UserAgent->new( cookie_jar =>HTTP::Cookies->new( file =>
'/tmp/cookies.txt', autosave => 1, ignore_discard => 1 ));

try ignore_discard => 1
Jul 19 '05 #2

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

1 post views Thread by Rajan Vijayaraghavan | last post: by
3 posts views Thread by Beryl Small | last post: by
2 posts views Thread by Prasad Dannani | last post: by
4 posts views Thread by Vlad Dogaru | last post: by
2 posts views Thread by jpgonzalezgarcia | last post: by
5 posts views Thread by Zeba | last post: by
reply views Thread by gheharukoh7 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.