473,219 Members | 2,052 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,219 software developers and data experts.

Not getting cookies in LWP

I'm trying to access a site with data that needs to be paged through, one
page at a time. It won't allow back buttons and you have to use the menu
links to get through. In other words, it is run through CGI (ASP, I think)
and must be able to keep track of sessions.

I have a simple program I've written using Perl and LWP. I tried a totally
innocuous site (TVLand.com) that I found had cookies. When I read in 1
page from this site, I got a cookie and it showed up in lwpcookies.txt.
Whenever I try this other site (which requires a password and account, so I
can't list it here, according to registration agreement -- and no, it isn't
pr0n!), I find that there is a line in the header to set a cookie in both
web pages. The only difference is that the one where the cookie isn't
storied doesn't list a domain name in the cookie line. Here's the header
lines with the cookie info (1st is TVLand, 2nd is private site):

Set-Cookie: JSESSIONID=M1LHYKX2DVXRMCQBAFML3UQ; domain=.tvland.com; path=/
Set-Cookie: JSESSIONID=0000CSPVV3Q5TXU2BUPIIDEWOCY:ulnfn1uq;Pa th=/

my program (listed below) prints out that I have a cookie. (Whenever I try
the site with Galeon, I can read a cookie for the site, too), but no cookie
shows up in lwpcookie.txt (the cookie file) at all.

Am I doing something wrong? I need to be sure that the cookie is persistant
in my program, but it never shows up in the cookie file. Is there a reason
for that, or am I doing something wrong? What can I do to make sure the
cookie from the 2nd site is stored for later -- and also read back when
needed by other pages?

Thanks!

Hal
--------------------------------------------------------------------
Program listing:

use LWP::UserAgent;
use HTTP::Cookies;

our $domain = "tvland.com";
our $locmenu = "schedule";
our $ua = LWP::UserAgent->new;
$ua->agent("Mozilla/4.0");
$ua->agent("MSIE/6.0");
$ua->cookie_jar(HTTP::Cookies->new(file =>"lwpcookies.txt", autosave =>
1));
$url = "HTTP://".$domain."/".$locmenu;
print "Url: $url\n";
$req = HTTP::Request->new(GET => $url);
$req->content_type("application/x-www-form-urlencoded");
$req->header('Accept' => 'text/html');
$res = $ua->request($req);
print "Cookie: ".$res->status_line."\n";
$page = $res->as_string;
# print "Page: $page\n";
Jul 19 '05 #1
1 6770
Hal Vaughan wrote:
I'm trying to access a site with data that needs to be paged through, one
page at a time. It won't allow back buttons and you have to use the menu
links to get through. In other words, it is run through CGI (ASP, I think)
and must be able to keep track of sessions.

I have a simple program I've written using Perl and LWP. I tried a totally
innocuous site (TVLand.com) that I found had cookies. When I read in 1
page from this site, I got a cookie and it showed up in lwpcookies.txt.
Whenever I try this other site (which requires a password and account, so I
can't list it here, according to registration agreement -- and no, it isn't
pr0n!), I find that there is a line in the header to set a cookie in both
web pages. The only difference is that the one where the cookie isn't
storied doesn't list a domain name in the cookie line. Here's the header
lines with the cookie info (1st is TVLand, 2nd is private site):

Set-Cookie: JSESSIONID=M1LHYKX2DVXRMCQBAFML3UQ; domain=.tvland.com; path=/
Set-Cookie: JSESSIONID=0000CSPVV3Q5TXU2BUPIIDEWOCY:ulnfn1uq;Pa th=/

my program (listed below) prints out that I have a cookie. (Whenever I try
the site with Galeon, I can read a cookie for the site, too), but no cookie
shows up in lwpcookie.txt (the cookie file) at all.

Am I doing something wrong? I need to be sure that the cookie is persistant
in my program, but it never shows up in the cookie file. Is there a reason
for that, or am I doing something wrong? What can I do to make sure the
cookie from the 2nd site is stored for later -- and also read back when
needed by other pages?

Thanks!

Hal
--------------------------------------------------------------------
Program listing:

use LWP::UserAgent;
use HTTP::Cookies;

our $domain = "tvland.com";
our $locmenu = "schedule";
our $ua = LWP::UserAgent->new;
$ua->agent("Mozilla/4.0");
$ua->agent("MSIE/6.0");
$ua->cookie_jar(HTTP::Cookies->new(file =>"lwpcookies.txt", autosave =>
1));
$url = "HTTP://".$domain."/".$locmenu;
print "Url: $url\n";
$req = HTTP::Request->new(GET => $url);
$req->content_type("application/x-www-form-urlencoded");
$req->header('Accept' => 'text/html');
$res = $ua->request($req);
print "Cookie: ".$res->status_line."\n";
$page = $res->as_string;
# print "Page: $page\n";


$ua = LWP::UserAgent->new( cookie_jar =>HTTP::Cookies->new( file =>
'/tmp/cookies.txt', autosave => 1, ignore_discard => 1 ));

try ignore_discard => 1
Jul 19 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Steve Mauldin | last post by:
This is a standard ASP application that has several pages at the root withthe global.asa. I set a session variable session("accountid") = "123456" within an asp page and then response.redirect to...
1
by: Rajan Vijayaraghavan | last post by:
Hi, I have a question with the JavaScript and CGI. This is the scenario. Say, a user is coming to page 'X'. The page 'X' is completely generated by a cgi bin program. The cgi-bin program is in C....
1
by: Mike | last post by:
I got the code below from an earlier post but I can't get it to work (I get an error on the "for (i=0; i<a.length; i++)" line) Anyone have code that works for cookies with keys? > Anyone...
3
by: Beryl Small | last post by:
Hi, I have a third party software that needs to send information to an .aspx page for processing to communicate with an SQL database. The software sends the information something like this: ...
3
by: rss | last post by:
SUMMARY: ========== I am unable to pass along a simple Cookie obtained from a HttpWebRequest call (Machine A's ASP.NET app) so that another Web Server (Machine B) recongnizes the cookie. I...
2
by: Prasad Dannani | last post by:
Hi, I want to know the browser settings like Image Enabled, Cookie Enabled etc., For Cookies and Script Request object has some properties but after disabling certain features also we are...
4
by: Vlad Dogaru | last post by:
Hello, I am trying to use cookies and Python to create a simple login example. But I am very disoriented at the existence of two cookie libraries, namely Cookie and cookielib. I have seen...
2
by: jpgonzalezgarcia | last post by:
hi. i am trying to make a script that will extract info from a protected page on ebay, which you can view only by logging in, which requires SSL 443. 1. get the url 2. get the login screen 3....
5
by: Zeba | last post by:
Hi ! I want to know the cookie value that was set in the previous page. Initially in my page in project B I used HttpCookie cookie = Request.Cookies.Get(somefile.COOKIE_NAME); where...
0
by: veera ravala | last post by:
ServiceNow is a powerful cloud-based platform that offers a wide range of services to help organizations manage their workflows, operations, and IT services more efficiently. At its core, ServiceNow...
0
by: VivesProcSPL | last post by:
Obviously, one of the original purposes of SQL is to make data query processing easy. The language uses many English-like terms and syntax in an effort to make it easy to learn, particularly for...
3
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 3 Jan 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). For other local times, please check World Time Buddy In...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: mar23 | last post by:
Here's the situation. I have a form called frmDiceInventory with subform called subfrmDice. The subform's control source is linked to a query called qryDiceInventory. I've been trying to pick up the...
2
by: jimatqsi | last post by:
The boss wants the word "CONFIDENTIAL" overlaying certain reports. He wants it large, slanted across the page, on every page, very light gray, outlined letters, not block letters. I thought Word Art...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.