I'm trying to access a site with data that needs to be paged through, one
page at a time. It won't allow back buttons and you have to use the menu
links to get through. In other words, it is run through CGI (ASP, I think)
and must be able to keep track of sessions.
I have a simple program I've written using Perl and LWP. I tried a totally
innocuous site (TVLand.com) that I found had cookies. When I read in 1
page from this site, I got a cookie and it showed up in lwpcookies.txt.
Whenever I try this other site (which requires a password and account, so I
can't list it here, according to registration agreement -- and no, it isn't
pr0n!), I find that there is a line in the header to set a cookie in both
web pages. The only difference is that the one where the cookie isn't
storied doesn't list a domain name in the cookie line. Here's the header
lines with the cookie info (1st is TVLand, 2nd is private site):
Set-Cookie: JSESSIONID=M1LHYKX2DVXRMCQBAFML3UQ; domain=.tvland.com; path=/
Set-Cookie: JSESSIONID=0000CSPVV3Q5TXU2BUPIIDEWOCY:ulnfn1uq;Pa th=/
my program (listed below) prints out that I have a cookie. (Whenever I try
the site with Galeon, I can read a cookie for the site, too), but no cookie
shows up in lwpcookie.txt (the cookie file) at all.
Am I doing something wrong? I need to be sure that the cookie is persistant
in my program, but it never shows up in the cookie file. Is there a reason
for that, or am I doing something wrong? What can I do to make sure the
cookie from the 2nd site is stored for later -- and also read back when
needed by other pages?
Thanks!
Hal
--------------------------------------------------------------------
Program listing:
use LWP::UserAgent;
use HTTP::Cookies;
our $domain = "tvland.com";
our $locmenu = "schedule";
our $ua = LWP::UserAgent->new;
$ua->agent("Mozilla/4.0");
$ua->agent("MSIE/6.0");
$ua->cookie_jar(HTTP::Cookies->new(file =>"lwpcookies.txt", autosave =>
1));
$url = "HTTP://".$domain."/".$locmenu;
print "Url: $url\n";
$req = HTTP::Request->new(GET => $url);
$req->content_type("application/x-www-form-urlencoded");
$req->header('Accept' => 'text/html');
$res = $ua->request($req);
print "Cookie: ".$res->status_line."\n";
$page = $res->as_string;
# print "Page: $page\n"; 1 6778
Hal Vaughan wrote: I'm trying to access a site with data that needs to be paged through, one page at a time. It won't allow back buttons and you have to use the menu links to get through. In other words, it is run through CGI (ASP, I think) and must be able to keep track of sessions.
I have a simple program I've written using Perl and LWP. I tried a totally innocuous site (TVLand.com) that I found had cookies. When I read in 1 page from this site, I got a cookie and it showed up in lwpcookies.txt. Whenever I try this other site (which requires a password and account, so I can't list it here, according to registration agreement -- and no, it isn't pr0n!), I find that there is a line in the header to set a cookie in both web pages. The only difference is that the one where the cookie isn't storied doesn't list a domain name in the cookie line. Here's the header lines with the cookie info (1st is TVLand, 2nd is private site):
Set-Cookie: JSESSIONID=M1LHYKX2DVXRMCQBAFML3UQ; domain=.tvland.com; path=/ Set-Cookie: JSESSIONID=0000CSPVV3Q5TXU2BUPIIDEWOCY:ulnfn1uq;Pa th=/
my program (listed below) prints out that I have a cookie. (Whenever I try the site with Galeon, I can read a cookie for the site, too), but no cookie shows up in lwpcookie.txt (the cookie file) at all.
Am I doing something wrong? I need to be sure that the cookie is persistant in my program, but it never shows up in the cookie file. Is there a reason for that, or am I doing something wrong? What can I do to make sure the cookie from the 2nd site is stored for later -- and also read back when needed by other pages?
Thanks!
Hal -------------------------------------------------------------------- Program listing:
use LWP::UserAgent; use HTTP::Cookies;
our $domain = "tvland.com"; our $locmenu = "schedule"; our $ua = LWP::UserAgent->new; $ua->agent("Mozilla/4.0"); $ua->agent("MSIE/6.0"); $ua->cookie_jar(HTTP::Cookies->new(file =>"lwpcookies.txt", autosave => 1)); $url = "HTTP://".$domain."/".$locmenu; print "Url: $url\n"; $req = HTTP::Request->new(GET => $url); $req->content_type("application/x-www-form-urlencoded"); $req->header('Accept' => 'text/html'); $res = $ua->request($req); print "Cookie: ".$res->status_line."\n"; $page = $res->as_string; # print "Page: $page\n";
$ua = LWP::UserAgent->new( cookie_jar =>HTTP::Cookies->new( file =>
'/tmp/cookies.txt', autosave => 1, ignore_discard => 1 ));
try ignore_discard => 1 This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Steve Mauldin |
last post by:
This is a standard ASP application that has several pages at the root
withthe global.asa. I set a session variable session("accountid") =
"123456" within an asp page and then response.redirect to...
|
by: Rajan Vijayaraghavan |
last post by:
Hi,
I have a question with the JavaScript and CGI. This is the
scenario.
Say, a user is coming to page 'X'. The page 'X' is completely
generated by a cgi bin program. The cgi-bin program is in C....
|
by: Mike |
last post by:
I got the code below from an earlier post but I can't get it to work
(I get an error on the "for (i=0; i<a.length; i++)" line) Anyone have
code that works for cookies with keys?
> Anyone...
|
by: Beryl Small |
last post by:
Hi, I have a third party software that needs to send
information to an .aspx page for processing to communicate
with an SQL database. The software sends the information
something like this:
...
|
by: rss |
last post by:
SUMMARY:
==========
I am unable to pass along a simple Cookie obtained from a HttpWebRequest
call (Machine A's ASP.NET app) so that another Web Server (Machine B)
recongnizes the cookie.
I...
|
by: Prasad Dannani |
last post by:
Hi,
I want to know the browser settings like Image Enabled, Cookie Enabled
etc.,
For Cookies and Script Request object has some properties but after
disabling certain features also we are...
|
by: Vlad Dogaru |
last post by:
Hello,
I am trying to use cookies and Python to create a simple login example.
But I am very disoriented at the existence of two cookie libraries,
namely Cookie and cookielib. I have seen...
|
by: jpgonzalezgarcia |
last post by:
hi. i am trying to make a script that will extract info from a
protected
page on ebay, which you can view only by logging in, which requires
SSL 443.
1. get the url
2. get the login screen
3....
|
by: Zeba |
last post by:
Hi !
I want to know the cookie value that was set in the previous page.
Initially in my page in project B I used
HttpCookie cookie = Request.Cookies.Get(somefile.COOKIE_NAME);
where...
|
by: DolphinDB |
last post by:
Tired of spending countless mintues downsampling your data? Look no further!
In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
|
by: ryjfgjl |
last post by:
ExcelToDatabase: batch import excel into database automatically...
|
by: isladogs |
last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM).
In this month's session, we are pleased to welcome back...
|
by: isladogs |
last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM).
In this month's session, we are pleased to welcome back...
|
by: Vimpel783 |
last post by:
Hello!
Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
|
by: CloudSolutions |
last post by:
Introduction:
For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
|
by: Defcon1945 |
last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
|
by: Shællîpôpï 09 |
last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome former...
| |