By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
455,852 Members | 1,426 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 455,852 IT Pros & Developers. It's quick & easy.

Managing multiple HTTP requests on same session

P: 12
Hi,

I am trying to write a script that simulate a nevigation on a particular web site.
Before I am getting the page that contains the information I need, I have to go through a series of preceding pages.

The page flow is as follow: login page -> page1 -> page2 ..etc.

The problem is that once I am doing the login to the webside and requesting for the next page, the web server doesn't recongnize me anymore. The response I get is that the login has expired. Therefore I am not getting the actual html on this page and I cannot move forward to the next page in the flow.

I guess that the web server doesn't recognize two subsequent requests as coming from the same session.

I tried to use a few packages of perl: LWP:UserAgent, POE:Component:Client:HTTP. Unfortunately, I couldn't solve that.

Do you have any idea ?

Thanks,
Ido.
Feb 21 '08 #1
Share this Question
Share on Google+
2 Replies


KevinADC
Expert 2.5K+
P: 4,059
I am pretty sure WWW::Mechanize can do this as long as there is no javascript invloved. Read the documentation and read any FAQS or Cookbooks that are also posted/linked to on CPAN. I can not help with WWW::Mechanize usage as I have no experience with the module.
Feb 21 '08 #2

Kelicula
Expert 100+
P: 176
Are you saving the cookies, that the remote site is sending you, and sending them back?
More than likely the remote site has sent you a cookie after the "login" page, which you're script will have to save and send back to the server, on all future request, just like a regular browser.

Here is an example server/client comm. with cookies.


Expand|Select|Wrap|Line Numbers
  1. client:
  2. GET www.somepage.com HTTP/1.1
  3. Host: localhost
  4. Accept: image/gif, image/x-bitmap, image/jpg, */*
  5. Connection: Keep-Alive
  6.  
  7. Server:
  8. HTTP/1.1 200 OK
  9. Date: Sat, 18 Mar 2008 20:35:35 GMT
  10. Server: Apache/1.3.9 (Unix)
  11. Set-Cookie: user_id=123456; path=/; expires=Sat, 18-Mar-05 19:06:19 GMT
  12.  
  13.  
  14. Then on all future request, client should echo:
  15. GET /somepage.html HTTP1.1
  16. Host: localhost
  17. Accept: image/gif, image/jpeg, */*
  18. Connection: Keep-Alive
  19. Cookie: user_id=123456
  20.  
  21.  
  22.  
Feb 22 '08 #3

Post your reply

Sign in to post your reply or Sign up for a free account.