HttpWebRequest & HttpWebResponse Cookies

[Cheung, Jeffrey Jing-Yen]

I have a windows form application that generates a request, downloads an image, and waits the user
to enter in login info. Unfortunately, this image is dynamic and based on session data. I have
read documents on the CookieCollection property of HttpWebRequest. Currently, I have the
functionality in my code to be able to accept cookies, and return them upon a new HttpWebRequest;
however, upon further inspection of the returning HttpWebResponse.Cookies property, I notice that
the cookies that are supposed to be there regarding my session data are not returned (one such
cookie named, JSESSIONID). What other methods are there available to me to retrieve cookies that
would contain session data?

Here is a code snippet (please excuse the wordwrap)

Dim strURL As String = "https://www.thisurlisatest.com/"
Dim objHTTPRequest As HttpWebRequest = CType(WebRequest.Create(strURL), HttpWebRequest)
objHTTPRequest.CookieContainer = New CookieContainer()
Dim objHTTPResponse As HttpWebResponse = CType(objHTTPRequest.GetResponse(), HttpWebResponse)
Dim objReceiveStream As Stream = objHTTPResponse.GetResponseStream()
' I successfully retrieve the cookies here
m_Cookies = objHTTPResponse.Cookies
Dim objCookie As Cookie
For Each objCookie In m_Cookies
Console.WriteLine(objCookie.Name + "=" + objCookie.Value)
Next

' This only outputs this: language_pref=en_US
' There should be more cookies that contain session id data

Thanks in advance for any light you can shed on this matter,

Jeff Cheung

[Cor]

Hi Cheung,

Do you have multiple webservers that you use the cookieCollection?

Otherwise you can have a look at the normal HttpRequest.cookie class

It is much easier to handle (if you dont forget to set the live time)

I hope this helps

Cor

[Cheung, Jeffrey Jing-Yen]

Cor,

I'm not passing the cookie collection to multiple webservers; however, this is a windows form
application. How do you propose that I get the System.Web.HttpRequest object in my form application?

I did find the solution to my problem; however, it brought upon a bigger issue for me unfortunately.
I ended up looking at the Set-Cookie item in the Headers property in the HttpWebResponse object,
and adding that value to the Cookie item in the Headers property in the HttpWebRequest object. This
works flawlessly as long as I do not pass a CookieCollection in the HttpWebRequest.

According to this link:

http://authors.aspalliance.com/damia...x.vb.view.aspx

if I populate the CookieCollection, this will override the Cookie header in the request object.

So my problem now is that I'm able to successfully login and retain my session, but any subsequent
page request returns nothing in the ResponseStream from HttpWebResponse.GetResponseStream().
Looking at the HttpWebResponse.StatusCode yields 200, but the content-length is always 0 whenever I
make any requests. Any ideas?

Thanks in advance,

Jeff Cheung

Cor wrote:

Hi Cheung,

Do you have multiple webservers that you use the cookieCollection?

Otherwise you can have a look at the normal HttpRequest.cookie class

It is much easier to handle (if you dont forget to set the live time)

I hope this helps

Cor

[Cor]

Hi Cheung,

I never used that cookiecontainer because the other ones are in my opinion
more simple.

But my problem with cookies is always that I forget to set the livetime of a
cookie, it disapears immidiatly.

I did not see that in your code, maybe it is your problem also, so before I
start checking things that I never did before, can you check that yourself?

Cor

[Cheung, Jeffrey Jing-Yen]

Cor,

Thanks again for your response. According to the RFC, the Set-Cookie header in the response is
responsible for setting the "live time" of the cookie. I say this with the assumption that you are
talking about the expires attribute for the cookie. The server should be responsible for setting
the expiry date of the cookie; however, I did set it to expire a month explicitly when I read your
message just to test it out. Unfortunately, I still get 0 bytes returned... But I know my session
is still intact because the response doesn't return any new Set-Cookie headers. If my session had
reset, then there would be a new Set-Cookie header containing the Session ID.

Regards,

Jeff Cheung

Cor wrote:

Hi Cheung,

I never used that cookiecontainer because the other ones are in my opinion
more simple.

But my problem with cookies is always that I forget to set the livetime of a
cookie, it disapears immidiatly.

I did not see that in your code, maybe it is your problem also, so before I
start checking things that I never did before, can you check that yourself?

Cor

[Cor]

Hi Jeff,

I become confused. I can not see how a windowform application can set a
cookie.

The cookie is for the connection from the webform and the user to check that
the right user answers (one of the reasons for a cookie).

Can you explain something more, because I think I am on the wrong track.

Cor

[Cor]

Hi Jeff,

I become confused. I can not see how a windowform application can set a
cookie.

The cookie is for the connection from the webform and the user to check that
the right user answers (one of the reasons for a cookie).

Can you explain something more, because I think I am on the wrong track.

Cor

[Cheung, Jeffrey Jing-Yen]

Cor,

Sure thing... My windows form application has HttpWebRequest and HttpWebResponse objects. You are
correct, since it's not a webform, I have no need to use the Response.Cookies method. The
differences are:

ASP.NET application
- You are writing the cookies to the client's browser

My application
- I am writing the cookies to the HttpWebRequest object, since the HttpWebRequest is going to be
making the calls to the webserver.

This goes back to my original question. The documentation in MSDN directs me to use the
CookieCollection in order to maintain any cookies in my subsequent requests since the HttpWebRequest
is designed to only make one request to a URI. In essence, my application is more of a web browser
with the exception that I do not have to render any of the html that I parse. My goal is to
impersonate a user logging in to a predefined site, and automating administrative tasks. Let me
know if this clears things up.

Regards,

Jeff Cheung

Cor wrote:

Hi Jeff,

I become confused. I can not see how a windowform application can set a
cookie.

The cookie is for the connection from the webform and the user to check that
the right user answers (one of the reasons for a cookie).

Can you explain something more, because I think I am on the wrong track.

Cor

[Cheung, Jeffrey Jing-Yen]

Cor,

Sure thing... My windows form application has HttpWebRequest and HttpWebResponse objects. You are
correct, since it's not a webform, I have no need to use the Response.Cookies method. The
differences are:

ASP.NET application
- You are writing the cookies to the client's browser

My application
- I am writing the cookies to the HttpWebRequest object, since the HttpWebRequest is going to be
making the calls to the webserver.

This goes back to my original question. The documentation in MSDN directs me to use the
CookieCollection in order to maintain any cookies in my subsequent requests since the HttpWebRequest
is designed to only make one request to a URI. In essence, my application is more of a web browser
with the exception that I do not have to render any of the html that I parse. My goal is to
impersonate a user logging in to a predefined site, and automating administrative tasks. Let me
know if this clears things up.

Regards,

Jeff Cheung

Cor wrote:

Hi Jeff,

I become confused. I can not see how a windowform application can set a
cookie.

The cookie is for the connection from the webform and the user to check that
the right user answers (one of the reasons for a cookie).

Can you explain something more, because I think I am on the wrong track.

Cor

[Cor]

Hi Jeff,

It soundsvery difficult, but when your program it is a kind of webbrowser,
it needs the cookies from the client.

As far as I know handles the classes you use the cookies from a client from
the side of the webserver.

But you can of course try to read the cookies from the client direct, I did
try that also but stopped, because I could not find an enumerating for the
cookie container and because that it can be on every computer (and specialy
every Microsoft OS) on another place I stopped with it.

But as far as I remember me the cookie is not that difficult file to read, I
thought just a kind of textfile.

Cor

Sure thing... My windows form application has HttpWebRequest and HttpWebResponse objects. You are correct, since it's not a webform, I have no need to use the Response.Cookies method. The differences are:

ASP.NET application
- You are writing the cookies to the client's browser

My application
- I am writing the cookies to the HttpWebRequest object, since the HttpWebRequest is going to be making the calls to the webserver.

This goes back to my original question. The documentation in MSDN directs me to use the CookieCollection in order to maintain any cookies in my subsequent requests since the HttpWebRequest is designed to only make one request to a URI. In essence, my application is more of a web browser with the exception that I do not have to render any of the html that I parse. My goal is to impersonate a user logging in to a predefined site, and automating administrative tasks. Let me know if this clears things up.

[Cheung, Jeffrey Jing-Yen]

Cor,

Thanks again for your response. I don't think that you fully understand my problem. I think that I
have the cookies functionality licked. I suspect the site that I am trying to connect to has
multiple redirects, so my code isn't accounting for those since the AllowAutoRedirect property is
set to true on my HttpWebRequest. I'll research this a little more because I just simply see more
cookies being written to my browser than in my application.

When you say that I can read the cookies from the client directly, you misunderstand my scenario.
It's not as simple as accessing the Request.Cookies collection. The "client" in this case is not
the user using the program. The "client" is the HttpWebRequest object. This class represents what
a normal browser would do - make web requests.

As far as the CookieContainer property, this is the "container" on the HttpWebRequest object. Think
of the HttpWebRequest object as a pseudo browser itself; it has its own method of storing cookies.
Since the CookieContainer property is just a collection of the object type "Coookie", enumerating
this property is very easy; however, my specific problem (you can try it out yourself in a Windows
Form application, you learn a lot about the HTTP protocol) is that the Cookies property on the
HttpWebResponse object doesn't return any cookies for the session data. They're just simply not
there. Because of this, I've had to resort to writing headers explicitly.

Regards,

Jeff Cheung
Cor wrote:

Hi Jeff,

It soundsvery difficult, but when your program it is a kind of webbrowser,
it needs the cookies from the client.

As far as I know handles the classes you use the cookies from a client from
the side of the webserver.

But you can of course try to read the cookies from the client direct, I did
try that also but stopped, because I could not find an enumerating for the
cookie container and because that it can be on every computer (and specialy
every Microsoft OS) on another place I stopped with it.

But as far as I remember me the cookie is not that difficult file to read, I
thought just a kind of textfile.

Cor

Sure thing... My windows form application has HttpWebRequest and

HttpWebResponse objects. You are

correct, since it's not a webform, I have no need to use the

Response.Cookies method. The

differences are:

ASP.NET application
- You are writing the cookies to the client's browser

My application
- I am writing the cookies to the HttpWebRequest object, since the

HttpWebRequest is going to be

making the calls to the webserver.

This goes back to my original question. The documentation in MSDN directs

me to use the

CookieCollection in order to maintain any cookies in my subsequent

requests since the HttpWebRequest

is designed to only make one request to a URI. In essence, my application

is more of a web browser

with the exception that I do not have to render any of the html that I

parse. My goal is to

impersonate a user logging in to a predefined site, and automating

administrative tasks. Let me

know if this clears things up.

[Tony Archer]

Could you please post the code you are using to accomplish this? I'm trying
to do something very similar.
"Eria Tlov" <ha********@vgpagagon.uk> wrote in message
news:e3**************@TK2MSFTNGP10.phx.gbl...

Well, I got something to work.
I create an HttpWebRequest with a cookie container, copy the session ID out to a string, and then write that string into the headers of subsequent
request. By subsequent request, I mean creation of new HttpWebRequest. I
still haven't figured out if you can navigate with one request. It seems to navigate with autoredirects to a page away from the RequestURI, but not
manually.
thanks,
Eria

[Tony Archer]

Could you please post the code you are using to accomplish this? I'm trying
to do something very similar.
"Eria Tlov" <ha********@vgpagagon.uk> wrote in message
news:e3**************@TK2MSFTNGP10.phx.gbl...

Well, I got something to work.
I create an HttpWebRequest with a cookie container, copy the session ID out to a string, and then write that string into the headers of subsequent
request. By subsequent request, I mean creation of new HttpWebRequest. I
still haven't figured out if you can navigate with one request. It seems to navigate with autoredirects to a page away from the RequestURI, but not
manually.
thanks,
Eria

[Eria Tlov]

Dim myReq As HttpWebRequest = WebRequest.Create(myURL)

myReq.CookieContainer = New CookieContainer()

Dim myResponse As HttpWebResponse = myReq.GetResponse

myResponse.Cookies = myReq.CookieContainer.GetCookies(myReq.RequestUri)

myAspSession = myResponse.Cookies(0).ToString

'Then for subsequent requests

Dim myReq As HttpWebRequest = WebRequest.Create(myNextURL)

myReq.Headers.Add("Cookie", myAspSession)

-Eria

"Tony Archer" <ta*****@nortelnetworks.com> wrote in message
news:OW*************@TK2MSFTNGP12.phx.gbl...

Could you please post the code you are using to accomplish this? I'm trying to do something very similar.
"Eria Tlov" <ha********@vgpagagon.uk> wrote in message
news:e3**************@TK2MSFTNGP10.phx.gbl...

Well, I got something to work.
I create an HttpWebRequest with a cookie container, copy the session ID out

to a string, and then write that string into the headers of subsequent
request. By subsequent request, I mean creation of new HttpWebRequest. I still haven't figured out if you can navigate with one request. It

seems to

navigate with autoredirects to a page away from the RequestURI, but not
manually.
thanks,
Eria

[Eria Tlov]

Dim myReq As HttpWebRequest = WebRequest.Create(myURL)

myReq.CookieContainer = New CookieContainer()

Dim myResponse As HttpWebResponse = myReq.GetResponse

myResponse.Cookies = myReq.CookieContainer.GetCookies(myReq.RequestUri)

myAspSession = myResponse.Cookies(0).ToString

'Then for subsequent requests

Dim myReq As HttpWebRequest = WebRequest.Create(myNextURL)

myReq.Headers.Add("Cookie", myAspSession)

-Eria

"Tony Archer" <ta*****@nortelnetworks.com> wrote in message
news:OW*************@TK2MSFTNGP12.phx.gbl...

Could you please post the code you are using to accomplish this? I'm trying to do something very similar.
"Eria Tlov" <ha********@vgpagagon.uk> wrote in message
news:e3**************@TK2MSFTNGP10.phx.gbl...

Well, I got something to work.
I create an HttpWebRequest with a cookie container, copy the session ID out

to a string, and then write that string into the headers of subsequent
request. By subsequent request, I mean creation of new HttpWebRequest. I still haven't figured out if you can navigate with one request. It

seems to

navigate with autoredirects to a page away from the RequestURI, but not
manually.
thanks,
Eria

[Tony Archer]

Could you please post the code you are using to accomplish this? I'm trying
to do something very similar.
"Eria Tlov" <ha********@vgpagagon.uk> wrote in message
news:e3**************@TK2MSFTNGP10.phx.gbl...

Well, I got something to work.
I create an HttpWebRequest with a cookie container, copy the session ID out to a string, and then write that string into the headers of subsequent
request. By subsequent request, I mean creation of new HttpWebRequest. I
still haven't figured out if you can navigate with one request. It seems to navigate with autoredirects to a page away from the RequestURI, but not
manually.
thanks,
Eria

[Eria Tlov]

Dim myReq As HttpWebRequest = WebRequest.Create(myURL)

myReq.CookieContainer = New CookieContainer()

Dim myResponse As HttpWebResponse = myReq.GetResponse

myResponse.Cookies = myReq.CookieContainer.GetCookies(myReq.RequestUri)

myAspSession = myResponse.Cookies(0).ToString

'Then for subsequent requests

Dim myReq As HttpWebRequest = WebRequest.Create(myNextURL)

myReq.Headers.Add("Cookie", myAspSession)

-Eria

"Tony Archer" <ta*****@nortelnetworks.com> wrote in message
news:OW*************@TK2MSFTNGP12.phx.gbl...

Could you please post the code you are using to accomplish this? I'm trying to do something very similar.
"Eria Tlov" <ha********@vgpagagon.uk> wrote in message
news:e3**************@TK2MSFTNGP10.phx.gbl...

Well, I got something to work.
I create an HttpWebRequest with a cookie container, copy the session ID out

to a string, and then write that string into the headers of subsequent
request. By subsequent request, I mean creation of new HttpWebRequest. I still haven't figured out if you can navigate with one request. It

seems to

navigate with autoredirects to a page away from the RequestURI, but not
manually.
thanks,
Eria