473,659 Members | 2,920 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

httpwebrequest, cookies, session

Hi Everybody,

I have a problem that is driving me crazy and I am hoping for some help
here.

I am building an application (vb.net, vs2003) that crawls selected websites.
The application works fine except for one thing: Some sites rely on session
cookies and I don't seem to be returning the cookies.

The IIS5 log looks like this when I access the same page with

IE:
12:13:15 192.168.0.2 CLIENT GET /pm4/aboutus.asp - 200 71
Mozilla/4.0+(compatible ;+MSIE+6.0;+Win dows+NT+5.1;+SV 1;+.NET+CLR+1.0 .3705;+.NET+CLR +1.1.4322;+.NET +CLR+2.0.40607)
signinfrom=;+AS PSESSIONIDCAABT ABS=EMFHLDECGHE EOCJBFMLLMDFD;+ ASPSESSIONIDACA CQBBS=NMONBFECJ PICHEGFGACFEIJG
-

My Application:
11:40:42 192.168.0.2 CLIENT GET /pm4/aboutus.asp - 200 40
Mozilla/4.0+(compatible ;+MSIE+6.0;+Win dows+NT+5.1;+Q3 12461;+.NET+CLR +1.0.3705)
- -

As you see the session cookies are missing.

The relevant part of my code looks like this:

Dim req As HttpWebRequest = CType(WebReques t.Create(url), HttpWebRequest)
req.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461;
..NET CLR 1.0.3705)"
req.CookieConta iner = New CookieContainer
req.KeepAlive = True
req.Headers.Set ("Pragma", "no-cache")
req.Timeout = 30000
req.ContentType = "applicatio n/x-www-form-urlencoded"
Dim ReceiveStream As Stream
Dim encode As Encoding
Dim sr As StreamReader
result = req.GetResponse ()
ReceiveStream = result.GetRespo nseStream()
encode = System.Text.Enc oding.GetEncodi ng("utf-8")
sr = New StreamReader(Re ceiveStream, encode)
etc...

If I access a .aspx page it confirms that my application supports cookies,
javascript, etc. It basically accepts that I emulate an IE.
But the cookie thing doesn't work.

What I am doing wrong?

Yours,
Nov 21 '05 #1
0 1333

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
3406
by: TJO | last post by:
Can someone at MS please reply to this. I am trying to post data so a web form via ssl with the following code. I keep getting this error: "The underlying connection was closed: Could not establish secure channel for SSL/TLS" private void mainHttpCalls(string postData) { HttpWebRequest objRequest1 ; HttpWebRequest objRequest2 ;
2
4810
by: Keith Patrick | last post by:
I'm trying to programmatically post data to another page within my ASP.Net app. Not POSTing is not an option (I can't store this data in my session, context, hidden fields, or anything else...I've exhausted all my other options, so I have to get this scenario working). Basically, if I POST the data normally, it works fine, except the target page has a new session ID, so I lost the session data that I *do* transfer around. However, if I...
11
15867
by: Keith Patrick | last post by:
Could someone explain to me the relationship between these two classes? I am ripping my hair out trying to divert an HttpRequest to a new location via an HttpWebRequest, but I cannot get my session xfer to work, possibly due to the cookies not being compatible. I've spent over a week trying to get the HWR to integrate nicely with my app, but I cannot get the session to transfer, and it'd probably take me even longer if I tried to digest...
1
2777
by: davvel | last post by:
It has been 6 days of re-writing the same code in different ways to try and avoid a getResponse Timeout which haunted me for much too long now. I am trying to do a very simple thing which is prooving the most difficult in my career. Scrape a screen which has several session variables within it, and send this page to an e-mail. I have two pages in my application webform1.aspx and webform2.aspx, in webform1.aspx I set several session...
4
10040
by: Andy Fish | last post by:
Hi, I have a web app that I want to make calls back into itself (preserving the session cookies etc). I am using a System.Net.HttpWebRequest object to make the request, but I am having trouble getting the cookies into it. In the current (i.e. incoming) request, the cookies are stored as System.Web.HttpCookie objects in a System.Web.HttpCookieCollection. However,
16
11043
by: Cheung, Jeffrey Jing-Yen | last post by:
I have a windows form application that generates a request, downloads an image, and waits the user to enter in login info. Unfortunately, this image is dynamic and based on session data. I have read documents on the CookieCollection property of HttpWebRequest. Currently, I have the functionality in my code to be able to accept cookies, and return them upon a new HttpWebRequest; however, upon further inspection of the returning...
0
1509
by: Alex Papadimoulis | last post by:
Hey Group, I'm in the process of converting an ASP-based site to an ASP.NET site and built a control that wraps around an ASP page. The control simply does a GET to the same server to render the ASP content, and then just writes it to the page. In development and testing, there were no problems. Once deployed to production, after the site runs for a few hours, it starts to slow down and throw countless "Operation Has Timed Out"...
0
2032
by: Alex Papadimoulis | last post by:
Hey Group, I'm in the process of converting an ASP-based site to an ASP.NET site and built a control that wraps around an ASP page. The control simply does a GET to the same server to render the ASP content, and then just writes it to the page. In development and testing, there were no problems. Once deployed to production, after the site runs for a few hours, it starts to slow down and throw countless "Operation Has Timed Out"...
3
3122
by: yoni | last post by:
Hi, I am using HttpWebRequest to get the content of a web page. the call to this web page is creating a session on the server. then, I wish to get the content of a 2nd web page on the same site, but this 2nd web page reads the session created on the 1st page. so when i call it with HttpWebRequest, it lost the session. anybody has a tip or a code snippet that shows me how do i make the server retain the session between two HttpWebRequest...
10
29970
by: rlueneberg | last post by:
I am trying to foward the old sessionID using "Session.SessionID" to an HttpWebRequest CookieContainer so that I can capture the requested page session variables but it is not working as it is supposed to. The HttpResponse object always returns a different sessionID from the old one which I am trying to force. Why is objRequest not carrying over the old SessionID? private String ReadHtmlPage(string url) { String result = string.Empty;
0
8851
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
1
8528
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8627
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7356
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6179
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5649
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4175
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4335
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
2752
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.