473,287 Members | 1,643 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,287 software developers and data experts.

HTTP Post and HttpWebRequest object

We have implemented a 'HTTP Post' client in C# that posts Xml documents to a
webserver. This is working fine if the post rate is one post for every 20
seconds. But if the post rate is increased to one post for every 10 seconds
the client start getting error 403 'forbidden' from the webserver after a
short period of time. The webserver is IIS. The choking of the client/server
communication when doing high frequency posting is due to that we create a
new HttpWebRequest object for each post. We have tried to reuse the
HttpWebRequest object but with no success. Only if using 'HTTP Get' are we
able to resue the HttpWebRequest object and able to 'Get' one document per
second. But also for 'Http Get' we observe the same behavior with the
client/server communication choking when creating a new HttpWebRequest object
for each 'HTTP Get'. I.e. the 'HTTP Get' behavior led us to belive that it is
the recreation of a HttpWebRequest object for each request/post that is the
problem.

We need a way to reuse the HttpWebRequest object for 'HTTP Post' or a more
light weight creation of the HttpWebRequest objects so that the client is
able to post one document per seconds for an extensive period of time.

Is this a know problem?

What is the best practice to do high frequency HTTP Post's using C# and IIS?

regards, /jansenh
May 28 '06 #1
3 4870
I think you should see IIS Log file first, this will give you insight into
exact problem. But I can advise you few thingsL
This kind of error sometime occurs when you do HTTP Protocol Violation. This
can be due to the incorrect or incomplete UserAgent header. Try this line

request.UserAgent="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET
CLR 1.1.4322)"

Moreover HTTP protocol violation can be ignored, if you specify
UseUnsafeHeaderParsing=true in the configuration file.
HttpWebRequest.UseUnsafeHeaderParsing property is internal and read-only and
can't be changed for particular instance of HttpWebRequest.

If you feel that my post help you please provide feedback for me.
"JansenH" wrote:
We have implemented a 'HTTP Post' client in C# that posts Xml documents to a
webserver. This is working fine if the post rate is one post for every 20
seconds. But if the post rate is increased to one post for every 10 seconds
the client start getting error 403 'forbidden' from the webserver after a
short period of time. The webserver is IIS. The choking of the client/server
communication when doing high frequency posting is due to that we create a
new HttpWebRequest object for each post. We have tried to reuse the
HttpWebRequest object but with no success. Only if using 'HTTP Get' are we
able to resue the HttpWebRequest object and able to 'Get' one document per
second. But also for 'Http Get' we observe the same behavior with the
client/server communication choking when creating a new HttpWebRequest object
for each 'HTTP Get'. I.e. the 'HTTP Get' behavior led us to belive that it is
the recreation of a HttpWebRequest object for each request/post that is the
problem.

We need a way to reuse the HttpWebRequest object for 'HTTP Post' or a more
light weight creation of the HttpWebRequest objects so that the client is
able to post one document per seconds for an extensive period of time.

Is this a know problem?

What is the best practice to do high frequency HTTP Post's using C# and IIS?

regards, /jansenh

May 28 '06 #2
Thus wrote jansenh,
We have implemented a 'HTTP Post' client in C# that posts Xml
documents to a webserver. This is working fine if the post rate is one
post for every 20 seconds. But if the post rate is increased to one
post for every 10 seconds the client start getting error 403
'forbidden' from the webserver after a short period of time. The
webserver is IIS. The choking of the client/server communication when
doing high frequency posting is due to that we create a new
HttpWebRequest object for each post. We have tried to reuse the
HttpWebRequest object but with no success. Only if using 'HTTP Get'
are we able to resue the HttpWebRequest object and able to 'Get' one
document per second. But also for 'Http Get' we observe the same
behavior with the client/server communication choking when creating a
new HttpWebRequest object for each 'HTTP Get'. I.e. the 'HTTP Get'
behavior led us to belive that it is the recreation of a
HttpWebRequest object for each request/post that is the problem.

We need a way to reuse the HttpWebRequest object for 'HTTP Post' or a
more light weight creation of the HttpWebRequest objects so that the
client is able to post one document per seconds for an extensive
period of time.


I don't see why the number of actual HttpWebRequest instances has anything
to do with a server-side problem. Simply create one instance per request
-- that's the underlying model. Can you post your code?

Cheers,
--
Joerg Jooss
ne********@joergjooss.de
May 31 '06 #3
Thus wrote Altaf,
I think you should see IIS Log file first, this will give you insight
into
exact problem. But I can advise you few thingsL
This kind of error sometime occurs when you do HTTP Protocol
Violation. This
can be due to the incorrect or incomplete UserAgent header. Try this
line
request.UserAgent="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0;
.NET CLR 1.1.4322)"
Not providing a User-Agent header isn't a protocol violation. But it's true
that many web applications rely on this header, and blissfully crash if it's
not there ;-)
Moreover HTTP protocol violation can be ignored, if you specify
UseUnsafeHeaderParsing=true in the configuration file.
HttpWebRequest.UseUnsafeHeaderParsing property is internal and
read-only and can't be changed for particular instance of
HttpWebRequest.


If there was a protocol violation, the OP should receive a WebException.

Cheers,
--
Joerg Jooss
ne********@joergjooss.de
May 31 '06 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: TJO | last post by:
Can someone at MS please reply to this. I am trying to post data so a web form via ssl with the following code. I keep getting this error: "The underlying connection was closed: Could not...
3
by: john | last post by:
is there a .net object that lets me use the http verbs like get, put, post?
3
by: ME | last post by:
Hi; I am getting "Unhandled Exception: System.Net.WebException: The remote server returned an erro r: (401) Unauthorized." when I am trying to get a page via post. Code follows...
1
by: Juan Dent | last post by:
Hi, The Request.SaveAs method stores in a file the complete HTTP request including the HttpVersion. However when I try to reproduce in memory the same thing, I cannot find the "HTTP/1.1"...
1
by: Dave Brown | last post by:
I am attempting to post to a url (https://FakeURL/logon.asp) using the HttpWebRequest class. The response for a succesful post will contain the html for the logon user's default page. We've...
0
by: WIWA | last post by:
Hi, I want to login to a password protected website and fetch the content of the page behind. I have based my code on http://weblogs.asp.net/jdennany/archive/2005/04/23/403971.aspx. When I use...
3
by: Patrick Fogarty | last post by:
I am programming what is to be a web service client that will use an HTTP-POST to request and retrieve data. The remote server (written in java for what it's worth) requires basic authentication...
2
by: =?Utf-8?B?UGF1bA==?= | last post by:
I am using ASP.Net 2.0 and VB.Net (although C#is ok also). I want to create an object/method/function that will take a URL as an input parameter and then return all of the HTML in that page. I...
1
by: ERobishaw | last post by:
Using Winform app writing a HTTP Post using the following... on the server side, I get no form fields. Request.Form.AllKeys.Length = 0. If I put the field/value paris in the URL I can use...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.