473,575 Members | 3,390 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

GetRequestStrea m slows down?

I have a multithreaded application that is posting XML requests to a remote
web site. As more and more calls are made to
HttpWebRequest. GetRequestStrea m, the length of time increases for each call
from an initial 2-3 seconds to anywhere from 15-30 seconds. What am I doing
that is causing this method call's duration to increase and how do I make it
run faster? Below is some pseudo-code for the method that is called in a
multi-threaded fashion...

- Jim

private void MultithreadedMe thod()
{
string url = ConfigurationMa nager.AppSettin gs["providerUr l"];

request = (HttpWebRequest )WebRequest.Cre ate(url);
request.Method = "POST";
request.Content Type = "text/xml";

startCount = Environment.Tic kCount;
using (Stream s = request.GetRequ estStream())
{
elapsedCount = Environment.Tic kCount - startCount;
//the elapsed time increases with each call
EventLog.WriteE ntry("request.G etRequestStream () = " + elapsedCount,
EventLogEntryTy pe.Information) ;

using (StreamWriter writer = new StreamWriter(s) )
{
writer.WriteLin e(requestXml);
writer.Flush();
writer.Close();
}

s.Close();
}
request.Timeout = requestTimeoutM illiseconds;
response = request.GetResp onse();

//handle the response here
}

Jan 19 '06 #1
8 6313
Are these calls concurrent or consecutive. In 1.1 these requests were
made on the thread pool, so you might be running into some concurrency
issues.

Additionally, depending on the size of the page returned, you might also
be running in to GC issues -- so for example, if the content returned is
over 85k (assuming it being put into one big string variable at some
point) that memory has the potential of being put into the large object
heap which is only swept and not compacted .... so you might be running
into some memory issues.

Also, the s.Close is redundant and repetitive ... it's getting called in
IDispose after you exit from your using block.

Also, make sure that you are closing the response object that you get
back from request.GetResp onse(); .... not doing so can lead to some
unmitigated nastiness

John

Jim Toth wrote:
I have a multithreaded application that is posting XML requests to a remote
web site. As more and more calls are made to
HttpWebRequest. GetRequestStrea m, the length of time increases for each call
from an initial 2-3 seconds to anywhere from 15-30 seconds. What am I doing
that is causing this method call's duration to increase and how do I make it
run faster? Below is some pseudo-code for the method that is called in a
multi-threaded fashion...

- Jim

private void MultithreadedMe thod()
{
string url = ConfigurationMa nager.AppSettin gs["providerUr l"];

request = (HttpWebRequest )WebRequest.Cre ate(url);
request.Method = "POST";
request.Content Type = "text/xml";

startCount = Environment.Tic kCount;
using (Stream s = request.GetRequ estStream())
{
elapsedCount = Environment.Tic kCount - startCount;
//the elapsed time increases with each call
EventLog.WriteE ntry("request.G etRequestStream () = " + elapsedCount,
EventLogEntryTy pe.Information) ;

using (StreamWriter writer = new StreamWriter(s) )
{
writer.WriteLin e(requestXml);
writer.Flush();
writer.Close();
}

s.Close();
}
request.Timeout = requestTimeoutM illiseconds;
response = request.GetResp onse();

//handle the response here
}

Jan 19 '06 #2
Aside from the other comments, the title of your method "MultithreadedM ethod"
is confusing, as it isn't multithreaded at all the way it is written.
If you really have a lot of this going on, you probably want to look at the
asynchronous BeginXXX / EndXXX overloads which use the Threadpool "under the
hood" - or use a custom threadpool.
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com


"Jim Toth" wrote:
I have a multithreaded application that is posting XML requests to a remote
web site. As more and more calls are made to
HttpWebRequest. GetRequestStrea m, the length of time increases for each call
from an initial 2-3 seconds to anywhere from 15-30 seconds. What am I doing
that is causing this method call's duration to increase and how do I make it
run faster? Below is some pseudo-code for the method that is called in a
multi-threaded fashion...

- Jim

private void MultithreadedMe thod()
{
string url = ConfigurationMa nager.AppSettin gs["providerUr l"];

request = (HttpWebRequest )WebRequest.Cre ate(url);
request.Method = "POST";
request.Content Type = "text/xml";

startCount = Environment.Tic kCount;
using (Stream s = request.GetRequ estStream())
{
elapsedCount = Environment.Tic kCount - startCount;
//the elapsed time increases with each call
EventLog.WriteE ntry("request.G etRequestStream () = " + elapsedCount,
EventLogEntryTy pe.Information) ;

using (StreamWriter writer = new StreamWriter(s) )
{
writer.WriteLin e(requestXml);
writer.Flush();
writer.Close();
}

s.Close();
}
request.Timeout = requestTimeoutM illiseconds;
response = request.GetResp onse();

//handle the response here
}

Jan 20 '06 #3
What I meant by "MultithreadedM ethod" is that the method is called many times
on many different threads.

I've seen the asynchronous calls for GetRequestStrea m... will they get me
around potential contention issues mentioned?

"Peter Bromberg [C# MVP]" wrote:
Aside from the other comments, the title of your method "MultithreadedM ethod"
is confusing, as it isn't multithreaded at all the way it is written.
If you really have a lot of this going on, you probably want to look at the
asynchronous BeginXXX / EndXXX overloads which use the Threadpool "under the
hood" - or use a custom threadpool.
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com


"Jim Toth" wrote:
I have a multithreaded application that is posting XML requests to a remote
web site. As more and more calls are made to
HttpWebRequest. GetRequestStrea m, the length of time increases for each call
from an initial 2-3 seconds to anywhere from 15-30 seconds. What am I doing
that is causing this method call's duration to increase and how do I make it
run faster? Below is some pseudo-code for the method that is called in a
multi-threaded fashion...

- Jim

private void MultithreadedMe thod()
{
string url = ConfigurationMa nager.AppSettin gs["providerUr l"];

request = (HttpWebRequest )WebRequest.Cre ate(url);
request.Method = "POST";
request.Content Type = "text/xml";

startCount = Environment.Tic kCount;
using (Stream s = request.GetRequ estStream())
{
elapsedCount = Environment.Tic kCount - startCount;
//the elapsed time increases with each call
EventLog.WriteE ntry("request.G etRequestStream () = " + elapsedCount,
EventLogEntryTy pe.Information) ;

using (StreamWriter writer = new StreamWriter(s) )
{
writer.WriteLin e(requestXml);
writer.Flush();
writer.Close();
}

s.Close();
}
request.Timeout = requestTimeoutM illiseconds;
response = request.GetResp onse();

//handle the response here
}

Jan 20 '06 #4
Correct, which is a misnomer. If it is called many different times, you
should probably be using the Threadpool and the asynchronous
IAsyncResult result = request.BeginGe tResponse(
new AsyncCallback(U pdateItem),stat e);

semantics. This will speed up your operation by an order of magnitude since
it offers parallelism (multiple requests in operation at the same time).
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com


"Jim Toth" wrote:
What I meant by "MultithreadedM ethod" is that the method is called many times
on many different threads.

I've seen the asynchronous calls for GetRequestStrea m... will they get me
around potential contention issues mentioned?

"Peter Bromberg [C# MVP]" wrote:
Aside from the other comments, the title of your method "MultithreadedM ethod"
is confusing, as it isn't multithreaded at all the way it is written.
If you really have a lot of this going on, you probably want to look at the
asynchronous BeginXXX / EndXXX overloads which use the Threadpool "under the
hood" - or use a custom threadpool.
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com


"Jim Toth" wrote:
I have a multithreaded application that is posting XML requests to a remote
web site. As more and more calls are made to
HttpWebRequest. GetRequestStrea m, the length of time increases for each call
from an initial 2-3 seconds to anywhere from 15-30 seconds. What am I doing
that is causing this method call's duration to increase and how do I make it
run faster? Below is some pseudo-code for the method that is called in a
multi-threaded fashion...

- Jim

private void MultithreadedMe thod()
{
string url = ConfigurationMa nager.AppSettin gs["providerUr l"];

request = (HttpWebRequest )WebRequest.Cre ate(url);
request.Method = "POST";
request.Content Type = "text/xml";

startCount = Environment.Tic kCount;
using (Stream s = request.GetRequ estStream())
{
elapsedCount = Environment.Tic kCount - startCount;
//the elapsed time increases with each call
EventLog.WriteE ntry("request.G etRequestStream () = " + elapsedCount,
EventLogEntryTy pe.Information) ;

using (StreamWriter writer = new StreamWriter(s) )
{
writer.WriteLin e(requestXml);
writer.Flush();
writer.Close();
}

s.Close();
}
request.Timeout = requestTimeoutM illiseconds;
response = request.GetResp onse();

//handle the response here
}

Jan 20 '06 #5
If you are using the 1.1 or earlier version of the Framework, never make
calls on WebClient or WebRequest from within the threadpool -- as the
calls on those objects are also implemented using the threadpool (even
if you are calling them synchronously) ...... the result is a very high
potential for a deadlock.
Peter Bromberg [C# MVP] wrote:
Correct, which is a misnomer. If it is called many different times, you
should probably be using the Threadpool and the asynchronous
IAsyncResult result = request.BeginGe tResponse(
new AsyncCallback(U pdateItem),stat e);

semantics. This will speed up your operation by an order of magnitude since
it offers parallelism (multiple requests in operation at the same time).
Peter

Jan 20 '06 #6
Perhaps I should have stated my suggestion more clearly. What I meant was,
use the asynchronous BeginGetRespons e etc. method, which use the .NET
threadpool.
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com


"John Murray" wrote:
If you are using the 1.1 or earlier version of the Framework, never make
calls on WebClient or WebRequest from within the threadpool -- as the
calls on those objects are also implemented using the threadpool (even
if you are calling them synchronously) ...... the result is a very high
potential for a deadlock.
Peter Bromberg [C# MVP] wrote:
Correct, which is a misnomer. If it is called many different times, you
should probably be using the Threadpool and the asynchronous
IAsyncResult result = request.BeginGe tResponse(
new AsyncCallback(U pdateItem),stat e);

semantics. This will speed up your operation by an order of magnitude since
it offers parallelism (multiple requests in operation at the same time).
Peter

Jan 20 '06 #7
It looks like I'm running into some issues with
ServicePointMan ager.DefaultCon nectionLimit, which is defaulted to 2, because
I'm sending data to the same web site URL over and over. I tried bumping
this up to 10, 15, and 20 and had to bump up the number of threads in the
ThreadPool as a result, however I'm seeing only marginal gains in
performance. Is there a configuration setting, either for .NET or Windows
Server 2003, that I should be looking at, or am I missing something else?

"Peter Bromberg [C# MVP]" wrote:
Correct, which is a misnomer. If it is called many different times, you
should probably be using the Threadpool and the asynchronous
IAsyncResult result = request.BeginGe tResponse(
new AsyncCallback(U pdateItem),stat e);

semantics. This will speed up your operation by an order of magnitude since
it offers parallelism (multiple requests in operation at the same time).
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com


"Jim Toth" wrote:
What I meant by "MultithreadedM ethod" is that the method is called many times
on many different threads.

I've seen the asynchronous calls for GetRequestStrea m... will they get me
around potential contention issues mentioned?

"Peter Bromberg [C# MVP]" wrote:
Aside from the other comments, the title of your method "MultithreadedM ethod"
is confusing, as it isn't multithreaded at all the way it is written.
If you really have a lot of this going on, you probably want to look at the
asynchronous BeginXXX / EndXXX overloads which use the Threadpool "under the
hood" - or use a custom threadpool.
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com


"Jim Toth" wrote:

> I have a multithreaded application that is posting XML requests to a remote
> web site. As more and more calls are made to
> HttpWebRequest. GetRequestStrea m, the length of time increases for each call
> from an initial 2-3 seconds to anywhere from 15-30 seconds. What am I doing
> that is causing this method call's duration to increase and how do I make it
> run faster? Below is some pseudo-code for the method that is called in a
> multi-threaded fashion...
>
> - Jim
>
> private void MultithreadedMe thod()
> {
> string url = ConfigurationMa nager.AppSettin gs["providerUr l"];
>
> request = (HttpWebRequest )WebRequest.Cre ate(url);
> request.Method = "POST";
> request.Content Type = "text/xml";
>
> startCount = Environment.Tic kCount;
> using (Stream s = request.GetRequ estStream())
> {
> elapsedCount = Environment.Tic kCount - startCount;
> //the elapsed time increases with each call
> EventLog.WriteE ntry("request.G etRequestStream () = " + elapsedCount,
> EventLogEntryTy pe.Information) ;
>
> using (StreamWriter writer = new StreamWriter(s) )
> {
> writer.WriteLin e(requestXml);
> writer.Flush();
> writer.Close();
> }
>
> s.Close();
> }
>
>
> request.Timeout = requestTimeoutM illiseconds;
> response = request.GetResp onse();
>
> //handle the response here
> }
>

Jan 20 '06 #8
make sure you close the Response object along with any streams /
streamreaders each time. Aside from that, I've seen a lot of old newsgroups
threads on upping the DefaultConnecti onLimit with mixed results. It could be
at the server too, you know.
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com


"Jim Toth" wrote:
I have a multithreaded application that is posting XML requests to a remote
web site. As more and more calls are made to
HttpWebRequest. GetRequestStrea m, the length of time increases for each call
from an initial 2-3 seconds to anywhere from 15-30 seconds. What am I doing
that is causing this method call's duration to increase and how do I make it
run faster? Below is some pseudo-code for the method that is called in a
multi-threaded fashion...

- Jim

private void MultithreadedMe thod()
{
string url = ConfigurationMa nager.AppSettin gs["providerUr l"];

request = (HttpWebRequest )WebRequest.Cre ate(url);
request.Method = "POST";
request.Content Type = "text/xml";

startCount = Environment.Tic kCount;
using (Stream s = request.GetRequ estStream())
{
elapsedCount = Environment.Tic kCount - startCount;
//the elapsed time increases with each call
EventLog.WriteE ntry("request.G etRequestStream () = " + elapsedCount,
EventLogEntryTy pe.Information) ;

using (StreamWriter writer = new StreamWriter(s) )
{
writer.WriteLin e(requestXml);
writer.Flush();
writer.Close();
}

s.Close();
}
request.Timeout = requestTimeoutM illiseconds;
response = request.GetResp onse();

//handle the response here
}

Jan 20 '06 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
2554
by: Narine | last post by:
Hi All, We're running SQL Server 2000, SP3. I have a stored procedure that consists of a single Select statement. It selects a bunch of columns one of which is a column of data type TEXT. SP takes 30 sec to run which causes timeouts on the Front End. When I comment out the Text column from the select it only takes 1 Sec.
0
1058
by: AM | last post by:
Hi, I use Xorg's XWin in rootless mode on cygwin on XP Pro for quite a bit of my work. Unfortunately it seems to slow down SQL Query Analyzer's UI terribly. If I exit out of X and/or kill it, the UI reverts back to normal ie responds as it normally would. The slowdown usually happens if I stop using Query ANalyzer for sometime and then try...
5
5007
by: Gerald Khin | last post by:
I encountered a performance problem with UPDATE statement and foreign key constraints. There are two tables involved: A parent table A and a child table B: create table A( ID CHAR(15) NOT NULL CONSTRAINT APK PRIMARY KEY, val NUMERIC(10)) create table B(ID CHAR(15) NOT NULL CONSTRAINT BPK PRIMARY KEY,
7
1636
by: Piet Van Vlierberghe | last post by:
We are a software firm developing in C++ and we recently made the step from Microsoft Visual Studio 6 to Microsoft Visual Studio .NET 2003. We are really excited about the quality of the compiler; however there seem to be some issues with the debugger or the combination of our code with the debugger that makes debugging very slow and...
9
1759
by: cow3 | last post by:
Is there anyone who can help me with this: I have written a fairly complicated vb application of hydrological model that does a lot of number crunching. The model creates a set of object variables before entering the main loop and then main calculations are done within the main loop using the object variables previously created. The...
14
18682
by: Paul_Madden via DotNetMonster.com | last post by:
Basically I have a listbox to which I add simple STRING items- I have a progress bar which I increment whenever I populate another portion of the complete set of items I wish to add. What I observe is that as more and more are added, population of the list box takes longer and longer. ie the first 10th of the item set are added much much...
1
1331
by: Jose Walker | last post by:
Hi there, I've an ASP.NET 2 web application that I want to publish, but it happened that the application slows down in performance compared with the version managed by VS.NET 2005. When the application is being developed and tested inside VS.NET 2005 is really fast; once deployed by the publishing option slows down and time to time...
5
1461
by: liquid106 | last post by:
Hi I'm using the following code in an Excel 2003 (Windows XP) Macro to populate formulas down the rows of a spreadsheet a certain number of times (variable - numContacts). Everything works fine to begin with as the macro adds rows very rapidly, but after about 3000 lines, it starts to slow down. nunContacts is around the 22,000 figure so with...
0
959
by: kimiraikkonen | last post by:
Hi, I noticed that if i view some jpeg files via Picturebox control (.net 2.0) placed in to a single form, while dragging the form around the screen, then the form slutters, dragging slows down and jitters occur. With same jpeg files other 3rd party picture viewers works very well when compared to a .net 2.0 picturebox control. (original...
0
7845
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
7775
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
8281
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
1
7868
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
6515
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5664
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
5338
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3778
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
1
1382
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.