473,320 Members | 1,900 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Asynchronous WebRequests?

I've have several webrequests being called consecutively from an .aspx that
return XML from sources outside the company. When the page runs it can take
anywhere between 30-60 seconds.

I heard about making webrequests asynchronosly and found:

http://samples.gotdotnet.com/quickst.../GETAsync.aspx

I modified the code to add a couple requests right after each other as
listed below.

It worked but...

1.) What is the drawback of this technique in terms of performance in the
long run?
2.) Is there a way to determine when all of these webrequests complete and
just flush the response instead of waiting for the Thread.Sleep to complete?

HttpWebRequest wreq;
IAsyncResult r;

wreq = (HttpWebRequest) WebRequest.Create("http://somexmlsource1");
r = (IAsyncResult) wreq.BeginGetResponse(new
AsyncCallback(this.RespCallback), wreq);

wreq = (HttpWebRequest) WebRequest.Create("http://somexmlsource2");
r = (IAsyncResult) wreq.BeginGetResponse(new
AsyncCallback(this.RespCallback), wreq);

wreq = (HttpWebRequest) WebRequest.Create("http://somexmlsource3");
r = (IAsyncResult) wreq.BeginGetResponse(new
AsyncCallback(this.RespCallback), wreq);

Thread.Sleep(30000); <!--Wait 30 seconds to allow requests to complete
Nov 19 '05 #1
4 1472
You'll be using 4 threads instead of 1 thread to service a single user
request - that will limit your scalability. On the oher hand, when the
site isn't highly utilized you'll have a dramatic performance
increase.

I touch a bit on this subject in the following posts:
http://odetocode.com/Blogs/scott/arc...6/16/1656.aspx
http://odetocode.com/Blogs/scott/arc...6/23/1877.aspx

Notice don't need the Sleep call at all - you can simply have the
thread block on EndGetResponse calls. That is the simplest approach.

--
Scott
http://www.OdeToCode.com/blogs/scott/
On Thu, 23 Jun 2005 13:23:02 -0700, "Dave"
<Da**@discussions.microsoft.com> wrote:
I've have several webrequests being called consecutively from an .aspx that
return XML from sources outside the company. When the page runs it can take
anywhere between 30-60 seconds.

I heard about making webrequests asynchronosly and found:

http://samples.gotdotnet.com/quickst.../GETAsync.aspx

I modified the code to add a couple requests right after each other as
listed below.

It worked but...

1.) What is the drawback of this technique in terms of performance in the
long run?
2.) Is there a way to determine when all of these webrequests complete and
just flush the response instead of waiting for the Thread.Sleep to complete?

HttpWebRequest wreq;
IAsyncResult r;

wreq = (HttpWebRequest) WebRequest.Create("http://somexmlsource1");
r = (IAsyncResult) wreq.BeginGetResponse(new
AsyncCallback(this.RespCallback), wreq);

wreq = (HttpWebRequest) WebRequest.Create("http://somexmlsource2");
r = (IAsyncResult) wreq.BeginGetResponse(new
AsyncCallback(this.RespCallback), wreq);

wreq = (HttpWebRequest) WebRequest.Create("http://somexmlsource3");
r = (IAsyncResult) wreq.BeginGetResponse(new
AsyncCallback(this.RespCallback), wreq);

Thread.Sleep(30000); <!--Wait 30 seconds to allow requests to complete


Nov 19 '05 #2
If the client code (i.e WebRequest) is running on NT, then NT I/O completion
port mechanism is used. This will actually improve your performance, because
you are now simultaneously downloading 4 pages at once, instead of each page
sequentially. Also, the completions of the async I/O is being signalled
using I/O Completion ports, which is a very fast way of doing async I/O.

But there are some factors that could affect your performance.

First, if all the requests are going to the same server, and if the server
you are accessing is a HTTP/1.1 server, then there is a connection limit of
2. So, the first two requests will start downloading data, and the third one
will block until one of the first two requests completes. You can change
this behavior by increasing the connection limit using
ServicePoint/ServicePointManager.

Secondly, if the page which download the request is being hit multiple
times, then you might have multiple HTTP requests going on. If the page
being downloaded from the backend server is huge, or if the network
bandwidth is slow, then you might have a lot of outstanding http requests.
So, you might see a lot of handles being created, and some memory being used
in terms of Non Paged Pool.

Third, you will have to make sure that you close the connection once a
request is done. You can do this by calling Close() on the response stream,
or the HttpWebResponse object. If you dont do this, then the connection will
not be freed up, and you will block subsequent requests.

Finally, you dont need to do a THread.Sleep() for the requests to finish.
You can signal an Event from your callback, and wait for the event in your
main thread. THat is a much better way of achieving synchronization.

--
feroze

-----------------
This posting is provided as-is. It offers no warranties and assigns no
rights.

See http://weblogs.asp.net/feroze_daud for System.Net related posts.
----------------

"Scott Allen" <sc***@nospam.odetocode.com> wrote in message
news:i7********************************@4ax.com...
You'll be using 4 threads instead of 1 thread to service a single user
request - that will limit your scalability. On the oher hand, when the
site isn't highly utilized you'll have a dramatic performance
increase.

I touch a bit on this subject in the following posts:
http://odetocode.com/Blogs/scott/arc...6/16/1656.aspx
http://odetocode.com/Blogs/scott/arc...6/23/1877.aspx

Notice don't need the Sleep call at all - you can simply have the
thread block on EndGetResponse calls. That is the simplest approach.

--
Scott
http://www.OdeToCode.com/blogs/scott/
On Thu, 23 Jun 2005 13:23:02 -0700, "Dave"
<Da**@discussions.microsoft.com> wrote:
I've have several webrequests being called consecutively from an .aspx
that
return XML from sources outside the company. When the page runs it can
take
anywhere between 30-60 seconds.

I heard about making webrequests asynchronosly and found:

http://samples.gotdotnet.com/quickst.../GETAsync.aspx

I modified the code to add a couple requests right after each other as
listed below.

It worked but...

1.) What is the drawback of this technique in terms of performance in the
long run?
2.) Is there a way to determine when all of these webrequests complete and
just flush the response instead of waiting for the Thread.Sleep to
complete?

HttpWebRequest wreq;
IAsyncResult r;

wreq = (HttpWebRequest) WebRequest.Create("http://somexmlsource1");
r = (IAsyncResult) wreq.BeginGetResponse(new
AsyncCallback(this.RespCallback), wreq);

wreq = (HttpWebRequest) WebRequest.Create("http://somexmlsource2");
r = (IAsyncResult) wreq.BeginGetResponse(new
AsyncCallback(this.RespCallback), wreq);

wreq = (HttpWebRequest) WebRequest.Create("http://somexmlsource3");
r = (IAsyncResult) wreq.BeginGetResponse(new
AsyncCallback(this.RespCallback), wreq);

Thread.Sleep(30000); <!--Wait 30 seconds to allow requests to complete

Nov 19 '05 #3
Scott,

I'm stuck. I guess I'm not sure how to go about blocking the thread. I've
tried plugging the AsyncWaitHandles into an array and passing into the
WaitAll method ,as suggested to me from another post, to wait for all the
threads to complete. But it still doesn't wait for all the webrequests to
return.

How would the sample code have to change to do what you are talking about.
One thing I don't want is the Sleep() because if all the threads return
before the timer, I want to continue at that point.

Thanks, Dave.
"Scott Allen" wrote:
You'll be using 4 threads instead of 1 thread to service a single user
request - that will limit your scalability. On the oher hand, when the
site isn't highly utilized you'll have a dramatic performance
increase.

I touch a bit on this subject in the following posts:
http://odetocode.com/Blogs/scott/arc...6/16/1656.aspx
http://odetocode.com/Blogs/scott/arc...6/23/1877.aspx

Notice don't need the Sleep call at all - you can simply have the
thread block on EndGetResponse calls. That is the simplest approach.

--
Scott
http://www.OdeToCode.com/blogs/scott/
On Thu, 23 Jun 2005 13:23:02 -0700, "Dave"
<Da**@discussions.microsoft.com> wrote:
I've have several webrequests being called consecutively from an .aspx that
return XML from sources outside the company. When the page runs it can take
anywhere between 30-60 seconds.

I heard about making webrequests asynchronosly and found:

http://samples.gotdotnet.com/quickst.../GETAsync.aspx

I modified the code to add a couple requests right after each other as
listed below.

It worked but...

1.) What is the drawback of this technique in terms of performance in the
long run?
2.) Is there a way to determine when all of these webrequests complete and
just flush the response instead of waiting for the Thread.Sleep to complete?

HttpWebRequest wreq;
IAsyncResult r;

wreq = (HttpWebRequest) WebRequest.Create("http://somexmlsource1");
r = (IAsyncResult) wreq.BeginGetResponse(new
AsyncCallback(this.RespCallback), wreq);

wreq = (HttpWebRequest) WebRequest.Create("http://somexmlsource2");
r = (IAsyncResult) wreq.BeginGetResponse(new
AsyncCallback(this.RespCallback), wreq);

wreq = (HttpWebRequest) WebRequest.Create("http://somexmlsource3");
r = (IAsyncResult) wreq.BeginGetResponse(new
AsyncCallback(this.RespCallback), wreq);

Thread.Sleep(30000); <!--Wait 30 seconds to allow requests to complete


Nov 19 '05 #4
The WaitAll approach should work also (there are about 6 different
ways to solve this problem!). Could you post what your new code looks
like?

--
Scott
http://www.OdeToCode.com/blogs/scott/

On Tue, 28 Jun 2005 10:36:03 -0700, "Dave"
<Da**@discussions.microsoft.com> wrote:
Scott,

I'm stuck. I guess I'm not sure how to go about blocking the thread. I've
tried plugging the AsyncWaitHandles into an array and passing into the
WaitAll method ,as suggested to me from another post, to wait for all the
threads to complete. But it still doesn't wait for all the webrequests to
return.

How would the sample code have to change to do what you are talking about.
One thing I don't want is the Sleep() because if all the threads return
before the timer, I want to continue at that point.

Thanks, Dave.
"Scott Allen" wrote:
You'll be using 4 threads instead of 1 thread to service a single user
request - that will limit your scalability. On the oher hand, when the
site isn't highly utilized you'll have a dramatic performance
increase.

I touch a bit on this subject in the following posts:
http://odetocode.com/Blogs/scott/arc...6/16/1656.aspx
http://odetocode.com/Blogs/scott/arc...6/23/1877.aspx

Notice don't need the Sleep call at all - you can simply have the
thread block on EndGetResponse calls. That is the simplest approach.

--
Scott
http://www.OdeToCode.com/blogs/scott/
On Thu, 23 Jun 2005 13:23:02 -0700, "Dave"
<Da**@discussions.microsoft.com> wrote:
>I've have several webrequests being called consecutively from an .aspx that
>return XML from sources outside the company. When the page runs it can take
>anywhere between 30-60 seconds.
>
>I heard about making webrequests asynchronosly and found:
>
>http://samples.gotdotnet.com/quickst.../GETAsync.aspx
>
>I modified the code to add a couple requests right after each other as
>listed below.
>
>It worked but...
>
>1.) What is the drawback of this technique in terms of performance in the
>long run?
>2.) Is there a way to determine when all of these webrequests complete and
>just flush the response instead of waiting for the Thread.Sleep to complete?
>
>HttpWebRequest wreq;
>IAsyncResult r;
>
>wreq = (HttpWebRequest) WebRequest.Create("http://somexmlsource1");
>r = (IAsyncResult) wreq.BeginGetResponse(new
>AsyncCallback(this.RespCallback), wreq);
>
>wreq = (HttpWebRequest) WebRequest.Create("http://somexmlsource2");
>r = (IAsyncResult) wreq.BeginGetResponse(new
>AsyncCallback(this.RespCallback), wreq);
>
>wreq = (HttpWebRequest) WebRequest.Create("http://somexmlsource3");
>r = (IAsyncResult) wreq.BeginGetResponse(new
>AsyncCallback(this.RespCallback), wreq);
>
>Thread.Sleep(30000); <!--Wait 30 seconds to allow requests to complete



Nov 19 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Leo | last post by:
Version 7 fixpack 9, aix 5.1 In our database we have a table that that has a dedicated bufferpool for the data, and another for indexes. Actually we have three of these combinations and they...
0
by: DotNetShadow | last post by:
Hi Guys I came across this article which deals with Performance Considerations for Making Web Service Calls from ASPX Pages:...
1
by: Julian Hershel | last post by:
Reading about asynchronous programming (ms-help://MS.NETFrameworkSDK/cpguidenf/html/cpconasynchronousdesignpatterno verview.htm) I could not clarify some doubts. Hope you can help me. 1) Are...
3
by: usenetaccount | last post by:
In a newly created test app, to maximize client performance I tried to make two SOAP method calls in tandem (the soap methods execute some specified query), as each call includes a large amount of...
1
by: GM | last post by:
Hello, Is it possible to call 4 webrequests from an aspx file at the same time and then wait for the results of all 4 to come in. (4 webrequests to 4 different servers) I know that I can create...
1
by: dba123 | last post by:
I need to perform Asynchronous Inserts using DAAB. So far I have a method which does an insert but how can I do this Asyncronously so that it does not affect the load on our public production...
0
by: Bishoy George | last post by:
Hi, I have a asp.net 2.0 web application. I want to implement the asynchronous model through http handler in web.config ...
4
by: Morgan Cheng | last post by:
Since ASP.NET 2.0, asynchronous web service client can be implemented with event-based pattern, instead of original BeginXXX/EndXXX pattern. However, I didn't find any material about event-based...
2
by: Nicolas Le Gland | last post by:
Hello everyone here. This is my first post in this newsgroup, I hope I won't be to much off-topic. Feel free to redirect me to any better group. I am getting strange timing issues when...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.