Monitoring asp.net async calls

[=?Utf-8?B?Q2FybG8gRm9saW5p?=]

Hi,
I implemented asynchronous calls to a web resource (using HttpWebRequest)
from asp.net 2.0.
The request it's made asyncronously (I see that beginGetRespons e returns
immediately).
The number of worker thread and completionPortT hreads are over 300.
Making 200 calls I see that 100 are queued.
There's a counter telling how much async calls are handled?
How can I raise the number of request handled?

Thanks in advance for your help

--
Carlo Folini

[bruce barker]

the default settings only allow 2 concurrent connections to a web server
(proper net-etiquette). you need to bump these up. also iis has a max
number of open connections (MaxConnections ), you need to bump this up.
also network driver has a limit (MaxUserPort). you will need to bump
this up.
-- bruce (sqlwork.com)


Carlo Folini wrote:

Hi,
I implemented asynchronous calls to a web resource (using HttpWebRequest)
from asp.net 2.0.
The request it's made asyncronously (I see that beginGetRespons e returns
immediately).
The number of worker thread and completionPortT hreads are over 300.
Making 200 calls I see that 100 are queued.
There's a counter telling how much async calls are handled?
How can I raise the number of request handled?

Thanks in advance for your help

[Steven Cheng[MSFT]]

Hello Carlo,

For your scenario, if you're using the "BeginGetRespon se" to send http
request asynchronously at client-side, that means the client-side http
request calling is under asynchronous mode. At server-side, the ASP.NET
runtime engine will still treat each comming requests(from your
asynchronous client) as normal requests and process them synchronously.
Therefore, you can still use those standard performance counters to trace
the requests being processed. e.g.

=============== ===============
Requests Executing
The number of requests currently executing.

Requests Failed
The total number of failed requests. Any status codes greater than or equal
to 400 will increment this counter. Requests that cause a 401 status code
increment this counter and the Requests Not Authorized counter. Requests
that cause a 404 or 414 status code increment this counter and the Requests
Not Found counter.

Requests Succeeded
The number of requests that executed successfully (status code 200).
=============== =============== =

You can find more ASP.NET specfiic performance counters in the following
reference:

#Performance Counters for ASP.NET
http://msdn2.microsoft.com/en-us/lib...92(VS.71).aspx

BTW, are you performing some performance test against your ASP.NET
application? As Bruce has mentioned, by default each .net client has a
maxconnection limit(2) to given remote site(request from local machine
doesn't have this limitation). If your client app is sending request to
remote machine, you need to take care of this:

http://msdn2.microsoft.com/en-us/lib...ion.connection
managementeleme nt.maxconnectio n.aspx

Also, for ASP.NET server-side threading, since the <processModel>' s
"maxWorkerThrea ds" and "maxIoThrea ds" limit the concurrent threads of each
ASP.NET worker process, you can monitor your server machine's CPU
utilization. If the CPU utlization is still low and there are lots of
queued requests, that means you can still enlarge the max worker/io threads
number to improve your application's throughput.

Hope this helps some.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead

=============== =============== =============== =====

Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscripti...ult.aspx#notif
ications.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscripti...t/default.aspx.

=============== =============== =============== =====

This posting is provided "AS IS" with no warranties, and confers no rights.

[=?Utf-8?B?Q2FybG8gRm9saW5p?=]

Hi Bruce hi Steven,
in my scenario the client is an ASP.NEt 2.0 aspx page and the server is a
SOAP toolkit isapi.
In my aspx page I'm creating the soap payload 'manually' and send it via
HttpWebRequest.
I'm stress testing this solution.
The IIS 6 serving aspx page under load takes 40% of CPU so I have plenty of
room to augment the number of concurrent requests.

I tweaked all the parameters you told me.
My question is about monitoring the asynchronous behavior of the aspx page.
When the aspx page send the request to the ws the processing is stopped and
put back in a IIS queue.
Is it this queue the same that handles the 'normal' requests? (i think it's
a different one? I call it 'async queue' to differentiate from the IIS normal
request queue)
Is there some perf counter to monitor how much request are inthe 'async
queue'?

I'm also calling different ws that behaves differently in terms of
performance (sometimes a subsystem gets slow). I need to separate the request
for the various subsystem (i use connectiongroup s to use different
connections).
In this scenario if a subsystem gets slow the request to the corresponding
ws remains queued in the 'async queue'. The IIS queue is free to serve the
other requests or the request limit cause the 'Server Too Busy' error?

--
Carlo Folini
"Steven Cheng[MSFT]" wrote:

Hello Carlo,

For your scenario, if you're using the "BeginGetRespon se" to send http
request asynchronously at client-side, that means the client-side http
request calling is under asynchronous mode. At server-side, the ASP.NET
runtime engine will still treat each comming requests(from your
asynchronous client) as normal requests and process them synchronously.
Therefore, you can still use those standard performance counters to trace
the requests being processed. e.g.

=============== ===============
Requests Executing
The number of requests currently executing.

Requests Failed
The total number of failed requests. Any status codes greater than or equal
to 400 will increment this counter. Requests that cause a 401 status code
increment this counter and the Requests Not Authorized counter. Requests
that cause a 404 or 414 status code increment this counter and the Requests
Not Found counter.

Requests Succeeded
The number of requests that executed successfully (status code 200).
=============== =============== =

You can find more ASP.NET specfiic performance counters in the following
reference:

#Performance Counters for ASP.NET
http://msdn2.microsoft.com/en-us/lib...92(VS.71).aspx

BTW, are you performing some performance test against your ASP.NET
application? As Bruce has mentioned, by default each .net client has a
maxconnection limit(2) to given remote site(request from local machine
doesn't have this limitation). If your client app is sending request to
remote machine, you need to take care of this:

http://msdn2.microsoft.com/en-us/lib...ion.connection
managementeleme nt.maxconnectio n.aspx

Also, for ASP.NET server-side threading, since the <processModel>' s
"maxWorkerThrea ds" and "maxIoThrea ds" limit the concurrent threads of each
ASP.NET worker process, you can monitor your server machine's CPU
utilization. If the CPU utlization is still low and there are lots of
queued requests, that means you can still enlarge the max worker/io threads
number to improve your application's throughput.

Hope this helps some.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead

=============== =============== =============== =====

Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscripti...ult.aspx#notif
ications.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscripti...t/default.aspx.

=============== =============== =============== =====

This posting is provided "AS IS" with no warranties, and confers no rights.

[Steven Cheng[MSFT]]

Thanks for your reply Carlo,

Now we got that your httpwebrequests call(asynchrono us) are actually made
in the client ASP.NET web application. Yes, such asynchronous call will
significantly affect the ASP.NET runtime's workerthreads and application
throughput.

As you mentioned that you use HttpWebRequest. BeginGetRespons e to make
asynchrous method call, then after the call, have you block the thread to
wait for the response or just simply ignore the response(let it be a
one-way style method call)? If you doesn't block it and wait for the
response(call EndGetResponse) , the ASP.NET main workerthread will run as
normal and be returned to the threadpool after the page lifecycle end.
Also, asynchronous call such as HttpWebRequest. BeginGetRespons e(or any
other BeginXXX like call) will use threads from .NET managed thread pool.

Therefore, for your scenario, there are two operations that will consume
..net CLR thread-pool threads:

** the ASP.NET requests which need thread-pool thread to handle it

** your httpwebrequest' s asynchronous call is running in thread-pool
thread.

Thus, if your maxWorkerThread and maxIOThread is not configured to the
proper value, you may encounter thoughput bottleneck.

Actually, ASP.NET runtime/.NET runtime doesn't care whether your ASP.NET
page is using asynchronous call, from its viewpoint, it only care whether
your task need a thread-pool thread to process. If so, it pickup a thread
from pool to process it. So what you need to take care is the currently
available thread-pool threads. This s quite easy to get through the
ThreadPool class, and .net doesn't add built-in counter for this, you can
simply create a custom counter as below:

#How To: Monitor the ASP.NET Thread Pool Using Custom Counters
http://msdn2.microsoft.com/en-gb/library/ms979194.aspx

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead
This posting is provided "AS IS" with no warranties, and confers no rights.

[=?Utf-8?B?Q2FybG8gRm9saW5p?=]

Hi Steven,
I didn't understand well what you are saying.

What I do on my aspx page is depicted in this post from Fritz Onion:
http://pluralsight.com/blogs/fritz/a...0/19/2892.aspx

I think that asp.net is aware of my async call.
When the "async point" is reached the aspx request is put back in the queue.
When the response from the ws come back the aspx request is picked up again
and handled by a (maybe different) worker thread to complete the rendering
phase.

Do you agree?

What it's not clear to me is where this aspx request is sent. Is it put back
in the http.sys queue or is the I/O thread pool managed by the .net runtime?

When I undertand this, I would like to know if there's a counter that counts
those async calls

--
Carlo Folini
"Steven Cheng[MSFT]" wrote:

Thanks for your reply Carlo,

Now we got that your httpwebrequests call(asynchrono us) are actually made
in the client ASP.NET web application. Yes, such asynchronous call will
significantly affect the ASP.NET runtime's workerthreads and application
throughput.

As you mentioned that you use HttpWebRequest. BeginGetRespons e to make
asynchrous method call, then after the call, have you block the thread to
wait for the response or just simply ignore the response(let it be a
one-way style method call)? If you doesn't block it and wait for the
response(call EndGetResponse) , the ASP.NET main workerthread will run as
normal and be returned to the threadpool after the page lifecycle end.
Also, asynchronous call such as HttpWebRequest. BeginGetRespons e(or any
other BeginXXX like call) will use threads from .NET managed thread pool.

Therefore, for your scenario, there are two operations that will consume
.net CLR thread-pool threads:

** the ASP.NET requests which need thread-pool thread to handle it

** your httpwebrequest' s asynchronous call is running in thread-pool
thread.

Thus, if your maxWorkerThread and maxIOThread is not configured to the
proper value, you may encounter thoughput bottleneck.

Actually, ASP.NET runtime/.NET runtime doesn't care whether your ASP.NET
page is using asynchronous call, from its viewpoint, it only care whether
your task need a thread-pool thread to process. If so, it pickup a thread
from pool to process it. So what you need to take care is the currently
available thread-pool threads. This s quite easy to get through the
ThreadPool class, and .net doesn't add built-in counter for this, you can
simply create a custom counter as below:

#How To: Monitor the ASP.NET Thread Pool Using Custom Counters
http://msdn2.microsoft.com/en-gb/library/ms979194.aspx

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead
This posting is provided "AS IS" with no warranties, and confers no rights.

[Steven Cheng[MSFT]]

Thanks for your reply Carlo,

Well, now I've got your actual scenario. Previously, I misunderstood your
"async page", as you mentioned "BeginGetRespon se", I assumed that you're
using normal page request to call the HttpWebRequest. BeginGetRespons e to
perform asynchronous operation.

For Async page you used, yes, you're right, when the async execution
begin(after PreRender event and before PreRenderComple te event), the worker
thread(for processing the page) is release and put back to managed
thread-pool. However, here in the async execution, you use Httpwebrequest
to perform another asynchronous operation, and based on my experience, such
asynchronous httpwebrequest call will also consume .NET threadpool thread.
I've checked the reflectored code of HttpWebRequest. BeginGetRespons e, it
will check the current ThreadPool's available workerthreads and IO threads.
Therefore, what I would suggest you do is create a custom counter which
use the " ThreadPool.GetA vailableThreads " method to monitor the current
available worker threads and IO thread in your ASP.NET application
process's CLR thread pool(as the following articles mentioned). This is the
most important statistics that will determine your ASP.NET application's
throughput (if there is no available threadpool thread, all the sequential
comming requests will be queued).

#How To: Monitor the ASP.NET Thread Pool Using Custom Counters
http://msdn2.microsoft.com/en-gb/library/ms979194.aspx

If you find that there is no significant improvement after using
asynchronous page execution, I think that means the WebRequest call will
also consume thread-pool thread so that using asynchronous execution won't
help much here.

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead
This posting is provided "AS IS" with no warranties, and confers no rights.

[Steven Cheng[MSFT]]

Hello Carlo,

Have you got any further idea or progress on this issue? As mentioned in
the previous message, the ThreadPool statistics are the primary flag of
your web applciation's throughput status. If you still haven't been able to
get it work or the performance monitor result doesn't quite reflect the
application's real status, there may has something else that impact the
performance. If you need to further thorough troubleshooting , I would
suggest you consider contact CSS for continual support. Please feel free
to let me know if there is still anything we can help.

http://msdn.microsoft.com/subscripti...t/default.aspx

Sincerely,

Steven Cheng

Microsoft MSDN Online Support Lead
This posting is provided "AS IS" with no warranties, and confers no rights.