By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
424,846 Members | 1,646 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 424,846 IT Pros & Developers. It's quick & easy.

Web Service won't respond and recycling the App Pool doesn't help

P: n/a
Bob
I've got a .NET Framework V1.1 web service running on a Windows 2003 Server
that has 2 web methods that are called from a web application on the same
server.

One is a fire-and-forget method that executes a long running process that
results in a database being updated.
The other is a normal syncronous method that returns configuration
information to the caller.

They both log their errors to the application event log.

The first thing the fire-and-forget web service does is validate that the
user is allowed to execute the web method. If the user is not allowed, an
error is logged to the event log and the web method terminates. Next it
updates a database indicating that a certain operation is taking place. After
that, it goes about executing a series of complex and time-consuming SQL
queries, hence the reason it's a web method, and the last thing it does is
update a database to indicate that operation has completed.

The second method simply returns the HttpContext.Current.User.Identity.Name
and the version number from the currently executing assembly.

The symptoms are: calling the fire-and-forget method never executes, but
doesn't generate an error. This is evident because the database that
indicates the operation is taking place is never updated.

Calling the second method results in a long wait followed by the error:
System.Net.WebException: The underlying connection was closed: An unexpected
error occurred on a receive.

Does anyone have any ideas about:

1) How to get the web service to start responding without rebooting the
server.

2) Preventing the web service from getting into this state to begin with.

Thanks.
Nov 23 '05 #1
Share this Question
Share on Google+
5 Replies


P: n/a
Bob,

I suspect that even fire-and-forget methods are going to be subject to the
same timeouts as web pages. Long running processes are best taken out of the
web and placed into services. We use MSMQ to sequence the requests taken by
the web service and route them to a background service. We can then make all
the webmethods synchronous returning a response indicating whether the
request has been accepted. Completion of the request is then signaled via
e-mail. This also allows us to balance the workload as we can control the
number of simultaneous, heavy processes running on the server.

For your second problem, have you been able to step into the web service in
Debug? Is it actually reaching your code?

"Bob" wrote:
I've got a .NET Framework V1.1 web service running on a Windows 2003 Server
that has 2 web methods that are called from a web application on the same
server.

One is a fire-and-forget method that executes a long running process that
results in a database being updated.
The other is a normal syncronous method that returns configuration
information to the caller.

They both log their errors to the application event log.

The first thing the fire-and-forget web service does is validate that the
user is allowed to execute the web method. If the user is not allowed, an
error is logged to the event log and the web method terminates. Next it
updates a database indicating that a certain operation is taking place. After
that, it goes about executing a series of complex and time-consuming SQL
queries, hence the reason it's a web method, and the last thing it does is
update a database to indicate that operation has completed.

The second method simply returns the HttpContext.Current.User.Identity.Name
and the version number from the currently executing assembly.

The symptoms are: calling the fire-and-forget method never executes, but
doesn't generate an error. This is evident because the database that
indicates the operation is taking place is never updated.

Calling the second method results in a long wait followed by the error:
System.Net.WebException: The underlying connection was closed: An unexpected
error occurred on a receive.

Does anyone have any ideas about:

1) How to get the web service to start responding without rebooting the
server.

2) Preventing the web service from getting into this state to begin with.

Thanks.

Nov 23 '05 #2

P: n/a
Bob


"Paul Hasell" wrote:
Bob,

I suspect that even fire-and-forget methods are going to be subject to the
same timeouts as web pages. Long running processes are best taken out of the
web and placed into services. We use MSMQ to sequence the requests taken by
the web service and route them to a background service. We can then make all
the webmethods synchronous returning a response indicating whether the
request has been accepted. Completion of the request is then signaled via
e-mail. This also allows us to balance the workload as we can control the
number of simultaneous, heavy processes running on the server.

In the case of the fire-and-forget web service, we have taken care of all
the timeout issues such that all the timeouts are set to 4 times the normal
run times. In addition, when there is a timeout, an error will be logged. No
error is being logged and no database updates are happening. The very first
database update is NOT in a transaction, so even if there is an error later
on, the first update would be visible in the database, and it's not.
For your second problem, have you been able to step into the web service in
Debug? Is it actually reaching your code?


We aren't allowed to debug on the server that is having problems, so no I
can't step through the web service.

I'm going to put in some writes to the event log to attempt to trace the
code. I've been hesitant to do so because making changes to the code will
usually cause the, "You changed your code, so it must be your code" finger
pointing.

Thanks for your response.

Nov 23 '05 #3

P: n/a
Bob,

Is their nothing in the Application/System/Security logs around the times of
the calls? It might be worth considering WSE 2.0 (if you've not already
installed this) which can be used to record a trace of web service calls on
the server. This can then be switched on/off via the web.config file.

"Bob" wrote:


"Paul Hasell" wrote:
Bob,

I suspect that even fire-and-forget methods are going to be subject to the
same timeouts as web pages. Long running processes are best taken out of the
web and placed into services. We use MSMQ to sequence the requests taken by
the web service and route them to a background service. We can then make all
the webmethods synchronous returning a response indicating whether the
request has been accepted. Completion of the request is then signaled via
e-mail. This also allows us to balance the workload as we can control the
number of simultaneous, heavy processes running on the server.


In the case of the fire-and-forget web service, we have taken care of all
the timeout issues such that all the timeouts are set to 4 times the normal
run times. In addition, when there is a timeout, an error will be logged. No
error is being logged and no database updates are happening. The very first
database update is NOT in a transaction, so even if there is an error later
on, the first update would be visible in the database, and it's not.
For your second problem, have you been able to step into the web service in
Debug? Is it actually reaching your code?


We aren't allowed to debug on the server that is having problems, so no I
can't step through the web service.

I'm going to put in some writes to the event log to attempt to trace the
code. I've been hesitant to do so because making changes to the code will
usually cause the, "You changed your code, so it must be your code" finger
pointing.

Thanks for your response.


Nov 23 '05 #4

P: n/a
Bob
Paul,

There is absolutely nothing in the event logs. Things are complicated by the
fact that we have a content switch that load balances to multiple servers. We
are pretty sure that the web service is NOT being executed as I added code to
write to the application event log as soon as one of the web method is
invoked. Nothing showed up in any of the event logs. We recycled the
application pool to force the code to be recompiled and then turned on
FileMon from SysInternals and seached the log for references to the web
service being read by the .NET runtime to be compiled and executed and it
didn't show anything.

We then did the same on our development server as a sanity check and FileMon
showed all sorts of .NET activity when we called the web service.

Thanks for the tip about WSE 2.0. I'll mention it in the meeting this
morning. Unfortunately, our company has segregated duties to such an extent
that it takes a meeting with people from 4 different groups to get the big
picture as to what may have been changed on the web servers and network gear
in the past week.

"Paul Hasell" wrote:
Bob,

Is their nothing in the Application/System/Security logs around the times of
the calls? It might be worth considering WSE 2.0 (if you've not already
installed this) which can be used to record a trace of web service calls on
the server. This can then be switched on/off via the web.config file.


Nov 23 '05 #5

P: n/a
Bob,

Have you got any IIS logging turned on? This might help to confirm whether
requests are getting through to IIS for the ASMX. There's an IE plug-in we
use called HTTPWatch (from www.simtec.ltd.uk) that we find useful for seeing
what's going on from the browser end and responses that it's getting. If your
accessing from another client application then HTTP Analyzer (from
www.ieinspector.com) might help as it monitors any/all HTTP requests from the
client.

"Bob" wrote:
Paul,

There is absolutely nothing in the event logs. Things are complicated by the
fact that we have a content switch that load balances to multiple servers. We
are pretty sure that the web service is NOT being executed as I added code to
write to the application event log as soon as one of the web method is
invoked. Nothing showed up in any of the event logs. We recycled the
application pool to force the code to be recompiled and then turned on
FileMon from SysInternals and seached the log for references to the web
service being read by the .NET runtime to be compiled and executed and it
didn't show anything.

We then did the same on our development server as a sanity check and FileMon
showed all sorts of .NET activity when we called the web service.

Thanks for the tip about WSE 2.0. I'll mention it in the meeting this
morning. Unfortunately, our company has segregated duties to such an extent
that it takes a meeting with people from 4 different groups to get the big
picture as to what may have been changed on the web servers and network gear
in the past week.

"Paul Hasell" wrote:
Bob,

Is their nothing in the Application/System/Security logs around the times of
the calls? It might be worth considering WSE 2.0 (if you've not already
installed this) which can be used to record a trace of web service calls on
the server. This can then be switched on/off via the web.config file.

Nov 23 '05 #6

This discussion thread is closed

Replies have been disabled for this discussion.