The state server has closed an expired TCP/IP connection. The IP
address of the client is 127.0.0.1. The expired Read operation began at
11/16/2007 14:31:49.
This one has been bothering me for quite some time... there didn't seem
to be any real solutions for this or they didn't seem to apply.
Microsoft discusses this issue in the Knowledge base, but it seems to
only happen under very heavy load, and their resolution didn't fix my
problem.
Finally, I figured it out, and I wanted to pass my knowledge on to the
masses.
The biggest clue was that sometimes it would start throwing these state
server errors in the event log at times when there was almost no
traffic on the website. This seemed to indicate to me that microsofts
KB article was dead wrong since they state that this can occur during
heavy load.
The culprit ended up being a "Memory Leak". (I use quotes because it
was not a leak in the traditional sense... its just that objects that I
thought should be GC'd were not getting GC'd)
If you have this issue, take a look at the memory usage on the box.
Does it seem to only go up and rarely go down... i.e. More is being
used than collected over time. I noticed that it went up rather
consistently throughout the day, and not nearly enough of it seemed to
be freed during a GC.
At about 900 Mb, the garbage collector started doing a Gen 2 every
second. A gen 2 on 900 Mb tends to be very CPU intensive, and would max
out our 4 processors to 100% for about a second. Eventually we had it
set up to recycle the app when it got that high, but that wasn't a real
solution.
Do you have many complicated object graphs that tend to be rather large
that are being stored in InProc session or in the ASP.NET cache? How
about web controls that have custom events which are handled by methods
in other web controls? These two were my biggest problems... for the
web controls, I simply unregistered the event handlers in the unload
event of the control. That helped a ton. In ASP.NET, I've found the
best way to write the pages is to load and release, keep as little as
possible in cache... try to use the ViewState rather than
Session/Cache... And be wary of how complicated your object graphs are
getting. Try re-querying the database forg grid paging, rather than
keeping all of the paged data in memory.
I ended up having to purchase the ANTS profiler to see what wasn't
getting relased in a timely fashion, and that helped quite a bit...
That with the Microsoft Application Test Center tool, I was able to fix
my problem... and NOW (finally!) no more of this state server TCP/IP
connection issue.
I hope that this helps somebody in the future
Josh Norris
Senior Application Developer
Henderson Engineers, Inc
