Troubleshooting ASP.Net memory usage issues?

[Daniel Peterson]

I'm responsible for a pair of IIS 6 webservers that run our production
ASP.Net application. We push code on a monthly basis, and about two weeks
after our October code push, we started to run into occasional webserver
errors.

The application pool is configured to run as a domain user which has
permissions to the web code. When we have issues, the webserver will return
the ASP.Net "Server Application Unavailable" page, and our app pool memory
usage is really high (anywhere from 2 to 4 times our normal usage). Non
asp.net content is served up just fine. If I recycle the application pool
or do an iisreset, memory usage drops back to normal and the site starts
working again.

Nothing is logged to the eventlog or http error log, and the only errors we
see from our application has been the occasional "Exception of type
System.OutOfMemoryException was thrown." The same page doesn't error every
time, and the function I've seen throw exceptions is incredibly generic and
used heavily throughout our entire application (or so our devs have told
me). Our application doesn't even log an error every time this has
happened.

How should I go about troubleshooting this? Our devs are saying it's a
server issue, but I'm leaning towards a code issue. These servers are
running Windows 2003 Server standard, fully updated as of this last weekend.

[Eliyahu Goldin]

What session management are you using. InProc sessions are major memory
consumers.

--
Eliyahu Goldin,
Software Developer
Microsoft MVP [ASP.NET]
http://msmvps.com/blogs/egoldin
http://usableasp.net
"Daniel Peterson" <py****@hotmail.comwrote in message
news:CB**********************************@microsof t.com...

I'm responsible for a pair of IIS 6 webservers that run our production
ASP.Net application. We push code on a monthly basis, and about two weeks
after our October code push, we started to run into occasional webserver
errors.

The application pool is configured to run as a domain user which has
permissions to the web code. When we have issues, the webserver will
return the ASP.Net "Server Application Unavailable" page, and our app pool
memory usage is really high (anywhere from 2 to 4 times our normal usage).
Non asp.net content is served up just fine. If I recycle the application
pool or do an iisreset, memory usage drops back to normal and the site
starts working again.

Nothing is logged to the eventlog or http error log, and the only errors
we see from our application has been the occasional "Exception of type
System.OutOfMemoryException was thrown." The same page doesn't error
every time, and the function I've seen throw exceptions is incredibly
generic and used heavily throughout our entire application (or so our devs
have told me). Our application doesn't even log an error every time this
has happened.

How should I go about troubleshooting this? Our devs are saying it's a
server issue, but I'm leaning towards a code issue. These servers are
running Windows 2003 Server standard, fully updated as of this last
weekend.

[Kevin Spencer]

Hi Daniel,

It's a code issue. It will be one of 2 possible reasons: 1. Memory Leak.
Yes, .Net applications can leak memory, if they use unmanaged resources that
are not disposed properly. 2. Memory Fragmentation. If the same block of
memory is re-used as, for example, an application-level collection that has
members added and removed over a long period of time, that memory can become
fragmented, necessitating the allocation of more memory to create contiguous
blocks when they are needed.

--
HTH,

Kevin Spencer
Chicken Salad Surgeon
Microsoft MVP

"Daniel Peterson" <py****@hotmail.comwrote in message
news:CB**********************************@microsof t.com...

I'm responsible for a pair of IIS 6 webservers that run our production
ASP.Net application. We push code on a monthly basis, and about two weeks
after our October code push, we started to run into occasional webserver
errors.

The application pool is configured to run as a domain user which has
permissions to the web code. When we have issues, the webserver will
return the ASP.Net "Server Application Unavailable" page, and our app pool
memory usage is really high (anywhere from 2 to 4 times our normal usage).
Non asp.net content is served up just fine. If I recycle the application
pool or do an iisreset, memory usage drops back to normal and the site
starts working again.

Nothing is logged to the eventlog or http error log, and the only errors
we see from our application has been the occasional "Exception of type
System.OutOfMemoryException was thrown." The same page doesn't error
every time, and the function I've seen throw exceptions is incredibly
generic and used heavily throughout our entire application (or so our devs
have told me). Our application doesn't even log an error every time this
has happened.

How should I go about troubleshooting this? Our devs are saying it's a
server issue, but I'm leaning towards a code issue. These servers are
running Windows 2003 Server standard, fully updated as of this last
weekend.

[Daniel Peterson]

We're using the session state db.

"Eliyahu Goldin" <RE**************************@mMvVpPsS.orgwrote in
message news:e0**************@TK2MSFTNGP06.phx.gbl...

What session management are you using. InProc sessions are major memory
consumers.

--
Eliyahu Goldin,
Software Developer
Microsoft MVP [ASP.NET]
http://msmvps.com/blogs/egoldin
http://usableasp.net
"Daniel Peterson" <py****@hotmail.comwrote in message
news:CB**********************************@microsof t.com...

>I'm responsible for a pair of IIS 6 webservers that run our production
ASP.Net application. We push code on a monthly basis, and about two
weeks after our October code push, we started to run into occasional
webserver errors.

The application pool is configured to run as a domain user which has
permissions to the web code. When we have issues, the webserver will
return the ASP.Net "Server Application Unavailable" page, and our app
pool memory usage is really high (anywhere from 2 to 4 times our normal
usage). Non asp.net content is served up just fine. If I recycle the
application pool or do an iisreset, memory usage drops back to normal and
the site starts working again.

Nothing is logged to the eventlog or http error log, and the only errors
we see from our application has been the occasional "Exception of type
System.OutOfMemoryException was thrown." The same page doesn't error
every time, and the function I've seen throw exceptions is incredibly
generic and used heavily throughout our entire application (or so our
devs have told me). Our application doesn't even log an error every time
this has happened.

How should I go about troubleshooting this? Our devs are saying it's a
server issue, but I'm leaning towards a code issue. These servers are
running Windows 2003 Server standard, fully updated as of this last
weekend.

[Daniel Peterson]

So what's the best route to troubleshoot, just bust out debugdiag and go
from there?

If it's option #2 as you mention below, would cycling the application pool
more frequently solve that? I think we're using the 1740 minute default
right now.

"Kevin Spencer" <un**********@nothinks.comwrote in message
news:#Y**************@TK2MSFTNGP03.phx.gbl...

Hi Daniel,

It's a code issue. It will be one of 2 possible reasons: 1. Memory Leak.
Yes, .Net applications can leak memory, if they use unmanaged resources
that are not disposed properly. 2. Memory Fragmentation. If the same block
of memory is re-used as, for example, an application-level collection that
has members added and removed over a long period of time, that memory can
become fragmented, necessitating the allocation of more memory to create
contiguous blocks when they are needed.

--
HTH,

Kevin Spencer
Chicken Salad Surgeon
Microsoft MVP

"Daniel Peterson" <py****@hotmail.comwrote in message
news:CB**********************************@microsof t.com...

>I'm responsible for a pair of IIS 6 webservers that run our production
ASP.Net application. We push code on a monthly basis, and about two
weeks after our October code push, we started to run into occasional
webserver errors.

The application pool is configured to run as a domain user which has
permissions to the web code. When we have issues, the webserver will
return the ASP.Net "Server Application Unavailable" page, and our app
pool memory usage is really high (anywhere from 2 to 4 times our normal
usage). Non asp.net content is served up just fine. If I recycle the
application pool or do an iisreset, memory usage drops back to normal and
the site starts working again.

Nothing is logged to the eventlog or http error log, and the only errors
we see from our application has been the occasional "Exception of type
System.OutOfMemoryException was thrown." The same page doesn't error
every time, and the function I've seen throw exceptions is incredibly
generic and used heavily throughout our entire application (or so our
devs have told me). Our application doesn't even log an error every time
this has happened.

How should I go about troubleshooting this? Our devs are saying it's a
server issue, but I'm leaning towards a code issue. These servers are
running Windows 2003 Server standard, fully updated as of this last
weekend.

[Kevin Spencer]

Hi Daniel,

You could set up some performance counters, but the first thing I think I
would do is look for evidence of the 2 conditions I specified in the code.
It might very well show up rather easily if you know what you're looking
for.

--
HTH,

Kevin Spencer
Chicken Salad Surgeon
Microsoft MVP

"Daniel Peterson" <py****@hotmail.comwrote in message
news:03**********************************@microsof t.com...

So what's the best route to troubleshoot, just bust out debugdiag and go
from there?

If it's option #2 as you mention below, would cycling the application pool
more frequently solve that? I think we're using the 1740 minute default
right now.

"Kevin Spencer" <un**********@nothinks.comwrote in message
news:#Y**************@TK2MSFTNGP03.phx.gbl...

>Hi Daniel,

It's a code issue. It will be one of 2 possible reasons: 1. Memory Leak.
Yes, .Net applications can leak memory, if they use unmanaged resources
that are not disposed properly. 2. Memory Fragmentation. If the same
block of memory is re-used as, for example, an application-level
collection that has members added and removed over a long period of time,
that memory can become fragmented, necessitating the allocation of more
memory to create contiguous blocks when they are needed.

--
HTH,

Kevin Spencer
Chicken Salad Surgeon
Microsoft MVP

"Daniel Peterson" <py****@hotmail.comwrote in message
news:CB**********************************@microso ft.com...

>>I'm responsible for a pair of IIS 6 webservers that run our production
ASP.Net application. We push code on a monthly basis, and about two
weeks after our October code push, we started to run into occasional
webserver errors.

The application pool is configured to run as a domain user which has
permissions to the web code. When we have issues, the webserver will
return the ASP.Net "Server Application Unavailable" page, and our app
pool memory usage is really high (anywhere from 2 to 4 times our normal
usage). Non asp.net content is served up just fine. If I recycle the
application pool or do an iisreset, memory usage drops back to normal
and the site starts working again.

Nothing is logged to the eventlog or http error log, and the only errors
we see from our application has been the occasional "Exception of type
System.OutOfMemoryException was thrown." The same page doesn't error
every time, and the function I've seen throw exceptions is incredibly
generic and used heavily throughout our entire application (or so our
devs have told me). Our application doesn't even log an error every
time this has happened.

How should I go about troubleshooting this? Our devs are saying it's a
server issue, but I'm leaning towards a code issue. These servers are
running Windows 2003 Server standard, fully updated as of this last
weekend.

[Scott Roberts]

I agree with Kevin, it's a code issue. Too many coders think they don't have
to worry about memory management anymore since "DotNet does it for me!".

Personally, I would start by looking at the Session and/or Application
object. It's likely that some coder decided it would be convenient to stuff
a huge collection of search results (or some such thing) into the session
object and now it's being held there until the session expires.

You could set up some performance counters, but the first thing I think I
would do is look for evidence of the 2 conditions I specified in the code.
It might very well show up rather easily if you know what you're looking
for.

[Dave Bush]

Another culprit could be IDisposable objects not being disposed. You
doing anything with the graphics object maybe?

Dave Bush
http://blog.dmbcllc.com

-----Original Message-----
From: Scott Roberts [mailto:sr******@no.spam.here-webworks-software.com]

Posted At: Friday, November 16, 2007 9:34 AM
Posted To: microsoft.public.dotnet.framework.aspnet
Conversation: Troubleshooting ASP.Net memory usage issues?
Subject: Re: Troubleshooting ASP.Net memory usage issues?

I agree with Kevin, it's a code issue. Too many coders think they don't
have
to worry about memory management anymore since "DotNet does it for me!".

Personally, I would start by looking at the Session and/or Application
object. It's likely that some coder decided it would be convenient to
stuff
a huge collection of search results (or some such thing) into the
session
object and now it's being held there until the session expires.

You could set up some performance counters, but the first thing I

think I

would do is look for evidence of the 2 conditions I specified in the

code.

It might very well show up rather easily if you know what you're

looking

for.

[Daniel Peterson]

Thanks for your help everyone. We found a 3rd party component with
debugdiag that appears to be leaking memory, although our devs seemed to
dismiss that as the issue, and they're still pointing to the servers. If
they don't want to look, I'm not sure how much more I can do.

They think that
http://support.microsoft.com/default...68&Product=asp
this is what's going on, although we're not seeing any of those errors or
eventlog messages.

Ugh.

"Scott Roberts" <sr******@no.spam.here-webworks-software.comwrote in
message news:et**************@TK2MSFTNGP04.phx.gbl...

I agree with Kevin, it's a code issue. Too many coders think they don't
have to worry about memory management anymore since "DotNet does it for
me!".

Personally, I would start by looking at the Session and/or Application
object. It's likely that some coder decided it would be convenient to
stuff a huge collection of search results (or some such thing) into the
session object and now it's being held there until the session expires.

>You could set up some performance counters, but the first thing I think I
would do is look for evidence of the 2 conditions I specified in the
code. It might very well show up rather easily if you know what you're
looking for.

[Scott Roberts]

Thanks for your help everyone. We found a 3rd party component with

debugdiag that appears to be leaking memory, although our devs seemed to
dismiss that as the issue, and they're still pointing to the servers. If
they don't want to look, I'm not sure how much more I can do.

In your original post you mentioned that the problem started after a new
version was published, right? Was this 3rd party component used in the old
version of the site? Was it upgraded for the new version? Does the new
version use some features of this component that were previously unused?

They think that
http://support.microsoft.com/default...68&Product=asp
this is what's going on, although we're not seeing any of those errors or
eventlog messages.

I don't see anything pertaining to large memory usage in that KB. Is it a
memory problem or a thread pool problem?

[Daniel Peterson]

To answer your questions:

1. Yes, we noticed the issues about 2 weeks after a release. The 3rd party
component was updated in that release, I don't think we're using any new
features, but I'll check.

2. I don't see anything in that KB article that describes the symptoms
we're seeing either. We're not getting any of the error messages that would
suggest a thread pool issue, I'm not quite sure how they ended up at that kb
article.

"Scott Roberts" <sr******@no.spam.here-webworks-software.comwrote in
message news:O1**************@TK2MSFTNGP06.phx.gbl...

>Thanks for your help everyone. We found a 3rd party component with
debugdiag that appears to be leaking memory, although our devs seemed to
dismiss that as the issue, and they're still pointing to the servers. If
they don't want to look, I'm not sure how much more I can do.

In your original post you mentioned that the problem started after a new
version was published, right? Was this 3rd party component used in the old
version of the site? Was it upgraded for the new version? Does the new
version use some features of this component that were previously unused?

>They think that
http://support.microsoft.com/default...68&Product=asp
this is what's going on, although we're not seeing any of those errors or
eventlog messages.

I don't see anything pertaining to large memory usage in that KB. Is it a
memory problem or a thread pool problem?