Interesting Memory Issue

[Jim]

I am developing an ASP.Net HTTP Module used to log requests and
responses. To offload the logging, the module saves the request
information to a queue which is accessed by another thread to actually
perform the logging. The request information is saved into an object
which is then put into the queue and pulled out of the queue by the
second thread. The second thread sets the object to null when it is
done processing it.

Everything is working just fine, but I am experiencing a memory
increase on the aspnet_wp process that does not seem to go down.
Looking further into what is actually occuring, I found out that the GC
only kicks in when ASP.Net kills the asp.net processing working thread.
Only then are the finalizers (only put in for debugging purposes)
called on the objects that end up in the queue.

Is this normal operation? The issue is that under high stress, the
memory goes up to 200mb and it takes it a long time to come back down.

[Kevin Spencer]

If the objects remain in the queue, they are still referenced *by* the
queue. Garbage Collection can only occur when all references to an object
are 0. Are these objects removed from the queue at any point?

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
Complex things are made up of
Lots of simple things.

"Jim" <ji*********@gmail.com> wrote in message
news:11*********************@g47g2000cwa.googlegro ups.com...

I am developing an ASP.Net HTTP Module used to log requests and
responses. To offload the logging, the module saves the request
information to a queue which is accessed by another thread to actually
perform the logging. The request information is saved into an object
which is then put into the queue and pulled out of the queue by the
second thread. The second thread sets the object to null when it is
done processing it.

Everything is working just fine, but I am experiencing a memory
increase on the aspnet_wp process that does not seem to go down.
Looking further into what is actually occuring, I found out that the GC
only kicks in when ASP.Net kills the asp.net processing working thread.
Only then are the finalizers (only put in for debugging purposes)
called on the objects that end up in the queue.

Is this normal operation? The issue is that under high stress, the
memory goes up to 200mb and it takes it a long time to come back down.

[Jim]

Kevin,

The objects are dequeued from the queue by a processing thread that is
actually doing the logging. Further, I see that the objects are
destroyed, but only when the asp.net worker thread is destroyed by
asp.net. I ran another test last night where the memory got to around
250mb and came back down about 3 hours after the test itself was done.

Obviously the issue is that asp.net will keep the worker threads around
as long as there is activity on the web application and that could
cause to a potential out of memory issue.

[Willy Denoyette [MVP]]

"Jim" <ji*********@gmail.com> wrote in message
news:11**********************@z14g2000cwz.googlegr oups.com...

Kevin,

The objects are dequeued from the queue by a processing thread that is
actually doing the logging. Further, I see that the objects are
destroyed, but only when the asp.net worker thread is destroyed by
asp.net. I ran another test last night where the memory got to around
250mb and came back down about 3 hours after the test itself was done.

Obviously the issue is that asp.net will keep the worker threads around
as long as there is activity on the web application and that could
cause to a potential out of memory issue.

What exactly do you mean with dequeued, do you realy set the queue entry to
null? Maybe a piece of code would help.
Just take a look at the GC counters with Permon.exe, you'll see that during
a period of 3 hours the GC has kicked in a thousand times, so it's clear
that you are still holding "the" (not necessarely the queued) objects alive,
a memory profiler can reveal which objects are long living.
Willy.

[Jim]

Willy,

Thanks for the reply.

Here is the portion of code that pulls the objects out of the queue.
What is interesting is that the oOperation object does not get
collected until the asp.net worker thread is destoryed. To know this, I
created a finalizer where I just set an internal collection to null.
That code only executes when the thread is destroyed.

object oQueueItem;
HttpOperation oOperation;
string sOperationContent;
string
sURI=moParentMonitor.AdapterConfigInfo.EventPostEC Config.PostingUrl;
if
(moParentMonitor.AdapterConfigInfo.EventPostECConf ig.PortNumber>0)
{
sURI+=":"+moParentMonitor.AdapterConfigInfo.EventP ostECConfig.PortNumber.ToString();
}
while ( moParentMonitor.AdapterConfigInfo.MonitoringActive )
{
if (moParentMonitor.EventsQueue.Count>0)
{
lock (moParentMonitor.EventsQueue.SyncRoot)
{
oQueueItem=moParentMonitor.EventsQueue.Dequeue();
}
oOperation=(HttpOperation)oQueueItem;
oQueueItem=null;
sOperationContent=oOperation.ToXml();
HttpPost(sURI,sOperationContent);
sOperationContent=null;
oOperation=null;
lock (moAdapterStatistics)
{
moAdapterStatistics.NumOfEventsPosted+=1;
}
}
Thread.Sleep(10);
}
}
finally
{
Thread.Sleep(10);
}

[Kevin Spencer]

Hi Jim,

Somewhere, something is keeping a reference to the object. That's what you
need to look for.

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
Complex things are made up of
Lots of simple things.

"Jim" <ji*********@gmail.com> wrote in message
news:11**********************@g47g2000cwa.googlegr oups.com...

Willy,

Thanks for the reply.

Here is the portion of code that pulls the objects out of the queue.
What is interesting is that the oOperation object does not get
collected until the asp.net worker thread is destoryed. To know this, I
created a finalizer where I just set an internal collection to null.
That code only executes when the thread is destroyed.

object oQueueItem;
HttpOperation oOperation;
string sOperationContent;
string
sURI=moParentMonitor.AdapterConfigInfo.EventPostEC Config.PostingUrl;
if
(moParentMonitor.AdapterConfigInfo.EventPostECConf ig.PortNumber>0)
{
sURI+=":"+moParentMonitor.AdapterConfigInfo.EventP ostECConfig.PortNumber.ToString();
}
while ( moParentMonitor.AdapterConfigInfo.MonitoringActive )
{
if (moParentMonitor.EventsQueue.Count>0)
{
lock (moParentMonitor.EventsQueue.SyncRoot)
{
oQueueItem=moParentMonitor.EventsQueue.Dequeue();
}
oOperation=(HttpOperation)oQueueItem;
oQueueItem=null;
sOperationContent=oOperation.ToXml();
HttpPost(sURI,sOperationContent);
sOperationContent=null;
oOperation=null;
lock (moAdapterStatistics)
{
moAdapterStatistics.NumOfEventsPosted+=1;
}
}
Thread.Sleep(10);
}
}
finally
{
Thread.Sleep(10);
}