Simple problem: Why does this cause a massive memory leak?

[Simon]

Hi all,

I have a huge memory leak problem in what is really very simple data
insert code.

In my app I'm trying to use a typed dataset to insert into a database.
It adds quite a few rows (say hundreds). Unfortunately it has a big ass
memory leak and I dont understand why.

The code below demonstrates the memory leak. It's like nothing is
cleaned up after PerformOperation is called.

According to .net memory profiler, after running the console app, there
are huge numbers of undisposed objects and meory usage is through the roof.

I've tried to attach a screenshot from .net memory profiler showing what
I mean

Any help would be very much appreciated

Thanks

Simon

class Program {

static void Main(string[] args) {
for (int i = 0; i < 1000; i++) {
PerformOperation();
}
}

private static void PerformOperation() {
GPSPositionTableAdapter da = new GPSPositionTableAdapter();
GPSPositionTable tblGPSPositions = new GPSPositionTable();
GPSPositionRow currentRow = null;

for (int i = 0; i < 50; i++) {

currentRow = tblGPSPositions.NewGPSPositionRow();

currentRow.DownloadID = 1;
currentRow.TTUReference = 9999;
currentRow.Timestamp = DateTime.Now;
currentRow.Latitude = 66.66666666666;
currentRow.Longitude = 66.66666666666;

tblGPSPositions.AddGPSPositionRow(currentRow);
}

da.Update(tblGPSPositions);
}

}

[Nicholas Paldino [.NET/C# MVP]]

Simon,

I am curious, why do you think there should be disposed objects? The GC
is not deterministic, and you cant say by looking at a program "I should
have disposed objects here". The only place you can even begin to make that
sort of assumption is where you have an explicit call to the static Collect
method on the GC class, which you do not have here.

What is it that you are looking to do?
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com

"Simon" <si***@nothanks.comwrote in message
news:uq**************@TK2MSFTNGP03.phx.gbl...

Hi all,

I have a huge memory leak problem in what is really very simple data
insert code.

In my app I'm trying to use a typed dataset to insert into a database. It
adds quite a few rows (say hundreds). Unfortunately it has a big ass
memory leak and I dont understand why.

The code below demonstrates the memory leak. It's like nothing is cleaned
up after PerformOperation is called.

According to .net memory profiler, after running the console app, there
are huge numbers of undisposed objects and meory usage is through the
roof.

I've tried to attach a screenshot from .net memory profiler showing what I
mean

Any help would be very much appreciated

Thanks

Simon

class Program {

static void Main(string[] args) {
for (int i = 0; i < 1000; i++) {
PerformOperation();
}
}

private static void PerformOperation() {
GPSPositionTableAdapter da = new GPSPositionTableAdapter();
GPSPositionTable tblGPSPositions = new GPSPositionTable();
GPSPositionRow currentRow = null;

for (int i = 0; i < 50; i++) {

currentRow = tblGPSPositions.NewGPSPositionRow();

currentRow.DownloadID = 1;
currentRow.TTUReference = 9999;
currentRow.Timestamp = DateTime.Now;
currentRow.Latitude = 66.66666666666;
currentRow.Longitude = 66.66666666666;

tblGPSPositions.AddGPSPositionRow(currentRow);
}

da.Update(tblGPSPositions);
}

}

[Ben Voigt]

According to .net memory profiler, after running the console app, there

are huge numbers of undisposed objects and meory usage is through the
roof.

If you want objects to be disposed, as opposed to finalized during
collection, you must call IDisposable.Dispose on each object. The using
clause is a convenient way to do this in most cases.

[Cor Ligthert [MVP]]

simon,

See inline

private static void PerformOperation() {
GPSPositionTableAdapter da = new GPSPositionTableAdapter();
GPSPositionTable tblGPSPositions = new GPSPositionTable();

This is te place where the old tblGSSPostitions should be given to the GC as
long as there are no rows anymore to it attached by instance by the
currentRow which is added to the tblGPSPpostitions.

Maybe will a simple clear before the new help in this.

Cor

GPSPositionRow currentRow = null;

for (int i = 0; i < 50; i++) {

currentRow = tblGPSPositions.NewGPSPositionRow();

currentRow.DownloadID = 1;
currentRow.TTUReference = 9999;
currentRow.Timestamp = DateTime.Now;
currentRow.Latitude = 66.66666666666;
currentRow.Longitude = 66.66666666666;

tblGPSPositions.AddGPSPositionRow(currentRow);
}

da.Update(tblGPSPositions);
}

}

[Simon]

Hi Guys,

Thanks for your thoughts on this.

The reason why its troubling me is that this application runs non-stop.
Over a period of say, 24hrs, the memory usage of the application
balloons to over 400 meg. If I leave it long enough, it will eventually
throw an exception due to running out of memory. Even if it wasn't
throwing a memory exception - why hasn't the garbage collector kicked in
much sooner?

Either way, I need to get it to stop using so much memory, and all the
undisposed instances in memory profiler seemed to be the culprit.

Is this behaviour normal? Can you tell me what to do?

Sincerest Thanks

Simon

[Simon]

Cor Ligthert [MVP] wrote:

simon,

See inline

>private static void PerformOperation() {
GPSPositionTableAdapter da = new GPSPositionTableAdapter();
GPSPositionTable tblGPSPositions = new GPSPositionTable();

This is te place where the old tblGSSPostitions should be given to the GC as
long as there are no rows anymore to it attached by instance by the
currentRow which is added to the tblGPSPpostitions.

Maybe will a simple clear before the new help in this.

Cor

> GPSPositionRow currentRow = null;

for (int i = 0; i < 50; i++) {

currentRow = tblGPSPositions.NewGPSPositionRow();

currentRow.DownloadID = 1;
currentRow.TTUReference = 9999;
currentRow.Timestamp = DateTime.Now;
currentRow.Latitude = 66.66666666666;
currentRow.Longitude = 66.66666666666;

tblGPSPositions.AddGPSPositionRow(currentRow);
}

da.Update(tblGPSPositions);
}

}

Hi Cor,

As I understand it, tblGPSPositions should be made available to the
garbage collector as soon as PerformOperation goes out of scope, not when

GPSPositionTable tblGPSPositions = new GPSPositionTable();

is called. Is this not the case?

Thanks

Simon

[Cor Ligthert [MVP]]

Simon,

Did you do what I wrote?

Cor

"Simon" <si***@nothanks.comschreef in bericht
news:Ov**************@TK2MSFTNGP03.phx.gbl...

Hi Guys,

Thanks for your thoughts on this.

The reason why its troubling me is that this application runs non-stop.
Over a period of say, 24hrs, the memory usage of the application balloons
to over 400 meg. If I leave it long enough, it will eventually throw an
exception due to running out of memory. Even if it wasn't throwing a
memory exception - why hasn't the garbage collector kicked in much sooner?

Either way, I need to get it to stop using so much memory, and all the
undisposed instances in memory profiler seemed to be the culprit.

Is this behaviour normal? Can you tell me what to do?

Sincerest Thanks

Simon

[Simon]

Hi Cor,

Thanks for your reply. I havent had a chance to try what you're
suggesting as the code is on another machine, but I can't see why it
would work...

PerformOperation is called say, 100 times and each time the datatable
and data adapter et al should go completely out of scope and therefore
be fair game for the GC.

If I call Clear on the data table just after it's create as you suggest,
I'd be calling clear on a completely new instance with nothing in it...

Am I perhaps misunderstanding something?

Many thanks

Simon

[Miha Markic [MVP C#]]

You should look for live (not disposed) instances - those are causing
problems.
Anyway, what live instances are there?

--
Miha Markic [MVP C#, INETA Country Leader for Slovenia]
RightHand .NET consulting & development www.rthand.com
Blog: http://cs.rthand.com/blogs/blog_with_righthand/
"Simon" <si***@nothanks.comwrote in message
news:uq**************@TK2MSFTNGP03.phx.gbl...

Hi all,

I have a huge memory leak problem in what is really very simple data
insert code.

In my app I'm trying to use a typed dataset to insert into a database. It
adds quite a few rows (say hundreds). Unfortunately it has a big ass
memory leak and I dont understand why.

The code below demonstrates the memory leak. It's like nothing is cleaned
up after PerformOperation is called.

According to .net memory profiler, after running the console app, there
are huge numbers of undisposed objects and meory usage is through the
roof.

I've tried to attach a screenshot from .net memory profiler showing what I
mean

[VJ]

Smion, how do you say there is a memory leak?.. Did you see the windows task
manager or use a memory profiler tool ???

VJ

"Simon" <si***@nothanks.comwrote in message
news:uq**************@TK2MSFTNGP03.phx.gbl...

Hi all,

I have a huge memory leak problem in what is really very simple data
insert code.

In my app I'm trying to use a typed dataset to insert into a database. It
adds quite a few rows (say hundreds). Unfortunately it has a big ass
memory leak and I dont understand why.

The code below demonstrates the memory leak. It's like nothing is cleaned
up after PerformOperation is called.

According to .net memory profiler, after running the console app, there
are huge numbers of undisposed objects and meory usage is through the
roof.

I've tried to attach a screenshot from .net memory profiler showing what I
mean

Any help would be very much appreciated

Thanks

Simon

class Program {

static void Main(string[] args) {
for (int i = 0; i < 1000; i++) {
PerformOperation();
}
}

private static void PerformOperation() {
GPSPositionTableAdapter da = new GPSPositionTableAdapter();
GPSPositionTable tblGPSPositions = new GPSPositionTable();
GPSPositionRow currentRow = null;

for (int i = 0; i < 50; i++) {

currentRow = tblGPSPositions.NewGPSPositionRow();

currentRow.DownloadID = 1;
currentRow.TTUReference = 9999;
currentRow.Timestamp = DateTime.Now;
currentRow.Latitude = 66.66666666666;
currentRow.Longitude = 66.66666666666;

tblGPSPositions.AddGPSPositionRow(currentRow);
}

da.Update(tblGPSPositions);
}

}

[Simon]

Hi Everyone,

I have a small confession to make - I was misunderstanding the readings
I was getting from the memory profiler. There is a memory leak, but it
isn't in the code that I posted. That code on its own doesnt cause a
memory leak.

I do still have a problem though that I don't know how to fix. It the
code I posted earlier, I ommitted (for clarity) the fact that I actually
download stuff using a COM component before adding it to the database.

It is this unmanaged code that is the source of the memory leak. Can
anyone give me any general advice on how to ensure that these COM
objects are cleaned up as I loop through each COM object?

Sorry for the confusion. You've all been very helpful though so thank
you everyone. You convinced me that the managed code shouldnt be the
source of the problem and made me look again at the unmanaged issue - an
area that I thought I had eliminated from my enquiries.

Sorry and thanks again

Simon

[VJ]

Implement IDisposable on COM objects and explicitly release unmaged
resources in this. And when using the com object instance, always call
dispose() when done.The link or example with MSDN IDisposible documentation
should help you get started. if you have questions ask.

VJ

"Simon" <si***@nothanks.comwrote in message
news:Ov**************@TK2MSFTNGP02.phx.gbl...

Hi Everyone,

I have a small confession to make - I was misunderstanding the readings I
was getting from the memory profiler. There is a memory leak, but it isn't
in the code that I posted. That code on its own doesnt cause a memory
leak.

I do still have a problem though that I don't know how to fix. It the code
I posted earlier, I ommitted (for clarity) the fact that I actually
download stuff using a COM component before adding it to the database.

It is this unmanaged code that is the source of the memory leak. Can
anyone give me any general advice on how to ensure that these COM objects
are cleaned up as I loop through each COM object?

Sorry for the confusion. You've all been very helpful though so thank you
everyone. You convinced me that the managed code shouldnt be the source of
the problem and made me look again at the unmanaged issue - an area that I
thought I had eliminated from my enquiries.

Sorry and thanks again

Simon

[Simon]

Hi VJ

Thanks for your reply.

I'm not sure how to "implement IDisposable" on these objects as they
don't have any such method. The COM objects are part of a third party
programming API that I have no control over.

How do I get rid of them if they dont have a Dispose() method or any
obvious equivelant?

Thanks again

Simon

[David Browne]

"Simon" <si***@nothanks.comwrote in message
news:Oc**************@TK2MSFTNGP04.phx.gbl...

Hi VJ

Thanks for your reply.

I'm not sure how to "implement IDisposable" on these objects as they don't
have any such method. The COM objects are part of a third party
programming API that I have no control over.

How do I get rid of them if they dont have a Dispose() method or any
obvious equivelant?

Yeah, that's nonsense. If the COM component really has a memory leak, you
would need to fix it in the COM component.

However, often .NET code using COM components have poor memory performance
because the Runtime Callable Wrappers that .NET uses to wrap the COM
interface pointers will not destroy the COM component until they are
finalized. This can cause lots of memory problems, although it's not
technically a leak. As a best practice, anytime you are using a COM
component from .NET, explicitly call
System.Runtime.InteropServices.Marshal.ReleaseComO bject when you are done
with the COM component. This will immediately decrement the reference count
on the COM component and should clean up any unmanaged memory associated
with the component.

David

[Simon]

Hi Guys,

Thanks for your help so far.

I have a couple of questions I'm hoping someone could help me with.

1. If I have a method that creates and uses some COM objects, will the
resources that those COM objects were using be fair game for release as
soon as they go out of scope? Or do I have to do something to get rid of
them?

2. Dave mentioned calling
System.Runtime.InteropServices.Marshal.ReleaseComO bject. I had found a
reference to this when googling, but discounted it because the official
documentation said that it should only be used for testing purposes.
Something about the behaviour being undefined under certain operating
conditions. Should I use this?

Surely there must be a well known way of getting rid of COM gubbins once
I'm done with it. It sure isn't happening automatically though....

Thanks all

Simon

[VJ]

Ok maybe I was not clear on what I said. The IDisposablie if for a .NET
Object not for COM. If the COM has a problem very true you fix it there..,
but any time you use Unmanaged code and reference in a .NET app, please wrap
it in a object with IDisposiable implementation and make sure you call
..Dispose() after done with the .NET object that has implemented the COM.

http://www.microsoft.com/mspress/boo...chap/5028.aspx

Also in addition to what David, said when using Native API Code in .NET make
sure they will have permissions to run, see below link what i mean.

http://msdn2.microsoft.com/en-us/lib...61(VS.80).aspx

All this helps you make your .NET app work well with native code, but again
if the native code has issues, have to fix it there.

VJ

"David Browne" <davidbaxterbrowne no potted me**@hotmail.comwrote in
message news:O%******************@TK2MSFTNGP03.phx.gbl...

>

"Simon" <si***@nothanks.comwrote in message
news:Oc**************@TK2MSFTNGP04.phx.gbl...

>Hi VJ

Thanks for your reply.

I'm not sure how to "implement IDisposable" on these objects as they
don't have any such method. The COM objects are part of a third party
programming API that I have no control over.

How do I get rid of them if they dont have a Dispose() method or any
obvious equivelant?

Yeah, that's nonsense. If the COM component really has a memory leak, you
would need to fix it in the COM component.

However, often .NET code using COM components have poor memory performance
because the Runtime Callable Wrappers that .NET uses to wrap the COM
interface pointers will not destroy the COM component until they are
finalized. This can cause lots of memory problems, although it's not
technically a leak. As a best practice, anytime you are using a COM
component from .NET, explicitly call
System.Runtime.InteropServices.Marshal.ReleaseComO bject when you are done
with the COM component. This will immediately decrement the reference
count on the COM component and should clean up any unmanaged memory
associated with the component.

David

[VJ]

Simon one more thing. If your COM object has methods to clean up itself
those have to be called. Just saying, I am sure you are already doing this.
See my recent post, this is where IDisposible is used which .NET and the GC
call after object is released, so you can do all cleaning.

VJ

"Simon" <si***@nothanks.comwrote in message
news:e9*************@TK2MSFTNGP02.phx.gbl...

Hi Guys,

Thanks for your help so far.

I have a couple of questions I'm hoping someone could help me with.

1. If I have a method that creates and uses some COM objects, will the
resources that those COM objects were using be fair game for release as
soon as they go out of scope? Or do I have to do something to get rid of
them?

2. Dave mentioned calling
System.Runtime.InteropServices.Marshal.ReleaseComO bject. I had found a
reference to this when googling, but discounted it because the official
documentation said that it should only be used for testing purposes.
Something about the behaviour being undefined under certain operating
conditions. Should I use this?

Surely there must be a well known way of getting rid of COM gubbins once
I'm done with it. It sure isn't happening automatically though....

Thanks all

Simon

[Cor Ligthert [MVP]]

Simon,

I mean before it is created.

GPSPositionTableAdapter da = new GPSPositionTableAdapter();
tblGSPositions.Clear();
'At least the table which will be replaces loses now his rows, and I hope
its references.
'Be aware that not the same table is used again, only the references is
replaced by another one with
'your sentence bellow. If there are references to the existing those will
stay as long as the program lives.
GPSPositionTable tblGPSPositions = new GPSPositionTable();

Cor

"Simon" <si***@nothanks.comschreef in bericht
news:eR*************@TK2MSFTNGP05.phx.gbl...

Hi Cor,

Thanks for your reply. I havent had a chance to try what you're suggesting
as the code is on another machine, but I can't see why it would work...

PerformOperation is called say, 100 times and each time the datatable and
data adapter et al should go completely out of scope and therefore be fair
game for the GC.

If I call Clear on the data table just after it's create as you suggest,
I'd be calling clear on a completely new instance with nothing in it...

Am I perhaps misunderstanding something?

Many thanks

Simon

[David Browne]

"Simon" <si***@nothanks.comwrote in message
news:e9*************@TK2MSFTNGP02.phx.gbl...

Hi Guys,

Thanks for your help so far.

I have a couple of questions I'm hoping someone could help me with.

1. If I have a method that creates and uses some COM objects, will the
resources that those COM objects were using be fair game for release as
soon as they go out of scope? Or do I have to do something to get rid of
them?

Yes, they might get cleaned up any time after the COM object goes out of
scope. But normally the Runtime Callable Wrapper must be Garbage-Collected
before the COM object can clean itself up. There's no way to tell when that
will happen. System.Runtime.InteropServices.Marshal.ReleaseComO bject should
destroy the COM object immediately.

>
2. Dave mentioned calling
System.Runtime.InteropServices.Marshal.ReleaseComO bject. I had found a
reference to this when googling, but discounted it because the official
documentation said that it should only be used for testing purposes.
Something about the behaviour being undefined under certain operating
conditions. Should I use this?

The official docs say nothing of the kind.

Marshal.ReleaseComObject Method
....
This method is used to explicitly control the lifetime of a COM object used
from managed code. You should use this method to free the underlying COM
object that holds references to resources in a timely manner or when objects
must be freed in a specific order.
http://msdn2.microsoft.com/en-us/lib...comobject.aspx

And yes, you should use this method to free the underlying COM object that
holds references to resources in a timely manner.

David

[Miha Markic [MVP C#]]

Cor,

There are two issues with your solution:
1. It won't compile as you are accessing a variable before its declaration
2. It doesn't matter whether you invoke Clear or not as reference goes out
of the scope anyway.

--
Miha Markic [MVP C#, INETA Country Leader for Slovenia]
RightHand .NET consulting & development www.rthand.com
Blog: http://cs.rthand.com/blogs/blog_with_righthand/

"Cor Ligthert [MVP]" <no************@planet.nlwrote in message
news:uE**************@TK2MSFTNGP04.phx.gbl...

Simon,

I mean before it is created.

GPSPositionTableAdapter da = new GPSPositionTableAdapter();
tblGSPositions.Clear();
'At least the table which will be replaces loses now his rows, and I hope
its references.
'Be aware that not the same table is used again, only the references is
replaced by another one with
'your sentence bellow. If there are references to the existing those will
stay as long as the program lives.
GPSPositionTable tblGPSPositions = new GPSPositionTable();

Cor

"Simon" <si***@nothanks.comschreef in bericht
news:eR*************@TK2MSFTNGP05.phx.gbl...

>Hi Cor,

Thanks for your reply. I havent had a chance to try what you're
suggesting as the code is on another machine, but I can't see why it
would work...

PerformOperation is called say, 100 times and each time the datatable and
data adapter et al should go completely out of scope and therefore be
fair game for the GC.

If I call Clear on the data table just after it's create as you suggest,
I'd be calling clear on a completely new instance with nothing in it...

Am I perhaps misunderstanding something?

Many thanks

Simon

[Simon Harvey]

Miha Markic [MVP C#] wrote:

Cor,

There are two issues with your solution:
1. It won't compile as you are accessing a variable before its declaration
2. It doesn't matter whether you invoke Clear or not as reference goes
out of the scope anyway.

Yeah - I've got to agree with that assesment I'm afraid. Unless I'm
being particularly thick, the code as suggested wouldnt even compile -

// Won't compile
tblGSPositions.Clear();
GPSPositionTable tblGPSPositions = new GPSPositionTable();

Thanks

Simon

[Cor Ligthert [MVP]]

Miha,

I agree with you that I missed the declaration part and this one won't go,
however the idea was that often the problem is that the old object stays,
because although it goes out of scope, holds or has references to it.

Maybe will this go with the declaration global. I trying this with the
shortes code I can think about.

cor

Cor,

There are two issues with your solution:
1. It won't compile as you are accessing a variable before its declaration
2. It doesn't matter whether you invoke Clear or not as reference goes out
of the scope anyway.

--
Miha Markic [MVP C#, INETA Country Leader for Slovenia]
RightHand .NET consulting & development www.rthand.com
Blog: http://cs.rthand.com/blogs/blog_with_righthand/

"Cor Ligthert [MVP]" <no************@planet.nlwrote in message
news:uE**************@TK2MSFTNGP04.phx.gbl...

>Simon,

I mean before it is created.

GPSPositionTableAdapter da = new GPSPositionTableAdapter();
tblGSPositions.Clear();
'At least the table which will be replaces loses now his rows, and I hope
its references.
'Be aware that not the same table is used again, only the references is
replaced by another one with
'your sentence bellow. If there are references to the existing those will
stay as long as the program lives.
GPSPositionTable tblGPSPositions = new GPSPositionTable();

Cor

"Simon" <si***@nothanks.comschreef in bericht
news:eR*************@TK2MSFTNGP05.phx.gbl...

>>Hi Cor,

Thanks for your reply. I havent had a chance to try what you're
suggesting as the code is on another machine, but I can't see why it
would work...

PerformOperation is called say, 100 times and each time the datatable
and data adapter et al should go completely out of scope and therefore
be fair game for the GC.

If I call Clear on the data table just after it's create as you suggest,
I'd be calling clear on a completely new instance with nothing in it...

Am I perhaps misunderstanding something?

Many thanks

Simon

[Miha Markic [MVP C#]]

"Cor Ligthert [MVP]" <no************@planet.nlwrote in message
news:Od**************@TK2MSFTNGP03.phx.gbl...

Miha,

I agree with you that I missed the declaration part and this one won't go,
however the idea was that often the problem is that the old object stays,
because although it goes out of scope, holds or has references to it.

If something is holding the reference (lets say parent) to instance you want
get rid of then you have to make sure to either
a) drop that reference
b) make sure every reference to parent is dropped
so the graph isn't referenced from outside.
Either way Clear would not help.

--
Miha Markic [MVP C#, INETA Country Leader for Slovenia]
RightHand .NET consulting & development www.rthand.com
Blog: http://cs.rthand.com/blogs/blog_with_righthand/