GC.Collect can be trusted?

[Ward Bekker]

Hi,

I'm wondering if the GC.Collect method really collects all objects
possible objects? Or is this still a "smart" process sometimes keeping
objects alive even if they can be garbage collected?

I need to know because I'm looking for memory leaks in an application.

It would be really helpful to be able to determine if an object after
manually invoking the GC.Collect is only kept alive because it still
being referenced by other alive objects in contrast to being alive
because the GC.Collect didn't found it necessary to collect the object,
even when asked explicitly.

Regards,

Ward

[Chris Mullins [MVP]]

"Christophe r Ireland" Wrote:

Can you please confirm to me that with the GC working well, any memory
leaks that occur while the application is running leave no "footprint" in
memory when the application is closed?

All current versions of Windows are pretty good about cleaning things up
once a process dies. I can't remember the last time I had a problem with
resources remaining allocated after I had exited a process.

Another thing I would be very grateful if you could clarify for me is
whether memory leaks which occur when the (100% managed) application is
running can be seen in perfmon's memory counter (or the task manager's
physical memory usage history, if it isn't the same thing).

Why do I feel like I'm being cross examined by an attorney looking to put
words in my mouth? :)

Perform's memory counter & Task Manager's memory viewer are terrible for
this stuff. Use the detailed CLR Performance Counters for information, it's
far more accurate.

Could I please try your patience one more time and ask you for what you
consider to be a good link (or even book!) on the matter?

http://msdn.microsoft.com/msdnmag/is...I/default.aspx
http://msdn.microsoft.com/msdnmag/is...2/default.aspx

This book has the best chapter on Garbage Collection that I've read. This is
where I learned most of what I know about how GC works:
http://www.amazon.com/Applied-Micros.../dp/0735614229

I've heard many good things about this one, but haven't read it myself:
http://www.amazon.com/o/ASIN/0735621...8QKPN1AZERGY02

--
Chris Mullins, MCSD.NET, MCPD:Enterprise , Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

[Christopher Ireland]

Willy Denoyette [MVP] wrote:

Note also, that this article was written years ago, and is based on
the first version of the framework and never updated since then.
These are the kind of "look how great our product is, it can even
find bugs in MSFT's products" articles, but they did not find
anything, this bug was known at the time they ran the profiler, all
they did was illustrate the impact of a bug in the Windows.Forms code
on the allocation pattern.

So it was a bug in the GC then?

You won't be able to find these kind of "bugs" using just a profiler,
you need a debugger, a great deal of understanding how the GC and the
CLR works, a lot of experience in debugging complex systems plus
quite some luck to finds small leaks like these. Granted, a profiler
is something you need in your toolbox, it's a great tool to
illustrate performance behaviors and uncover possible bottlenecks, it
can help you better understand your object allocation patterns, where
you allocate too many or to large objects, so you can adapt your
algorithms and allocation needs, but you won't ever be able to detect
small managed memory leaks by using a profiler alone.

I can tell you now that I have very little time to spend tracking down bugs
in the GC. What I'm after is being able to track down memory leaks that I
unwittingly create in the code I write. The question is: are the bugs I
introduce into my code through the allocation of too many or too large
objects really "memory leaks" (please, this is not a trick question, I'm
really trying to figure out what's going on here :-) ?

--
Thank you,

Christopher Ireland

[Christopher Ireland]

Jon Skeet [C# MVP] wrote:

If they don't, that's an operating system failure more than a CLR
failure. An operating system really, really, really should release any
memory held by a process when the process exits (assuming it's not
deliberately sharing memory with another process etc).

Ah, I see, thank you.

They may be visible there. It's a lot more reliable to use the CLR
performance counters in perfmon though.

If there was a memory leak in my code, what sort of symptoms should I be
looking for in perfmon's CLR performance counters?

--
Thank you,

Christopher Ireland

[Christopher Ireland]

Chris Mullins [MVP] wrote:

All current versions of Windows are pretty good about cleaning things
up once a process dies. I can't remember the last time I had a
problem with resources remaining allocated after I had exited a
process.

Ok, thank you for the information.

Why do I feel like I'm being cross examined by an attorney looking to
put words in my mouth? :)

I'm sorry for giving you that impression, but it really isn't my intention
at all. I'm much more of a sheep in wolf's clothing than the other way round
:-)

Perform's memory counter & Task Manager's memory viewer are terrible
for this stuff. Use the detailed CLR Performance Counters for
information, it's far more accurate.

Ok. What sort of things could the CLR performance counters show me that
would lead me to believe that my code has a memory leak?

http://msdn.microsoft.com/msdnmag/is...I/default.aspx
http://msdn.microsoft.com/msdnmag/is...2/default.aspx

This book has the best chapter on Garbage Collection that I've read.
This is where I learned most of what I know about how GC works:
http://www.amazon.com/Applied-Micros.../dp/0735614229

I've heard many good things about this one, but haven't read it
myself:
http://www.amazon.com/o/ASIN/0735621...8QKPN1AZERGY02

Fantastic! Again, many thanks for the tips!

--
Thank you,

Christopher Ireland

[Willy Denoyette [MVP]]

"Christophe r Ireland" <ci******@gmail .comwrote in message
news:3F******** *************** ***********@mic rosoft.com...

Willy Denoyette [MVP] wrote:

>Note also, that this article was written years ago, and is based on
the first version of the framework and never updated since then.
These are the kind of "look how great our product is, it can even
find bugs in MSFT's products" articles, but they did not find
anything, this bug was known at the time they ran the profiler, all
they did was illustrate the impact of a bug in the Windows.Forms code
on the allocation pattern.

So it was a bug in the GC then?

No, as I said, it was a bug in the Windows.Forms code.

>You won't be able to find these kind of "bugs" using just a profiler,
you need a debugger, a great deal of understanding how the GC and the
CLR works, a lot of experience in debugging complex systems plus
quite some luck to finds small leaks like these. Granted, a profiler
is something you need in your toolbox, it's a great tool to
illustrate performance behaviors and uncover possible bottlenecks, it
can help you better understand your object allocation patterns, where
you allocate too many or to large objects, so you can adapt your
algorithms and allocation needs, but you won't ever be able to detect
small managed memory leaks by using a profiler alone.

I can tell you now that I have very little time to spend tracking down
bugs in the GC. What I'm after is being able to track down memory leaks
that I unwittingly create in the code I write. The question is: are the
bugs I introduce into my code through the allocation of too many or too
large objects really "memory leaks" (please, this is not a trick question,
I'm really trying to figure out what's going on here :-) ?

Again, you don't have to, nor can you even track down bugs in the CLR's code
like the GC, not because you don't have the time to do so, but because you
are missing what I call essential when debugging a complex system like the
CLR, it's the source code and a deep understanding of the internals of the
CLR and the underlying platform. This is something you have to leave to the
CLR team at MS, really.
Allocating too many and/or too large objects ( too complex and/or too large
object hierachy) is basically a design mistake , which can possibly lead to
runtime errors and/or bad performing applications (that is, it becomes a
bug), but they are no leaks. Note, that while such applications can fail on
some systems, they can perfectly run on system that have a lot of memory/cpu
resources available. Involuntarily keeping object references alive when done
with them is a latent bug, you can recover from, not a real leak.
A "real" leak is something you can't recover from, for example, an object
that stays allocated in the heap, despite you no longer have live references
to it in your code, there is little you can do about this and when it
happens it will go undetected, unless it happens regularly in a long running
process.

Willy.

[Christopher Ireland]

Willy Denoyette [MVP] wrote:

No, as I said, it was a bug in the Windows.Forms code.

Ah, Ok. I read your phrase like this: "[to] illustrate the impact of a bug
[in the GC] in the Windows.Forms code [NumericUpDown being a class in
Windows.Forms]". My mistake, sorry.

Again, you don't have to, nor can you even track down bugs in the
CLR's code like the GC, not because you don't have the time to do so,
but because you are missing what I call essential when debugging a
complex system like the CLR, it's the source code and a deep
understanding of the internals of the CLR and the underlying
platform. This is something you have to leave to the CLR team at MS,
really.

Yes, I can understand that.

Involuntarily keeping object references alive when done with them is
a latent bug, you can recover from, not a real leak.

That's certainly the way I saw it before entering into this thread :-)

A "real" leak is something you can't recover from, for example, an
object that stays allocated in the heap, despite you no longer have
live references to it in your code, there is little you can do about
this and when it happens it will go undetected, unless it happens
regularly in a long running process.

If there was a "real" leak, as it were, what symptoms could I detect using
perfmon's counters (sorry for being so repetitive)?

Thanks Willy, you really are helping me clarify my ideas on this matter!

--
Thank you,

Christopher Ireland

[Willy Denoyette [MVP]]

"Christophe r Ireland" <ci******@gmail .comwrote in message
news:28******** *************** ***********@mic rosoft.com...

Willy Denoyette [MVP] wrote:

>No, as I said, it was a bug in the Windows.Forms code.

Ah, Ok. I read your phrase like this: "[to] illustrate the impact of a bug
[in the GC] in the Windows.Forms code [NumericUpDown being a class in
Windows.Forms]". My mistake, sorry.

>Again, you don't have to, nor can you even track down bugs in the
CLR's code like the GC, not because you don't have the time to do so,
but because you are missing what I call essential when debugging a
complex system like the CLR, it's the source code and a deep
understandin g of the internals of the CLR and the underlying
platform. This is something you have to leave to the CLR team at MS,
really.

Yes, I can understand that.

>Involuntaril y keeping object references alive when done with them is
a latent bug, you can recover from, not a real leak.

That's certainly the way I saw it before entering into this thread :-)

>A "real" leak is something you can't recover from, for example, an
object that stays allocated in the heap, despite you no longer have
live references to it in your code, there is little you can do about
this and when it happens it will go undetected, unless it happens
regularly in a long running process.

If there was a "real" leak, as it were, what symptoms could I detect using
perfmon's counters (sorry for being so repetitive)?

Well, perfmon is just another tool, you can get a reasonable idea about your
allocation scheme by watching them, however, the real art is to know which
ones to look at and the ones to ignore, and the ones who might be plain
wrong!.
Memory management is a complex issue, really. Basically, all memory
allocations/de-allocations are made by the OS itself ( by THE memory
manager), every process in Windows starts with a "default heap" at process
creation time, all heaps allocated to a process are reclaimed by the OS when
the process terminates, don't expect to find leaks at this level, this is
really a robust piece of code in the kernel.
Program code (running in user space) can allocate (reserve) space from this
process heap or allocate (request) another (private) heap from the OS.
Program code allocates/de-allocates process heap space by means of a WIN32
API call, program code also calls the C runtime to further allocate memory
space from the aforementioned process heaps, the CLR memory allocator does
exactly that, he calls the OS to reserve a number of process heaps, while he
uses the C Runtime to reserve space from these process heaps for the GC
generational heaps (Gen0, 1, 2 and LOH), the GC allocates/de-allocates
object space from these "CG heaps".
So, you see you need to know exactly what you are after, are you looking for
CLR object allocation/de-allocation counters? then you need to watch the
"CLR Memory "counters (like the Gen0 size, if you interested in the
unmanaged memory consumption, you'll have to watch the Memory counters (eg.
Private Memory counters). Note however, that the GC heaps are taken from the
process heaps so their allocated space is also reflected by the heap
counters (Private Memory).
Again all this is something that takes time to grasp, if you can't spend
some time to learn all this, you'll have a hard time when debugging at a
reasonably low level, so if you don't think you need this knowledge, no
problem, spend most of your time by getting your design at the right level
and stay away from low level debugging, or make sure you know someone who
knows it, so you can get some help, instead of wasting precious time.

Willy.

[Christopher Ireland]

Willy Denoyette [MVP] wrote:

So, you see you need to know exactly what you are after, are you
looking for CLR object allocation/de-allocation counters? then you
need to watch the "CLR Memory "counters (like the Gen0 size, if you
interested in the unmanaged memory consumption, you'll have to watch
the Memory counters (eg. Private Memory counters). Note however, that
the GC heaps are taken from the process heaps so their allocated
space is also reflected by the heap counters (Private Memory).
Again all this is something that takes time to grasp, if you can't
spend some time to learn all this, you'll have a hard time when
debugging at a reasonably low level, so if you don't think you need
this knowledge, no problem, spend most of your time by getting your
design at the right level and stay away from low level debugging, or
make sure you know someone who knows it, so you can get some help,
instead of wasting precious time.

My I first thank you Willy for all the time and effort you've put into
helping me here. I hope you don't feel that I've wasted your precious time!
Funnily enough, one of the most interesting articles I've seen on generation
0, 1 and 2 GC collections is in the CLRProfiler.doc that comes with the
CLRProfiler (funnily enough <g>). Damn shame I can't get the CLRProfiler to
work under Vista (all I get it "Waiting for application to start common
language runtime")!

Not to worry. I've come to the stage at which I need to understand more
about this area and have ordered the book suggested to me by Chris Mullins
(ISBN-10: 0735621632
). I might have to bother you again once I've read it, but with any luck
I'll be able to let you get on with other things for now!

--
Thank you again,

Christopher Ireland

[Willy Denoyette [MVP]]

"Christophe r Ireland" <ci******@gmail .comwrote in message
news:1B******** *************** ***********@mic rosoft.com...

Willy Denoyette [MVP] wrote:

>So, you see you need to know exactly what you are after, are you
looking for CLR object allocation/de-allocation counters? then you
need to watch the "CLR Memory "counters (like the Gen0 size, if you
interested in the unmanaged memory consumption, you'll have to watch
the Memory counters (eg. Private Memory counters). Note however, that
the GC heaps are taken from the process heaps so their allocated
space is also reflected by the heap counters (Private Memory).
Again all this is something that takes time to grasp, if you can't
spend some time to learn all this, you'll have a hard time when
debugging at a reasonably low level, so if you don't think you need
this knowledge, no problem, spend most of your time by getting your
design at the right level and stay away from low level debugging, or
make sure you know someone who knows it, so you can get some help,
instead of wasting precious time.

My I first thank you Willy for all the time and effort you've put into
helping me here. I hope you don't feel that I've wasted your precious
time!

Not at all, that's why we are here.

Funnily enough, one of the most interesting articles I've seen on
generation 0, 1 and 2 GC collections is in the CLRProfiler.doc that comes
with the CLRProfiler (funnily enough <g>). Damn shame I can't get the
CLRProfiler to work under Vista (all I get it "Waiting for application to
start common language runtime")!

I haven't used the CLRProfiler on Vista, it's not really a product (the
profiler) I care about (nor does MS).

Not to worry. I've come to the stage at which I need to understand more
about this area and have ordered the book suggested to me by Chris Mullins
(ISBN-10: 0735621632
). I might have to bother you again once I've read it, but with any luck
I'll be able to let you get on with other things for now!

No problem, if you have more questions, there will always be someone to help
you out I guess.

Willy.