GC.Collect can be trusted?

[Ward Bekker]

Hi,

I'm wondering if the GC.Collect method really collects all objects
possible objects? Or is this still a "smart" process sometimes keeping
objects alive even if they can be garbage collected?

I need to know because I'm looking for memory leaks in an application.

It would be really helpful to be able to determine if an object after
manually invoking the GC.Collect is only kept alive because it still
being referenced by other alive objects in contrast to being alive
because the GC.Collect didn't found it necessary to collect the object,
even when asked explicitly.

Regards,

Ward

[DeveloperX]

On 19 Apr, 11:37, Ward Bekker <wDotbekker@RemoveThisequanimityDotnl>
wrote:

Hi,

I'm wondering if the GC.Collect method really collects all objects
possible objects? Or is this still a "smart" process sometimes keeping
objects alive even if they can be garbage collected?

I need to know because I'm looking for memory leaks in an application.

It would be really helpful to be able to determine if an object after
manually invoking the GC.Collect is only kept alive because it still
being referenced by other alive objects in contrast to being alive
because the GC.Collect didn't found it necessary to collect the object,
even when asked explicitly.

Regards,

Ward

I believe it attempts to reclaim all objects. If you've got a memory
leak though you might want to take a look at the CLR profiler

http://www.microsoft.com/downloads/d...displaylang=en

thats for 1.1, there's a link for 2.0 at the bottom..

[=?Utf-8?B?UGV0ZXIgQnJvbWJlcmcgW0MjIE1WUF0=?=]

Ward,
Using GC.Collect to attempt to identify memory leaks is not a realistic
solution. Memory Leaks are generally caused by developers. Especially when
they fail to dispose of objects that need to be disposed, or if they fail to
implement similar patterns on their own objects where necessary.
There are a number of memory profilers you can use to track down this type
of issue. As well, FXCop can help you to identify bad coding patterns.
Peter

--
Site: http://www.eggheadcafe.com
UnBlog: http://petesbloggerama.blogspot.com
Short urls & more: http://ittyurl.net


"Ward Bekker" wrote:

Hi,

I'm wondering if the GC.Collect method really collects all objects
possible objects? Or is this still a "smart" process sometimes keeping
objects alive even if they can be garbage collected?

I need to know because I'm looking for memory leaks in an application.

It would be really helpful to be able to determine if an object after
manually invoking the GC.Collect is only kept alive because it still
being referenced by other alive objects in contrast to being alive
because the GC.Collect didn't found it necessary to collect the object,
even when asked explicitly.

Regards,

Ward

[Ward Bekker]

Let me explain a bit more. I am using a memory profiler, and see several
live objects that should be collected at the time I created a memory
snapshot.

I need to be sure that they are _really_ live objects because of a
mistake in our own code, or that they just alive because the GC.Collect
can't be trusted.
Peter Bromberg [C# MVP] wrote:

Ward,
Using GC.Collect to attempt to identify memory leaks is not a realistic
solution. Memory Leaks are generally caused by developers. Especially when
they fail to dispose of objects that need to be disposed, or if they fail to
implement similar patterns on their own objects where necessary.
There are a number of memory profilers you can use to track down this type
of issue. As well, FXCop can help you to identify bad coding patterns.
Peter

[Christopher Ireland]

Ward Bekker wrote:

Hi,

I'm wondering if the GC.Collect method really collects all objects
possible objects? Or is this still a "smart" process sometimes keeping
objects alive even if they can be garbage collected?

I need to know because I'm looking for memory leaks in an application.

I think you will find the following article useful and interesting:
http://support.microsoft.com/kb/318263/en-us

"Because of the garbage collection package that is implemented in the
Microsoft .NET Framework, it is not possible to have a memory leak in
managed code. This suggests two questions: How then can a memory leak occur?
Why does it appear that you have a memory leak?"

--
Thank you,

Christopher Ireland

[Ward Bekker]

Hi Cristopher,

My definition of a memory leak for managed frameworks:

All objects that should be garbage collected, but can't because they are
still referenced by other objects that will not be garbage collected ;-)

Christopher Ireland wrote:

Ward Bekker wrote:

>Hi,

I'm wondering if the GC.Collect method really collects all objects
possible objects? Or is this still a "smart" process sometimes keeping
objects alive even if they can be garbage collected?

I need to know because I'm looking for memory leaks in an application.

I think you will find the following article useful and interesting:
http://support.microsoft.com/kb/318263/en-us

"Because of the garbage collection package that is implemented in the
Microsoft .NET Framework, it is not possible to have a memory leak in
managed code. This suggests two questions: How then can a memory leak
occur? Why does it appear that you have a memory leak?"

[Christopher Ireland]

Ward Bekker wrote:

My definition of a memory leak for managed frameworks:

All objects that should be garbage collected, but can't because they
are still referenced by other objects that will not be garbage
collected ;-)

The article I quoted suggests to me that there are no managed objects that
can't be garbage collected!

--
Thank you,

Christopher Ireland

[Christof Nordiek]

"Christopher Ireland" <ci******@gmail.comschrieb im Newsbeitrag
news:3B**********************************@microsof t.com...

"Because of the garbage collection package that is implemented in the
Microsoft .NET Framework, it is not possible to have a memory leak in
managed code. This suggests two questions: How then can a memory leak
occur? Why does it appear that you have a memory leak?"

Though garbage collection is very usefull to provent memory leaks, this
quote is a bit to optimistic. Sure ther can be memory leaks in managed code.
E.g. all static fields aren't collected. So any object directly or
indirectly referenced by a static field wouldn't be collected. Also any
variables very high in the call stack can prevent referenced object from
being collected, (though the lifetime of a variable can be shortened by the
compiler.)
By this a memory leak in managed code can easily construckted.

But it's very unlikely to happen in real code. And it's much 'harder' to do
it purposelessly in managed code, than in unmanaged colde.

Regards
Christof

[Christopher Ireland]

Christof Nordiek wrote:

Though garbage collection is very usefull to provent memory leaks,
this quote is a bit to optimistic. Sure ther can be memory leaks in
managed code. E.g. all static fields aren't collected. So any object
directly or indirectly referenced by a static field wouldn't be
collected. Also any variables very high in the call stack can prevent
referenced object from being collected, (though the lifetime of a
variable can be shortened by the compiler.)
By this a memory leak in managed code can easily construckted.

Interesting. Are you suggesting that the Microsoft Article 318263 I posted a
link to is somehow incorrect or misleading? Do you have any Microsoft links
which specifically refer to memory leaks in managed code?

This is an important area for me to clear up with my clients and I would
appreciate being able to quote Microsoft sources.

--
Thank you,

Christopher Ireland

[Nicholas Paldino [.NET/C# MVP]]

Ward,

That's the thing though, if the object that you think should be garbage
collected is referenced by another object which is rooted (meaning you are
holding onto it some other way) and so on, then your object should NOT be
garbage collected. This is well-defined behavior for GC systems.

If you feel that your object should be collected at any time, then what
you really want are weak references. Take a look at the WeakReference
class.

Also, remember, if you have a delegate that points to a method on an
instance, then that instance is not eligible for GC. So if you subscribed
to an event, and didn't unsubscribe, then that instance is going to live on
as long as the object that publishes the event lives. A lot of developers
forget about this.

Hope this helps.
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com
"Ward Bekker" <wDotbekker@RemoveThisequanimityDotnlwrote in message
news:eS**************@TK2MSFTNGP05.phx.gbl...

Hi Cristopher,

My definition of a memory leak for managed frameworks:

All objects that should be garbage collected, but can't because they are
still referenced by other objects that will not be garbage collected ;-)

Christopher Ireland wrote:

>Ward Bekker wrote:

>>Hi,

I'm wondering if the GC.Collect method really collects all objects
possible objects? Or is this still a "smart" process sometimes keeping
objects alive even if they can be garbage collected?

I need to know because I'm looking for memory leaks in an application.

I think you will find the following article useful and interesting:
http://support.microsoft.com/kb/318263/en-us

"Because of the garbage collection package that is implemented in the
Microsoft .NET Framework, it is not possible to have a memory leak in
managed code. This suggests two questions: How then can a memory leak
occur? Why does it appear that you have a memory leak?"

[Jon Skeet [C# MVP]]

On Apr 19, 3:42 pm, "Nicholas Paldino [.NET/C# MVP]"
<[e-mail]:

That's the thing though, if the object that you think should be garbage
collected is referenced by another object which is rooted (meaning you are
holding onto it some other way) and so on, then your object should NOT be
garbage collected. This is well-defined behavior for GC systems.

Yes, and I think that's Ward's point. He sees (in the profiler) an
object which he know should be garbage. There are three possibilities
here:

1) It is garbage, but it hasn't been collected.
2) It's not garbage, but should be, and there's a bug.
3) It's not garbage, and shouldn't be.

I believe the original question was trying to determine whether
calling GC.Collect() is a good way of removing option 1 from the
equation.
I suspect it's a good but not absolutely foolproof way. Calling
GC.Collect(), then GC.WaitForPendingFinalizers(), then GC.Collect() is
probably better, but I suspect there are still some situations where
it wouldn't help.

Jon

[Christof Nordiek]

"Christopher Ireland" <ci******@gmail.comschrieb im Newsbeitrag
news:0B**********************************@microsof t.com...

Christof Nordiek wrote:

>Though garbage collection is very usefull to provent memory leaks,
this quote is a bit to optimistic. Sure ther can be memory leaks in
managed code. E.g. all static fields aren't collected. So any object
directly or indirectly referenced by a static field wouldn't be
collected. Also any variables very high in the call stack can prevent
referenced object from being collected, (though the lifetime of a
variable can be shortened by the compiler.)
By this a memory leak in managed code can easily construckted.

Interesting. Are you suggesting that the Microsoft Article 318263 I posted
a link to is somehow incorrect or misleading? Do you have any Microsoft
links which specifically refer to memory leaks in managed code?

Atleast it is a bit inaccurate about what is a memory leak.

Look furhter down in the same article:

"Additionally, a project may only appear to have a memory leak. This
condition can occur if many large objects (such as DataTable objects) are
declared and then added to a collection (such as a DataSet). The resources
that these objects own may never be released, and the resources are left
alive for the whole run of the program. This appears to be a leak, but
actually it is just a symptom of the way that memory is being allocated in
the program.

For example, you have a DataSet. Every time that a new query is run, you add
a new DataTable element to that DataSet to hold the data that is returned.
If there are large amounts of data that you never dispose of, the data stays
alive as long as the DataSet is still in use. If this occurs enough times,
it is possible to run out of memory. This is not a memory leak, but instead
it is a problem in managing the memory."

If all the data loaded into the dataset mentioned is really needed still in
memory than this application simply needs so much memory and I's guess, it's
very badly designed.

On the other hand, if the older datatables aren't used anymore, but simply
remain in memory because they are referenced by the dataset, then I surely
would call this a memory leak.

What you should be cautious about is, if a shortly used objects are
referenced by a long living objects. For all if the objects are great and/or
there are much of them. Then should not remove those references to late or
forget to remove them at all.

HTH
Christof

[Peter Duniho]

On Thu, 19 Apr 2007 05:17:48 -0700, Christopher Ireland
<ci******@gmail.comwrote:

Ward Bekker wrote:

>My definition of a memory leak for managed frameworks:

All objects that should be garbage collected, but can't because they
are still referenced by other objects that will not be garbage
collected ;-)

The article I quoted suggests to me that there are no managed objects
that can't be garbage collected!

Anything to which a reference still exists "can't be garbage collected".
Which is good, because otherwise all of your objects would disappear the
first time the garbage collector ran. The garbage collector collects
anything that *can* be garbage collected, and so obviously there must also
be things that "can't be garbage collected" (that is, those things that
you're still using).

That's what Ward was talking about.

You obviously are really talking about "UNREFERENCED managed objects that
can't be garbage collected", and I'd agree there are none of those. But
it seems to me that Ward made it pretty clear that's not what he's talking
about. He's talking about objects to which he still has references, but
*shouldn't*, and so they stay allocated even though they should have been
released.

It's a fundamentally different way of looking at the idea of a "memory
leak", but it's pretty much the only kind of memory leak you'd have with
managed objects and is still an example of a coding error (it's just that
the references are being kept when they shouldn't be, instead of being
lost when they shouldn't be :) ).

I hope that clarifies things.

Pete

[Jon Skeet [C# MVP]]

Christof Nordiek <[e-mail]:

Interesting. Are you suggesting that the Microsoft Article 318263 I posted
a link to is somehow incorrect or misleading? Do you have any Microsoft
links which specifically refer to memory leaks in managed code?

Atleast it is a bit inaccurate about what is a memory leak.

Look furhter down in the same article:

"Additionally, a project may only appear to have a memory leak. This
condition can occur if many large objects (such as DataTable objects) are
declared and then added to a collection (such as a DataSet). The resources
that these objects own may never be released, and the resources are left
alive for the whole run of the program. This appears to be a leak, but
actually it is just a symptom of the way that memory is being allocated in
the program.

Basically what it's saying is that bugs can cause things which look
like memory leaks. Quite what it would include in the category of
memory leaks in unmanaged code if it excludes bugs is beyond me... a
memory leak is a bug pretty much by definition.

In short, it's harder to have memory leaks in managed code, but far
from impossible.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Christopher Ireland]

Christof Nordiek wrote:

Look furhter down in the same article:

>On the other hand, if the older datatables aren't used anymore, but simply
remain in memory because they are referenced by the dataset, then I surely
would call this a memory leak.

The article also says:
"Each time that this occurs, the amount of memory that the program is using
increases. The memory does not decrease until the end of the program or the
release of the objects from the collection. When you watch the program on a
performance monitor, this appears to be a memory leak, but it is not. The
program still has control over the memory but has chosen not to release it.
The fact that the program still has control prevents this from being a
memory leak, but the fact that the program keeps increasing the amount of
memory used can make it appear to be a memory leak. "

This suggests to me that a memory leak is only a memory leak when the
program loses control over the memory. As far as I understand, in a managed
environment this never happens (except in the case of a bug in the managed
environment itself).

This does not excuse, however, badly designed managed code which keep
objects alive unnecessarily and which gives the *appearance* of a memory
leak.
--
Thank you,

Christopher Ireland

[Christopher Ireland]

Peter Duniho wrote:

That's what Ward was talking about.

Thank you Peter, I think that's how I understood it.

You obviously are really talking about "UNREFERENCED managed objects
that can't be garbage collected

No, I'm not.

>But it seems to me that Ward made it pretty clear that's not
what he's talking about. He's talking about objects to which he
still has references, but *shouldn't*, and so they stay allocated
even though they should have been released.

AFAIK, this can only mean a bug in the GC. Please see my reply to Christof.

It's a fundamentally different way of looking at the idea of a "memory
leak", but it's pretty much the only kind of memory leak you'd have
with managed objects and is still an example of a coding error (it's
just that the references are being kept when they shouldn't be,
instead of being lost when they shouldn't be :) ).

As I mentioned to Christof, I'm not excusing badly designed managed code and
I agree that such code can give rise to the *appearance* of a memory leak.

--
Thank you,

Christopher Ireland

[Christopher Ireland]

Jon Skeet [C# MVP] wrote:

Basically what it's saying is that bugs can cause things which look
like memory leaks.

That's certainly how I understood it.

>Quite what it would include in the category of
memory leaks in unmanaged code if it excludes bugs is beyond me... a
memory leak is a bug pretty much by definition.

In unmanaged code it would be a bug in your code, in managed code it would
be a bug in the GC, no?

In short, it's harder to have memory leaks in managed code, but far
from impossible.

I agree, but I think I was simply trying to say that real memory leaks in
managed code are due to bugs in the framework rather than bugs in your code
(making the distinction between bugs and badly designed code here (somehow
<g>)).

--
Thank you,

Christopher Ireland

[Jon Skeet [C# MVP]]

Christopher Ireland <ci******@gmail.comwrote:

Basically what it's saying is that bugs can cause things which look
like memory leaks.

That's certainly how I understood it.

Quite what it would include in the category of
memory leaks in unmanaged code if it excludes bugs is beyond me... a
memory leak is a bug pretty much by definition.

In unmanaged code it would be a bug in your code, in managed code it would
be a bug in the GC, no?

No - the kind of leak they're talking about is caused by a bug in your
code, by keeping a reference in another (uncollected) object.

As for bugs in the GC - a memory leak due to that would be similar to a
memory leak in MFC or a 3rd party library.

In short, it's harder to have memory leaks in managed code, but far
from impossible.

I agree, but I think I was simply trying to say that real memory leaks in
managed code are due to bugs in the framework rather than bugs in your code
(making the distinction between bugs and badly designed code here (somehow
<g>)).

If they *are* saying that, then I disagree - it's easy to assign
something to a static variable and forget about it. You'll then
potentially have what looks like a memory leak, and it would certainly
be due to a bug in your code. Not a failure to call free(), but a
failure to effectively make the object eligible for garbage collection.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Jon Skeet [C# MVP]]

Christopher Ireland <ci******@gmail.comwrote:

But it seems to me that Ward made it pretty clear that's not
what he's talking about. He's talking about objects to which he
still has references, but *shouldn't*, and so they stay allocated
even though they should have been released.

AFAIK, this can only mean a bug in the GC. Please see my reply to Christof.

No, it doesn't mean a bug in the GC. It almost certainly means a bug in
Ward's code, where he's got a reference to an object even though his
design (or whatever) says that he shouldn't. I believe he's trying to
track down that bug. Read Peter's sentence very carefully:

<quote>
He's talking about objects to which he still has references, but
*shouldn't*, and so they stay allocated even though they should have
been released.
</quote>

If he still has references to the object, it shouldn't be garbage
collected, so it can't be a GC bug.

It's a fundamentally different way of looking at the idea of a "memory
leak", but it's pretty much the only kind of memory leak you'd have
with managed objects and is still an example of a coding error (it's
just that the references are being kept when they shouldn't be,
instead of being lost when they shouldn't be :) ).

As I mentioned to Christof, I'm not excusing badly designed managed
code and I agree that such code can give rise to the *appearance* of
a memory leak.

And that's the situation I believe Ward is in.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Jon Skeet [C# MVP]]

Christopher Ireland <ci******@gmail.comwrote:

<snip>

This suggests to me that a memory leak is only a memory leak when the
program loses control over the memory. As far as I understand, in a managed
environment this never happens (except in the case of a bug in the managed
environment itself).

That seems like a fairly arbitrary definition of "memory leak" to me (I
know it's not yours) and not a terribly useful one. I find the
wikipedia statement clearer:

<quote>
In computer science, a memory leak is a particular kind of
unintentional memory consumption by a computer program where the
program fails to release memory when no longer needed.
</quote>

Certainly a *user* couldn't care less whether their memory is being
eaten due to references that are still available or due to memory which
no code knows about - and it doesn't make huge odds when it comes to
debugging, either.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Christopher Ireland]

Jon Skeet [C# MVP] wrote:

If they *are* saying that, then I disagree - it's easy to assign
something to a static variable and forget about it. You'll then
potentially have what looks like a memory leak, and it would certainly
be due to a bug in your code. Not a failure to call free(), but a
failure to effectively make the object eligible for garbage
collection.

I agree. It's a question of how terminology is understood. I understand that
in an environment where memory is managed, like the .NET Framework, then
memory which is not released when the application terminates is memory that
has failed to be effectively managed by the said environment.

Managed memory which accumulates while the application is running is not
what I would understand as a memory leak (the memory hasn't "leaked" out of
control of the application) but certainly looks like one and is due, as you
say, to not making the relevant objects eligible for garbage collection.

--
Thank you,

Christopher Ireland

[Christopher Ireland]

Jon Skeet [C# MVP] wrote:

No, it doesn't mean a bug in the GC. It almost certainly means a bug
in Ward's code, where he's got a reference to an object even though
his design (or whatever) says that he shouldn't. I believe he's
trying to track down that bug. Read Peter's sentence very carefully:

<quote>
He's talking about objects to which he still has references, but
*shouldn't*, and so they stay allocated even though they should have
been released.
</quote>

You're right, I hadn't read the sentence as carefully as I should have done
and yes, re-reading it makes it clear that the bug is not in the GC.

--
Thank you,

Christopher Ireland

"Peace comes from within. Do not seek it without."
Siddhartha Gautama

[Christopher Ireland]

Jon Skeet [C# MVP] wrote:

Certainly a *user* couldn't care less whether their memory is being
eaten due to references that are still available or due to memory
which no code knows about - and it doesn't make huge odds when it
comes to debugging, either.

No, but it does make a difference to whether or not you can effect a change
in how the memory is being used. If the memory is being used by references
that are still available then you can change it, if it due to a bug in the
GC then you can't. In an unmanaged environment you are completely
responsible for memory management and can therefore always effect a change,
AFAIK.

--
Thank you,

Christopher Ireland

[Jon Skeet [C# MVP]]

Christopher Ireland <ci******@gmail.comwrote:

Jon Skeet [C# MVP] wrote:

Certainly a *user* couldn't care less whether their memory is being
eaten due to references that are still available or due to memory
which no code knows about - and it doesn't make huge odds when it
comes to debugging, either.

No, but it does make a difference to whether or not you can effect a change
in how the memory is being used. If the memory is being used by references
that are still available then you can change it, if it due to a bug in the
GC then you can't. In an unmanaged environment you are completely
responsible for memory management and can therefore always effect a change,
AFAIK.

No, in neither case are you completely responsible.

Very, *very* few unmanaged Windows programs don't use either third
party libraries or Win32 calls. I view a memory leak in the GC as being
comparable to a memory leak in those third party libraries or Win32
calls - basically, you have a problem which is hard to deal with in
either case.

Likewise, in both managed code and unmanaged code, memory leaks are
more *likely* to be due to bugs in your own code.

The tools you might use to track down such problems will clearly be
different, but both kinds of leak are definitely possible in both
situations.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Christopher Ireland]

Jon Skeet [C# MVP] wrote:

No, in neither case are you completely responsible.

One is always completely responsible for one's code and it was never my
intention to suggest otherwise. What I'm saying is that in a managed memory
environment you are not responsible for the direct allocation and
deallocation of memory and therefore have to rely on an intermediate layer
that is. What this means is that there are instances in the running of a
program where an object is not in scope and where a programmer cannot
directly destroy it, giving rise to this object "unnecessarily" occupying
memory and therefore the appearance of a memory leak.

Very, *very* few unmanaged Windows programs don't use either third
party libraries or Win32 calls. I view a memory leak in the GC as
being comparable to a memory leak in those third party libraries or
Win32 calls - basically, you have a problem which is hard to deal
with in either case.

Likewise, in both managed code and unmanaged code, memory leaks are
more *likely* to be due to bugs in your own code.

I think I've been mistaken in overly referring to bugs in the GC and I agree
with you that this is by far the least likely scenario.

The tools you might use to track down such problems will clearly be
different, but both kinds of leak are definitely possible in both
situations.

Yes, the question is: once you know which object is unnecessarily occupying
space, can you guarantee its destruction before the program terminates? I
think this is the question that Ward is going to be asking himself once he's
tracked his object down.

--
Thank you,

Christopher Ireland

[Jon Skeet [C# MVP]]

Christopher Ireland <ci******@gmail.comwrote:

Jon Skeet [C# MVP] wrote:

No, in neither case are you completely responsible.

One is always completely responsible for one's code and it was never my
intention to suggest otherwise. What I'm saying is that in a managed memory
environment you are not responsible for the direct allocation and
deallocation of memory and therefore have to rely on an intermediate layer
that is. What this means is that there are instances in the running of a
program where an object is not in scope and where a programmer cannot
directly destroy it, giving rise to this object "unnecessarily" occupying
memory and therefore the appearance of a memory leak.

Indeed. My point was that even in the unmanaged world, while you will
be directly responsible for *some* (quite possibly most) of the memory
allocation, you're unlikely to be responsible for *all* of it.

Even if you're calling malloc()/free(), that's still going through an
intermediate layer. I believe free() in Windows had a subtle and rare
bug at some point, for instance...

<snip>

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Willy Denoyette [MVP]]

"Ward Bekker" <wDotbekker@RemoveThisequanimityDotnlwrote in message
news:eS**************@TK2MSFTNGP05.phx.gbl...

Hi Cristopher,

My definition of a memory leak for managed frameworks:

All objects that should be garbage collected, but can't because they are still referenced
by other objects that will not be garbage collected ;-)

Objects that are still referenced should not be GC, that would be a serious bug if the GC
collected such objects ;-)
The problem you are describing is not a real "memory leak", the problem is that you don't
know who's keeping a reference to the object, so you aren't able to release the object by
setting it's reference to null, this is an application bug disguised as a leak.
A "real leak " is part of the memory, occupied by a "non referenced object" staying
allocated on the GC heap after a GC run, if the GC can't deallocate the memory it will stay
in the heap until the process terminates, no-one is still under control of this chunk of
memory. Or otherwise, a leak in the managed heap is the result of a CLR bug, possibly a GC
bug.

Willy.

[Christopher Ireland]

Jon Skeet [C# MVP] wrote:

Indeed. My point was that even in the unmanaged world, while you will
be directly responsible for *some* (quite possibly most) of the memory
allocation, you're unlikely to be responsible for *all* of it.

Even if you're calling malloc()/free(), that's still going through an
intermediate layer. I believe free() in Windows had a subtle and rare
bug at some point, for instance...

Sure, although I think there is a fundamental qualitative and quantitive
difference between the intermediate layer that effects memory in unmanaged
applications and in managed applications. Non-deterministic memory
management by its very nature means that programmers cannot make completely
predictable changes to memory allocation through it.

--
Thank you,

Christopher Ireland

[Ward Bekker]

Hehe, my question caused quite some discussion. Thank you for your help!

Let me try to clarify a bit more:

1. I'm hunting for bugs in my own code, specifically for object graphs
that are still connected to the root object but are not needed any more.
These objects are correctly not garbage collected because the code did
not dereference them. I use among others Ants Profiler to look what
objects are still in memory and alive.

2. I want to make sure that all objects that can be garbage collected (
disconnected from the root object) _are_ not longer alive. The
documentation is not very clear that GC.Collect will actually throw out
the thrash.

Jon Skeet maybe explains it better what I'm trying to do: See
news://news.microsoft.com:119/11****...oglegroups.com

Besides Jon's suggestion, I also found this method:
GC.GetTotalMemory(true).

According to the documentation, the true argument tells to wait for the
garbage collection to finish before returning so the result will be more
accurate. I noticed a delay when there really was stuff to collect when
executing.

Maybe I should do both ways, just to make sure ;-)

Thank you very much,

Ward

[mpetrotta]

On Apr 20, 4:59 am, Ward Bekker <wDotbekker@RemoveThisequanimityDotnl>
wrote:

2. I want to make sure that all objects that can be garbage collected (
disconnected from the root object) _are_ not longer alive. The
documentation is not very clear that GC.Collect will actually throw out
the thrash.

Seems pretty clear to me, if lacking in detail:
"However, the Collect method does not guarantee that all inaccessible
memory is reclaimed."

Michael

[Willy Denoyette [MVP]]

<mp*******@gmail.comwrote in message
news:11*********************@n59g2000hsh.googlegro ups.com...

On Apr 20, 4:59 am, Ward Bekker <wDotbekker@RemoveThisequanimityDotnl>
wrote:

>2. I want to make sure that all objects that can be garbage collected (
disconnected from the root object) _are_ not longer alive. The
documentation is not very clear that GC.Collect will actually throw out
the thrash.

Seems pretty clear to me, if lacking in detail:
"However, the Collect method does not guarantee that all inaccessible
memory is reclaimed."

Reclaimed by who? The task of the GC is to collect the inaccessible objects and if possible
compact the heap. It's not the task of the GC to return the memory to the OS, the GC can't
even do that, it's up to the memory manager (another CLR component) to keep track of the
extra segments allocated from the process heap and (eventually) return these when they don't
contain any GC heap data (managed objects and other CLR data).
Anyway, the memory manager won't ever return the default process heap segments occupied by
the managed heap (32 MB or 64 MB ) to the OS

Willy.

[mpetrotta]

On Apr 20, 1:04 pm, "Willy Denoyette [MVP]"
<[e-mail]:

<[e-mail] in message

news:11*********************@n59g2000hsh.googlegro ups.com...

On Apr 20, 4:59 am, Ward Bekker <wDotbekker@RemoveThisequanimityDotnl>
wrote:

2. I want to make sure that all objects that can be garbage collected (
disconnected from the root object) _are_ not longer alive. The
documentation is not very clear that GC.Collect will actually throw out
the thrash.

Seems pretty clear to me, if lacking in detail:
"However, the Collect method does not guarantee that all inaccessible
memory is reclaimed."

Reclaimed by who? The task of the GC is to collect the inaccessible objects and if possible
compact the heap. It's not the task of the GC to return the memory to the OS, the GC can't
even do that, it's up to the memory manager (another CLR component) to keep track of the
extra segments allocated from the process heap and (eventually) return these when they don't
contain any GC heap data (managed objects and other CLR data).
Anyway, the memory manager won't ever return the default process heap segments occupied by
the managed heap (32 MB or 64 MB ) to the OS

I don't see that it matters. The OP is using Ants Profiler (and its
"Live Objects" view) to profile his memory usage. I'm not very
familiar with that tool, so I just tried out the demo. That view
lists all objects on the managed heap (collected or not, apparently).
Once collected, objects should no longer appear in that view. Whether
the memory they consumed is still being used by the CLR is irrelevant
(for the OP, at any rate).

Michael

[Willy Denoyette [MVP]]

<mp*******@gmail.comwrote in message
news:11**********************@e65g2000hsc.googlegr oups.com...

On Apr 20, 1:04 pm, "Willy Denoyette [MVP]"
<[e-mail]:

><[e-mail] in message

news:11*********************@n59g2000hsh.googlegr oups.com...

On Apr 20, 4:59 am, Ward Bekker <wDotbekker@RemoveThisequanimityDotnl>
wrote:
2. I want to make sure that all objects that can be garbage collected (
disconnected from the root object) _are_ not longer alive. The
documentation is not very clear that GC.Collect will actually throw out
the thrash.

Seems pretty clear to me, if lacking in detail:
"However, the Collect method does not guarantee that all inaccessible
memory is reclaimed."

Reclaimed by who? The task of the GC is to collect the inaccessible objects and if
possible
compact the heap. It's not the task of the GC to return the memory to the OS, the GC
can't
even do that, it's up to the memory manager (another CLR component) to keep track of the
extra segments allocated from the process heap and (eventually) return these when they
don't
contain any GC heap data (managed objects and other CLR data).
Anyway, the memory manager won't ever return the default process heap segments occupied
by
the managed heap (32 MB or 64 MB ) to the OS

I don't see that it matters. The OP is using Ants Profiler (and its
"Live Objects" view) to profile his memory usage. I'm not very
familiar with that tool, so I just tried out the demo. That view
lists all objects on the managed heap (collected or not, apparently).
Once collected, objects should no longer appear in that view. Whether
the memory they consumed is still being used by the CLR is irrelevant
(for the OP, at any rate).

You have too much trust in such tools. First, collected objects should not appear in that
list, note however that NO single tool is able to show in real time what's happening in the
CLR, they are simply relying on the counter data provided by the CLR, these (and other info)
are supplied by the CLR via a call-back interface, however, the data isn't updated in
real-time fashion, more whenever you attach a profiler you are introducing side effects, the
CLR behaves differently when a profiler or debugger is attached.
A profiler by the way is no means to detect memory leaks, they are meant to profile
application behavior, this includes allocation patterns (what parts of the program is
allocating what kind and how many objects, what's their life time etc..), but they can't
show you leaked memory, debuggers are (and here I mean unmanaged debuggers not managed ones)
meant to find these kind of bugs, but keep in mind they too are quite intrusive, the crate
side effects the influence the behavior of the application. Only way to find a leak is by
taking a snapshot dump and investigate that dump off line using the right tools for managed
applications.
Willy.
..

[Christopher Ireland]

Willy,

Only way to find a leak is by taking a
snapshot dump and investigate that dump off line using the right
tools for managed applications.

Would you mind telling me how to create such a dump and which tools you
would use to analyse it?

--
Thank you,

Christopher Ireland

[Chris Mullins [MVP]]

I talk a fair bit about how to get a dump here:
http://www.coversant.net/Coversant/B...8/Default.aspx

To analyze the dump, you can either use:
- WinDbg + Son of Strike (as I talk about in the blog). This is very painful
though.
- Scitech Memory Profiler (http://memprofiler.com/). Learning their GUI will
take a while, but once you do, the power is nothing short of amazing.

The SciTech one will also compare dumps - so you can get you app in a steady
state, capture a dump, perform a an action, and capture a 2nd dump. I've
gotten alot of use out of this...

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"Christopher Ireland" <ci******@gmail.comwrote in message
news:56**********************************@microsof t.com...

Willy,

>Only way to find a leak is by taking a
snapshot dump and investigate that dump off line using the right
tools for managed applications.

Would you mind telling me how to create such a dump and which tools you
would use to analyse it?

--
Thank you,

Christopher Ireland

[Christopher Ireland]

Chris,

- Scitech Memory Profiler (http://memprofiler.com/). Learning their GUI
will take a while, but once you do, the power is nothing short of amazing.

The SciTech one will also compare dumps - so you can get you app in a
steady state, capture a dump, perform a an action, and capture a 2nd dump.
I've gotten alot of use out of this...

Many thanks for your answer! In fact, I've been using AQTime
(http://www.automatedqa.com/products/aqtime/) to profile the performance of
my applications for some time now, but this is the first time I've wanted to
use its allocation profiler. It's interesting to note what AutomatedQA say
about memory leaks in .NET applications here:
http://automatedqa.com/techpapers/ne...n_profiler.asp

They seem to suggest that memory leaks in .NET are a problem of the GC not
working properly. Would you agree with this analysis?

Thank you,

Christopher Ireland.

[Chris Mullins [MVP]]

I don't agree with that at all, really.

The scenario in their article they are repeatadly calling a bug, isn't a GC
bug at all - it's a bug in the UpDown control. That control isn't removing
an event handler, thus keeping the control rooted for way longer than the
developer intended. The GC is working perfectly. It's a great article on
"how to" for tracking down a leak though....

I've yet to come across a single instance where the GC wasn't working as
designed. Given the type of work I do - large scale, high performance, high
availability server applications - and the amount of time I spend in
profilers (both performance and memory) - I know the area pretty well.

There have been a number of times where apps were leaking memory, but in all
cases it turned out to be developer bugs. Sometimes very subtle application
bugs or race conditions, but bugs nonetheless.

In this day and age, GC is pretty good. It lets junior devs build small apps
in a "fire and forget" type of way. Unfortuantly, as complexity grows, that
"fire and forget" approach has to be replaced with a much deeper
understanding of how things work.

At the end of the day, as professional developers building complex
applications, we need to have a good understanding to how Garbage Collection
works. We need to understand Dispose patterns, Finalization, how the Managed
Heap works, and other related technologies. We need to understand what a
root path is, and how it keeps things from being collected. At the more
esoteric level, if you're building really complex apps, you need someone
around who understands how Pinning and Fragmentation work, Weak References,
Object Resurrection, the differences between the Worksation and Server CLR,
what Concurrent GC is, what the Large Object Heap is, and all the other good
struff that's in there.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"Christopher Ireland" <ci******@gmail.comwrote in message
news:uG**************@TK2MSFTNGP03.phx.gbl...

Chris,

>- Scitech Memory Profiler (http://memprofiler.com/). Learning their GUI
will take a while, but once you do, the power is nothing short of
amazing.

The SciTech one will also compare dumps - so you can get you app in a
steady state, capture a dump, perform a an action, and capture a 2nd
dump. I've gotten alot of use out of this...

Many thanks for your answer! In fact, I've been using AQTime
(http://www.automatedqa.com/products/aqtime/) to profile the performance
of my applications for some time now, but this is the first time I've
wanted to use its allocation profiler. It's interesting to note what
AutomatedQA say about memory leaks in .NET applications here:
http://automatedqa.com/techpapers/ne...n_profiler.asp

They seem to suggest that memory leaks in .NET are a problem of the GC not
working properly. Would you agree with this analysis?

Thank you,

Christopher Ireland.

[Christopher Ireland]

Chris Mullins [MVP] wrote:

There have been a number of times where apps were leaking memory, but
in all cases it turned out to be developer bugs. Sometimes very
subtle application bugs or race conditions, but bugs nonetheless.

Can you please confirm to me that with the GC working well, any memory leaks
that occur while the application is running leave no "footprint" in memory
when the application is closed? Another thing I would be very grateful if
you could clarify for me is whether memory leaks which occur when the (100%
managed) application is running can be seen in perfmon's memory counter (or
the task manager's physical memory usage history, if it isn't the same
thing).

In this day and age, GC is pretty good. It lets junior devs build
small apps in a "fire and forget" type of way. Unfortuantly, as
complexity grows, that "fire and forget" approach has to be replaced
with a much deeper understanding of how things work.

Yes, I agree.

At the end of the day, as professional developers building complex
applications, we need to have a good understanding to how Garbage
Collection works. We need to understand Dispose patterns,
Finalization, how the Managed Heap works, and other related
technologies. We need to understand what a root path is, and how it
keeps things from being collected. At the more esoteric level, if
you're building really complex apps, you need someone around who
understands how Pinning and Fragmentation work, Weak References,
Object Resurrection, the differences between the Worksation and
Server CLR, what Concurrent GC is, what the Large Object Heap is, and
all the other good struff that's in there.

Great, these are exactly the kind of keywords I was after. Could I please
try your patience one more time and ask you for what you consider to be a
good link (or even book!) on the matter?

--
Many thanks,

Christopher Ireland

[Willy Denoyette [MVP]]

"Chris Mullins [MVP]" <cm******@yahoo.comwrote in message
news:ua**************@TK2MSFTNGP05.phx.gbl...

>I don't agree with that at all, really.

The scenario in their article they are repeatadly calling a bug, isn't a
GC bug at all - it's a bug in the UpDown control. That control isn't
removing an event handler, thus keeping the control rooted for way longer
than the developer intended. The GC is working perfectly. It's a great
article on "how to" for tracking down a leak though....

I've yet to come across a single instance where the GC wasn't working as
designed. Given the type of work I do - large scale, high performance,
high availability server applications - and the amount of time I spend in
profilers (both performance and memory) - I know the area pretty well.

There have been a number of times where apps were leaking memory, but in
all cases it turned out to be developer bugs. Sometimes very subtle
application bugs or race conditions, but bugs nonetheless.

In this day and age, GC is pretty good. It lets junior devs build small
apps in a "fire and forget" type of way. Unfortuantly, as complexity
grows, that "fire and forget" approach has to be replaced with a much
deeper understanding of how things work.

At the end of the day, as professional developers building complex
applications, we need to have a good understanding to how Garbage
Collection works. We need to understand Dispose patterns, Finalization,
how the Managed Heap works, and other related technologies. We need to
understand what a root path is, and how it keeps things from being
collected. At the more esoteric level, if you're building really complex
apps, you need someone around who understands how Pinning and
Fragmentation work, Weak References, Object Resurrection, the differences
between the Worksation and Server CLR, what Concurrent GC is, what the
Large Object Heap is, and all the other good struff that's in there.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

Note also, that this article was written years ago, and is based on the
first version of the framework and never updated since then.
These are the kind of "look how great our product is, it can even find bugs
in MSFT's products" articles, but they did not find anything, this bug was
known at the time they ran the profiler, all they did was illustrate the
impact of a bug in the Windows.Forms code on the allocation pattern.
You won't be able to find these kind of "bugs" using just a profiler, you
need a debugger, a great deal of understanding how the GC and the CLR works,
a lot of experience in debugging complex systems plus quite some luck to
finds small leaks like these. Granted, a profiler is something you need in
your toolbox, it's a great tool to illustrate performance behaviors and
uncover possible bottlenecks, it can help you better understand your object
allocation patterns, where you allocate too many or to large objects, so you
can adapt your algorithms and allocation needs, but you won't ever be able
to detect small managed memory leaks by using a profiler alone.

Willy.

"Christopher Ireland" <ci******@gmail.comwrote in message
news:uG**************@TK2MSFTNGP03.phx.gbl...

>Chris,

>>- Scitech Memory Profiler (http://memprofiler.com/). Learning their GUI
will take a while, but once you do, the power is nothing short of
amazing.

The SciTech one will also compare dumps - so you can get you app in a
steady state, capture a dump, perform a an action, and capture a 2nd
dump. I've gotten alot of use out of this...

Many thanks for your answer! In fact, I've been using AQTime
(http://www.automatedqa.com/products/aqtime/) to profile the performance
of my applications for some time now, but this is the first time I've
wanted to use its allocation profiler. It's interesting to note what
AutomatedQA say about memory leaks in .NET applications here:
http://automatedqa.com/techpapers/ne...n_profiler.asp

They seem to suggest that memory leaks in .NET are a problem of the GC
not working properly. Would you agree with this analysis?

Thank you,

Christopher Ireland.

[Jon Skeet [C# MVP]]

Christopher Ireland <ci******@gmail.comwrote:

There have been a number of times where apps were leaking memory, but
in all cases it turned out to be developer bugs. Sometimes very
subtle application bugs or race conditions, but bugs nonetheless.

Can you please confirm to me that with the GC working well, any memory leaks
that occur while the application is running leave no "footprint" in memory
when the application is closed?

If they don't, that's an operating system failure more than a CLR
failure. An operating system really, really, really should release any
memory held by a process when the process exits (assuming it's not
deliberately sharing memory with another process etc).

Another thing I would be very grateful if you could clarify for me is
whether memory leaks which occur when the (100% managed) application
is running can be seen in perfmon's memory counter (or the task
manager's physical memory usage history, if it isn't the same thing).

They may be visible there. It's a lot more reliable to use the CLR
performance counters in perfmon though.

<snip>

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Chris Mullins [MVP]]

"Christopher Ireland" Wrote:

Can you please confirm to me that with the GC working well, any memory
leaks that occur while the application is running leave no "footprint" in
memory when the application is closed?

All current versions of Windows are pretty good about cleaning things up
once a process dies. I can't remember the last time I had a problem with
resources remaining allocated after I had exited a process.

Another thing I would be very grateful if you could clarify for me is
whether memory leaks which occur when the (100% managed) application is
running can be seen in perfmon's memory counter (or the task manager's
physical memory usage history, if it isn't the same thing).

Why do I feel like I'm being cross examined by an attorney looking to put
words in my mouth? :)

Perform's memory counter & Task Manager's memory viewer are terrible for
this stuff. Use the detailed CLR Performance Counters for information, it's
far more accurate.

Could I please try your patience one more time and ask you for what you
consider to be a good link (or even book!) on the matter?

http://msdn.microsoft.com/msdnmag/is...I/default.aspx
http://msdn.microsoft.com/msdnmag/is...2/default.aspx

This book has the best chapter on Garbage Collection that I've read. This is
where I learned most of what I know about how GC works:
http://www.amazon.com/Applied-Micros.../dp/0735614229

I've heard many good things about this one, but haven't read it myself:
http://www.amazon.com/o/ASIN/0735621...8QKPN1AZERGY02

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

[Christopher Ireland]

Willy Denoyette [MVP] wrote:

Note also, that this article was written years ago, and is based on
the first version of the framework and never updated since then.
These are the kind of "look how great our product is, it can even
find bugs in MSFT's products" articles, but they did not find
anything, this bug was known at the time they ran the profiler, all
they did was illustrate the impact of a bug in the Windows.Forms code
on the allocation pattern.

So it was a bug in the GC then?

You won't be able to find these kind of "bugs" using just a profiler,
you need a debugger, a great deal of understanding how the GC and the
CLR works, a lot of experience in debugging complex systems plus
quite some luck to finds small leaks like these. Granted, a profiler
is something you need in your toolbox, it's a great tool to
illustrate performance behaviors and uncover possible bottlenecks, it
can help you better understand your object allocation patterns, where
you allocate too many or to large objects, so you can adapt your
algorithms and allocation needs, but you won't ever be able to detect
small managed memory leaks by using a profiler alone.

I can tell you now that I have very little time to spend tracking down bugs
in the GC. What I'm after is being able to track down memory leaks that I
unwittingly create in the code I write. The question is: are the bugs I
introduce into my code through the allocation of too many or too large
objects really "memory leaks" (please, this is not a trick question, I'm
really trying to figure out what's going on here :-) ?

--
Thank you,

Christopher Ireland

[Christopher Ireland]

Jon Skeet [C# MVP] wrote:

If they don't, that's an operating system failure more than a CLR
failure. An operating system really, really, really should release any
memory held by a process when the process exits (assuming it's not
deliberately sharing memory with another process etc).

Ah, I see, thank you.

They may be visible there. It's a lot more reliable to use the CLR
performance counters in perfmon though.

If there was a memory leak in my code, what sort of symptoms should I be
looking for in perfmon's CLR performance counters?

--
Thank you,

Christopher Ireland

[Christopher Ireland]

Chris Mullins [MVP] wrote:

All current versions of Windows are pretty good about cleaning things
up once a process dies. I can't remember the last time I had a
problem with resources remaining allocated after I had exited a
process.

Ok, thank you for the information.

Why do I feel like I'm being cross examined by an attorney looking to
put words in my mouth? :)

I'm sorry for giving you that impression, but it really isn't my intention
at all. I'm much more of a sheep in wolf's clothing than the other way round
:-)

Perform's memory counter & Task Manager's memory viewer are terrible
for this stuff. Use the detailed CLR Performance Counters for
information, it's far more accurate.

Ok. What sort of things could the CLR performance counters show me that
would lead me to believe that my code has a memory leak?

http://msdn.microsoft.com/msdnmag/is...I/default.aspx
http://msdn.microsoft.com/msdnmag/is...2/default.aspx

This book has the best chapter on Garbage Collection that I've read.
This is where I learned most of what I know about how GC works:
http://www.amazon.com/Applied-Micros.../dp/0735614229

I've heard many good things about this one, but haven't read it
myself:
http://www.amazon.com/o/ASIN/0735621...8QKPN1AZERGY02

Fantastic! Again, many thanks for the tips!

--
Thank you,

Christopher Ireland

[Willy Denoyette [MVP]]

"Christopher Ireland" <ci******@gmail.comwrote in message
news:3F**********************************@microsof t.com...

Willy Denoyette [MVP] wrote:

>Note also, that this article was written years ago, and is based on
the first version of the framework and never updated since then.
These are the kind of "look how great our product is, it can even
find bugs in MSFT's products" articles, but they did not find
anything, this bug was known at the time they ran the profiler, all
they did was illustrate the impact of a bug in the Windows.Forms code
on the allocation pattern.

So it was a bug in the GC then?

No, as I said, it was a bug in the Windows.Forms code.

>You won't be able to find these kind of "bugs" using just a profiler,
you need a debugger, a great deal of understanding how the GC and the
CLR works, a lot of experience in debugging complex systems plus
quite some luck to finds small leaks like these. Granted, a profiler
is something you need in your toolbox, it's a great tool to
illustrate performance behaviors and uncover possible bottlenecks, it
can help you better understand your object allocation patterns, where
you allocate too many or to large objects, so you can adapt your
algorithms and allocation needs, but you won't ever be able to detect
small managed memory leaks by using a profiler alone.

I can tell you now that I have very little time to spend tracking down
bugs in the GC. What I'm after is being able to track down memory leaks
that I unwittingly create in the code I write. The question is: are the
bugs I introduce into my code through the allocation of too many or too
large objects really "memory leaks" (please, this is not a trick question,
I'm really trying to figure out what's going on here :-) ?

Again, you don't have to, nor can you even track down bugs in the CLR's code
like the GC, not because you don't have the time to do so, but because you
are missing what I call essential when debugging a complex system like the
CLR, it's the source code and a deep understanding of the internals of the
CLR and the underlying platform. This is something you have to leave to the
CLR team at MS, really.
Allocating too many and/or too large objects ( too complex and/or too large
object hierachy) is basically a design mistake , which can possibly lead to
runtime errors and/or bad performing applications (that is, it becomes a
bug), but they are no leaks. Note, that while such applications can fail on
some systems, they can perfectly run on system that have a lot of memory/cpu
resources available. Involuntarily keeping object references alive when done
with them is a latent bug, you can recover from, not a real leak.
A "real" leak is something you can't recover from, for example, an object
that stays allocated in the heap, despite you no longer have live references
to it in your code, there is little you can do about this and when it
happens it will go undetected, unless it happens regularly in a long running
process.

Willy.

[Christopher Ireland]

Willy Denoyette [MVP] wrote:

No, as I said, it was a bug in the Windows.Forms code.

Ah, Ok. I read your phrase like this: "[to] illustrate the impact of a bug
[in the GC] in the Windows.Forms code [NumericUpDown being a class in
Windows.Forms]". My mistake, sorry.

Again, you don't have to, nor can you even track down bugs in the
CLR's code like the GC, not because you don't have the time to do so,
but because you are missing what I call essential when debugging a
complex system like the CLR, it's the source code and a deep
understanding of the internals of the CLR and the underlying
platform. This is something you have to leave to the CLR team at MS,
really.

Yes, I can understand that.

Involuntarily keeping object references alive when done with them is
a latent bug, you can recover from, not a real leak.

That's certainly the way I saw it before entering into this thread :-)

A "real" leak is something you can't recover from, for example, an
object that stays allocated in the heap, despite you no longer have
live references to it in your code, there is little you can do about
this and when it happens it will go undetected, unless it happens
regularly in a long running process.

If there was a "real" leak, as it were, what symptoms could I detect using
perfmon's counters (sorry for being so repetitive)?

Thanks Willy, you really are helping me clarify my ideas on this matter!

--
Thank you,

Christopher Ireland

[Willy Denoyette [MVP]]

"Christopher Ireland" <ci******@gmail.comwrote in message
news:28**********************************@microsof t.com...

Willy Denoyette [MVP] wrote:

>No, as I said, it was a bug in the Windows.Forms code.

Ah, Ok. I read your phrase like this: "[to] illustrate the impact of a bug
[in the GC] in the Windows.Forms code [NumericUpDown being a class in
Windows.Forms]". My mistake, sorry.

>Again, you don't have to, nor can you even track down bugs in the
CLR's code like the GC, not because you don't have the time to do so,
but because you are missing what I call essential when debugging a
complex system like the CLR, it's the source code and a deep
understanding of the internals of the CLR and the underlying
platform. This is something you have to leave to the CLR team at MS,
really.

Yes, I can understand that.

>Involuntarily keeping object references alive when done with them is
a latent bug, you can recover from, not a real leak.

That's certainly the way I saw it before entering into this thread :-)

>A "real" leak is something you can't recover from, for example, an
object that stays allocated in the heap, despite you no longer have
live references to it in your code, there is little you can do about
this and when it happens it will go undetected, unless it happens
regularly in a long running process.

If there was a "real" leak, as it were, what symptoms could I detect using
perfmon's counters (sorry for being so repetitive)?

Well, perfmon is just another tool, you can get a reasonable idea about your
allocation scheme by watching them, however, the real art is to know which
ones to look at and the ones to ignore, and the ones who might be plain
wrong!.
Memory management is a complex issue, really. Basically, all memory
allocations/de-allocations are made by the OS itself ( by THE memory
manager), every process in Windows starts with a "default heap" at process
creation time, all heaps allocated to a process are reclaimed by the OS when
the process terminates, don't expect to find leaks at this level, this is
really a robust piece of code in the kernel.
Program code (running in user space) can allocate (reserve) space from this
process heap or allocate (request) another (private) heap from the OS.
Program code allocates/de-allocates process heap space by means of a WIN32
API call, program code also calls the C runtime to further allocate memory
space from the aforementioned process heaps, the CLR memory allocator does
exactly that, he calls the OS to reserve a number of process heaps, while he
uses the C Runtime to reserve space from these process heaps for the GC
generational heaps (Gen0, 1, 2 and LOH), the GC allocates/de-allocates
object space from these "CG heaps".
So, you see you need to know exactly what you are after, are you looking for
CLR object allocation/de-allocation counters? then you need to watch the
"CLR Memory "counters (like the Gen0 size, if you interested in the
unmanaged memory consumption, you'll have to watch the Memory counters (eg.
Private Memory counters). Note however, that the GC heaps are taken from the
process heaps so their allocated space is also reflected by the heap
counters (Private Memory).
Again all this is something that takes time to grasp, if you can't spend
some time to learn all this, you'll have a hard time when debugging at a
reasonably low level, so if you don't think you need this knowledge, no
problem, spend most of your time by getting your design at the right level
and stay away from low level debugging, or make sure you know someone who
knows it, so you can get some help, instead of wasting precious time.

Willy.

[Christopher Ireland]

Willy Denoyette [MVP] wrote:

So, you see you need to know exactly what you are after, are you
looking for CLR object allocation/de-allocation counters? then you
need to watch the "CLR Memory "counters (like the Gen0 size, if you
interested in the unmanaged memory consumption, you'll have to watch
the Memory counters (eg. Private Memory counters). Note however, that
the GC heaps are taken from the process heaps so their allocated
space is also reflected by the heap counters (Private Memory).
Again all this is something that takes time to grasp, if you can't
spend some time to learn all this, you'll have a hard time when
debugging at a reasonably low level, so if you don't think you need
this knowledge, no problem, spend most of your time by getting your
design at the right level and stay away from low level debugging, or
make sure you know someone who knows it, so you can get some help,
instead of wasting precious time.

My I first thank you Willy for all the time and effort you've put into
helping me here. I hope you don't feel that I've wasted your precious time!
Funnily enough, one of the most interesting articles I've seen on generation
0, 1 and 2 GC collections is in the CLRProfiler.doc that comes with the
CLRProfiler (funnily enough <g>). Damn shame I can't get the CLRProfiler to
work under Vista (all I get it "Waiting for application to start common
language runtime")!

Not to worry. I've come to the stage at which I need to understand more
about this area and have ordered the book suggested to me by Chris Mullins
(ISBN-10: 0735621632
). I might have to bother you again once I've read it, but with any luck
I'll be able to let you get on with other things for now!

--
Thank you again,

Christopher Ireland

[Willy Denoyette [MVP]]

"Christopher Ireland" <ci******@gmail.comwrote in message
news:1B**********************************@microsof t.com...

Willy Denoyette [MVP] wrote:

>So, you see you need to know exactly what you are after, are you
looking for CLR object allocation/de-allocation counters? then you
need to watch the "CLR Memory "counters (like the Gen0 size, if you
interested in the unmanaged memory consumption, you'll have to watch
the Memory counters (eg. Private Memory counters). Note however, that
the GC heaps are taken from the process heaps so their allocated
space is also reflected by the heap counters (Private Memory).
Again all this is something that takes time to grasp, if you can't
spend some time to learn all this, you'll have a hard time when
debugging at a reasonably low level, so if you don't think you need
this knowledge, no problem, spend most of your time by getting your
design at the right level and stay away from low level debugging, or
make sure you know someone who knows it, so you can get some help,
instead of wasting precious time.

My I first thank you Willy for all the time and effort you've put into
helping me here. I hope you don't feel that I've wasted your precious
time!

Not at all, that's why we are here.

Funnily enough, one of the most interesting articles I've seen on
generation 0, 1 and 2 GC collections is in the CLRProfiler.doc that comes
with the CLRProfiler (funnily enough <g>). Damn shame I can't get the
CLRProfiler to work under Vista (all I get it "Waiting for application to
start common language runtime")!

I haven't used the CLRProfiler on Vista, it's not really a product (the
profiler) I care about (nor does MS).

Not to worry. I've come to the stage at which I need to understand more
about this area and have ordered the book suggested to me by Chris Mullins
(ISBN-10: 0735621632
). I might have to bother you again once I've read it, but with any luck
I'll be able to let you get on with other things for now!

No problem, if you have more questions, there will always be someone to help
you out I guess.

Willy.