473,320 Members | 1,948 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Is this a true statement?

If I use Perfmon to look at the number of GC Handles for my application, and
it stays about the same over several hours, I don't have an issue with
freeing up unmanaged resources in my app. Is that true?

I have an application that is using up memory. Using Perfmon, # Bytes in
all heaps stays around 2.5MB after running for hours. TaskMgr shows my
process to be using ~75MB and growing steadily. If the memory isn't being
used by any of the .Net heaps, and I'm not increasing the number of GC
Handles, where the heck is the memory being consumed?

Could someone please take pity on me and point me toward the, probably
simple, explanation for this? I keep thinking I understand the way garbage
collection works, and the promotion from gen0 to gen1 to gen2, but I must be
missing something fundamental here.

TIA,
Chris
Nov 16 '05 #1
9 1288
"Chris" <di*****@hotmail.com> wrote in message
news:-M********************@giganews.com...
If I use Perfmon to look at the number of GC Handles for my application, and
it stays about the same over several hours, I don't have an issue with
freeing up unmanaged resources in my app. Is that true?
I do not see any logical way to equate 'GC Handles'
to 'unmanaged resources'. The number of each is, in
general, independent. So, no, your assertion is not true.
I have an application that is using up memory. Using Perfmon, # Bytes in
all heaps stays around 2.5MB after running for hours. TaskMgr shows my
process to be using ~75MB and growing steadily. If the memory isn't being
used by any of the .Net heaps, and I'm not increasing the number of GC
Handles, where the heck is the memory being consumed?
Maybe you are leaking OS handles.
Could someone please take pity on me and point me toward the, probably
simple, explanation for this? I keep thinking I understand the way garbage
collection works, and the promotion from gen0 to gen1 to gen2, but I must be
missing something fundamental here.
Perhaps you have assumed that worries about resource
leaks are history, now that a GC system is here. Sorry
to be the bearer of bad news, but 't'aint so. From your
description, I can divine that you are using some kind of
resource other than CLR memory, since that is pretty
much the only resource that is (almost) automatically
managed without programmer effort.

My bet would be that, when you get this fixed, a few
more finalizers, IDisposable implementation, and C#
'using' constructs will be in place.
TIA,
Chris


Good luck. Let us know what you find.

--
--Larry Brasfield
email: do***********************@hotmail.com
Above views may belong only to me.
Nov 16 '05 #2
GC Handles doesn't relate to unamanged resources.

However, TaskManager just shows the working set, which can be misleading.
In other words it shows memory that Windows hasn't felt any need to reclaim
as yet, but this doesn't necessarily mean it's not been released by your
app.

Do you notice the working set as reported by TaskManager going down when you
minimize the app? If the memory consumption continues to rise overall over
time even when you periodically minimize and restore the application window,
then you may not be releasing unmanaged resources.

--Bob

"Chris" <di*****@hotmail.com> wrote in message
news:-M********************@giganews.com...
If I use Perfmon to look at the number of GC Handles for my application,
and
it stays about the same over several hours, I don't have an issue with
freeing up unmanaged resources in my app. Is that true?

I have an application that is using up memory. Using Perfmon, # Bytes in
all heaps stays around 2.5MB after running for hours. TaskMgr shows my
process to be using ~75MB and growing steadily. If the memory isn't being
used by any of the .Net heaps, and I'm not increasing the number of GC
Handles, where the heck is the memory being consumed?

Could someone please take pity on me and point me toward the, probably
simple, explanation for this? I keep thinking I understand the way
garbage
collection works, and the promotion from gen0 to gen1 to gen2, but I must
be
missing something fundamental here.

TIA,
Chris

Nov 16 '05 #3
If the managed heaps are NOT growing (did you look at the LH heap!!), the
memory increase is due to the growing of the non managed portion.
This part of the process memory is taken by all code modules and data, that
is:
1 - all loaded modules (managed and unmanaged DLL's)
2 - all memory allocated using the Marshal class unmanaged allocators like
Marshal.AllocHGlobal and Marshal,AllocCoTaskMem, Marshal.Copy etc.
3 - all memory allocated using native OS allocators from unmanaged code.
4 - all memory used by your thread stacks (up to 1MB per thread).

So if your unmanaged memory consumption keeps growing, check for COM interop
leaks or PInvoke leaks. Release your COM objects when done with it. When
using PInvoke make sure you don't have to free unmanaged memory, and don't
use PInvoke to call undocumented functions or functions that possibly
allocate without any means for the caller to de-allocate.
If you are using Marhal class allocators make sure you free the allocated
memory when done.
Keep the number of threads (manual and threadpool) running (or waiting) as
low as possible, make sure some of them aren't blocked waiting for an event
that won't happen.

Willy.


"Chris" <di*****@hotmail.com> wrote in message
news:-M********************@giganews.com...
If I use Perfmon to look at the number of GC Handles for my application,
and
it stays about the same over several hours, I don't have an issue with
freeing up unmanaged resources in my app. Is that true?

I have an application that is using up memory. Using Perfmon, # Bytes in
all heaps stays around 2.5MB after running for hours. TaskMgr shows my
process to be using ~75MB and growing steadily. If the memory isn't being
used by any of the .Net heaps, and I'm not increasing the number of GC
Handles, where the heck is the memory being consumed?

Could someone please take pity on me and point me toward the, probably
simple, explanation for this? I keep thinking I understand the way
garbage
collection works, and the promotion from gen0 to gen1 to gen2, but I must
be
missing something fundamental here.

TIA,
Chris

Nov 16 '05 #4
"Larry Brasfield" <do***********************@hotmail.com> wrote in message
news:ug**************@TK2MSFTNGP09.phx.gbl...
"Chris" <di*****@hotmail.com> wrote in message
news:-M********************@giganews.com...
If I use Perfmon to look at the number of GC Handles for my application, and it stays about the same over several hours, I don't have an issue with
freeing up unmanaged resources in my app. Is that true?


I do not see any logical way to equate 'GC Handles'
to 'unmanaged resources'. The number of each is, in
general, independent. So, no, your assertion is not true.


Thanks Larry. I'm just going by what I get when I hit "Explain" in Perfmon
for the # GC Handles counter. Here is what it says...

"This counter displays the current number of GC Handles in use. GCHandles
are handles to resources external to the CLR and the managed environment.
Handles occupy small amounts of memory in the GCHeap but potentially
expensive unmanaged resources."

-Chris
Nov 16 '05 #5
"Bob Grommes" <bo*@bobgrommes.com> wrote in message
news:eo**************@TK2MSFTNGP14.phx.gbl...
GC Handles doesn't relate to unamanged resources.

However, TaskManager just shows the working set, which can be misleading.
In other words it shows memory that Windows hasn't felt any need to reclaim as yet, but this doesn't necessarily mean it's not been released by your
app.

Do you notice the working set as reported by TaskManager going down when you minimize the app? If the memory consumption continues to rise overall over time even when you periodically minimize and restore the application window, then you may not be releasing unmanaged resources.


Hi Bob,

My app is running as a Windows Service. It will continue to consume memory
until the system is unusable.

-Chris
Nov 16 '05 #6
"Willy Denoyette [MVP]" <wi*************@pandora.be> wrote in message
news:%2****************@TK2MSFTNGP10.phx.gbl...
If the managed heaps are NOT growing (did you look at the LH heap!!), the
memory increase is due to the growing of the non managed portion.
Yep, I did check the large object heap and it stays very small. I came to
the same conclusion on the memory being used by unmanaged code and just
wanted to run it by others to be sure.
This part of the process memory is taken by all code modules and data, that is:
1 - all loaded modules (managed and unmanaged DLL's)
2 - all memory allocated using the Marshal class unmanaged allocators like
Marshal.AllocHGlobal and Marshal,AllocCoTaskMem, Marshal.Copy etc.
3 - all memory allocated using native OS allocators from unmanaged code.
4 - all memory used by your thread stacks (up to 1MB per thread).

So if your unmanaged memory consumption keeps growing, check for COM interop leaks or PInvoke leaks. Release your COM objects when done with it. When
using PInvoke make sure you don't have to free unmanaged memory, and don't
use PInvoke to call undocumented functions or functions that possibly
allocate without any means for the caller to de-allocate.
If you are using Marhal class allocators make sure you free the allocated
memory when done.
Keep the number of threads (manual and threadpool) running (or waiting) as
low as possible, make sure some of them aren't blocked waiting for an event that won't happen.


Thanks for all the above, Willy. My app is actually fairly simple. It uses
an FTP client library to check a remote site every couple minutes and see if
a certain file has been updated. If it has, I download it, parse it, and
write the info to a database. I don't directly interact with any unmanaged
code, but I know the DB libraries and IO calls do.

Most of the classes in this app have actually been running in a Production
environment 24/7 for months now. I recently added some functionality, did a
little refactoring, and changed the DB drivers I use to communicate with our
Sybase DB. I suspect my change from using ODBC drivers to a ADO .Net Data
Provider from Sybase, is where my problem lives, but I can't figure out what
I'm doing wrong. It looks like I'm calling dispose on all my commands,
connections and data adapters. I do allow a few tables to live, and be
recreated with new, each time I process the downloaded file.

-Chris
Nov 16 '05 #7

"Chris" <di*****@hotmail.com> wrote in message
news:Iv********************@giganews.com...
"Willy Denoyette [MVP]" <wi*************@pandora.be> wrote in message
news:%2****************@TK2MSFTNGP10.phx.gbl...
If the managed heaps are NOT growing (did you look at the LH heap!!), the
memory increase is due to the growing of the non managed portion.


Yep, I did check the large object heap and it stays very small. I came to
the same conclusion on the memory being used by unmanaged code and just
wanted to run it by others to be sure.
This part of the process memory is taken by all code modules and data,

that
is:
1 - all loaded modules (managed and unmanaged DLL's)
2 - all memory allocated using the Marshal class unmanaged allocators
like
Marshal.AllocHGlobal and Marshal,AllocCoTaskMem, Marshal.Copy etc.
3 - all memory allocated using native OS allocators from unmanaged code.
4 - all memory used by your thread stacks (up to 1MB per thread).

So if your unmanaged memory consumption keeps growing, check for COM

interop
leaks or PInvoke leaks. Release your COM objects when done with it. When
using PInvoke make sure you don't have to free unmanaged memory, and
don't
use PInvoke to call undocumented functions or functions that possibly
allocate without any means for the caller to de-allocate.
If you are using Marhal class allocators make sure you free the allocated
memory when done.
Keep the number of threads (manual and threadpool) running (or waiting)
as
low as possible, make sure some of them aren't blocked waiting for an

event
that won't happen.


Thanks for all the above, Willy. My app is actually fairly simple. It
uses
an FTP client library to check a remote site every couple minutes and see
if
a certain file has been updated. If it has, I download it, parse it, and
write the info to a database. I don't directly interact with any
unmanaged
code, but I know the DB libraries and IO calls do.

Most of the classes in this app have actually been running in a Production
environment 24/7 for months now. I recently added some functionality, did
a
little refactoring, and changed the DB drivers I use to communicate with
our
Sybase DB. I suspect my change from using ODBC drivers to a ADO .Net Data
Provider from Sybase, is where my problem lives, but I can't figure out
what
I'm doing wrong. It looks like I'm calling dispose on all my commands,
connections and data adapters. I do allow a few tables to live, and be
recreated with new, each time I process the downloaded file.

-Chris


Chris,

If you suspect the Sysbase provider, couldn't you write a simple testcase
that exercises the same DB calls against a test DB using the same provider?
That way it's much simpler to debug and watch your memory consumption using
native tools like windbg and VADump.

Willy.


Nov 16 '05 #8
> Chris,

If you suspect the Sysbase provider, couldn't you write a simple testcase
that exercises the same DB calls against a test DB using the same provider? That way it's much simpler to debug and watch your memory consumption using native tools like windbg and VADump.

Willy.


Yes, I think that's what I need to do now. I'm such a newbie that I didn't
know about vadump and windbg though. I've been giving myself a crash course
today but I still have a whole lot of reading to do. You wouldn't happen to
know how to translate the stats below into meaningful class names would you?
The 33652 pages in private Heap 3 would appear to be my problem, but I
haven't figured out yet where to go from here.

Thanks again for your help with this.

-Chris

Heap Working Set Contributions
237 pages from Process Heap (class 0x00000000)
0x00130000 - 0x00230000 215 pages
0x08050000 - 0x08150000 22 pages
0 pages from UNKNOWN Heap 0 (class 0x00008000)
0x00230000 - 0x00240000 0 pages
0 pages from Private Heap 1 (class 0x00001000)
0x002F0000 - 0x00300000 0 pages
3 pages from Private Heap 2 (class 0x00001000)
0x00510000 - 0x00520000 3 pages
33652 pages from Private Heap 3 (class 0x00001000)
0x00520000 - 0x00530000 1 pages
0x06890000 - 0x06990000 232 pages
0x086D0000 - 0x088D0000 511 pages
0x089D0000 - 0x08DD0000 1023 pages
0x08ED0000 - 0x096D0000 2047 pages
0x09BD0000 - 0x0ABD0000 4095 pages
0x0BED0000 - 0x0DED0000 8190 pages
0x11010000 - 0x15010000 16383 pages
0x17910000 - 0x1F910000 1170 pages
3 pages from Private Heap 4 (class 0x00001000)
0x06D40000 - 0x06D50000 3 pages
1 pages from Private Heap 5 (class 0x00001000)
0x07520000 - 0x07560000 1 pages
3 pages from Private Heap 6 (class 0x00001000)
0x07560000 - 0x07660000 3 pages
6 pages from Private Heap 7 (class 0x00001000)
0x07EA0000 - 0x07EE0000 6 pages
4 pages from Private Heap 8 (class 0x00001000)
0x07EE0000 - 0x07F20000 4 pages
1 pages from Private Heap 9 (class 0x00001000)
0x07F30000 - 0x07F40000 1 pages
9 pages from Private Heap 10 (class 0x00001000)
0x08040000 - 0x08050000 9 pages
2 pages from Private Heap 11 (class 0x00001000)
0x08150000 - 0x08190000 2 pages
Nov 16 '05 #9
"Chris" <di*****@hotmail.com> wrote in message
news:-M********************@giganews.com...
If I use Perfmon to look at the number of GC Handles for my application, and it stays about the same over several hours, I don't have an issue with
freeing up unmanaged resources in my app. Is that true?

I have an application that is using up memory. Using Perfmon, # Bytes in
all heaps stays around 2.5MB after running for hours. TaskMgr shows my
process to be using ~75MB and growing steadily. If the memory isn't being
used by any of the .Net heaps, and I'm not increasing the number of GC
Handles, where the heck is the memory being consumed?

Could someone please take pity on me and point me toward the, probably
simple, explanation for this? I keep thinking I understand the way garbage collection works, and the promotion from gen0 to gen1 to gen2, but I must be missing something fundamental here.


Looks like I finally got this beat, after spending my entire weekend on it.
The DataReader implementation, in the Sybase ADO .Net Data Provider,
apparently doesn't clean up after itself if you supply a commandbehavior
when constructing it. I had no problem with what I was doing with the
OdbcDataReader but started seeing this memory leak when I moved it to the
Sybase specific version. Took me a LONG time to decide to try the default
constructor though.

-Chris
Nov 16 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Eric Linders | last post by:
Hello, We have a form that is collecting user input. There are three fields for each phone number (area code, prefix and suffix). I'm trying to use a simple if statement to confirm if the first...
46
by: Scott Chapman | last post by:
There seems to be an inconsistency here: Python 2.3.2 (#1, Oct 3 2003, 19:04:58) on linux2 >>> 1 == True True >>> 3 == True False >>> if 1: print "true" ....
3
by: drs | last post by:
I just upgraded my Python install, and for the first time have True and False rather than 1 and 0. I was playing around at the command line to test how they work (for instance, "if 9:" and "if...
14
by: root | last post by:
Hi group, Apologies in advance if this has been asked somewhere before, but I haven't managed to get anything from the Google archives - I've been getting introductory guides to C++ all day...
5
by: Steve | last post by:
Hello, I've been a PHP programmer for a number of years and have just started to learn JS. My Employer (a water analysis lab) wants what should be a very simple .js written that basically takes...
48
by: Skybuck Flying | last post by:
Hi, I came across this C code which I wanted to understand etc it looked like this: if (-1) etc It made me wonder what the result would be... true or false ? In C and Delphi
22
by: Paminu | last post by:
As I remember if(1) evaluates to true and all other numbers including 0 evaluate to false. But where do I find out about this for sure?? I have looked through K&R, all the C for dummies books...
30
by: Jason | last post by:
I am fairly new to ASP--I have been using it about 2 months. I did these tests (below), and it doesn't make sense to me. False is equal to 0, and that's fine. True should be equal to 1, but it's...
40
by: nufuhsus | last post by:
Hello all, First let me appologise if this has been answered but I could not find an acurate answer to this interesting problem. If the following is true: C:\Python25\rg.py>python Python...
12
by: phelle25 | last post by:
i made a program in else-if statement about this problem. but my teacher requires to give a return statement about this. i am new with in programming and we had just tackle a return statement and i...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.