Memory Leak

[Peter]

I am using VS2008.

I have a Windows Service application which creates Crystal Reports. This is
a multi theaded application which can run several reports at one time.
My problem - there is a memory leak someplace. I can not detect the memory
leak by running several reports by hand, but when I run tha app as a
servrice and process few hundred reports there is significant memory leak.
The application can consume over 1GB of memory where it should not go over
200 mb. I have few other applications that use Crystal Reports and those
applications do not have memory leak, the big difference is those
applications are single threaded.

My question is are there any free tools I can use to find the memory leak?

I have tried http://www.softprodigy.com but I could not get it to work it
just locks up.

Thank You
Peter

[Cor Ligthert[MVP]]

Peter,

Always the first questions on Memory Leaks in the dotnet newsgroup

How do you know that there is a Memory Leak?

Cor

"Peter" <cz****@nospam.nospamschreef in bericht
news:OS**************@TK2MSFTNGP04.phx.gbl...

>I am using VS2008.

I have a Windows Service application which creates Crystal Reports. This
is a multi theaded application which can run several reports at one time.
My problem - there is a memory leak someplace. I can not detect the memory
leak by running several reports by hand, but when I run tha app as a
servrice and process few hundred reports there is significant memory leak.
The application can consume over 1GB of memory where it should not go over
200 mb. I have few other applications that use Crystal Reports and those
applications do not have memory leak, the big difference is those
applications are single threaded.

My question is are there any free tools I can use to find the memory leak?

I have tried http://www.softprodigy.com but I could not get it to work it
just locks up.

Thank You
Peter

[Jeffrey Tan[MSFT]]

Hi Peter ,

Regarding .Net memory leak issue, an important step is determing if it is a
managed leak or unmanaged memory leak.

Can you tell me what memory counter do you get "over 1G memory" result?
Windows has a lot of memory counters which may cause fake memory leak
conclusions. Basically, we should monitor "Process/Private Bytes"(or "VM
Size" column in task manager.) counter to determine memory leak. Note: we
should not use the "Mem Usage" column in task manager which is misleading.
I highly recommend you to download Process Explorer tool from the link
below since it is more powerful tool than task manager:
http://technet.microsoft.com/en-us/s.../bb896653.aspx

Once the "Private Bytes" is keeping increasing without decrease, it means
that the application is leaking memory. Now, you can monitor the ".Net CLR
Memory/# Bytes in all Heaps" counter. If the "Private Bytes" increases
while the ".Net CLR Memory" is stable, it indicates an unmanaged memory
leak. Otherwise, it basically means a managed memory leak. The article
below talks more information:
"I have a memory leak!!! What do i do? (defining the "where")"
http://blogs.msdn.com/tess/archive/2...leak-what-do-i
-do-defining-the-where.aspx

I will wait for your further feedback to determine if it is a managed or
unmanaged leak since they need totally different approach for
troubleshooting. Thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support

Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscripti...ult.aspx#notif
ications.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscripti...t/default.aspx.
==================================================
This posting is provided "AS IS" with no warranties, and confers no rights.

[Peter]

"Cor Ligthert[MVP]" <no************@planet.nlwrote in message
news:19**********************************@microsof t.com...

Peter,

Always the first questions on Memory Leaks in the dotnet newsgroup

How do you know that there is a Memory Leak?

Cor

"Peter" <cz****@nospam.nospamschreef in bericht
news:OS**************@TK2MSFTNGP04.phx.gbl...

>>I am using VS2008.

I have a Windows Service application which creates Crystal Reports. This
is a multi theaded application which can run several reports at one time.
My problem - there is a memory leak someplace. I can not detect the
memory leak by running several reports by hand, but when I run tha app as
a servrice and process few hundred reports there is significant memory
leak. The application can consume over 1GB of memory where it should not
go over 200 mb. I have few other applications that use Crystal Reports
and those applications do not have memory leak, the big difference is
those applications are single threaded.

My question is are there any free tools I can use to find the memory
leak?

I have tried http://www.softprodigy.com but I could not get it to work it
just locks up.

Thank You
Peter

I do not know if it's the memory leak - that's what I am trying to find out.

The reason I am starting with this assumption is because when I look at the
"Mem Usage" in the Windows Task Manager the memory keeps growing and never
ever goes down and eventually the system goes to a craw until I restart the
application. Plus my other simullar applications when they run the "Mem
Usage" goes up and down and eventually goes back to the almost original
starting number when the applications is not processing anything.

[Peter]

I get "over 1G" on both "Mem Usage" and "VM Size", the VM Size is slightly
lower but it's consistant with "Mem Usage" so in this case "at least it
looks like Mem Usage is just as reliable"

I have also downlaoded the Process Exlprer and "Private Bytes" keep going up
and up, it does go down slightly at times but not as much as it goes up.
""Jeffrey Tan[MSFT]"" <je***@online.microsoft.comwrote in message
news:qL**************@TK2MSFTNGHUB02.phx.gbl...

Hi Peter ,

Regarding .Net memory leak issue, an important step is determing if it is
a
managed leak or unmanaged memory leak.

Can you tell me what memory counter do you get "over 1G memory" result?
Windows has a lot of memory counters which may cause fake memory leak
conclusions. Basically, we should monitor "Process/Private Bytes"(or "VM
Size" column in task manager.) counter to determine memory leak. Note: we
should not use the "Mem Usage" column in task manager which is misleading.
I highly recommend you to download Process Explorer tool from the link
below since it is more powerful tool than task manager:
http://technet.microsoft.com/en-us/s.../bb896653.aspx

Once the "Private Bytes" is keeping increasing without decrease, it means
that the application is leaking memory. Now, you can monitor the ".Net CLR
Memory/# Bytes in all Heaps" counter. If the "Private Bytes" increases
while the ".Net CLR Memory" is stable, it indicates an unmanaged memory
leak. Otherwise, it basically means a managed memory leak. The article
below talks more information:
"I have a memory leak!!! What do i do? (defining the "where")"
http://blogs.msdn.com/tess/archive/2...leak-what-do-i
-do-defining-the-where.aspx

I will wait for your further feedback to determine if it is a managed or
unmanaged leak since they need totally different approach for
troubleshooting. Thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support

Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscripti...ult.aspx#notif
ications.

Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscripti...t/default.aspx.
==================================================
This posting is provided "AS IS" with no warranties, and confers no
rights.

[Jeffrey Tan[MSFT]]

Hi Peter,

Thanks for your feedback.

This should mean a memory leak in your application. Then, have you checked
".Net CLR Memory/# Bytes in all Heaps" counter? It will help us to
distinguish between managed/unmanaged leak in the application.

The "Mem Usage" actually means the "Working Set Size" which is the physical
RAM that a process currently uses instead of virtual memory. In modern
Virtual Memory OS, the physical RAM usage normally makes no sense to
application developer since the kernel VMM manages this. The "Working Set"
increases/decreases based on the OS paging activities and the process
working set commit. For example, if you bring a GUI application to the
foreground, it "Mem Usage" will go up immediately while many other
background applications' "Mem Usage" may decrease. This has nothing to do
with the memory leak. Also, although one applicatin is leaking memory, if
it has reached the "Working Set" charge limit, you will find that "Mem
Usage" not change anymore, but the process is still leaking. So, "Mem
Usage" is basically a useless counter.

Thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no rights.

[Peter]

The Private Bytes counter - Keeps increasing
The # Bytes in all Heaps - Keeps increasing
Gen 0 heap size - is stable (increases and decreases)
Gen 1 heap size - is stable (increases and decreases)
Gen 2 heap size - Keeps increasing

""Jeffrey Tan[MSFT]"" <je***@online.microsoft.comwrote in message
news:Sx**************@TK2MSFTNGHUB02.phx.gbl...

Hi Peter,

Thanks for your feedback.

This should mean a memory leak in your application. Then, have you checked
".Net CLR Memory/# Bytes in all Heaps" counter? It will help us to
distinguish between managed/unmanaged leak in the application.

The "Mem Usage" actually means the "Working Set Size" which is the
physical
RAM that a process currently uses instead of virtual memory. In modern
Virtual Memory OS, the physical RAM usage normally makes no sense to
application developer since the kernel VMM manages this. The "Working Set"
increases/decreases based on the OS paging activities and the process
working set commit. For example, if you bring a GUI application to the
foreground, it "Mem Usage" will go up immediately while many other
background applications' "Mem Usage" may decrease. This has nothing to do
with the memory leak. Also, although one applicatin is leaking memory, if
it has reached the "Working Set" charge limit, you will find that "Mem
Usage" not change anymore, but the process is still leaking. So, "Mem
Usage" is basically a useless counter.

Thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no
rights.

[Jeffrey Tan[MSFT]]

Hi Peter,

Thanks for your feedback.

Ok, this sounds like a managed memory leak in .Net.

To troubleshoot the managed memory leak, we normally use windbg+SOS.dll
extension which understands the internal data structure of .Net objects.
The basic idea is: first dumping all the managed objects on the managed
heap sorted by size. Then we can understand what type of object costs the
most of memory(which is the possible culprit for leaking). Then, we can use
"!gcroot" command to traverse all the root graph to find the call graph
that is holding the reference to the leaking objects. The articles talk
about the detailed steps regarding how to perform these steps:
"Tracking down managed memory leaks (how to find a GC leak)"
http://blogs.msdn.com/ricom/archive/...10/279612.aspx
".NET Memory Leak Case Study: The Event Handlers That Made The Memory
Baloon"
http://blogs.msdn.com/tess/archive/2...case-study-the
-event-handlers-that-made-the-memory-baloon.aspx

Hope it helps.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no rights.

[Peter]

I've downloaded vadump.exe from Microsoft but I am getting error
'FAILURE query working set 24' when I run it.
and
Where can I find these tools like !dumpheap and !eeheap

I have VS2008 Archtect addition installed on my machine, but I don't have
those programs.

""Jeffrey Tan[MSFT]"" <je***@online.microsoft.comwrote in message
news:Tg**************@TK2MSFTNGHUB02.phx.gbl...

Hi Peter,

Thanks for your feedback.

Ok, this sounds like a managed memory leak in .Net.

To troubleshoot the managed memory leak, we normally use windbg+SOS.dll
extension which understands the internal data structure of .Net objects.
The basic idea is: first dumping all the managed objects on the managed
heap sorted by size. Then we can understand what type of object costs the
most of memory(which is the possible culprit for leaking). Then, we can
use
"!gcroot" command to traverse all the root graph to find the call graph
that is holding the reference to the leaking objects. The articles talk
about the detailed steps regarding how to perform these steps:
"Tracking down managed memory leaks (how to find a GC leak)"
http://blogs.msdn.com/ricom/archive/...10/279612.aspx
".NET Memory Leak Case Study: The Event Handlers That Made The Memory
Baloon"
http://blogs.msdn.com/tess/archive/2...case-study-the
-event-handlers-that-made-the-memory-baloon.aspx

Hope it helps.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no
rights.

[Jeffrey Tan[MSFT]]

Hi Peter,

Thanks for your feedback.

Actually, there is no need to run vadump.exe because we have used different
counters from Process Explorer to confirm the memory leak. So we may ignore
the first 3 steps in the link below and start from the step 4:
http://blogs.msdn.com/ricom/archive/...10/279612.aspx

!dumpheap and !eeheap are both the windbg extension commands exported by
SOS.dll which is written by the CLR team for internal debugging purpose.
So, we need to install windbg to debug this memory leak instead of using VS
debugger. Yes, we can use VS debugger with SOS.dll, but it is inconvenient.
The link below talks about how to use VS debugger+SOS.dll for .Net
debugging:
".NET Finalizer Memory Leak: Debugging with sos.dll in Visual Studio"
http://blogs.msdn.com/tess/archive/2...mory-leak-debu
gging-with-sos-dll-in-visual-studio.aspx

I prefer windbg because it is more powerful and convenient for production
debugging than VS debugger and it is free. If you have never used windbg
before, you can follow the basic setup in the blog I written below:
"How to debug application crash/hang in production environment"
http://blogs.msdn.com/msdnts/archive...pplication-cra
sh-hang-in-production-environment.aspx

After completing the windbg installation and configuration, you can follow
to start from "Step 4: Attach Windbg and load SOS" in the first blog link
above.

If there is anything unclear, please feel free to tell me, thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no rights.

[Peter]

Thanks for our help

I have fallowed the instructions and it seems to be working, but I do not
know what the results mean, I don't know how to read them
Here's a partial output, but I do not know what to do next.

!DumpHeap -stat

00687e2c 3572 285760
CrystalDecisions.CrystalReports.Engine.ReportDocum ent
00d9c33c 21518 860720
com.crystaldecisions.common.keycode.KeycodeDefinit ionDataXML+feature
79101fe4 22600 1265600 System.Collections.Hashtable
790fd8c4 22576 1351096 System.String
0015bbc0 295 2175560 Free
7912d9bc 22600 3270432 System.Collections.Hashtable+bucket[]

!DumpHeap -type System.Collections.Hashtable+bucket[]
Address MT Size
00fd15bc 7912d9bc 144
00fd1684 7912d9bc 144
00fd174c 7912d9bc 144
00fd18c0 7912d9bc 144
00fd1a60 7912d9bc 144
00fd1bd0 7912d9bc 144
00fd1cc4 7912d9bc 144
00fd1d8c 7912d9bc 144
00fd2168 7912d9bc 144
00fd2230 7912d9bc 144
00fd28c4 7912d9bc 216
00fd30ec 7912d9bc 144
00fd442c 7912d9bc 144
00fd4d3c 7912d9bc 456
00fd4f3c 7912d9bc 456
00fdeca0 7912d9bc 2880
00fe2ae8 7912d9bc 288
00fe2e4c 7912d9bc 144
00fe306c 7912d9bc 144
00fe3134 7912d9bc 144
00fe34a4 7912d9bc 144
00fe3de4 7912d9bc 144
00fe4c40 7912d9bc 144
00fe5438 7912d9bc 1296
010bb2e4 7912d9bc 144
010bb574 7912d9bc 144
010bb84c 7912d9bc 144
010bb978 7912d9bc 144
010bbb3c 7912d9bc 144
010bbc74 7912d9bc 144
010bbe30 7912d9bc 144
010bc180 7912d9bc 144
010bcbf8 7912d9bc 144
010c2dc0 7912d9bc 1296
010cde14 7912d9bc 144

!gcroot 00fdeca0

Note: Roots found on stacks may be false positives. Run "!help gcroot" for
more info.
Scan Thread 0 OSTHread 1490
Scan Thread 2 OSTHread 15dc
Scan Thread 5 OSTHread 11b0
Scan Thread 13 OSTHread de0
Scan Thread 14 OSTHread 1c68
Scan Thread 15 OSTHread 1e50
DOMAIN(001652D0):HANDLE(WeakSh):6612c4:Root:010d0f 48(System.Diagnostics.SourceSwitch)->
010d11e0(System.Diagnostics.SwitchElementsCollecti on)->
00fd536c(System.Configuration.RuntimeConfiguration Record)->
00fd619c(System.Collections.Hashtable)->
00fdeca0(System.Collections.Hashtable+bucket[])
""Jeffrey Tan[MSFT]"" <je***@online.microsoft.comwrote in message
news:pr**************@TK2MSFTNGHUB02.phx.gbl...

Hi Peter,

Thanks for your feedback.

Actually, there is no need to run vadump.exe because we have used
different
counters from Process Explorer to confirm the memory leak. So we may
ignore
the first 3 steps in the link below and start from the step 4:
http://blogs.msdn.com/ricom/archive/...10/279612.aspx

!dumpheap and !eeheap are both the windbg extension commands exported by
SOS.dll which is written by the CLR team for internal debugging purpose.
So, we need to install windbg to debug this memory leak instead of using
VS
debugger. Yes, we can use VS debugger with SOS.dll, but it is
inconvenient.
The link below talks about how to use VS debugger+SOS.dll for .Net
debugging:
".NET Finalizer Memory Leak: Debugging with sos.dll in Visual Studio"
http://blogs.msdn.com/tess/archive/2...mory-leak-debu
gging-with-sos-dll-in-visual-studio.aspx

I prefer windbg because it is more powerful and convenient for production
debugging than VS debugger and it is free. If you have never used windbg
before, you can follow the basic setup in the blog I written below:
"How to debug application crash/hang in production environment"
http://blogs.msdn.com/msdnts/archive...pplication-cra
sh-hang-in-production-environment.aspx

After completing the windbg installation and configuration, you can follow
to start from "Step 4: Attach Windbg and load SOS" in the first blog link
above.

If there is anything unclear, please feel free to tell me, thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no
rights.

[Jeffrey Tan[MSFT]]

Hi Peter,

Thanks for your feedback.

The "!DumpHeap - stat" command will group all the objects on the heap
based on their types. The output is sorted by the occupied space of each
type(3rd column), so we can find out the objects that occupy the largest
memory space. For detailed explanation of this command, please refer to:
"!dumpheap -stat explained¡* (debugging .net leaks)"
http://blogs.msdn.com/tess/archive/2...25/496973.aspx.

In your output, is "System.Collections.Hashtable+bucket[]" the last line in
the output? From the output, "System.Collections.Hashtable+bucket[]" type
only occupes about 3MB of memory(3270432), so it may not be the culprit of
the leaking problem.

Can you paste the last several lines of the output? There should be a
summary line stating the total objects and total heap object sizes, like
"Total 9,712,896 objects, Total size: 1,032,127,612".

Thanks!

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no rights.

[Peter]

Here are the results from !dumpheap -stat
!dumpheap -stat

00d951f4 13173 316152 CrystalDecisions.Shared.ExportOptions
790fa9d8 23429 374864 System.__ComObject
04e7c68c 12603 403296
CrystalDecisions.CrystalReports.Engine.DataSourceC acheItem
04e7715c 12603 403296 CrystalDecisions.CrystalReports.Engine.Table
00687f64 13172 1053760
CrystalDecisions.CrystalReports.Engine.ReportDocum ent
00164450 2373 2024340 Free
790fd8c4 60981 3590148 System.String
79101fe4 94954 5317424 System.Collections.Hashtable
7912d9bc 94956 13689264 System.Collections.Hashtable+bucket[]
Total 449066 objects

!dumpheap -mt 7912d9bc

total 94956 objects
Statistics:
MT Count TotalSize Class Name
7912d9bc 94956 13689264 System.Collections.Hashtable+bucket[]
Total 94956 objects
""Jeffrey Tan[MSFT]"" <je***@online.microsoft.comwrote in message
news:wU****************@TK2MSFTNGHUB02.phx.gbl...

Hi Peter,

Thanks for your feedback.

The "!DumpHeap - stat" command will group all the objects on the heap
based on their types. The output is sorted by the occupied space of each
type(3rd column), so we can find out the objects that occupy the largest
memory space. For detailed explanation of this command, please refer to:
"!dumpheap -stat explained¡* (debugging .net leaks)"
http://blogs.msdn.com/tess/archive/2...25/496973.aspx.

In your output, is "System.Collections.Hashtable+bucket[]" the last line
in
the output? From the output, "System.Collections.Hashtable+bucket[]" type
only occupes about 3MB of memory(3270432), so it may not be the culprit of
the leaking problem.

Can you paste the last several lines of the output? There should be a
summary line stating the total objects and total heap object sizes, like
"Total 9,712,896 objects, Total size: 1,032,127,612".

Thanks!

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no
rights.

[Jeffrey Tan[MSFT]]

Hi Peter,

Thanks for your feedback.

The command output indicates that the largest object size on the GC heap is
of type "System.Collections.Hashtable+bucket[]". However, it only costs
less than 14M memory: "13689264". So, it should not be the culprit of the
memory leaking problem.

Can you issue command "!eeheap" in your application? This command will dump
a summary report of all the managed heaps in the application. I suspect the
GC heap is not leaking that much memory yet. You may paste the "!eeheap"
command output here for analysis. Thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no rights.

[Peter]

Here it is the !eeheap
Loader Heap:
--------------------------------------
System Domain: 7a3bc8b8
LowFrequencyHeap: Size: 0x0(0)bytes.
HighFrequencyHeap: 00672000(8000:1000) Size: 0x1000(4096)bytes.
StubHeap: 0067a000(2000:2000) 04150000(10000:7000) Size: 0x9000(36864)bytes.
Virtual Call Stub Heap:
IndcellHeap: Size: 0x0(0)bytes.
LookupHeap: Size: 0x0(0)bytes.
ResolveHeap: Size: 0x0(0)bytes.
DispatchHeap: Size: 0x0(0)bytes.
CacheEntryHeap: Size: 0x0(0)bytes.
Total size: 0xa000(40960)bytes
--------------------------------------
Shared Domain: 7a3bc560
LowFrequencyHeap: 006a0000(2000:1000) Size: 0x1000(4096)bytes.
HighFrequencyHeap: 006a2000(8000:1000) Size: 0x1000(4096)bytes.
StubHeap: 006aa000(2000:1000) Size: 0x1000(4096)bytes.
Virtual Call Stub Heap:
IndcellHeap: 006b0000(2000:1000) Size: 0x1000(4096)bytes.
LookupHeap: 006b5000(2000:1000) Size: 0x1000(4096)bytes.
ResolveHeap: 006bb000(5000:1000) Size: 0x1000(4096)bytes.
DispatchHeap: 006b7000(4000:1000) Size: 0x1000(4096)bytes.
CacheEntryHeap: 006b2000(3000:1000) Size: 0x1000(4096)bytes.
Total size: 0x7000(28672)bytes
--------------------------------------
Domain 1: 15a790
LowFrequencyHeap: 00680000(2000:2000) 00b90000(10000:d000)
00d80000(10000:10000) 00dc0000(10000:f000) 03890000(10000:f000)
039a0000(10000:f000) 00ac0000(10000:f000) 077b0000(10000:10000)
07ee0000(10000:10000) 08860000(10000:10000) 07000000(10000:d000)
05c90000(20000:20000) 05e40000(10000:c000) Size: 0xc4000(802816)bytes.
Wasted: 0xa000(40960)bytes.
HighFrequencyHeap: 00682000(8000:8000) 00d90000(10000:10000)
04e70000(10000:10000) 07ab0000(10000:10000) 07ed0000(10000:10000)
07f50000(10000:10000) 055c0000(10000:10000) 05cc0000(10000:a000) Size:
0x72000(466944)bytes.
StubHeap: 0068a000(2000:2000) 07ac0000(10000:3000) Size: 0x5000(20480)bytes.
Virtual Call Stub Heap:
IndcellHeap: 00690000(2000:1000) Size: 0x1000(4096)bytes.
LookupHeap: 00696000(1000:1000) Size: 0x1000(4096)bytes.
ResolveHeap: 0069a000(6000:5000) Size: 0x5000(20480)bytes.
DispatchHeap: 00697000(3000:2000) Size: 0x2000(8192)bytes.
CacheEntryHeap: 00692000(4000:1000) Size: 0x1000(4096)bytes.
Total size: 0x144000(1327104)bytes
--------------------------------------
Jit code heap:
LoaderCodeHeap: 05d30000(10000:8000) Size: 0x8000(32768)bytes.
LoaderCodeHeap: 05cb0000(10000:10000) Size: 0x10000(65536)bytes.
LoaderCodeHeap: 050a0000(10000:10000) Size: 0x10000(65536)bytes.
LoaderCodeHeap: 06fe0000(10000:10000) Size: 0x10000(65536)bytes.
LoaderCodeHeap: 088b0000(10000:10000) Size: 0x10000(65536)bytes.
LoaderCodeHeap: 077e0000(10000:f000) Size: 0xf000(61440)bytes.
LoaderCodeHeap: 05290000(10000:10000) Size: 0x10000(65536)bytes.
LoaderCodeHeap: 00b50000(10000:f000) Size: 0xf000(61440)bytes.
LoaderCodeHeap: 00ba0000(10000:e000) Size: 0xe000(57344)bytes.
Total size: 0x84000(540672)bytes
--------------------------------------
Module Thunk heaps:
Module 790c2000: Size: 0x0(0)bytes.
Module 006a239c: Size: 0x0(0)bytes.
Module 006a2010: Size: 0x0(0)bytes.
Module 00682c3c: Size: 0x0(0)bytes.
Module 67a30000: Size: 0x0(0)bytes.
Module 7a726000: Size: 0x0(0)bytes.
Module 00683e94: Size: 0x0(0)bytes.
Module 006851d4: Size: 0x0(0)bytes.
Module 006862f4: Size: 0x0(0)bytes.
Module 7b454000: Size: 0x0(0)bytes.
Module 7ae74000: Size: 0x0(0)bytes.
Module 00d90084: Size: 0x0(0)bytes.
Module 00d9070c: Size: 0x0(0)bytes.
Module 00d90ea4: Size: 0x0(0)bytes.
Module 648ea000: Size: 0x0(0)bytes.
Module 639f8000: Size: 0x0(0)bytes.
Module 00d92568: Size: 0x0(0)bytes.
Module 653f2000: 03860000(10000:1000) Size: 0x1000(4096)bytes.
Module 00d95f14: Size: 0x0(0)bytes.
Module 00d97404: Size: 0x0(0)bytes.
Module 00d983c0: Size: 0x0(0)bytes.
Module 00d9901c: Size: 0x0(0)bytes.
Module 00d995b8: Size: 0x0(0)bytes.
Module 00d9ab64: Size: 0x0(0)bytes.
Module 6638c000: Size: 0x0(0)bytes.
Module 67b2e000: Size: 0x0(0)bytes.
Module 67426000: Size: 0x0(0)bytes.
Module 04cf4000: 04d50000(10000:2000) Size: 0x2000(8192)bytes.
Module 6037e000: 077a0000(10000:2000) Size: 0x2000(8192)bytes.
Module 04e7a46c: Size: 0x0(0)bytes.
Module 65da0000: Size: 0x0(0)bytes.
Module 07ed601c: Size: 0x0(0)bytes.
Module 07ed6ca4: Size: 0x0(0)bytes.
Module 07f51f54: Size: 0x0(0)bytes.
Module 07f5fb04: Size: 0x0(0)bytes.
Module 055c341c: Size: 0x0(0)bytes.
Total size: 0x5000(20480)bytes
--------------------------------------
Module Lookup Table heaps:
Module 790c2000: Size: 0x0(0)bytes.
Module 006a239c: Size: 0x0(0)bytes.
Module 006a2010: Size: 0x0(0)bytes.
Module 00682c3c: Size: 0x0(0)bytes.
Module 67a30000: Size: 0x0(0)bytes.
Module 7a726000: Size: 0x0(0)bytes.
Module 00683e94: Size: 0x0(0)bytes.
Module 006851d4: Size: 0x0(0)bytes.
Module 006862f4: Size: 0x0(0)bytes.
Module 7b454000: Size: 0x0(0)bytes.
Module 7ae74000: Size: 0x0(0)bytes.
Module 00d90084: Size: 0x0(0)bytes.
Module 00d9070c: Size: 0x0(0)bytes.
Module 00d90ea4: Size: 0x0(0)bytes.
Module 648ea000: Size: 0x0(0)bytes.
Module 639f8000: Size: 0x0(0)bytes.
Module 00d92568: Size: 0x0(0)bytes.
Module 653f2000: Size: 0x0(0)bytes.
Module 00d95f14: Size: 0x0(0)bytes.
Module 00d97404: Size: 0x0(0)bytes.
Module 00d983c0: Size: 0x0(0)bytes.
Module 00d9901c: Size: 0x0(0)bytes.
Module 00d995b8: Size: 0x0(0)bytes.
Module 00d9ab64: Size: 0x0(0)bytes.
Module 6638c000: Size: 0x0(0)bytes.
Module 67b2e000: Size: 0x0(0)bytes.
Module 67426000: Size: 0x0(0)bytes.
Module 04cf4000: Size: 0x0(0)bytes.
Module 6037e000: Size: 0x0(0)bytes.
Module 04e7a46c: Size: 0x0(0)bytes.
Module 65da0000: Size: 0x0(0)bytes.
Module 07ed601c: Size: 0x0(0)bytes.
Module 07ed6ca4: Size: 0x0(0)bytes.
Module 07f51f54: Size: 0x0(0)bytes.
Module 07f5fb04: Size: 0x0(0)bytes.
Module 055c341c: Size: 0x0(0)bytes.
Total size: 0x0(0)bytes
--------------------------------------
Total LoaderHeap size: 0x1de000(1957888)bytes
=======================================
Number of GC Heaps: 1
generation 0 starts at 0x1e7800f4
generation 1 starts at 0x1e770098
generation 2 starts at 0x00fd1000
ephemeral segment allocation context: none
segment begin allocated size
001960a0 7b463c40 7b47a744 0x00016b04(92932)
0017c2a8 7a733370 7a754b98 0x00021828(137256)
00170ec0 790d8620 790f7d8c 0x0001f76c(128876)
00fd0000 00fd1000 01e3aa38 0x00e69a38(15112760)
1e5e0000 1e5e1000 1e7bc100 0x001db100(1945856)
Large object heap starts at 0x01fd1000
segment begin allocated size
01fd0000 01fd1000 0211c910 0x0014b910(1358096)
Total Size 0x11e7ee0(18775776)
------------------------------
GC Heap Size 0x11e7ee0(18775776)

""Jeffrey Tan[MSFT]"" <je***@online.microsoft.comwrote in message
news:W6**************@TK2MSFTNGHUB02.phx.gbl...

Hi Peter,

Thanks for your feedback.

The command output indicates that the largest object size on the GC heap
is
of type "System.Collections.Hashtable+bucket[]". However, it only costs
less than 14M memory: "13689264". So, it should not be the culprit of the
memory leaking problem.

Can you issue command "!eeheap" in your application? This command will
dump
a summary report of all the managed heaps in the application. I suspect
the
GC heap is not leaking that much memory yet. You may paste the "!eeheap"
command output here for analysis. Thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no
rights.

[Jeffrey Tan[MSFT]]

Hi Peter,

Thanks for your feedback.

As the !eeheap output stated, the GC heap only costs more than 18M of
memory(18775776) which is not the culprit for the memory leak(as you
original stated, the memory leak costs more than 1Gb of memory). So, I
should conclude this as an unmanaged memory leak.

Does your application use any unmanaged code? For example, do you use
p/invoke or COM interop in the application? Do you use any 3rd party
component that may cause the unmanaged memory leak?

Also, I recommend you to issue "!address -summary" command in windbg and
paste the result here. This command will generate a memory usage summary
report of the entire process from OS point of view instead of .Net point of
view.

Thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no rights.

[Peter]

I let the program run longer before I took the readings and here are the
results, the 1 Gb only occurs after the program runs for few days. This
time the Windows Task Manager reported 905644 K

The program does not use any p/Invoke or COM directly, but it's using
Crystal Reports .NET library and I don't know if Crystal Reports is using
any p/Invoike..

!eeheap

Loader Heap:
--------------------------------------
System Domain: 7a3bc8b8
LowFrequencyHeap: Size: 0x0(0)bytes.
HighFrequencyHeap: 00672000(8000:1000) Size: 0x1000(4096)bytes.
StubHeap: 0067a000(2000:2000) 04150000(10000:8000) Size: 0xa000(40960)bytes.
Virtual Call Stub Heap:
IndcellHeap: Size: 0x0(0)bytes.
LookupHeap: Size: 0x0(0)bytes.
ResolveHeap: Size: 0x0(0)bytes.
DispatchHeap: Size: 0x0(0)bytes.
CacheEntryHeap: Size: 0x0(0)bytes.
Total size: 0xb000(45056)bytes
--------------------------------------
Shared Domain: 7a3bc560
LowFrequencyHeap: 006a0000(2000:1000) Size: 0x1000(4096)bytes.
HighFrequencyHeap: 006a2000(8000:1000) Size: 0x1000(4096)bytes.
StubHeap: 006aa000(2000:1000) Size: 0x1000(4096)bytes.
Virtual Call Stub Heap:
IndcellHeap: 006b0000(2000:1000) Size: 0x1000(4096)bytes.
LookupHeap: 006b5000(2000:1000) Size: 0x1000(4096)bytes.
ResolveHeap: 006bb000(5000:1000) Size: 0x1000(4096)bytes.
DispatchHeap: 006b7000(4000:1000) Size: 0x1000(4096)bytes.
CacheEntryHeap: 006b2000(3000:1000) Size: 0x1000(4096)bytes.
Total size: 0x7000(28672)bytes
--------------------------------------
Domain 1: 15a790
LowFrequencyHeap: 00680000(2000:2000) 00b90000(10000:d000)
00d80000(10000:10000) 00dc0000(10000:f000) 03890000(10000:f000)
039a0000(10000:f000) 00a80000(10000:10000) 07d00000(10000:10000)
09a80000(10000:10000) 087a0000(10000:f000) 05230000(10000:f000)
058d0000(20000:20000) 05d50000(10000:c000) Size: 0xc6000(811008)bytes.
Wasted: 0x8000(32768)bytes.
HighFrequencyHeap: 00682000(8000:8000) 00d90000(10000:10000)
04e70000(10000:10000) 09a20000(10000:10000) 09a30000(10000:10000)
09ab0000(10000:10000) 05100000(10000:10000) 058f0000(10000:a000) Size:
0x72000(466944)bytes.
StubHeap: 0068a000(2000:2000) 07d30000(10000:4000) Size: 0x6000(24576)bytes.
Virtual Call Stub Heap:
IndcellHeap: 00690000(2000:1000) Size: 0x1000(4096)bytes.
LookupHeap: 00696000(1000:1000) Size: 0x1000(4096)bytes.
ResolveHeap: 0069a000(6000:5000) Size: 0x5000(20480)bytes.
DispatchHeap: 00697000(3000:2000) Size: 0x2000(8192)bytes.
CacheEntryHeap: 00692000(4000:1000) Size: 0x1000(4096)bytes.
Total size: 0x147000(1339392)bytes
--------------------------------------
Jit code heap:
LoaderCodeHeap: 05990000(10000:a000) Size: 0xa000(40960)bytes.
LoaderCodeHeap: 05900000(10000:10000) Size: 0x10000(65536)bytes.
LoaderCodeHeap: 05200000(10000:10000) Size: 0x10000(65536)bytes.
LoaderCodeHeap: 087d0000(10000:10000) Size: 0x10000(65536)bytes.
LoaderCodeHeap: 08790000(10000:10000) Size: 0x10000(65536)bytes.
LoaderCodeHeap: 07cf0000(10000:10000) Size: 0x10000(65536)bytes.
LoaderCodeHeap: 05360000(10000:10000) Size: 0x10000(65536)bytes.
LoaderCodeHeap: 00b10000(10000:d000) Size: 0xd000(53248)bytes.
LoaderCodeHeap: 00ba0000(10000:d000) Size: 0xd000(53248)bytes.
Total size: 0x84000(540672)bytes
--------------------------------------
Module Thunk heaps:
Module 790c2000: Size: 0x0(0)bytes.
Module 006a239c: Size: 0x0(0)bytes.
Module 006a2010: Size: 0x0(0)bytes.
Module 00682c3c: Size: 0x0(0)bytes.
Module 67a30000: Size: 0x0(0)bytes.
Module 7a726000: Size: 0x0(0)bytes.
Module 00683e94: Size: 0x0(0)bytes.
Module 006851d4: Size: 0x0(0)bytes.
Module 006862f4: Size: 0x0(0)bytes.
Module 7b454000: Size: 0x0(0)bytes.
Module 7ae74000: Size: 0x0(0)bytes.
Module 00d90084: Size: 0x0(0)bytes.
Module 00d9070c: Size: 0x0(0)bytes.
Module 00d90ea4: Size: 0x0(0)bytes.
Module 648ea000: Size: 0x0(0)bytes.
Module 639f8000: Size: 0x0(0)bytes.
Module 00d92568: Size: 0x0(0)bytes.
Module 653f2000: 03860000(10000:1000) Size: 0x1000(4096)bytes.
Module 00d95f14: Size: 0x0(0)bytes.
Module 00d97404: Size: 0x0(0)bytes.
Module 00d983c0: Size: 0x0(0)bytes.
Module 00d9901c: Size: 0x0(0)bytes.
Module 00d995b8: Size: 0x0(0)bytes.
Module 00d9ab64: Size: 0x0(0)bytes.
Module 6638c000: Size: 0x0(0)bytes.
Module 67b2e000: Size: 0x0(0)bytes.
Module 67426000: Size: 0x0(0)bytes.
Module 04cf4000: 04d50000(10000:2000) Size: 0x2000(8192)bytes.
Module 6037e000: 07ce0000(10000:2000) Size: 0x2000(8192)bytes.
Module 04e7a4e4: Size: 0x0(0)bytes.
Module 65da0000: Size: 0x0(0)bytes.
Module 09a36184: Size: 0x0(0)bytes.
Module 09a36e0c: Size: 0x0(0)bytes.
Module 09abcab4: Size: 0x0(0)bytes.
Module 09abfaac: Size: 0x0(0)bytes.
Module 05103434: Size: 0x0(0)bytes.
Total size: 0x5000(20480)bytes
--------------------------------------
Module Lookup Table heaps:
Module 790c2000: Size: 0x0(0)bytes.
Module 006a239c: Size: 0x0(0)bytes.
Module 006a2010: Size: 0x0(0)bytes.
Module 00682c3c: Size: 0x0(0)bytes.
Module 67a30000: Size: 0x0(0)bytes.
Module 7a726000: Size: 0x0(0)bytes.
Module 00683e94: Size: 0x0(0)bytes.
Module 006851d4: Size: 0x0(0)bytes.
Module 006862f4: Size: 0x0(0)bytes.
Module 7b454000: Size: 0x0(0)bytes.
Module 7ae74000: Size: 0x0(0)bytes.
Module 00d90084: Size: 0x0(0)bytes.
Module 00d9070c: Size: 0x0(0)bytes.
Module 00d90ea4: Size: 0x0(0)bytes.
Module 648ea000: Size: 0x0(0)bytes.
Module 639f8000: Size: 0x0(0)bytes.
Module 00d92568: Size: 0x0(0)bytes.
Module 653f2000: Size: 0x0(0)bytes.
Module 00d95f14: Size: 0x0(0)bytes.
Module 00d97404: Size: 0x0(0)bytes.
Module 00d983c0: Size: 0x0(0)bytes.
Module 00d9901c: Size: 0x0(0)bytes.
Module 00d995b8: Size: 0x0(0)bytes.
Module 00d9ab64: Size: 0x0(0)bytes.
Module 6638c000: Size: 0x0(0)bytes.
Module 67b2e000: Size: 0x0(0)bytes.
Module 67426000: Size: 0x0(0)bytes.
Module 04cf4000: Size: 0x0(0)bytes.
Module 6037e000: Size: 0x0(0)bytes.
Module 04e7a4e4: Size: 0x0(0)bytes.
Module 65da0000: Size: 0x0(0)bytes.
Module 09a36184: Size: 0x0(0)bytes.
Module 09a36e0c: Size: 0x0(0)bytes.
Module 09abcab4: Size: 0x0(0)bytes.
Module 09abfaac: Size: 0x0(0)bytes.
Module 05103434: Size: 0x0(0)bytes.
Total size: 0x0(0)bytes
--------------------------------------
Total LoaderHeap size: 0x1e2000(1974272)bytes
=======================================
Number of GC Heaps: 1
generation 0 starts at 0x215e4520
generation 1 starts at 0x215e39e8
generation 2 starts at 0x00fd1000
ephemeral segment allocation context: none
segment begin allocated size
001960a0 7b463c40 7b47a744 0x00016b04(92932)
0017c2a8 7a733370 7a754b98 0x00021828(137256)
00170ec0 790d8620 790f7d8c 0x0001f76c(128876)
00fd0000 00fd1000 01fcc05c 0x00ffb05c(16756828)
1f9e0000 1f9e1000 209a0744 0x00fbf744(16512836)
1e1e0000 1e1e1000 1f1ad534 0x00fcc534(16565556)
209e0000 209e1000 21631628 0x00c50628(12912168)
Large object heap starts at 0x01fd1000
segment begin allocated size
01fd0000 01fd1000 02278b48 0x002a7b48(2784072)
Total Size 0x3ed68dc(65890524)
------------------------------
GC Heap Size 0x3ed68dc(65890524)
----------------------------------------------------------------------------------------------------------------------------------------------------------------
!address -summary

-------------------- Usage SUMMARY --------------------------
TotSize ( KB) Pct(Tots) Pct(Busy) Usage
a2dc000 ( 166768) : 07.95% 14.11% : RegionUsageIsVAD
37dcd000 ( 915252) : 43.64% 00.00% : RegionUsageFree
abe3000 ( 176012) : 08.39% 14.89% : RegionUsageImage
1210000 ( 18496) : 00.88% 01.57% : RegionUsageStack
13000 ( 76) : 00.00% 00.01% : RegionUsageTeb
3213e000 ( 820472) : 39.12% 69.42% : RegionUsageHeap
0 ( 0) : 00.00% 00.00% : RegionUsagePageHeap
1000 ( 4) : 00.00% 00.00% : RegionUsagePeb
1000 ( 4) : 00.00% 00.00% : RegionUsageProcessParametrs
1000 ( 4) : 00.00% 00.00% : RegionUsageEnvironmentBlock
Tot: 7fff0000 (2097088 KB) Busy: 48223000 (1181836 KB)

-------------------- Type SUMMARY --------------------------
TotSize ( KB) Pct(Tots) Usage
37dcd000 ( 915252) : 43.64% : <free>
b4b3000 ( 185036) : 08.82% : MEM_IMAGE
759000 ( 7524) : 00.36% : MEM_MAPPED
3c617000 ( 989276) : 47.17% : MEM_PRIVATE

-------------------- State SUMMARY --------------------------
TotSize ( KB) Pct(Tots) Usage
40baf000 ( 1060540) : 50.57% : MEM_COMMIT
37dcd000 ( 915252) : 43.64% : MEM_FREE
7674000 ( 121296) : 05.78% : MEM_RESERVE

Largest free region: Base 6dffa000 - Size 03ae6000 (60312 KB)
""Jeffrey Tan[MSFT]"" <je***@online.microsoft.comwrote in message
news:1K**************@TK2MSFTNGHUB02.phx.gbl...

Hi Peter,

Thanks for your feedback.

As the !eeheap output stated, the GC heap only costs more than 18M of
memory(18775776) which is not the culprit for the memory leak(as you
original stated, the memory leak costs more than 1Gb of memory). So, I
should conclude this as an unmanaged memory leak.

Does your application use any unmanaged code? For example, do you use
p/invoke or COM interop in the application? Do you use any 3rd party
component that may cause the unmanaged memory leak?

Also, I recommend you to issue "!address -summary" command in windbg and
paste the result here. This command will generate a memory usage summary
report of the entire process from OS point of view instead of .Net point
of
view.

Thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no
rights.

[Jeffrey Tan[MSFT]]

Hi Peter,

Thanks for the further information, it is quite useful.

From the "!eeheap" output we can see that GC heap only costs 66M of memory
which is much less than the 900M total memory usage. From the "!address
-summary" output, we finally caught the culprit RegionUsageHeap which costs
more than 800M(820472KB) of memory. Since RegionUsageHeap indicates the
Windows native heap, this should be an unmanaged memory leak.

In this scenario, I would recommend you to use umdh.exe to troubleshooting
the native heap memory leak. UMDH is a tool followed with windbg package.
It will can turn-on and query the Windows native heap manager memory
tracking ability. During the memory leaking period, you may use UMDH to
take two snapshots of the heap manager states. Then, you may use the
build-in function of UMDH to compare these two snapshots and find out the
memory leaking point. The KB below demonstrates the detailed steps:
"Umdhtools.exe: How to use Umdh.exe to find memory leaks"
http://support.microsoft.com/default.aspx/kb/268343

Hope it helps.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no rights.

[Peter]

I do not understand what I have to do.

I went to http://www.microsoft.com/whdc/DevToo...default.mspx#c
and downloaded the ISO file created CD and installed it on the hard drive,
but I do not have gflags or Umdh.exe anywhere on the hard drive so obviously
I am installing the wrong thing.

What should I install?
""Jeffrey Tan[MSFT]"" <je***@online.microsoft.comwrote in message
news:m4****************@TK2MSFTNGHUB02.phx.gbl...

Hi Peter,

Thanks for the further information, it is quite useful.

From the "!eeheap" output we can see that GC heap only costs 66M of memory
which is much less than the 900M total memory usage. From the "!address
-summary" output, we finally caught the culprit RegionUsageHeap which
costs
more than 800M(820472KB) of memory. Since RegionUsageHeap indicates the
Windows native heap, this should be an unmanaged memory leak.

In this scenario, I would recommend you to use umdh.exe to troubleshooting
the native heap memory leak. UMDH is a tool followed with windbg package.
It will can turn-on and query the Windows native heap manager memory
tracking ability. During the memory leaking period, you may use UMDH to
take two snapshots of the heap manager states. Then, you may use the
build-in function of UMDH to compare these two snapshots and find out the
memory leaking point. The KB below demonstrates the detailed steps:
"Umdhtools.exe: How to use Umdh.exe to find memory leaks"
http://support.microsoft.com/default.aspx/kb/268343

Hope it helps.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no
rights.

[Jeffrey Tan[MSFT]]

Hi Peter,

Thanks for the feedback.

I assume you have installed windbg yet because you can use "!address
-summary" command. If so, there is no need to install it again. Umdh.exe is
a tool followed with windbg, so if you have installed windbg, umdh is
there. You can find the umdh.exe by going to windbg's installation
directory, such as "D:\Debugging Tools for Windows (x86)". You will find
umdh.exe in this directory. Gflags.exe is in this directory either.

Thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no rights.

[Peter]

I have ran umdh application and created the log files but I do not know how
to interpert them.

the log files are over 1 MB zipped so I can't send the zip file to the news
group.

""Jeffrey Tan[MSFT]"" <je***@online.microsoft.comwrote in message
news:3m**************@TK2MSFTNGHUB02.phx.gbl...

Hi Peter,

Thanks for the feedback.

I assume you have installed windbg yet because you can use "!address
-summary" command. If so, there is no need to install it again. Umdh.exe
is
a tool followed with windbg, so if you have installed windbg, umdh is
there. You can find the umdh.exe by going to windbg's installation
directory, such as "D:\Debugging Tools for Windows (x86)". You will find
umdh.exe in this directory. Gflags.exe is in this directory either.

Thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no
rights.

[Jeffrey Tan[MSFT]]

Hi Peter,

Thanks for your feedback.

First, you should create two log files for the application. You need to
take one log and perform some operation to reproduce the memory leak, and
then take a second log file.

After getting two log files, we may use the build-in analyze function of
UMDH.exe to compare these two log files. This is documented in the "Use
Umdh.exe to Compare UMDH Logs" section in the KB below:
http://support.microsoft.com/kb/q268343/

Let me explain the basic idea/logic here: the log file contains all the
Win32 native heap allocation stack traces(which explains the root cause of
each allocation), you can think of them as snapshot of the heap. Many of
the allocations are legal and are not the culprit of the memory leak, so we
need a way to find out the culprit allocation for the leaking. That is the
reason for comparing two log files. By comparing these two log files, the
UMDH will output the allocations that did not happen in the first snapshot
but happen between first and second snapshot(new allocations). These new
allocations are very likely the culprit for the memory leak.

After the comparison, the generated output file should big a much smaller
one. Can you paste the comparison output file here for analysis?

Also, the "UMDH Output Explained" section in the KB above explains how to
interpret the output if you want to analyze it yourself.

Thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no rights.

[Peter]

I've tried to paste the comparison file bu it's still complaing the message
is too big to post to news group.
But I am pastting the end part of the file, don't know if it will do any
good.

Here's the output from the comparison

- 26000 ( 0 - 26000) 0 allocs BackTrace7331
- 26000 ( 0 - 26000) 0 allocs BackTrace7301
- 26520 ( 6528 - 33048) 544 allocs BackTrace578A
- 27104 ( 46496 - 73600) 604 allocs BackTrace578E
- 29352 ( 0 - 29352) 0 allocs BackTrace6C66
- 31972 ( 0 - 31972) 0 allocs BackTrace5453
- 35776 ( 0 - 35776) 0 allocs BackTrace7330
- 35776 ( 0 - 35776) 0 allocs BackTrace7300
- 37104 ( 0 - 37104) 0 allocs BackTrace538D
- 38880 ( 0 - 38880) 0 allocs BackTrace72FB
- 38880 ( 0 - 38880) 0 allocs BackTrace732B
- 41435 ( 0 - 41435) 0 allocs BackTraceBED0
- 42626 ( 0 - 42626) 0 allocs BackTrace7D62
- 43168 ( 18656 - 61824) 638 allocs BackTrace5790
- 43420 ( 0 - 43420) 0 allocs BackTrace7D64
- 43680 ( 0 - 43680) 0 allocs BackTrace7D5F
- 46304 ( 4704 - 51008) 147 allocs BackTrace57CE
- 46992 ( 47872 - 94864) 544 allocs BackTrace5786
- 47344 ( 26224 - 73568) 149 allocs BackTrace6270
- 48544 ( 0 - 48544) 0 allocs BackTrace6C5F
- 48840 ( 0 - 48840) 0 allocs BackTrace72FA
- 49674 ( 2174 - 51848) 85 allocs BackTrace31CE
- 51216 ( 0 - 51216) 0 allocs BackTrace732A
- 62624 ( 32560 - 95184) 1037 allocs BackTrace5D38
- 63488 ( 0 - 63488) 0 allocs BackTrace7D5E
- 68992 ( 18912 - 87904) 591 allocs BackTrace5791
- 70238 ( 278 - 70516) 11 allocs BackTrace2F00
- 70848 ( 16160 - 87008) 505 allocs BackTrace5787
- 110186 ( 0 - 110186) 0 allocs BackTrace6C64
- 110186 ( 0 - 110186) 0 allocs BackTrace6C62
- 151652 ( 0 - 151652) 0 allocs BackTrace6C5E
- 1383069 ( 41467 - 1424536) 586 allocs BackTraceF48
Total increase == -3437444 requested + -1574084 overhead = -5011528

Also:
I've ran the program and took the snap shot 2 times - 1 before and 1 after,
but the output does not make any sense to me.
For Instance I've serverched for BackTrace5787 in the 'after' snapshot and
this is what I found

20 bytes + 18 at 6AC3DC0 by BackTrace5787
7C8531E4
7C83D97A
7C8339E1
7C833D2D
77E6786F
6100119E
610011E5
61036F44
610013CE
43454C16

30 bytes + 18 at 6AFA3D8 by BackTrace5748
10 bytes + 18 at 6AFA420 by BackTrace578E
10 bytes + 18 at 6AFA448 by BackTrace5790
20 bytes + 18 at 6AFA470 by BackTrace5791
20 bytes + 18 at 6AFA4A8 by BackTrace57CE
20 bytes + 18 at 6AFA4E0 by BackTrace5787
C bytes + 1C at 6AFA518 by BackTrace578A
10 bytes + 18 at 6AFA540 by BackTrace5790
20 bytes + 18 at 6AFA568 by BackTrace5791
20 bytes + 18 at 6AFA5A0 by BackTrace57CE
So I don't know how would I find the location im my application.
""Jeffrey Tan[MSFT]"" <je***@online.microsoft.comwrote in message
news:X6**************@TK2MSFTNGHUB02.phx.gbl...

Hi Peter,

Thanks for your feedback.

First, you should create two log files for the application. You need to
take one log and perform some operation to reproduce the memory leak, and
then take a second log file.

After getting two log files, we may use the build-in analyze function of
UMDH.exe to compare these two log files. This is documented in the "Use
Umdh.exe to Compare UMDH Logs" section in the KB below:
http://support.microsoft.com/kb/q268343/

Let me explain the basic idea/logic here: the log file contains all the
Win32 native heap allocation stack traces(which explains the root cause of
each allocation), you can think of them as snapshot of the heap. Many of
the allocations are legal and are not the culprit of the memory leak, so
we
need a way to find out the culprit allocation for the leaking. That is the
reason for comparing two log files. By comparing these two log files, the
UMDH will output the allocations that did not happen in the first snapshot
but happen between first and second snapshot(new allocations). These new
allocations are very likely the culprit for the memory leak.

After the comparison, the generated output file should big a much smaller
one. Can you paste the comparison output file here for analysis?

Also, the "UMDH Output Explained" section in the KB above explains how to
interpret the output if you want to analyze it yourself.

Thanks.

Best regards,
Jeffrey Tan
Microsoft Online Community Support
=========================================
Delighting our customers is our #1 priority. We welcome your comments and
suggestions about how we can improve the support we provide to you. Please
feel free to let my manager know what you think of the level of service
provided. You can send feedback directly to my manager at:
ms****@microsoft.com.

This posting is provided "AS IS" with no warranties, and confers no
rights.