Large memory footprint of dotnet application

[assi]

Hello all

We are developing a large dotnet application, which includes ~ 120
assemblies. (total size of all binaries is ~ 20MB). Our application
also references the following dotnet assemblies: System,System.XML,
System.Windows.Forms, System.Drawing, System.Data,
System.Design. We use dotnet framework 1.1

During initialization, the application scans a directory and loads the
assemblies (using Assembly.LoadFrom). It then scans all types in the
single module in the assembly (using GetModules and Module.GetTypes),
and registers them in application collections by identifying the
properties of each class using reflection.

During the initialization process the memory footprint (both private
bytes and working set) rises to about 220MB, when
#Bytes in all heaps' perfmon counter shows 30MB. This occurs when the
assemlies are scanned, not when they are loaded. The high footprint
causes page faults when the application executes on
some of the target platforms, which has only 256M memory.

To try to account for the 190MB not in the managed heaps, we tried to
estimate the size fo the assemblies in memory when loading. We
pre-jitted the assemblies and got a 1:7 ratio between the
native image and the assembly, which Accoutns for 20MB (total size of
assemblies) x 7 = 140, to which we can add the system assemblies.

Our questions are:

1) How can we calculate the memory footprint of the assemblies? Are
their any tools to do this? (we tried vadump.exe,however we do nto
fully understand the counters)?

2) Does IL code remain in memory, assuming we do not make any
additional reflection calls after startup? Can this behavior
Be controlled? Does pre-jitting change this behavior?
3) How can we reduce the memory footprint?
- Will linking the assemblies into a single module with multiple
dlls help?
- From reading some articles we understand that the dotnet runtime
pre-allocates large amouts of memory from the unamaged heap if it
notices the apllication is making many managed allocations, and not all
of this memory if later released. Can this behavoir be controlled?

Any ideas, anyone?

assi barak

[AMercer]

Questions like yours keep coming up in these discussion groups. That
suggests to me that MS should make clearer their memory management practices.
I suggest you search these discussion groups for the following phrases:
WaitForPendingFinalizers
GetTotalMemory
SetProcessWorkingSetSize
You will find interesting reading. The first two are in the GC class, the
third is a win32 api.

I do two things when I want to know what's up with memory. Maybe you should
try them as well, because I have stopped worrying about large footprints
since I know my memory usage is not so bad after all. First, I call
SetProcessWorkingSetSize with -1 and -1 to trim the working set. Then
Diagnostics.Process.GetCurrentProcess.WorkingSet will return a much smaller
number. Second, I do three iterations of garbage collections like this:
Dim i As Integer
Dim l As Long
For i = 1 To 3
GC.Collect()
GC.WaitForPendingFinalizers()
l = GC.GetTotalMemory(True)
Next
At this point, GetTotalMemory returns a low number (not always the lowest,
but such is life).

I only do these things in a development and debugging environment to see if
I have a problem. In production, I think it is better practice to let
MS/Windows/.Net manage memory rather than having me do it.
"assi" wrote:

Hello all

We are developing a large dotnet application, which includes ~ 120
assemblies. (total size of all binaries is ~ 20MB). Our application
also references the following dotnet assemblies: System,System.XML,
System.Windows.Forms, System.Drawing, System.Data,
System.Design. We use dotnet framework 1.1

During initialization, the application scans a directory and loads the
assemblies (using Assembly.LoadFrom). It then scans all types in the
single module in the assembly (using GetModules and Module.GetTypes),
and registers them in application collections by identifying the
properties of each class using reflection.

During the initialization process the memory footprint (both private
bytes and working set) rises to about 220MB, when
#Bytes in all heaps' perfmon counter shows 30MB. This occurs when the
assemlies are scanned, not when they are loaded. The high footprint
causes page faults when the application executes on
some of the target platforms, which has only 256M memory.

To try to account for the 190MB not in the managed heaps, we tried to
estimate the size fo the assemblies in memory when loading. We
pre-jitted the assemblies and got a 1:7 ratio between the
native image and the assembly, which Accoutns for 20MB (total size of
assemblies) x 7 = 140, to which we can add the system assemblies.

Our questions are:

1) How can we calculate the memory footprint of the assemblies? Are
their any tools to do this? (we tried vadump.exe,however we do nto
fully understand the counters)?

2) Does IL code remain in memory, assuming we do not make any
additional reflection calls after startup? Can this behavior
Be controlled? Does pre-jitting change this behavior?
3) How can we reduce the memory footprint?
- Will linking the assemblies into a single module with multiple
dlls help?
- From reading some articles we understand that the dotnet runtime
pre-allocates large amouts of memory from the unamaged heap if it
notices the apllication is making many managed allocations, and not all
of this memory if later released. Can this behavoir be controlled?

Any ideas, anyone?

assi barak

[Willy Denoyette [MVP]]

"assi" <as********@hotmail.com> wrote in message
news:11**********************@o13g2000cwo.googlegr oups.com...

Hello all

We are developing a large dotnet application, which includes ~ 120
assemblies. (total size of all binaries is ~ 20MB). Our application
also references the following dotnet assemblies: System,System.XML,
System.Windows.Forms, System.Drawing, System.Data,
System.Design. We use dotnet framework 1.1

During initialization, the application scans a directory and loads the
assemblies (using Assembly.LoadFrom). It then scans all types in the
single module in the assembly (using GetModules and Module.GetTypes),
and registers them in application collections by identifying the
properties of each class using reflection.

During the initialization process the memory footprint (both private
bytes and working set) rises to about 220MB, when
#Bytes in all heaps' perfmon counter shows 30MB. This occurs when the
assemlies are scanned, not when they are loaded. The high footprint
causes page faults when the application executes on
some of the target platforms, which has only 256M memory.

To try to account for the 190MB not in the managed heaps, we tried to
estimate the size fo the assemblies in memory when loading. We
pre-jitted the assemblies and got a 1:7 ratio between the
native image and the assembly, which Accoutns for 20MB (total size of
assemblies) x 7 = 140, to which we can add the system assemblies.

Our questions are:

1) How can we calculate the memory footprint of the assemblies? Are
their any tools to do this? (we tried vadump.exe,however we do nto
fully understand the counters)?

2) Does IL code remain in memory, assuming we do not make any
additional reflection calls after startup? Can this behavior
Be controlled? Does pre-jitting change this behavior?
3) How can we reduce the memory footprint?
- Will linking the assemblies into a single module with multiple
dlls help?
- From reading some articles we understand that the dotnet runtime
pre-allocates large amouts of memory from the unamaged heap if it
notices the apllication is making many managed allocations, and not all
of this memory if later released. Can this behavoir be controlled?

Any ideas, anyone?

assi barak

Don't know why you are loading all assemblies and scanning them to get the
attributes of all classes.
By doing so, not only do you consume a lot of memory for the DLL's (IL and
metadata) and the administrative data structures needed for the assemblies
loaded, you also consume a fixed amount of 30MB for the "scanned application
collections" before you even have executed a single method of the real
application code - so there is little JITTED code bytes in your private
bytes. Note that the loaded assemblies (your private and the loaded
Framework assemblies), the JITTED code ,the CLR code and Data pages and the
30MB of managed heap is the minimum fixed part of your WS, and this for the
whole duration of your application.

Willy.