Memory Leak in .NET 1.1

[MAG1301]

I've detected memory leaks in our huge .NET 1.1 C# application but
couldn't localize them directly. So I've reduced the code to the
following console application:

using System;
using System.IO;

namespace MemLeak
{
class MemLeak
{
[STAThread]
static void Main(string[] args)
{
try
{
throw new NullReferenceException();
}
catch
{
}

for(int i=0; i<1000; i++)
{
using(StreamWriter newMasterfile =
new StreamWriter(@"c:\XXX.tmp"))
{
newMasterfile.Close();
}
}

Console.ReadLine();
}
}
}

Also you have to create an "App.config" file which is distributed by
VS2003 into the application directory with the name of the application
followed by ".config". There may be any content, the file can be empty
too - this will not influence our result.

Compile the "Release" configuration and you'll see with a tool like
"NetMemoryProfiler" that there are almost 1000 StreamWriter reserved at
"Console.ReadLine". You can wait as long as you want - they'll never be
freed. If you don't have such a tool: increase the number of loops.
Then you can use the task manager to see how the memory grows (look at
the virtual memory!). With an endless loop there will be an "Out of
virtual memory" error at the end.

This problem is not only bounded to StreamWriter. We've seen this for
database objects, simple strings and other.

We have tested this with several Windows XP SP2 Systems with different
configurations. Also with the latest fixes for Windows and .NET.

The funny thing: there will be no leaks when you use the "Debug"
configuration. Also you can omit the exception or the "App.config"
file. In these cases the memory will be freed correctly. But we will
comile in "Release" configuration, use the "App.config" and Exceptions
can't be avoided.

The last option is to switch the attribute "[STAThread]" to
"[MTAThread]". The memory is ok in the small example (why?) and also in
our huge application. But I'm not sure with the COM-components we use.
So I prefer the "[STAThread]" attribute.

There are some methods which you can tell the gabage collector to clean
up: "GC.GetTotalMemory(...)", "GC.WaitForPendingFinalizers()" and
"GC.Collect()". You can find combinations in the small example to get
rid of the memory. But this is very unreliable and will not work
correctly in our huge application where late binding and remoting is
used.

After some days of trial and error, I am really at my wits' end over
this problem. So is there anybody who can help me?

Many thanks in advance
Martin Gaus

[Ludwig]

On 28 May 2006 05:00:33 -0700, MA*****@gmx.net wrote:

I've detected memory leaks in our huge .NET 1.1 C# application but
couldn't localize them directly. So I've reduced the code to the
following console application:

using System;
using System.IO;

namespace MemLeak
{
class MemLeak
{
[STAThread]
static void Main(string[] args)
{
try
{
throw new NullReferenceException();
}
catch
{
}

for(int i=0; i<1000; i++)
{
using(StreamWriter newMasterfile =
new StreamWriter(@"c:\XXX.tmp"))
{
newMasterfile.Close();
}
}

Console.ReadLine();
}
}
}

Also you have to create an "App.config" file which is distributed by
VS2003 into the application directory with the name of the application
followed by ".config". There may be any content, the file can be empty
too - this will not influence our result.

Compile the "Release" configuration and you'll see with a tool like
"NetMemoryProfiler" that there are almost 1000 StreamWriter reserved at
"Console.ReadLine". You can wait as long as you want - they'll never be
freed. If you don't have such a tool: increase the number of loops.
Then you can use the task manager to see how the memory grows (look at
the virtual memory!). With an endless loop there will be an "Out of
virtual memory" error at the end.

This problem is not only bounded to StreamWriter. We've seen this for
database objects, simple strings and other.

We have tested this with several Windows XP SP2 Systems with different
configurations. Also with the latest fixes for Windows and .NET.

The funny thing: there will be no leaks when you use the "Debug"
configuration. Also you can omit the exception or the "App.config"
file. In these cases the memory will be freed correctly. But we will
comile in "Release" configuration, use the "App.config" and Exceptions
can't be avoided.

The last option is to switch the attribute "[STAThread]" to
"[MTAThread]". The memory is ok in the small example (why?) and also in
our huge application. But I'm not sure with the COM-components we use.
So I prefer the "[STAThread]" attribute.

There are some methods which you can tell the gabage collector to clean
up: "GC.GetTotalMemory(...)", "GC.WaitForPendingFinalizers()" and
"GC.Collect()". You can find combinations in the small example to get
rid of the memory. But this is very unreliable and will not work
correctly in our huge application where late binding and remoting is
used.

After some days of trial and error, I am really at my wits' end over
this problem. So is there anybody who can help me?

Many thanks in advance
Martin Gaus

just wondering (without trying it)

newMasterfile.Close();
newMasterfile=null;

does this make any difference?
--
Ludwig Stuyck
http://www.coders-lab.be

[Martin]

using(StreamWriter newMasterfile = new StreamWriter( @"c:\XXX.tmp" ))
{
newMasterfile.Close();
newMasterfile = null;
}

Something like this is not possible. The compiler complains that
"newMasterfile" is write protected. But I've tested the following with
the same result (with leaks):

StreamWriter newMasterfile = new StreamWriter( @"c:\XXX.tmp" );
newMasterfile.Close();
newMasterfile = null;

[Greg Young]

There does not seem to be a memory leak here (atleast on my system) .. try
the included code .. on my system it gets up to about 8mb of memory .. then
flattens out (if there were a memory leak here I can assure you that running
10,000,000 times as opposed to 1000 would show a severe difference in memory
usage)

Most likely the garbage collector has just not collected those items yet
(and with no new items being created, and no pressure on the memory it
already has allocated why should it hurry?). Once some pressure is placed
upon it the gc seems to pick things up without issue

In your larger system my first place to look given the things you mention
would be the large object heap to see if it is growing .. if it is, whats in
it?

Cheers,

Greg Young
MVP - C#

using System;
using System.IO;

namespace MemLeak
{
class MemLeak
{
[STAThread]
static void Main(string[] args)
{
try
{
throw new NullReferenceException();
}
catch
{
}

for(int i=0; i<10000000; i++)
{
using(StreamWriter newMasterfile =
new StreamWriter(@"c:\XXX.tmp"))
{
newMasterfile.Close();
}
}

Console.ReadLine();
}
}
}
<MA*****@gmx.net> wrote in message
news:11**********************@j33g2000cwa.googlegr oups.com...

I've detected memory leaks in our huge .NET 1.1 C# application but
couldn't localize them directly. So I've reduced the code to the
following console application:

using System;
using System.IO;

namespace MemLeak
{
class MemLeak
{
[STAThread]
static void Main(string[] args)
{
try
{
throw new NullReferenceException();
}
catch
{
}

for(int i=0; i<1000; i++)
{
using(StreamWriter newMasterfile =
new StreamWriter(@"c:\XXX.tmp"))
{
newMasterfile.Close();
}
}

Console.ReadLine();
}
}
}

Also you have to create an "App.config" file which is distributed by
VS2003 into the application directory with the name of the application
followed by ".config". There may be any content, the file can be empty
too - this will not influence our result.

Compile the "Release" configuration and you'll see with a tool like
"NetMemoryProfiler" that there are almost 1000 StreamWriter reserved at
"Console.ReadLine". You can wait as long as you want - they'll never be
freed. If you don't have such a tool: increase the number of loops.
Then you can use the task manager to see how the memory grows (look at
the virtual memory!). With an endless loop there will be an "Out of
virtual memory" error at the end.

This problem is not only bounded to StreamWriter. We've seen this for
database objects, simple strings and other.

We have tested this with several Windows XP SP2 Systems with different
configurations. Also with the latest fixes for Windows and .NET.

The funny thing: there will be no leaks when you use the "Debug"
configuration. Also you can omit the exception or the "App.config"
file. In these cases the memory will be freed correctly. But we will
comile in "Release" configuration, use the "App.config" and Exceptions
can't be avoided.

The last option is to switch the attribute "[STAThread]" to
"[MTAThread]". The memory is ok in the small example (why?) and also in
our huge application. But I'm not sure with the COM-components we use.
So I prefer the "[STAThread]" attribute.

There are some methods which you can tell the gabage collector to clean
up: "GC.GetTotalMemory(...)", "GC.WaitForPendingFinalizers()" and
"GC.Collect()". You can find combinations in the small example to get
rid of the memory. But this is very unreliable and will not work
correctly in our huge application where late binding and remoting is
used.

After some days of trial and error, I am really at my wits' end over
this problem. So is there anybody who can help me?

Many thanks in advance
Martin Gaus

[Ludwig]

On 28 May 2006 05:25:54 -0700, "Martin" <MA*****@gmx.net> wrote:

using(StreamWriter newMasterfile = new StreamWriter( @"c:\XXX.tmp" ))
{
newMasterfile.Close();
newMasterfile = null;
}

Something like this is not possible. The compiler complains that
"newMasterfile" is write protected. But I've tested the following with
the same result (with leaks):

StreamWriter newMasterfile = new StreamWriter( @"c:\XXX.tmp" );
newMasterfile.Close();
newMasterfile = null;

Well I tried it and used the performance monitor to monitor the
available bytes. I used a loop of 1000000000.
Result: number of available bytes decreased slowly. After 10 minutes
it went up again. After 20 minutes I see no evidence of a memory leak.
--
Ludwig Stuyck
http://www.coders-lab.be

[Nicholas Paldino [.NET/C# MVP]]

With this example, why ^wouldn't^ there be a bunch of StreamWriter
objects? The way that GC works, unless there was a prior GC, those objects
will reside in memory. Just because you close them doesn't mean that the
instances are still on the heap. Only the resources that they manage which
are critical (in this case, the file handle for the underlying stream) are
released when Close is called.

Then, you call ReadLine, which uses P/Invoke ultimately. Since it is
waiting in an unmanaged piece of code (and more than likely, you are running
the CLR in a workstation configuration with the GC occuring on the executing
thread), you aren't going to be able to have a GC occur.

What you are seeing in this example is completely predictable and
expected.
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com

<MA*****@gmx.net> wrote in message
news:11**********************@j33g2000cwa.googlegr oups.com...

I've detected memory leaks in our huge .NET 1.1 C# application but
couldn't localize them directly. So I've reduced the code to the
following console application:

using System;
using System.IO;

namespace MemLeak
{
class MemLeak
{
[STAThread]
static void Main(string[] args)
{
try
{
throw new NullReferenceException();
}
catch
{
}

for(int i=0; i<1000; i++)
{
using(StreamWriter newMasterfile =
new StreamWriter(@"c:\XXX.tmp"))
{
newMasterfile.Close();
}
}

Console.ReadLine();
}
}
}

Also you have to create an "App.config" file which is distributed by
VS2003 into the application directory with the name of the application
followed by ".config". There may be any content, the file can be empty
too - this will not influence our result.

Compile the "Release" configuration and you'll see with a tool like
"NetMemoryProfiler" that there are almost 1000 StreamWriter reserved at
"Console.ReadLine". You can wait as long as you want - they'll never be
freed. If you don't have such a tool: increase the number of loops.
Then you can use the task manager to see how the memory grows (look at
the virtual memory!). With an endless loop there will be an "Out of
virtual memory" error at the end.

This problem is not only bounded to StreamWriter. We've seen this for
database objects, simple strings and other.

We have tested this with several Windows XP SP2 Systems with different
configurations. Also with the latest fixes for Windows and .NET.

The funny thing: there will be no leaks when you use the "Debug"
configuration. Also you can omit the exception or the "App.config"
file. In these cases the memory will be freed correctly. But we will
comile in "Release" configuration, use the "App.config" and Exceptions
can't be avoided.

The last option is to switch the attribute "[STAThread]" to
"[MTAThread]". The memory is ok in the small example (why?) and also in
our huge application. But I'm not sure with the COM-components we use.
So I prefer the "[STAThread]" attribute.

There are some methods which you can tell the gabage collector to clean
up: "GC.GetTotalMemory(...)", "GC.WaitForPendingFinalizers()" and
"GC.Collect()". You can find combinations in the small example to get
rid of the memory. But this is very unreliable and will not work
correctly in our huge application where late binding and remoting is
used.

After some days of trial and error, I am really at my wits' end over
this problem. So is there anybody who can help me?

Many thanks in advance
Martin Gaus

[Martin]

I'm wondering why you can't approve my observations.
I've seen this on several installations now.

Just to be sure:
- it must be the "Release" configuration
- in directory "bin\Release" there must be an application cofiguration
file.
I have two files there:
MemoryLeak.exe
MemoryLeak.exe.config

When I start my tests the virtual memory of the process will grow in
one minute from 5MB to 17MB and so on until all the virtual memory is
completly used. There will never be a gabage collection neither when
the memory is under pressure.

May I send you my solution incl. binary?

Martin

[Lebesgue]

And how would you explain that the problem goes away when they delete the
app.config file from the bin directory?

"Nicholas Paldino [.NET/C# MVP]" <mv*@spam.guard.caspershouse.com> wrote in
message news:OA**************@TK2MSFTNGP03.phx.gbl...

With this example, why ^wouldn't^ there be a bunch of StreamWriter
objects? The way that GC works, unless there was a prior GC, those
objects will reside in memory. Just because you close them doesn't mean
that the instances are still on the heap. Only the resources that they
manage which are critical (in this case, the file handle for the
underlying stream) are released when Close is called.

Then, you call ReadLine, which uses P/Invoke ultimately. Since it is
waiting in an unmanaged piece of code (and more than likely, you are
running the CLR in a workstation configuration with the GC occuring on the
executing thread), you aren't going to be able to have a GC occur.

What you are seeing in this example is completely predictable and
expected.
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com

<MA*****@gmx.net> wrote in message
news:11**********************@j33g2000cwa.googlegr oups.com...

I've detected memory leaks in our huge .NET 1.1 C# application but
couldn't localize them directly. So I've reduced the code to the
following console application:

using System;
using System.IO;

namespace MemLeak
{
class MemLeak
{
[STAThread]
static void Main(string[] args)
{
try
{
throw new NullReferenceException();
}
catch
{
}

for(int i=0; i<1000; i++)
{
using(StreamWriter newMasterfile =
new StreamWriter(@"c:\XXX.tmp"))
{
newMasterfile.Close();
}
}

Console.ReadLine();
}
}
}

Also you have to create an "App.config" file which is distributed by
VS2003 into the application directory with the name of the application
followed by ".config". There may be any content, the file can be empty
too - this will not influence our result.

Compile the "Release" configuration and you'll see with a tool like
"NetMemoryProfiler" that there are almost 1000 StreamWriter reserved at
"Console.ReadLine". You can wait as long as you want - they'll never be
freed. If you don't have such a tool: increase the number of loops.
Then you can use the task manager to see how the memory grows (look at
the virtual memory!). With an endless loop there will be an "Out of
virtual memory" error at the end.

This problem is not only bounded to StreamWriter. We've seen this for
database objects, simple strings and other.

We have tested this with several Windows XP SP2 Systems with different
configurations. Also with the latest fixes for Windows and .NET.

The funny thing: there will be no leaks when you use the "Debug"
configuration. Also you can omit the exception or the "App.config"
file. In these cases the memory will be freed correctly. But we will
comile in "Release" configuration, use the "App.config" and Exceptions
can't be avoided.

The last option is to switch the attribute "[STAThread]" to
"[MTAThread]". The memory is ok in the small example (why?) and also in
our huge application. But I'm not sure with the COM-components we use.
So I prefer the "[STAThread]" attribute.

There are some methods which you can tell the gabage collector to clean
up: "GC.GetTotalMemory(...)", "GC.WaitForPendingFinalizers()" and
"GC.Collect()". You can find combinations in the small example to get
rid of the memory. But this is very unreliable and will not work
correctly in our huge application where late binding and remoting is
used.

After some days of trial and error, I am really at my wits' end over
this problem. So is there anybody who can help me?

Many thanks in advance
Martin Gaus

[Artur-ek]

"Greg Young" <dr*******************@hotmail.com> wrote in message
news:uQ**************@TK2MSFTNGP04.phx.gbl...

There does not seem to be a memory leak here (atleast on my system)

the same here, no leak

System: Win 2003
Framework: 1.1.4322.2300
compiled in release mode, with config file in directory.

[Willy Denoyette [MVP]]

<MA*****@gmx.net> wrote in message
news:11**********************@j33g2000cwa.googlegr oups.com...
| I've detected memory leaks in our huge .NET 1.1 C# application but
| couldn't localize them directly. So I've reduced the code to the
| following console application:
|
| using System;
| using System.IO;
|
| namespace MemLeak
| {
| class MemLeak
| {
| [STAThread]
| static void Main(string[] args)
| {
| try
| {
| throw new NullReferenceException();
| }
| catch
| {
| }
|
| for(int i=0; i<1000; i++)
| {
| using(StreamWriter newMasterfile =
| new StreamWriter(@"c:\XXX.tmp"))
| {
| newMasterfile.Close();
| }
| }
|
| Console.ReadLine();
| }
| }
| }
|
| Also you have to create an "App.config" file which is distributed by
| VS2003 into the application directory with the name of the application
| followed by ".config". There may be any content, the file can be empty
| too - this will not influence our result.
|
| Compile the "Release" configuration and you'll see with a tool like
| "NetMemoryProfiler" that there are almost 1000 StreamWriter reserved at
| "Console.ReadLine". You can wait as long as you want - they'll never be
| freed. If you don't have such a tool: increase the number of loops.
| Then you can use the task manager to see how the memory grows (look at
| the virtual memory!). With an endless loop there will be an "Out of
| virtual memory" error at the end.
|
| This problem is not only bounded to StreamWriter. We've seen this for
| database objects, simple strings and other.
|
| We have tested this with several Windows XP SP2 Systems with different
| configurations. Also with the latest fixes for Windows and .NET.
|
| The funny thing: there will be no leaks when you use the "Debug"
| configuration. Also you can omit the exception or the "App.config"
| file. In these cases the memory will be freed correctly. But we will
| comile in "Release" configuration, use the "App.config" and Exceptions
| can't be avoided.
|
| The last option is to switch the attribute "[STAThread]" to
| "[MTAThread]". The memory is ok in the small example (why?) and also in
| our huge application. But I'm not sure with the COM-components we use.
| So I prefer the "[STAThread]" attribute.
|
| There are some methods which you can tell the gabage collector to clean
| up: "GC.GetTotalMemory(...)", "GC.WaitForPendingFinalizers()" and
| "GC.Collect()". You can find combinations in the small example to get
| rid of the memory. But this is very unreliable and will not work
| correctly in our huge application where late binding and remoting is
| used.
|
| After some days of trial and error, I am really at my wits' end over
| this problem. So is there anybody who can help me?
|
| Many thanks in advance
| Martin Gaus
|

The small sample you posted does not leak, nor using V1.1.4322 nor using
v2.0., with or without a config file.
Your conclusion stems from the fact that you don't really understand how the
GC works, let met explain:

1. The only point in time a GC can occur is when objects are being created
on the GC heap, when waiting on a ReadLine(), no more objects are getting
instantiated, so, no GC can ever occur as long as you are waiting for a
keybord input.
That means that the StreamWriter instances who are not yet collected at the
moment you reach ReadLine, won't be cleaned-up before the process terminates
(well, the next GC).
Now, if you simply run this program from the console and watch the 'CLR
Memory' performance counters, you will see a number of Gen0 collections
occurring while executing the loop. That means that a bunch of StreamWriter
objects will be removed from the heap, while a number of StreamWrtiter
objects are still on the heap when executing ReadLine and as I said before
they stay there until the next GC (here because of the AD unload).
If you run this with a profiler attached, the behavior changes dramatically,
the CLR switches to a less aggressive mode of GC when a
(managed)debugger/profiler is attached, that means that it's possible that
no Gen0 collections occurred during the loop, leaving all StreamWriter
objects on the heap at the moment you reach ReadLine().
Ok, why the different behavior when there is a config file? Well, the
allocation/de-allocation scheme changed because you created a lot more
objects before you execute the loop, possibly triggering a GC run before the
loop has started and as such preventing a GC run during the loop.
Lesson learned, don't use the CLR memory profiler to monior such simple
programs, start using the non-invasive tools like perfmon to monitor your
memory allocation pattern before you start profiling using any managed
profilers, and if you do, make sure you understand what you are looking at.

2. you need to set the apartment state of the main thread to be compatible
with your COM objects requirements, seems like you don't know what they are.
When you set this to STA, be sure you never block this thread or you will
prevent the finalizer to clean-up the finalizable objects, possibly creating
a huge memory leak.

Willy.

[Jon Skeet [C# MVP]]

Willy Denoyette [MVP] <wi*************@telenet.be> wrote:

<snip>

1. The only point in time a GC can occur is when objects are being created
on the GC heap, when waiting on a ReadLine(), no more objects are getting
instantiated, so, no GC can ever occur as long as you are waiting for a
keybord input.

That's not quite true, IMO. I believe GC will kick in when there's
memory pressure, too. Here are two programs which demonstrate it
between them:

Test.cs:
using System;

class Test
{
~Test()
{
Console.WriteLine ("Finalizer");
}

static void Main()
{
Test t = new Test();
Console.WriteLine ("Created");
Console.ReadLine();
}
}
UseMemory.cs:
using System.Collections;

class UseMemory
{
static void Main()
{
ArrayList list = new ArrayList();
while (true)
{
list.Add (new byte[10000]);
}
}
}

From one console, compile and run Test.cs. You'll see (or may see)
"Created" on its own. You can then leave that console alone for a
couple of minutes, and nothing is printed.

Then compile and run UseMemory.cs. Shortly, you should see "Finalizer"
being printed in the original console window - at least, that's what
happens on my box.

While this isn't absolute proof (due to finalizers and GC being
different things) it's at least extremely suggestive.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Willy Denoyette [MVP]]

True, when there is physical memory pressure (high memory threshold) and you
are running on XP or higher, the OS will notify the CLR who will start to
trim it's working set, but before doing this, the CLR will trigger a full
collection, that means it will trigger the GC at the next allocation request
or whenever the program enters the CLR (or returns from unmanaged), but in
your case (unless you run a server GC), the GC cannot run as it's main
thread blocked in unmanaged code (result of the ReadLine()), so the CLR
proceeds by starting the finalizer thread (running on it's own thread), this
is exactly what you noticed when running your code.

I didn't want to complicate things any further by introducing "external
triggers", so I didn't mention these, I didn't count on you to come back on
this, I should have known better ;-).

Willy.

"Jon Skeet [C# MVP]" <sk***@pobox.com> wrote in message
news:MP************************@msnews.microsoft.c om...
| Willy Denoyette [MVP] <wi*************@telenet.be> wrote:
|
| <snip>
|
| > 1. The only point in time a GC can occur is when objects are being
created
| > on the GC heap, when waiting on a ReadLine(), no more objects are
getting
| > instantiated, so, no GC can ever occur as long as you are waiting for a
| > keybord input.
|
| That's not quite true, IMO. I believe GC will kick in when there's
| memory pressure, too. Here are two programs which demonstrate it
| between them:
|
| Test.cs:
| using System;
|
| class Test
| {
| ~Test()
| {
| Console.WriteLine ("Finalizer");
| }
|
| static void Main()
| {
| Test t = new Test();
| Console.WriteLine ("Created");
| Console.ReadLine();
| }
| }
|
|
| UseMemory.cs:
| using System.Collections;
|
| class UseMemory
| {
| static void Main()
| {
| ArrayList list = new ArrayList();
| while (true)
| {
| list.Add (new byte[10000]);
| }
| }
| }
|
| From one console, compile and run Test.cs. You'll see (or may see)
| "Created" on its own. You can then leave that console alone for a
| couple of minutes, and nothing is printed.
|
| Then compile and run UseMemory.cs. Shortly, you should see "Finalizer"
| being printed in the original console window - at least, that's what
| happens on my box.
|
| While this isn't absolute proof (due to finalizers and GC being
| different things) it's at least extremely suggestive.
|
| --
| Jon Skeet - <sk***@pobox.com>
| http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
| If replying to the group, please do not mail me too

[Martin]

Thats all fine. But why do I ran out of virtual memory when I have an
endless loop?

....
while(true)
{
using(StreamWriter newMasterfile =
new StreamWriter(@"c:\XXX.tmp"))
{
newMasterfile.Close();
}
}
....

[Jon Skeet [C# MVP]]

Martin <MA*****@gmx.net> wrote:

Thats all fine. But why do I ran out of virtual memory when I have an
endless loop?

...
while(true)
{
using(StreamWriter newMasterfile =
new StreamWriter(@"c:\XXX.tmp"))
{
newMasterfile.Close();
}
}

That I don't know - I can't reproduce it. How quickly do you leak
memory? Should I be able to easily see it in task manager?

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Martin]

I think so.
About 10MB per minute.
Intel P4 mobile, 2 GHz.

[Willy Denoyette [MVP]]

"Martin" <MA*****@gmx.net> wrote in message
news:11**********************@y43g2000cwc.googlegr oups.com...
| Thats all fine. But why do I ran out of virtual memory when I have an
| endless loop?
|
| ...
| while(true)
| {
| using(StreamWriter newMasterfile =
| new StreamWriter(@"c:\XXX.tmp"))
| {
| newMasterfile.Close();
| }
| }
| ...
|

Don't know, it doesn't happen when I run your code, are you sure you are
running the exact same code as you posted?

Willy.

[Greg Young]

Why post you exact solution and we can check it out? The code posted at
present when brought here (with or without an app.config does not seem to
have any problems)

Cheers,

Greg Young
MVP - C#
"Martin" <MA*****@gmx.net> wrote in message
news:11**********************@y43g2000cwc.googlegr oups.com...

Thats all fine. But why do I ran out of virtual memory when I have an
endless loop?

...
while(true)
{
using(StreamWriter newMasterfile =
new StreamWriter(@"c:\XXX.tmp"))
{
newMasterfile.Close();
}
}
...

[Martin]

Please give me a place where I can upload a zip file with the solution.

[Greg Young]

There are lots of free services such as http://www.savefile.com/ which offer
this exact functionality.

Cheers,

Greg Young
"Martin" <MA*****@gmx.net> wrote in message
news:11**********************@i39g2000cwa.googlegr oups.com...

Please give me a place where I can upload a zip file with the solution.

[Martin]

Good morning and thank you Greg!

you'll find the files there:
http://savefile.com/projects/831693

There are 2 files:
- MemoryLeak1: File with the original problem as described
- MemoryLeak2: A threading test with free main thread

Just start the compiled release.
Is there anyone who can reproduce my problem?

Martin

[Greg Young]

I am showing the same behavior for the first example (6mb usage when done
running)

The second levelled off at around 6.3 mb after a minute or so.

Cheers,

Greg Young
MVP - C#

"Martin" <MA*****@gmx.net> wrote in message
news:11*********************@y43g2000cwc.googlegro ups.com...

Good morning and thank you Greg!

you'll find the files there:
http://savefile.com/projects/831693

There are 2 files:
- MemoryLeak1: File with the original problem as described
- MemoryLeak2: A threading test with free main thread

Just start the compiled release.
Is there anyone who can reproduce my problem?

Martin

[Lebesgue]

I think I found the source of your problem:

try this:
for(int i=0; i<1000000; i++)
{
using(StreamWriter newMasterfile =
new StreamWriter(@"c:\XXX.tmp"))
{
newMasterfile.Close();
}
}

Console.WriteLine("done"); //<<<----

Console.ReadLine();

Does the memory usage grow after you see "done" on the console?
Didn't you just think the program was on the ReadLine line while it was
still in the loop?
It takes quite long to execute the whole loop - the memory usage is rising
all the time and then it remains the same - because GC can't run because the
only thread is stuck on the ReadLine() and it has no reason to run, because
there's no memory pressure.

"Martin" <MA*****@gmx.net> wrote in message
news:11*********************@y43g2000cwc.googlegro ups.com...

Good morning and thank you Greg!

you'll find the files there:
http://savefile.com/projects/831693

There are 2 files:
- MemoryLeak1: File with the original problem as described
- MemoryLeak2: A threading test with free main thread

Just start the compiled release.
Is there anyone who can reproduce my problem?

Martin

[Willy Denoyette [MVP]]

Hmmm, looks like you don't know how the GC works, I tried to explain what
basically happens when you run this sample, for more details please refer to
my other reply in this thread.
Also, did you actually run the code?, and did you watch memory consumption
using perfmon?
The ReadLine is executed after the loop terminates, the GC cannot run while
in ReadLine(), but, while executing the loop, the GC kicks in several times
(per ~40 StreamWriter instances), so what you should see is a sawtooth-like
memory allocation pattern, starting at a certain level (~5Mb) and going up
to a maximum of ~6Mb (your mileage may vary).
Anyway, none of the programs posted are leaking.

Willy.

"Lebesgue" <le******@gmail.com> wrote in message
news:%2****************@TK2MSFTNGP03.phx.gbl...
|I think I found the source of your problem:
|
| try this:
| for(int i=0; i<1000000; i++)
| {
| using(StreamWriter newMasterfile =
| new StreamWriter(@"c:\XXX.tmp"))
| {
| newMasterfile.Close();
| }
| }
|
| Console.WriteLine("done"); //<<<----
|
| Console.ReadLine();
|
| Does the memory usage grow after you see "done" on the console?
| Didn't you just think the program was on the ReadLine line while it was
| still in the loop?
| It takes quite long to execute the whole loop - the memory usage is rising
| all the time and then it remains the same - because GC can't run because
the
| only thread is stuck on the ReadLine() and it has no reason to run,
because
| there's no memory pressure.
|
| "Martin" <MA*****@gmx.net> wrote in message
| news:11*********************@y43g2000cwc.googlegro ups.com...
| > Good morning and thank you Greg!
| >
| > you'll find the files there:
| > http://savefile.com/projects/831693
| >
| > There are 2 files:
| > - MemoryLeak1: File with the original problem as described
| > - MemoryLeak2: A threading test with free main thread
| >
| > Just start the compiled release.
| > Is there anyone who can reproduce my problem?
| >
| > Martin
| >
|
|

[Lebesgue]

I know how the GC works and wrote what I wrote for the sake of simplicity.
I actually ran the code and saw no real memory leak - that's what I wanted
to express in my post. I am sorry you misinterpreted it.

"Willy Denoyette [MVP]" <wi*************@telenet.be> wrote in message
news:e6**************@TK2MSFTNGP03.phx.gbl...

Hmmm, looks like you don't know how the GC works, I tried to explain what
basically happens when you run this sample, for more details please refer
to
my other reply in this thread.
Also, did you actually run the code?, and did you watch memory consumption
using perfmon?
The ReadLine is executed after the loop terminates, the GC cannot run
while
in ReadLine(), but, while executing the loop, the GC kicks in several
times
(per ~40 StreamWriter instances), so what you should see is a
sawtooth-like
memory allocation pattern, starting at a certain level (~5Mb) and going up
to a maximum of ~6Mb (your mileage may vary).
Anyway, none of the programs posted are leaking.

Willy.

"Lebesgue" <le******@gmail.com> wrote in message
news:%2****************@TK2MSFTNGP03.phx.gbl...
|I think I found the source of your problem:
|
| try this:
| for(int i=0; i<1000000; i++)
| {
| using(StreamWriter newMasterfile =
| new StreamWriter(@"c:\XXX.tmp"))
| {
| newMasterfile.Close();
| }
| }
|
| Console.WriteLine("done"); //<<<----
|
| Console.ReadLine();
|
| Does the memory usage grow after you see "done" on the console?
| Didn't you just think the program was on the ReadLine line while it was
| still in the loop?
| It takes quite long to execute the whole loop - the memory usage is
rising
| all the time and then it remains the same - because GC can't run because
the
| only thread is stuck on the ReadLine() and it has no reason to run,
because
| there's no memory pressure.
|
| "Martin" <MA*****@gmx.net> wrote in message
| news:11*********************@y43g2000cwc.googlegro ups.com...
| > Good morning and thank you Greg!
| >
| > you'll find the files there:
| > http://savefile.com/projects/831693
| >
| > There are 2 files:
| > - MemoryLeak1: File with the original problem as described
| > - MemoryLeak2: A threading test with free main thread
| >
| > Just start the compiled release.
| > Is there anyone who can reproduce my problem?
| >
| > Martin
| >
|
|

[Willy Denoyette [MVP]]

"Lebesgue" <le******@gmail.com> wrote in message
news:en**************@TK2MSFTNGP04.phx.gbl...
|I know how the GC works and wrote what I wrote for the sake of simplicity.
| I actually ran the code and saw no real memory leak - that's what I wanted
| to express in my post. I am sorry you misinterpreted it.
|

Hmmmm, how could I correctly interpret this:

| It takes quite long to execute the whole loop - the memory usage is
rising
| all the time and then it remains the same - because GC can't run because
the

if you know how the GC works, you must known that the memory doesn't rise
all the time while executing the loop, because the GC kicks in several times
during execution.

Willy.

[Martin]

I've recognised now that we don't have a real memory leak. Anytime and
anyhow the StreamWriters will be kicked. Possibly I have to use on of
the GC methods - and that is not very reliable but the memory is free.
I always thought I don't have care about things like this because .NET
is good enough. Maybe I'll go back to the unmanaged world. There I know
what's going on :-(

But I've nevertheless have the problem: Our program is working like the
example (and that's running out of memory). The main thread of out
program is waiting in a console.readline() while the work (creating the
streamwriter) is done in a thread.

Now I look for an alternative construction. I'll avoid the
console.readline() in the main thread as you've suggested. Please look
at my secon example:
http://savefile.com/projects/831693

Why do I have the problem there? The main thread is looping delayed and
free to call the GC. The work (creating the streamwriter) is done in a
Thread. The breaking condition (console.readline) is in a second
thread. All the things I learned from the postings before is that I
will not have a problem in this situation. So why is my system running
out of virtual memory (also under pressure)?

Has anyone thought about the fact that we have the problem only in the
release configuration not in the debug?

Martin

[Martin]

To all of you who never have see an "out of virtual memory exception"
see:
http://www.savefile.com/projects/831693
I've placed some screenshots there. They're from the second example.

Maybe there is hope for you:
I've tested 10 systems yesterday with different configurations:
notebooks, VMware images, pcs and so on.

Nine of them leaked but there was one that was some different. You wont
believe: it was not leaking. It was a naked (no SP) WinXP (German)
VMWare Image. I had only installed the .NET 1.1 framework. So who is
the bastard on the others?

Please can you send me your system configuration to compare?

Thanks for all
Martin

[Willy Denoyette [MVP]]

"Martin" <MA*****@gmx.net> wrote in message
news:11**********************@r44g2000cwb.googlegr oups.com...
| I've recognised now that we don't have a real memory leak.

Sure not, but what is your problem?

Anytime and
| anyhow the StreamWriters will be kicked. Possibly I have to use on of
| the GC methods -

No you don't, the GC methods are only needed in extreme rare cases, calling
them disturbs the GC allocation scheme when done without any good reason.

and that is not very reliable but the memory is free.
| I always thought I don't have care about things like this because .NET
| is good enough.

Sure it is.

Maybe I'll go back to the unmanaged world. There I know
| what's going on :-(
|

I hope you do, but I know that memory leaks are common issues in unmanaged
world :-).

| But I've nevertheless have the problem: Our program is working like the
| example (and that's running out of memory).

Not when I run it with V1.1.4322 SP1, on clean (no anti-virus crap) XP SP2 +
latest security updates, and on W2K3 SP1 latest security updates V1.1.4322
SP1 (ran it for 15 minutes).

The memory consumption stabilizes after a couple of minutes arround ~5.812
Kb "Memory usage" ~4.740Kb "VM size" .
The memory usage in debug mode is somewhat higher (~6800Kb/5356Kb)

You must have some other problem on your system(s).

Willy.

[Martin]

That's interesting.
I've a XP without SPs and the Framework 1.1.4322.573 also without SP.
It's stable! There must be something between those two.

[Willy Denoyette [MVP]]

"Martin" <MA*****@gmx.net> wrote in message
news:11**********************@j73g2000cwa.googlegr oups.com...
| That's interesting.
| I've a XP without SPs and the Framework 1.1.4322.573 also without SP.
| It's stable! There must be something between those two.
|
For me it's stable on 1.1 with SP1 and without SP1 and V2.0.50727 too.
Really I don't believe your problem has anything to do with .NET, just try
to install SP1 on the box with V1.1 and try again, you can even install V2
and test again. Watch the CLR performance counters (CLR memory/ gen0, 1 and
2 collection frequency), the GC should run every once and a while.

Another suggestion is to stopt Norton AV, I noticed in the screendumps that
you run this on the system you used to run the failing test, could be wrong
though.

Willy.

[Martin]

I had systems with other virus software and systems without.
Thay all had the problem.

Do you have installed VS2003, Office or something like this?

try to install SP1 on the box with V1.1 and try again,
I'll do
you can even install V2 and test again.
It's install. As I told no prblem with 1.1 debug, 2.0 debug and
release.
There's only a problem with 1.1 release.
Watch the CLR performance counters (CLR memory/ gen0, 1 and
2 collection frequency), the GC should run every once and a while

It's running and counting but the objects are not freed. When I do the
GC.WaitForPendingFinalizers() it seems I reactivate something and it
cleans up. From this point everything works fine - no objects remain
anymore.