How does "new" work in a loop?

[Tony Sinclair]

I'm just learning C#. I'm writing a program (using Visual C# 2005 on
WinXP) to combine several files into one (HKSplit is a popular
freeware program that does this, but it requires all input and output
to be within one directory, and I want to be able to combine files
from different directories into another directory of my choice).

My program seems to work fine, but I'm wondering about this loop:
for (int i = 0; i < numFiles; i++)
{
// read next input file

FileStream fs = new FileStream(fileNames[i],
FileMode.Open, FileAccess.Read, FileShare.Read);
Byte[] inputBuffer = new Byte[fs.Length];

fs.Read(inputBuffer, 0, (int)fs.Length);
fs.Close();

//append to output stream previously opened as fsOut

fsOut.Write(inputBuffer, 0, (int) inputBuffer.Length);
progBar.Value++;
} // for int i

As you can see, the objects fs and inputBuffer are both created as
"new" each time through the loop, which could be many times. I didn't
think this would work; I just tried it to see what kind of error
message I would get, and I was surprised when it ran. Every test run
has produced perfect results.

So what is happening here? Is the memory being reused, or am I piling
up objects on the heap that will only go away when my program ends, or
am I creating a huge memory leak?

I can see that fs might go away after fs.Close(), but I don't
understand why I'm allowed to recreate the byte array over and over,
without ever disposing of it. I have verifed with the debugger that
the array has a different size each time the input file size changes,
so it really is being reallocated each time through the loop, rather
than just being reused. I've tried to find explanations of how "new"
works in a loop, but I haven't been able to so far. Any help,
including pointers to the VS docs or a popular book on C#, would be
appreciated.

[John J. Hughes II]

Jon,

Thanks for the debate, at this point I have not changed my mind but it does
give me food for thought. The next time I have some slow time I will
research the matter further taking you points and other points in this
thread into consideration.

But as a last comment: In one of you other messages in this thread you made
the following comment:

then the JIT wouldn't work out that first could never become true after
the first iteration and bigObject would therefore never be used after
that point. That's one of the few situations where it might make sense
to set a local variable to null.

Since I have some rather long running threads which create some long lived
variables it's possible that setting some of them to null cause memory to be
returned which was being held before.

Regards,
John

"Jon Skeet [C# MVP]" <sk***@pobox.comwrote in message
news:MP************************@msnews.microsoft.c om...

John J. Hughes II <[e-mail]:

>I do agree the memory is not marked... poor verbiage on my part.

I don't think your example really proves anything since you are calling
garbage collection.

Well, I can make an example which ends up garbage collecting due to
other activity if you want. It'll do the same thing. Just change the
call to GC.Collect() to

for (int i=0; i < 10000000; i++)
{
byte[] b = new byte[1000];
}

and you'll see the same thing.

>I have no argument that when GC runs it will clean up
memory that is not being used. I personally believe that all references
to
a variable are not removed in a timely fashion unless you tell them too
be.
The key here is timely.

It's not a matter of the reference being removed. It's a case of the
release-mode garbage collector ignoring variables which are no longer
relevant.

>Again as I have said I had a problem with memory creep, the only change I
did was add using statements the problem slowed down but was not
eliminated.

And *that* can have a significant impact - because many classes which
implement IDisposable also have finalizers which are suppressed when
you call Dispose. That really *does* affect when the memory can be
freed, and can make a big difference.

>The second change was to add value=null statement (shotgun blast style)
and
the problem went away. Since it was a production system I used great
care
to change as little as possible so I really don't think I fixed any other
problems.

I'm afraid I still don't believe you saw what you claimed to be seeing
- not on a production system. You *would* see improvements in a
debugger, but that's a different matter.

>If at some point in the near future if I can give you code which proves
my
point I will be happy too but the last time I had the problem it required
a
system running full blown for 14 days on average.

That being said I may have gotten my head wet and decided it was raining
when it was snowing. I decide to use an umbrella and my head it not wet
now.

I really suspect you were mistaken, I'm afraid.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Tony Sinclair]

On Fri, 07 Jul 2006 11:09:58 -0700, Tony Sinclair <[e-mail]:

>My sincere gratitude to everyone who responded.

I'm afraid the debate on my question sailed over my head quite some
time ago, but in case anyone is interested, I can give you the actual
results of my program.

I used essentially the same code in my OP on an internet file that I
downloaded, which comprised 50+ segments of 24 MB each, so each time
through my loop, I was allocating a 24 MB buffer as "new." (I do
intended to incorporate the improvements suggested, especially the
using statement, but I haven't gotten to it yet.) I watched the
memory data with the MS Task manager as I started and ran my program.

When it started, the program quickly grabbed an extra 24MB from the
memory pool. It never went more than 1MB above that for the rest of
the run, and the file assembled perfectly. There was no shortage of
memory at the time (I have 1GB of physical memory, and Task Manager
showed about 2500MB of virtual memory available. When I started my
program, about 700MB of this was committed).

I conclude that even in a short loop, with no shortage of memory, and
with no hints from me that might help speed it up, the GC acts quickly
enough to dispose of the old buffer as soon as it's unneeded. I note
that even though the loop is short in lines, the CPU probably has a
lot of time on its hands while I am writing to the output buffer. I
might try this test again with a high-CPU task running in the
background and see what happens.

Thanks again to everyone for their help.