Hi,
a c# app of mine that parses 30,000 xml files writes large
amounts of data to flat file (> 2GB) using a streamwriter
object (sw). everything works fine except that the memory
used by the app grows linearly during execution and
eventually crashes the computer. without going into too
much detail of my code, i made the following observations:
- if you comment out the sw.Write(x) statement (which is
inside the loop that parses the xml files), memory doesn't
increase. thus it must be something to do with the
streamwriter or file I/O.
- if you change x in sw.Write(x) to be a constant, memory
doesn't increase, but if x is an expression or variable,
memory does increase. Not sure what this means.
I've tried many things to solve this problem and am
completely stuck. I've tried making sw = null every so
often within the loop, making a new streamwriter and
output file (so that the file sizes never exceed 10MB),
and calling GC.Collect() to try and force the compiler to
clean up memory, but there is no effect. I've tried using
{} statements which don't do anything either. The only
thing I can think of is to re-run the entire application
multiple times and pass command line parameters to
indicate where the last one left off, but this is
obviously not an ideal solution.
I did get it to work by using the
Scripting.FileSystemObejct via COM instead of
streamwriter, but it's prohibitively slow. However this
does indicate that there are no memory leaks within the
rest of my code, and that the problem is with streamwriter
somehow.
Any thoughts? Any help at all is greatly appreciated!!!!!!!
thanks in advance
Rob
