StreamWriter is not reliable

[yfguo]

for who have any idea about my question.

I have a multithread middle tier engine and use StreamWriter to log events to console and files. I run the engine for whole night (6 - 10 hours) and there are 8 batch log files generated. Sometimes some log files (one or two of 8) is cut off on the very last portion. I thought it might be because buffer is not flushed out. But I use AutoFlush and Flush. The problem still exist. Do some body have the same problem with StreamWriter? The problem is I cannot re-produce the problem when on debug mode and runnning in my local machine. It seems always happening when the app compiled and deployed to a server machine.

[jhaxo]

I have never had an issue with StreamWriter itself, I think you should take a higher-level look to see if you have an application problem. dead lock, or file access or plain-old-fashion running out of disk space.

[yfguo]

Thank your reply.

But I have no space problem. The space still has 10 GB. No security, permission problem, because it has already been written several hundreds bytes log. I used to guess it is a problem of dead lock, because I use multithreading. However I already took care of the thread safe issue. Because of the complication of the multithread logging, so it is still the candidate. Another candidate is buffer flushing problem.

Any other thoughts?

[Shashi Sadasivan]

Is each thread writing a seperate log file?
or can more than one thread edit/create the same log file?

If more than 2 threads can write to the same log file, what you are experiencing is the need to centralize writing to a file.

Its similar to have a document open in one window, and then trying to edit the same document in another one.

Try using a static/singleton class to handle writing of events to the logs.

[yfguo]

it is writing to one log file for multithreads and I am using a static WriteLogEntry method.

Today I created a dummy project to see if it is the problem of multithreading. But it looks I can run 1000 threads without any problem (message lost) to WriteLogEntry to one log file.

Actually there is only one batch log file of the 7 batch log files missing the log message from middle somewhere from where there are no any logs. All these 8 batch jobs run in sequent (not parallel) for over 6 hours.

[Shashi Sadasivan]

Is there a possibility that you could provide us with that sample project you made, so that we can have more information what is going wrong.

Also please insert appropriate comments in the code if you decide to provide it

[krishnabhargav]

Are you doing a Close() on the StreamWriter object you are using.

Go to visual Studio->Debug->Exceptions->Managed Debugging Assists and enable StreamWriterBufferedDataLost

This would raise an exception in case an S W object is not closed before finalize().

Read this article ..
http://blogs.msdn.com/bclteam/archiv...13/214405.aspx

90% of the times its Close() that gives problems, atleast for me. I use StreamWriter to log 1000's of URL for my research and usually i do not lose any data.

[yfguo]

I think the problem has been resovled. The problem I guess is because I did not close the StreamWriter. When I add code to close the streaWirter, the problem goes away.

Thanks for all you guys' comments and help.