473,322 Members | 1,314 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

close and open file again or keep open -- for log file?

Hi,

Both way works, I'd just ask some experts which way is better?

My application creates a log file daily.

Now each time when I write a log, I will open the file and append to the
end. Ocz, if the file is not exist(e.g. another day), it will creates the
file first.

Now I am thinking keep the file strem open, I think this should can gain
better performance. Ocz I will close the file strem when applicatoin exist
and/or in de-constructor, and when switch to another log file(at mid-night,
another day begins).

Either way, I will open the log file in ReadWrite share mode.

Which way is common practise?

Thanks a lot!
Ryan
Dec 11 '06 #1
5 11126
Hi,

Ryan Liu wrote:
Hi,

Both way works, I'd just ask some experts which way is better?

My application creates a log file daily.

Now each time when I write a log, I will open the file and append to the
end. Ocz, if the file is not exist(e.g. another day), it will creates the
file first.

Now I am thinking keep the file strem open, I think this should can gain
better performance. Ocz I will close the file strem when applicatoin exist
and/or in de-constructor, and when switch to another log file(at mid-night,
another day begins).

Either way, I will open the log file in ReadWrite share mode.

Which way is common practise?

Thanks a lot!
Ryan
I think it's not a good idea. If your application crashes, the stream
won't be closed. Keeping the file open complicates access through
another application (typically, you need to open the log file in a
viewer or editor when you need to debug the application on a production
machine, and typically you have only notepad available in this
circumstance).

Maybe you should consider using log4net, which has excellent
performances AFAICS and which handles opening and closing the log files
itself. Additionally, it's fairly easy to add listeners to log to a
remote machine, for example.

HTH,
Laurent
--
Laurent Bugnion, GalaSoft
Software engineering: http://www.galasoft-LB.ch
PhotoAlbum: http://www.galasoft-LB.ch/pictures
Support children in Calcutta: http://www.calcutta-espoir.ch
Dec 11 '06 #2
"Laurent Bugnion" <ga*********@bluewin.chwrote in message
news:%2****************@TK2MSFTNGP02.phx.gbl...
I think it's not a good idea. If your application crashes, the stream
won't be closed.
Clarification: the stream may not be *flushed*. It definitely will be
closed, if the application crashes. (Of course, one hopes that the
application crashing is a rare circumstance). Data may be lost, but any
data that has actually been written out to the file will be safely stored
there, even if the application crashes.
Keeping the file open complicates access through another application
(typically, you need to open the log file in a viewer or editor when you
need to debug the application on a production machine, and typically you
have only notepad available in this circumstance).
Very true. If the desire is to access the log file while the application is
running, via some independent program, keeping the file open for write
access may prevent that access.

Of course, similarly one would want to make sure to use some means to view
the logfile that doesn't also keep it open (notepad, as you suggest, is
appropriate for this). Otherwise, the application itself may find itself
unable to write to the file.

This could happen in any case, since the file needs to be kept open at least
long enough to read it to make a new copy (in memory as notepad does, or to
a different file). So it may be worthwhile to include logic in the logging
code to either queue and retry logging attempts, or start a new file if the
current one can't be opened.

Another thing to keep in mind is how often the log file will be written to.
If the application is likely to be generating log entries on a near-constant
basis, keeping the file open may be important for performance reasons. It's
not VERY expensive to open and close a file, but it's expensive enough if
you're doing it many times a second.

Of course, one could take the Windows event manager approach, and use an
independent process to handle logging as well as viewing the logs. That
would, of course, greatly complicate the implementation and it may be that
something like "log4net" (as mentioned already) would be more appropriate in
that situation (I wouldn't know, since this is the first I've heard of it
:) ).

Pete
Dec 11 '06 #3
Hi,

Peter Duniho wrote:
"Laurent Bugnion" <ga*********@bluewin.chwrote in message
news:%2****************@TK2MSFTNGP02.phx.gbl...
>I think it's not a good idea. If your application crashes, the stream
won't be closed.

Clarification: the stream may not be *flushed*. It definitely will be
closed, if the application crashes. (Of course, one hopes that the
application crashing is a rare circumstance). Data may be lost, but any
data that has actually been written out to the file will be safely stored
there, even if the application crashes.
Yes, sorry.
Of course, similarly one would want to make sure to use some means to view
the logfile that doesn't also keep it open (notepad, as you suggest, is
appropriate for this). Otherwise, the application itself may find itself
unable to write to the file.
I typically use a web browser to view log files. It makes it easy to
refresh the view when the application write more entries to the file,
and it doesn't block the access.
Of course, one could take the Windows event manager approach, and use an
independent process to handle logging as well as viewing the logs. That
would, of course, greatly complicate the implementation and it may be that
something like "log4net" (as mentioned already) would be more appropriate in
that situation (I wouldn't know, since this is the first I've heard of it
:) ).

Pete
log4net is a quite well known logging component. It's a port from log4j
(for Java) to .NET. It's free and easy to customize, and generally more
friendly than the built-in Trace and Debug classes. That said, it's 3rd
party, so one may want to use the built-in classes anyway...
http://logging.apache.org/log4net/

HTH,
Laurent
--
Laurent Bugnion, GalaSoft
Software engineering: http://www.galasoft-LB.ch
PhotoAlbum: http://www.galasoft-LB.ch/pictures
Support children in Calcutta: http://www.calcutta-espoir.ch
Dec 11 '06 #4
Thanks , Laurent and Peter!

So that's why I want to close the stream in the finally of main() and also
in static de-constructor, and hope that will help when the application
crashes. (So when it crashes, even finally might not be executed?)

And for the same reason I open file stream in shared read/write mode, so
even the stream is not properly closed, it can still be read and write next
time.

Will those two tricks help?

And is that really costly to open a file and append at the end? I am afraid
it will read whole file (could be big) then write. If it can locate and
right "jump" to the end of file without reading file, it should not that
costly. I just don't know the underneath implementation.

Thanks a lot!

"Laurent Bugnion" <ga*********@bluewin.ch????
news:u1**************@TK2MSFTNGP03.phx.gbl...
Hi,

Peter Duniho wrote:
"Laurent Bugnion" <ga*********@bluewin.chwrote in message
news:%2****************@TK2MSFTNGP02.phx.gbl...
I think it's not a good idea. If your application crashes, the stream
won't be closed.
Clarification: the stream may not be *flushed*. It definitely will be
closed, if the application crashes. (Of course, one hopes that the
application crashing is a rare circumstance). Data may be lost, but any
data that has actually been written out to the file will be safely
stored
there, even if the application crashes.

Yes, sorry.
Of course, similarly one would want to make sure to use some means to
view
the logfile that doesn't also keep it open (notepad, as you suggest, is
appropriate for this). Otherwise, the application itself may find
itself
unable to write to the file.

I typically use a web browser to view log files. It makes it easy to
refresh the view when the application write more entries to the file,
and it doesn't block the access.
Of course, one could take the Windows event manager approach, and use an
independent process to handle logging as well as viewing the logs. That
would, of course, greatly complicate the implementation and it may be
that
something like "log4net" (as mentioned already) would be more
appropriate in
that situation (I wouldn't know, since this is the first I've heard of
it
:) ).

Pete

log4net is a quite well known logging component. It's a port from log4j
(for Java) to .NET. It's free and easy to customize, and generally more
friendly than the built-in Trace and Debug classes. That said, it's 3rd
party, so one may want to use the built-in classes anyway...
http://logging.apache.org/log4net/

HTH,
Laurent
--
Laurent Bugnion, GalaSoft
Software engineering: http://www.galasoft-LB.ch
PhotoAlbum: http://www.galasoft-LB.ch/pictures
Support children in Calcutta: http://www.calcutta-espoir.ch

Dec 11 '06 #5
"Ryan Liu" <ad********@online.sh.cnwrote in message
news:eE**************@TK2MSFTNGP03.phx.gbl...
Thanks , Laurent and Peter!

So that's why I want to close the stream in the finally of main() and also
in static de-constructor, and hope that will help when the application
crashes. (So when it crashes, even finally might not be executed?)
It depends on why it crashes. If the error is in your own code, or in .NET
code, you have an excellent chance of having the opportunity to clean up in
the finally clause of your main() function. If the entire computer faults
(video driver problem, for example) or the process is terminated abnormally
by external means ("End Process", for example) then no.

However, one presumes that on a system where you want a continuously running
process for which a log file is very important, you are minimizing your
exposure to buggy third-party code. Hopefully the scenarios in which your
application doesn't get a chance to clean up are even more rare than the
scenarios in which your application itself might crash, and presumably those
latter scenarios are themselves exceedingly rare.

If not, your time would be better spent figuring out how to ensure that they
*are* exceedinly rare, rather than worrying about the log file. :)
And for the same reason I open file stream in shared read/write mode, so
even the stream is not properly closed, it can still be read and write
next
time.
I have seen Windows fail to clean up an opened file correctly. However,
never after a simple application crash, and infrequently in any case (I've
seen it happen maybe three times since I first started using Windows NT,
almost 15 years ago). How an application opens the file doesn't have
anything to do with access to the file after that application crashes.
Normally, Windows will clean things up for you and the file will be returned
to a normal, closed state ready for access by other processes.

So, no...using a shared file access mode has no bearing on access to the
file after the application crashes.
And is that really costly to open a file and append at the end? I am
afraid
it will read whole file (could be big) then write. If it can locate and
right "jump" to the end of file without reading file, it should not that
costly. I just don't know the underneath implementation.
Seeking to a particular position in the file is relatively fast. The size
of the file shouldn't affect performance much, if at all. It's just that
those operations (open, seek, and close) aren't free and relative to
whatever work you could be doing on the computer they may even be fairly
expensive. If you do them every few seconds or even less frequently, I
would guess that they would not be a noticeable problem. But doing them
several times a second or more often could easily take i/o and CPU time away
from more important work.

If it's something that really matters in your situation, the best thing to
do is to measure it. Measure your performance (throughput, whatever is
relevant in your situation) without logging, and measure logging with and
without keeping the file open. This will give you some real data as to what
the best compromise will be for you.

Keeping in mind, of course, that over the years as hardware performance
changes -- CPUs get faster or more numerous, disk i/o gets faster, etc. --
the exact ratios you've measured will change, possibly affecting the
analysis as well. This measurement is most useful only with the computer
configuration on which you've done the measurement, but it should be
instructive regardless.

If it's not important enough to do some basic measurements, it's probably
not important enough to worry much at all about the performance of different
implementations. :)

Pete
Dec 11 '06 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Andras Gilicz | last post by:
Hi VB fans I'm working on a relatively large project in VB6 with about a dozen forms, including graphs, labels, text boxes, etc. The software itself is actually a flow simulator with more or...
1
by: Robin Tucker | last post by:
Just a quick question about connection management. My application will never need more than 1 or 2 connections about at any given time. Also, I do not expect many users to be connected at any...
2
by: Irvin Amoraal | last post by:
Process: I have a form which uploads a file from client to server written in PHP. When the user presses the submit button, I use the "onSubmit" event to execute javascript to open a child window...
6
by: chon | last post by:
I have an ASP page that is sent a file location as a parameter. It opens this file, loads the ADODB.Stream object and does a binary write to the page forcing the download dialog to appear. This...
5
by: Yoav | last post by:
I use the following code to the console output: def get_console(cmd): try: p_in, p_out, p_err = os.popen3(cmd) except: pass out_str = '' for obj in p_out: out_str = out_str + obj
4
by: Bill | last post by:
I need help closing a CMD window when it is executed from Access. 1) The batch file is called from Access. 2) Access closes, 3) the batch runs a copy of the access database (creating a backup)...
3
by: Bill Nguyen | last post by:
When my app attempts to save a particular file in the same VB session, I got an error message . If I close the app and run again, the file is overwritten ok. I guess I need to close the file...
11
by: Dave | last post by:
For some reason, the below lines only work on select machines. All machines are running IE6. IE SP's and OS's vary. When it doesn't work, default.aspx (the page that this code is in) opens and...
6
by: FAQ server | last post by:
----------------------------------------------------------------------- FAQ Topic - How do I close a window and why does it not work on the first one?...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.