473,326 Members | 2,099 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

Design question...

I am writing a reusable class for logging.
My goal is to make it as fast and as robust as possible, while keeping
memory usage to the lowest. All members are static.

For some reason I'm stuck on the following design question.
Obviously you need a stream to write the log file.
Should this stream be created every time the log needs to be written, or
should the stream be a member variable of the logging class and only be
opened and closed once per application lifetime.

Can someone suggest the right way to go here and why?
This seems like a simple question but for some reason I can't decide
which is the right way to do this. Does it even matter?
Nov 17 '05 #1
18 1997
"Nick Z." <pa*****@gmail.com> wrote in message
news:OO**************@fe10.lga...
I am writing a reusable class for logging.
My goal is to make it as fast and as robust as possible, while keeping
memory usage to the lowest. All members are static.

For some reason I'm stuck on the following design question.
Obviously you need a stream to write the log file.
Should this stream be created every time the log needs to be written, or
should the stream be a member variable of the logging class and only be
opened and closed once per application lifetime.

Can someone suggest the right way to go here and why?
This seems like a simple question but for some reason I can't decide which
is the right way to do this. Does it even matter?


For logs I open and close the stream each time. If someone happens to
terminate the app premeturely you need to be able to see the log.

Michael
Nov 17 '05 #2
Michael C wrote:
"Nick Z." <pa*****@gmail.com> wrote in message
news:OO**************@fe10.lga...
I am writing a reusable class for logging.
My goal is to make it as fast and as robust as possible, while keeping
memory usage to the lowest. All members are static.

For some reason I'm stuck on the following design question.
Obviously you need a stream to write the log file.
Should this stream be created every time the log needs to be written, or
should the stream be a member variable of the logging class and only be
opened and closed once per application lifetime.

Can someone suggest the right way to go here and why?
This seems like a simple question but for some reason I can't decide which
is the right way to do this. Does it even matter?

For logs I open and close the stream each time. If someone happens to
terminate the app premeturely you need to be able to see the log.

Michael


Hmmm, true. But what if you force the stream to flush after each write.
I guess I want to know what the overhead is for creating the stream
every time.
Nov 17 '05 #3
Hi Nick,
I am sure you have good reason to write your own logging class, but have
you looked at some of the out of the box logging .Net offers you such as a
TextWriterTraceListener, you can then write to log files using the:

System.Diagnostics.Trace or
System.Diagnostics.Debug classes.

Mark

"Nick Z." wrote:
I am writing a reusable class for logging.
My goal is to make it as fast and as robust as possible, while keeping
memory usage to the lowest. All members are static.

For some reason I'm stuck on the following design question.
Obviously you need a stream to write the log file.
Should this stream be created every time the log needs to be written, or
should the stream be a member variable of the logging class and only be
opened and closed once per application lifetime.

Can someone suggest the right way to go here and why?
This seems like a simple question but for some reason I can't decide
which is the right way to do this. Does it even matter?

Nov 17 '05 #4
Hey Mark,

Thanks for you input. As far as I know using System.Diagnostics only
provides the ability to save event to system's application EventLog, the
one accessible through administrative tools. Or am I wrong? Would I be
able to easily retrieve/collect events generated by my application
specifically on demand? (for error reporting etc.)

The reason I want to implement my own logging class is making the log
file as easy to use and process as possible yet as non intrusive as
possible. I wanted to use xml and basically provide some structure to
the log file making it easier to parse without adversely affecting
performance of the logging class. I want to make it extremely fast so I
don't feel uncomfortable posting events to the log. Would creating a
separate thread to do file access be a smart thing to do or do the
stream writers (XmlTextWriter to be more precise) run asynchronously (in
another thread) anyway?

Thanks,
Nick Z.
Mark R. Dawson wrote:
Hi Nick,
I am sure you have good reason to write your own logging class, but have
you looked at some of the out of the box logging .Net offers you such as a
TextWriterTraceListener, you can then write to log files using the:

System.Diagnostics.Trace or
System.Diagnostics.Debug classes.

Mark

"Nick Z." wrote:

I am writing a reusable class for logging.
My goal is to make it as fast and as robust as possible, while keeping
memory usage to the lowest. All members are static.

For some reason I'm stuck on the following design question.
Obviously you need a stream to write the log file.
Should this stream be created every time the log needs to be written, or
should the stream be a member variable of the logging class and only be
opened and closed once per application lifetime.

Can someone suggest the right way to go here and why?
This seems like a simple question but for some reason I can't decide
which is the right way to do this. Does it even matter?

Nov 17 '05 #5
Hi Nick,
writing logs to the event log is just one of the places you can write the
System.Diagnostics.Trace output to, this would be if you used an
EventLogTraceListener. If you want to output info easily to a file or the
console you can use a TextWriterTraceListener class.

If you are using .Net 2.0 then it has another Trace writer you can use
called an XmlWriterTraceListener which will write your info to an Xml File.

You can very easily add or remove the TraceListeners from your app by
modifying the App.Config file (or Web.Config for the web) and adding a new
trace listener (search Google for more info on how to do this).

You could also write your own custom trace listener by inheriting from
the abstract class System.Diagnostics.TraceListener. For an example see:
http://www.codeproject.com/dotnet/cu...elisteners.asp

I recently did this to allow me to log trace messages over a network in
real-time using Remoting. I like you, was worried about affecting the
performance of the application by using tracing. What I did was to have a
queue, when the trace event occurs it simply places a message in the queue,
then I have a thread which loops through all the messages in the queue and
processes them, this way there is no performance hit for the app and the
thread can take it's time.
Hope that helps
Mark R Dawson

"Nick Z." wrote:
Hey Mark,

Thanks for you input. As far as I know using System.Diagnostics only
provides the ability to save event to system's application EventLog, the
one accessible through administrative tools. Or am I wrong? Would I be
able to easily retrieve/collect events generated by my application
specifically on demand? (for error reporting etc.)

The reason I want to implement my own logging class is making the log
file as easy to use and process as possible yet as non intrusive as
possible. I wanted to use xml and basically provide some structure to
the log file making it easier to parse without adversely affecting
performance of the logging class. I want to make it extremely fast so I
don't feel uncomfortable posting events to the log. Would creating a
separate thread to do file access be a smart thing to do or do the
stream writers (XmlTextWriter to be more precise) run asynchronously (in
another thread) anyway?

Thanks,
Nick Z.
Mark R. Dawson wrote:
Hi Nick,
I am sure you have good reason to write your own logging class, but have
you looked at some of the out of the box logging .Net offers you such as a
TextWriterTraceListener, you can then write to log files using the:

System.Diagnostics.Trace or
System.Diagnostics.Debug classes.

Mark

"Nick Z." wrote:

I am writing a reusable class for logging.
My goal is to make it as fast and as robust as possible, while keeping
memory usage to the lowest. All members are static.

For some reason I'm stuck on the following design question.
Obviously you need a stream to write the log file.
Should this stream be created every time the log needs to be written, or
should the stream be a member variable of the logging class and only be
opened and closed once per application lifetime.

Can someone suggest the right way to go here and why?
This seems like a simple question but for some reason I can't decide
which is the right way to do this. Does it even matter?

Nov 17 '05 #6
I wrote a little test program to find out if opening and closing a file
1000 times introduces any major overhead, or slows down a loop at all.

it does not. well, it does, but not much at all. I have a p4-3ghz,
1.5gb RAM. This is using .NET 2.0 beta2.

test 1 opens a file, logs to it 1000 times, then closes the file.
test 2 opens a file, logts to it, closes it, then does all that 999 more
times.

here are the results:
0ms for the first test.
187ms for the second test.

Here's the source:

using System;
using System.IO;
using System.Collections.Generic;
using System.Text;

namespace fileopenclosetest {
class Program {
static void Main(string[] args) {

DateTime beforeFirst = DateTime.Now;

FileStream fs = new FileStream("test1.txt", FileMode.OpenOrCreate);
StreamWriter sw = new StreamWriter(fs);

for (int i = 0; i < 1000; i++) {
sw.WriteLine("this is a log line. It is logline " + i + ".");
}

sw.Close();
fs.Close();

DateTime afterFirst = DateTime.Now;

TimeSpan ts = afterFirst - beforeFirst;

Console.WriteLine(ts.Milliseconds + "ms for the first test.");

/////////////////////// second test.

DateTime beforeSecond = DateTime.Now;

for (int i = 0; i < 1000; i++) {

FileStream fs2 = new FileStream("test2.txt", FileMode.Append);
StreamWriter sw2 = new StreamWriter(fs2);

sw2.WriteLine("this is a log line. It is logline " + i + ".");
sw2.Close();
fs2.Close();

sw2=null;
fs2=null;

}

DateTime afterSecond = DateTime.Now;

TimeSpan ts2 = afterSecond - beforeSecond;

Console.WriteLine(ts2.Milliseconds + "ms for the second test.");

}
}
}

Nick Z. wrote:
Michael C wrote:
"Nick Z." <pa*****@gmail.com> wrote in message
news:OO**************@fe10.lga...
I am writing a reusable class for logging.
My goal is to make it as fast and as robust as possible, while
keeping memory usage to the lowest. All members are static.

For some reason I'm stuck on the following design question.
Obviously you need a stream to write the log file.
Should this stream be created every time the log needs to be written,
or should the stream be a member variable of the logging class and
only be opened and closed once per application lifetime.

Can someone suggest the right way to go here and why?
This seems like a simple question but for some reason I can't decide
which is the right way to do this. Does it even matter?


For logs I open and close the stream each time. If someone happens to
terminate the app premeturely you need to be able to see the log.

Michael


Hmmm, true. But what if you force the stream to flush after each write.
I guess I want to know what the overhead is for creating the stream
every time.

Nov 17 '05 #7
I’ve done something similar to this using a queue to add the messages to log
and let a separate thread do the logging. It works great!
--
Don DenUyl
Diamond Systems
"Mark R. Dawson" wrote:
Hi Nick,
writing logs to the event log is just one of the places you can write the
System.Diagnostics.Trace output to, this would be if you used an
EventLogTraceListener. If you want to output info easily to a file or the
console you can use a TextWriterTraceListener class.

If you are using .Net 2.0 then it has another Trace writer you can use
called an XmlWriterTraceListener which will write your info to an Xml File.

You can very easily add or remove the TraceListeners from your app by
modifying the App.Config file (or Web.Config for the web) and adding a new
trace listener (search Google for more info on how to do this).

You could also write your own custom trace listener by inheriting from
the abstract class System.Diagnostics.TraceListener. For an example see:
http://www.codeproject.com/dotnet/cu...elisteners.asp

I recently did this to allow me to log trace messages over a network in
real-time using Remoting. I like you, was worried about affecting the
performance of the application by using tracing. What I did was to have a
queue, when the trace event occurs it simply places a message in the queue,
then I have a thread which loops through all the messages in the queue and
processes them, this way there is no performance hit for the app and the
thread can take it's time.
Hope that helps
Mark R Dawson

"Nick Z." wrote:
Hey Mark,

Thanks for you input. As far as I know using System.Diagnostics only
provides the ability to save event to system's application EventLog, the
one accessible through administrative tools. Or am I wrong? Would I be
able to easily retrieve/collect events generated by my application
specifically on demand? (for error reporting etc.)

The reason I want to implement my own logging class is making the log
file as easy to use and process as possible yet as non intrusive as
possible. I wanted to use xml and basically provide some structure to
the log file making it easier to parse without adversely affecting
performance of the logging class. I want to make it extremely fast so I
don't feel uncomfortable posting events to the log. Would creating a
separate thread to do file access be a smart thing to do or do the
stream writers (XmlTextWriter to be more precise) run asynchronously (in
another thread) anyway?

Thanks,
Nick Z.
Mark R. Dawson wrote:
Hi Nick,
I am sure you have good reason to write your own logging class, but have
you looked at some of the out of the box logging .Net offers you such as a
TextWriterTraceListener, you can then write to log files using the:

System.Diagnostics.Trace or
System.Diagnostics.Debug classes.

Mark

"Nick Z." wrote:
>I am writing a reusable class for logging.
>My goal is to make it as fast and as robust as possible, while keeping
>memory usage to the lowest. All members are static.
>
>For some reason I'm stuck on the following design question.
>Obviously you need a stream to write the log file.
>Should this stream be created every time the log needs to be written, or
>should the stream be a member variable of the logging class and only be
>opened and closed once per application lifetime.
>
>Can someone suggest the right way to go here and why?
>This seems like a simple question but for some reason I can't decide
>which is the right way to do this. Does it even matter?
>

Nov 17 '05 #8
Thank you very much for the benchmarks!

187 per 1000 calls will probably be negligible for most applications.

Nick Z.

jeremiah johnson wrote:
I wrote a little test program to find out if opening and closing a file
1000 times introduces any major overhead, or slows down a loop at all.

it does not. well, it does, but not much at all. I have a p4-3ghz,
1.5gb RAM. This is using .NET 2.0 beta2.

test 1 opens a file, logs to it 1000 times, then closes the file.
test 2 opens a file, logts to it, closes it, then does all that 999 more
times.

here are the results:
0ms for the first test.
187ms for the second test.

Here's the source:

using System;
using System.IO;
using System.Collections.Generic;
using System.Text;

namespace fileopenclosetest {
class Program {
static void Main(string[] args) {

DateTime beforeFirst = DateTime.Now;

FileStream fs = new FileStream("test1.txt", FileMode.OpenOrCreate);
StreamWriter sw = new StreamWriter(fs);

for (int i = 0; i < 1000; i++) {
sw.WriteLine("this is a log line. It is logline " + i + ".");
}

sw.Close();
fs.Close();

DateTime afterFirst = DateTime.Now;

TimeSpan ts = afterFirst - beforeFirst;

Console.WriteLine(ts.Milliseconds + "ms for the first test.");

/////////////////////// second test.

DateTime beforeSecond = DateTime.Now;

for (int i = 0; i < 1000; i++) {

FileStream fs2 = new FileStream("test2.txt", FileMode.Append);
StreamWriter sw2 = new StreamWriter(fs2);

sw2.WriteLine("this is a log line. It is logline " + i + ".");
sw2.Close();
fs2.Close();

sw2=null;
fs2=null;

}

DateTime afterSecond = DateTime.Now;

TimeSpan ts2 = afterSecond - beforeSecond;

Console.WriteLine(ts2.Milliseconds + "ms for the second test.");
}
}
}

Nick Z. wrote:
Michael C wrote:
"Nick Z." <pa*****@gmail.com> wrote in message
news:OO**************@fe10.lga...

I am writing a reusable class for logging.
My goal is to make it as fast and as robust as possible, while
keeping memory usage to the lowest. All members are static.

For some reason I'm stuck on the following design question.
Obviously you need a stream to write the log file.
Should this stream be created every time the log needs to be
written, or should the stream be a member variable of the logging
class and only be opened and closed once per application lifetime.

Can someone suggest the right way to go here and why?
This seems like a simple question but for some reason I can't decide
which is the right way to do this. Does it even matter?


For logs I open and close the stream each time. If someone happens to
terminate the app premeturely you need to be able to see the log.

Michael


Hmmm, true. But what if you force the stream to flush after each write.
I guess I want to know what the overhead is for creating the stream
every time.

Nov 17 '05 #9
Tried microsoft application blocks for logging

-------
Regards ,
C#, VB.NET , SQL SERVER , UML , DESIGN Patterns Interview question book
http://www.geocities.com/dotnetinterviews/
My Interview Blog
http://spaces.msn.com/members/dotnetinterviews/

Nov 17 '05 #10
Mark R. Dawson wrote:
Hi Nick,
writing logs to the event log is just one of the places you can write the
System.Diagnostics.Trace output to, this would be if you used an
EventLogTraceListener. If you want to output info easily to a file or the
console you can use a TextWriterTraceListener class.

If you are using .Net 2.0 then it has another Trace writer you can use
called an XmlWriterTraceListener which will write your info to an Xml File.

You can very easily add or remove the TraceListeners from your app by
modifying the App.Config file (or Web.Config for the web) and adding a new
trace listener (search Google for more info on how to do this).

You could also write your own custom trace listener by inheriting from
the abstract class System.Diagnostics.TraceListener. For an example see:
http://www.codeproject.com/dotnet/cu...elisteners.asp
This looks interesting. However, what would be the advantage of
implementing a TraceListener? XmlWriterTraceListener sounds like a great
idea, but its only supported in 2.0. For a class like this I'd rather
have it support all versions of the framework.

I recently did this to allow me to log trace messages over a network in
real-time using Remoting. I like you, was worried about affecting the
performance of the application by using tracing. What I did was to have a
queue, when the trace event occurs it simply places a message in the queue,
then I have a thread which loops through all the messages in the queue and
processes them, this way there is no performance hit for the app and the
thread can take it's time.

I am probably going to do the same. This sounds very efficient.
However, isn't the writing done in a separate thread anyway? So in a
sense aren't you implementing a thread that simply starts another thread
to write the log.

Just out of curiosity, when the thread that is writing the messages runs
out of messages to write, does it terminate or wait for new messages to
come in. If it waits how could that be implemented?

Hope that helps
Mark R Dawson

"Nick Z." wrote:

Hey Mark,

Thanks for you input. As far as I know using System.Diagnostics only
provides the ability to save event to system's application EventLog, the
one accessible through administrative tools. Or am I wrong? Would I be
able to easily retrieve/collect events generated by my application
specifically on demand? (for error reporting etc.)

The reason I want to implement my own logging class is making the log
file as easy to use and process as possible yet as non intrusive as
possible. I wanted to use xml and basically provide some structure to
the log file making it easier to parse without adversely affecting
performance of the logging class. I want to make it extremely fast so I
don't feel uncomfortable posting events to the log. Would creating a
separate thread to do file access be a smart thing to do or do the
stream writers (XmlTextWriter to be more precise) run asynchronously (in
another thread) anyway?

Thanks,
Nick Z.
Mark R. Dawson wrote:

Hi Nick,
I am sure you have good reason to write your own logging class, but have
you looked at some of the out of the box logging .Net offers you such as a
TextWriterTraceListener, you can then write to log files using the:

System.Diagnostics.Trace or
System.Diagnostics.Debug classes.

Mark

"Nick Z." wrote:

I am writing a reusable class for logging.
My goal is to make it as fast and as robust as possible, while keeping
memory usage to the lowest. All members are static.

For some reason I'm stuck on the following design question.
Obviously you need a stream to write the log file.
Should this stream be created every time the log needs to be written, or
should the stream be a member variable of the logging class and only be
opened and closed once per application lifetime.

Can someone suggest the right way to go here and why?
This seems like a simple question but for some reason I can't decide
which is the right way to do this. Does it even matter?

Nov 17 '05 #11
Also, doesn't Trace require that the user change something in the config
file? What exactly does this mean?

From the codeproject article: (I believe I saw something similar in the
official docs as well)
"Trace functions can be compiled into a program and shipped to
customers, so in case your users encounter a problem, they can activate
trace by simply editing application configuration file."

Thank you,
Nick Z.

Mark R. Dawson wrote:
Hi Nick,
writing logs to the event log is just one of the places you can write the
System.Diagnostics.Trace output to, this would be if you used an
EventLogTraceListener. If you want to output info easily to a file or the
console you can use a TextWriterTraceListener class.

If you are using .Net 2.0 then it has another Trace writer you can use
called an XmlWriterTraceListener which will write your info to an Xml File.

You can very easily add or remove the TraceListeners from your app by
modifying the App.Config file (or Web.Config for the web) and adding a new
trace listener (search Google for more info on how to do this).

You could also write your own custom trace listener by inheriting from
the abstract class System.Diagnostics.TraceListener. For an example see:
http://www.codeproject.com/dotnet/cu...elisteners.asp

I recently did this to allow me to log trace messages over a network in
real-time using Remoting. I like you, was worried about affecting the
performance of the application by using tracing. What I did was to have a
queue, when the trace event occurs it simply places a message in the queue,
then I have a thread which loops through all the messages in the queue and
processes them, this way there is no performance hit for the app and the
thread can take it's time.
Hope that helps
Mark R Dawson

"Nick Z." wrote:

Hey Mark,

Thanks for you input. As far as I know using System.Diagnostics only
provides the ability to save event to system's application EventLog, the
one accessible through administrative tools. Or am I wrong? Would I be
able to easily retrieve/collect events generated by my application
specifically on demand? (for error reporting etc.)

The reason I want to implement my own logging class is making the log
file as easy to use and process as possible yet as non intrusive as
possible. I wanted to use xml and basically provide some structure to
the log file making it easier to parse without adversely affecting
performance of the logging class. I want to make it extremely fast so I
don't feel uncomfortable posting events to the log. Would creating a
separate thread to do file access be a smart thing to do or do the
stream writers (XmlTextWriter to be more precise) run asynchronously (in
another thread) anyway?

Thanks,
Nick Z.
Mark R. Dawson wrote:

Hi Nick,
I am sure you have good reason to write your own logging class, but have
you looked at some of the out of the box logging .Net offers you such as a
TextWriterTraceListener, you can then write to log files using the:

System.Diagnostics.Trace or
System.Diagnostics.Debug classes.

Mark

"Nick Z." wrote:

I am writing a reusable class for logging.
My goal is to make it as fast and as robust as possible, while keeping
memory usage to the lowest. All members are static.

For some reason I'm stuck on the following design question.
Obviously you need a stream to write the log file.
Should this stream be created every time the log needs to be written, or
should the stream be a member variable of the logging class and only be
opened and closed once per application lifetime.

Can someone suggest the right way to go here and why?
This seems like a simple question but for some reason I can't decide
which is the right way to do this. Does it even matter?

Nov 17 '05 #12
I'd also have a look at log4net (http://logging.apache.org/log4net/) which
is an port of a java logging framework - fairly simple to use in practice.

Paul

"Nick Z." <pa*****@gmail.com> wrote in message
news:eG*****************@fe12.lga...
Hey Mark,

Thanks for you input. As far as I know using System.Diagnostics only
provides the ability to save event to system's application EventLog, the
one accessible through administrative tools. Or am I wrong? Would I be
able to easily retrieve/collect events generated by my application
specifically on demand? (for error reporting etc.)

The reason I want to implement my own logging class is making the log file
as easy to use and process as possible yet as non intrusive as possible. I
wanted to use xml and basically provide some structure to the log file
making it easier to parse without adversely affecting performance of the
logging class. I want to make it extremely fast so I don't feel
uncomfortable posting events to the log. Would creating a separate thread
to do file access be a smart thing to do or do the stream writers
(XmlTextWriter to be more precise) run asynchronously (in another thread)
anyway?

Thanks,
Nick Z.
Mark R. Dawson wrote:
Hi Nick,
I am sure you have good reason to write your own logging class, but have
you looked at some of the out of the box logging .Net offers you such as
a TextWriterTraceListener, you can then write to log files using the:

System.Diagnostics.Trace or
System.Diagnostics.Debug classes. Mark

"Nick Z." wrote:

I am writing a reusable class for logging.
My goal is to make it as fast and as robust as possible, while keeping
memory usage to the lowest. All members are static.

For some reason I'm stuck on the following design question.
Obviously you need a stream to write the log file.
Should this stream be created every time the log needs to be written, or
should the stream be a member variable of the logging class and only be
opened and closed once per application lifetime.

Can someone suggest the right way to go here and why?
This seems like a simple question but for some reason I can't decide
which is the right way to do this. Does it even matter?

Nov 17 '05 #13
Hi Nick,
The advantage of writing a class that Inherits from TraceListener is that
you can write your own custom listener that is not provided as standard by
Microsoft, I did this to send my trace messages over a network using remoting.

One way to achieve asyncronosity is to have the main app thread write data
into the log queue, then a seperate thread sits and waits for the data and
pulls it out of the queue and performs processing. This consumer thread will
loop until it is told to stop, when there is no data it will just wait until
more data is added to the log. In the example below I used an AutoResetEvent
to make the thread wait until the Producer added more data to the queue:
using System;
using System.Threading;
using System.Collections;

namespace ConsoleApplication5
{
class Program
{
static void Main(string[] args)
{
MessageManager mm = new MessageManager();
Random random = new Random();

int numberofMessages = 100;
int count = 0;
while (count++ < numberofMessages)
{
mm.AddMessage("hello: " + count.ToString());
Thread.Sleep((int)(random.NextDouble() * 300));
}
}
}

public class MessageManager
{
private AutoResetEvent _consumerWait;
private Queue _queue;
private Thread _consumer;
private bool _continueProcessing;

public MessageManager()
{
_queue = new Queue();
_consumerWait = new AutoResetEvent(false);
_continueProcessing = true;

_consumer = new Thread(new ThreadStart(ConsumeMessages));
_consumer.IsBackground = true;
_consumer.Start();
}

private void Stop()
{
_continueProcessing = false;
_consumerWait.Set();
}

private void ConsumeMessages()
{
//Loop until we no longer want to process messages
while (_continueProcessing)
{
//process all the current messages
while (_queue.Count > 0)
{
string message = (string)_queue.Dequeue();
Console.WriteLine(message);
}

//wait until there is data to process
_consumerWait.WaitOne();
}
}

public void AddMessage(string message)
{
//Add message to the queue
_queue.Enqueue(message);

//Signal there is more data
_consumerWait.Set();
}
}
Hope that helps
Mark R Dawson
http://www.markdawson.org

"Nick Z." wrote:
Mark R. Dawson wrote:
Hi Nick,
writing logs to the event log is just one of the places you can write the
System.Diagnostics.Trace output to, this would be if you used an
EventLogTraceListener. If you want to output info easily to a file or the
console you can use a TextWriterTraceListener class.

If you are using .Net 2.0 then it has another Trace writer you can use
called an XmlWriterTraceListener which will write your info to an Xml File.

You can very easily add or remove the TraceListeners from your app by
modifying the App.Config file (or Web.Config for the web) and adding a new
trace listener (search Google for more info on how to do this).

You could also write your own custom trace listener by inheriting from
the abstract class System.Diagnostics.TraceListener. For an example see:
http://www.codeproject.com/dotnet/cu...elisteners.asp


This looks interesting. However, what would be the advantage of
implementing a TraceListener? XmlWriterTraceListener sounds like a great
idea, but its only supported in 2.0. For a class like this I'd rather
have it support all versions of the framework.

I recently did this to allow me to log trace messages over a network in
real-time using Remoting. I like you, was worried about affecting the
performance of the application by using tracing. What I did was to have a
queue, when the trace event occurs it simply places a message in the queue,
then I have a thread which loops through all the messages in the queue and
processes them, this way there is no performance hit for the app and the
thread can take it's time.


I am probably going to do the same. This sounds very efficient.
However, isn't the writing done in a separate thread anyway? So in a
sense aren't you implementing a thread that simply starts another thread
to write the log.

Just out of curiosity, when the thread that is writing the messages runs
out of messages to write, does it terminate or wait for new messages to
come in. If it waits how could that be implemented?

Hope that helps
Mark R Dawson

"Nick Z." wrote:

Hey Mark,

Thanks for you input. As far as I know using System.Diagnostics only
provides the ability to save event to system's application EventLog, the
one accessible through administrative tools. Or am I wrong? Would I be
able to easily retrieve/collect events generated by my application
specifically on demand? (for error reporting etc.)

The reason I want to implement my own logging class is making the log
file as easy to use and process as possible yet as non intrusive as
possible. I wanted to use xml and basically provide some structure to
the log file making it easier to parse without adversely affecting
performance of the logging class. I want to make it extremely fast so I
don't feel uncomfortable posting events to the log. Would creating a
separate thread to do file access be a smart thing to do or do the
stream writers (XmlTextWriter to be more precise) run asynchronously (in
another thread) anyway?

Thanks,
Nick Z.
Mark R. Dawson wrote:
Hi Nick,
I am sure you have good reason to write your own logging class, but have
you looked at some of the out of the box logging .Net offers you such as a
TextWriterTraceListener, you can then write to log files using the:

System.Diagnostics.Trace or
System.Diagnostics.Debug classes.

Mark

"Nick Z." wrote:

>I am writing a reusable class for logging.
>My goal is to make it as fast and as robust as possible, while keeping
>memory usage to the lowest. All members are static.
>
>For some reason I'm stuck on the following design question.
>Obviously you need a stream to write the log file.
>Should this stream be created every time the log needs to be written, or
>should the stream be a member variable of the logging class and only be
>opened and closed once per application lifetime.
>
>Can someone suggest the right way to go here and why?
>This seems like a simple question but for some reason I can't decide
>which is the right way to do this. Does it even matter?
>

Nov 17 '05 #14
I looked at log4net, but I think its just too much (bloated) for a
simple logging solution. It has features that I will never use.

Paul Hatcher wrote:
I'd also have a look at log4net (http://logging.apache.org/log4net/) which
is an port of a java logging framework - fairly simple to use in practice.

Paul

"Nick Z." <pa*****@gmail.com> wrote in message
news:eG*****************@fe12.lga...
Hey Mark,

Thanks for you input. As far as I know using System.Diagnostics only
provides the ability to save event to system's application EventLog, the
one accessible through administrative tools. Or am I wrong? Would I be
able to easily retrieve/collect events generated by my application
specifically on demand? (for error reporting etc.)

The reason I want to implement my own logging class is making the log file
as easy to use and process as possible yet as non intrusive as possible. I
wanted to use xml and basically provide some structure to the log file
making it easier to parse without adversely affecting performance of the
logging class. I want to make it extremely fast so I don't feel
uncomfortable posting events to the log. Would creating a separate thread
to do file access be a smart thing to do or do the stream writers
(XmlTextWriter to be more precise) run asynchronously (in another thread)
anyway?

Thanks,
Nick Z.
Mark R. Dawson wrote:
Hi Nick,
I am sure you have good reason to write your own logging class, but have
you looked at some of the out of the box logging .Net offers you such as
a TextWriterTraceListener, you can then write to log files using the:

System.Diagnostics.Trace or
System.Diagnostics.Debug classes. Mark

"Nick Z." wrote:
I am writing a reusable class for logging.
My goal is to make it as fast and as robust as possible, while keeping
memory usage to the lowest. All members are static.

For some reason I'm stuck on the following design question.
Obviously you need a stream to write the log file.
Should this stream be created every time the log needs to be written, or
should the stream be a member variable of the logging class and only be
opened and closed once per application lifetime.

Can someone suggest the right way to go here and why?
This seems like a simple question but for some reason I can't decide
which is the right way to do this. Does it even matter?


Nov 17 '05 #15
Thank you very much for the code!
Very clear right to the point. Basically you need some synchronization
object for the thread in your case it was AutoResetEvent, got it.

Thanks,
Nick Z.

Mark R. Dawson wrote:
Hi Nick,
The advantage of writing a class that Inherits from TraceListener is that
you can write your own custom listener that is not provided as standard by
Microsoft, I did this to send my trace messages over a network using remoting.

One way to achieve asyncronosity is to have the main app thread write data
into the log queue, then a seperate thread sits and waits for the data and
pulls it out of the queue and performs processing. This consumer thread will
loop until it is told to stop, when there is no data it will just wait until
more data is added to the log. In the example below I used an AutoResetEvent
to make the thread wait until the Producer added more data to the queue:
using System;
using System.Threading;
using System.Collections;

namespace ConsoleApplication5
{
class Program
{
static void Main(string[] args)
{
MessageManager mm = new MessageManager();
Random random = new Random();

int numberofMessages = 100;
int count = 0;
while (count++ < numberofMessages)
{
mm.AddMessage("hello: " + count.ToString());
Thread.Sleep((int)(random.NextDouble() * 300));
}
}
}

public class MessageManager
{
private AutoResetEvent _consumerWait;
private Queue _queue;
private Thread _consumer;
private bool _continueProcessing;

public MessageManager()
{
_queue = new Queue();
_consumerWait = new AutoResetEvent(false);
_continueProcessing = true;

_consumer = new Thread(new ThreadStart(ConsumeMessages));
_consumer.IsBackground = true;
_consumer.Start();
}

private void Stop()
{
_continueProcessing = false;
_consumerWait.Set();
}

private void ConsumeMessages()
{
//Loop until we no longer want to process messages
while (_continueProcessing)
{
//process all the current messages
while (_queue.Count > 0)
{
string message = (string)_queue.Dequeue();
Console.WriteLine(message);
}

//wait until there is data to process
_consumerWait.WaitOne();
}
}

public void AddMessage(string message)
{
//Add message to the queue
_queue.Enqueue(message);

//Signal there is more data
_consumerWait.Set();
}
}
Hope that helps
Mark R Dawson
http://www.markdawson.org

"Nick Z." wrote:
Mark R. Dawson wrote:
Hi Nick,
writing logs to the event log is just one of the places you can write the
System.Diagnostics.Trace output to, this would be if you used an
EventLogTraceListener. If you want to output info easily to a file or the
console you can use a TextWriterTraceListener class.

If you are using .Net 2.0 then it has another Trace writer you can use
called an XmlWriterTraceListener which will write your info to an Xml File.

You can very easily add or remove the TraceListeners from your app by
modifying the App.Config file (or Web.Config for the web) and adding a new
trace listener (search Google for more info on how to do this).

You could also write your own custom trace listener by inheriting from
the abstract class System.Diagnostics.TraceListener. For an example see:
http://www.codeproject.com/dotnet/cu...elisteners.asp


This looks interesting. However, what would be the advantage of
implementing a TraceListener? XmlWriterTraceListener sounds like a great
idea, but its only supported in 2.0. For a class like this I'd rather
have it support all versions of the framework.

I recently did this to allow me to log trace messages over a network in
real-time using Remoting. I like you, was worried about affecting the
performance of the application by using tracing. What I did was to have a
queue, when the trace event occurs it simply places a message in the queue,
then I have a thread which loops through all the messages in the queue and
processes them, this way there is no performance hit for the app and the
thread can take it's time.


I am probably going to do the same. This sounds very efficient.
However, isn't the writing done in a separate thread anyway? So in a
sense aren't you implementing a thread that simply starts another thread
to write the log.

Just out of curiosity, when the thread that is writing the messages runs
out of messages to write, does it terminate or wait for new messages to
come in. If it waits how could that be implemented?

Hope that helps
Mark R Dawson

"Nick Z." wrote:
>Hey Mark,
>
>Thanks for you input. As far as I know using System.Diagnostics only
>provides the ability to save event to system's application EventLog, the
>one accessible through administrative tools. Or am I wrong? Would I be
>able to easily retrieve/collect events generated by my application
>specifically on demand? (for error reporting etc.)
>
>The reason I want to implement my own logging class is making the log
>file as easy to use and process as possible yet as non intrusive as
>possible. I wanted to use xml and basically provide some structure to
>the log file making it easier to parse without adversely affecting
>performance of the logging class. I want to make it extremely fast so I
>don't feel uncomfortable posting events to the log. Would creating a
>separate thread to do file access be a smart thing to do or do the
>stream writers (XmlTextWriter to be more precise) run asynchronously (in
>another thread) anyway?
>
>Thanks,
>Nick Z.
>
>
>Mark R. Dawson wrote:
>
>
>>Hi Nick,
>> I am sure you have good reason to write your own logging class, but have
>>you looked at some of the out of the box logging .Net offers you such as a
>>TextWriterTraceListener, you can then write to log files using the:
>>
>>System.Diagnostics.Trace or
>>System.Diagnostics.Debug classes.
>>
>>Mark
>>
>>"Nick Z." wrote:
>>
>>
>>
>>>I am writing a reusable class for logging.
>>>My goal is to make it as fast and as robust as possible, while keeping
>>>memory usage to the lowest. All members are static.
>>>
>>>For some reason I'm stuck on the following design question.
>>>Obviously you need a stream to write the log file.
>>>Should this stream be created every time the log needs to be written, or
>>>should the stream be a member variable of the logging class and only be
>>>opened and closed once per application lifetime.
>>>
>>>Can someone suggest the right way to go here and why?
>>>This seems like a simple question but for some reason I can't decide
>>>which is the right way to do this. Does it even matter?
>>>
>


Nov 17 '05 #16
Nick Z. wrote:
Thank you very much for the benchmarks!

187 per 1000 calls will probably be negligible for most applications.

Nick Z.


That's 187 Milliseconds - less than a fifth of a second for all 1000
opens and all 1000 closes.

jeremiah
Nov 17 '05 #17
jeremiah johnson wrote:
Nick Z. wrote:
Thank you very much for the benchmarks!

187 per 1000 calls will probably be negligible for most applications.

Nick Z.


That's 187 Milliseconds - less than a fifth of a second for all 1000
opens and all 1000 closes.

jeremiah

Yeah, thats what I meant, skipped a word, oops.
Nov 17 '05 #18
When opening a XmlTextWriter along with a stream and writing 1000 times
it takes 1400-1600 ms, you still think its a good idea to open/close
every time?

If I keep the stream open throughout the lifetime of the class it takes
10-80 ms to write 1000 times.

Is it bad design to have a static resource open throughout a lifetime of
a class with all static members? I mean it sure seems "nicer"
opening/closing every time, but its just that much slower and I dont
want to ever think about performance of that class again. I just want to
make it as fast as reliable as possible the first time. However, I
dont'n want to get used to bad design practices. So my question is why
is keeping the stream open a bad idea?

Thanks,
Nick Z.

Nick Z. wrote:
jeremiah johnson wrote:
Nick Z. wrote:
Thank you very much for the benchmarks!

187 per 1000 calls will probably be negligible for most applications.

Nick Z.


That's 187 Milliseconds - less than a fifth of a second for all 1000
opens and all 1000 closes.

jeremiah


Yeah, thats what I meant, skipped a word, oops.

Nov 17 '05 #19

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Don Vaillancourt | last post by:
Hello all, Over the years as I design more database schemas the more I come up with patterns in database design. The more patterns I recognize the more I want to try to design some kind of...
9
by: sk | last post by:
I have an applicaton in which I collect data for different parameters for a set of devices. The data are entered into a single table, each set of name, value pairs time-stamped and associated with...
2
by: Test User | last post by:
Hi all, (please excuse the crosspost as I'm trying to reach as many people as possible) I am somewhat familiar with Access 2000, but my latest project has me stumped. So, I defer to you...
6
by: rodchar | last post by:
Hey all, I'm trying to understand Master/Detail concepts in VB.NET. If I do a data adapter fill for both customer and orders from Northwind where should that dataset live? What client is...
17
by: tshad | last post by:
Many (if not most) have said that code-behind is best if working in teams - which does seem logical. How do you deal with the flow of the work? I have someone who is good at designing, but...
17
by: roN | last post by:
Hi, I'm creating a Website with divs and i do have some troubles, to make it looking the same way in Firefox and IE (tested with IE7). I checked it with the e3c validator and it says: " This...
6
by: JoeC | last post by:
I have a question about designing objects and programming. What is the best way to design objects? Create objects debug them and later if you need some new features just use inhereitance. Often...
0
by: | last post by:
I have a question about spawning and displaying subordinate list controls within a list control. I'm also interested in feedback about the design of my search application. Lots of code is at the...
19
by: neelsmail | last post by:
Hi, I have been working on C++ for some time now, and I think I have a flair for design (which just might be only my imagination over- stretched.. :) ). So, I tried to find a design...
8
by: indrawati.yahya | last post by:
In a recent job interview, the interviewer asked me how I'd design classes for the following problem: let's consider a hypothetical firewall, which filters network packets by either IP address,...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.