C# Threads

An application doesn't switch threads. It runs in one thread. It can spawn
others. It sounds like you're running the debugger, which may be jumping
around from one thread to another, but the reason that the files are not
being written has nothing to do with the application switching threads.

It also seems to me that you're having the parent application do too much of
the work involved in these individual threads, such as telling them when to
run, managing a timer for each one, etc.

We have several services which run in one service. They run at different
intervals, but they all share a few things in common. Each must be able to
be started, keep track of its own timing, and perform at task at intervals.
So I created a base class that has a timer and a timer interval. The base
abstract class has a Start, Stop, and Run method. The Start method calls the
Run method. The Stop method safely stops the class from running. The class
raises events which can inform a listener (the Service in which they run)
what it is about to do, doing, or has just finished doing. The class is
configurable as to interval and a number of other things, and sets its own
interval using an app.config file (although it could be configured in other
ways). In other words, the class minds its own business. The abstract method
is the Run method. It defines exactly what the derived class is supposed to
do, and must be explicitly implemented in each inherited class. The Run
method runs in a separate thread. It is not accessible, except by the Start
and Stop methods.

So, I can host these classes in a Service or an application, any kind of app
in other words, and the app can manage the classes as much as it needs to.
It can start them, and stop them. It can configure them when they are
stopped. It can respond to the events they raise. Everything is neatly
encapsulated. And I can re-use the base class in any similar type of
application (generally designed for use in services). I can add as many of
these to a single service as I need. And it is simple to manage.

Threading is a tricky business. The simpler you can make it, the better.

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
Ambiguity has a certain quality to it.

Offhand, it sounds like your Scheduler is doing too much of the work that
these threads of yours should be doing for themselves.
<Ra*******@hotmail.com> wrote in message
news:11**********************@g43g2000cwa.googlegr oups.com...

Hi guys,
I would like some advice on thread programming using C#.

I am writing an application that communicates with a panel over
ethernet, collects data and writes it to a file. The way the data is
collected is that we have different schedules (so one set of data is
collected say every second, another set of data might be collected
every 30 seconds, and so on).
The way I am approaching this is that I have created a class called
Scheduler, which consists of an ArrayList which has the individual
schedules. Each individual schedule has a timer object which would fire
after a specified amount of time (again this would be different for
each different schedule). When it fires, it would set a flag. In the
scheduler I have a function called Run, which goes through each
individual schedule item and see if the flag is set. If it is, it would
call a method in the individual schedule (called DoServicing()) and
clear the flag. The DoServicing() method calls the communication
function and then writes the flag.
Now I am creating the scheduler on a separate thread when the user
presses a Start button, and the scheduler just keeps calling Run()
forever, until the user presses a stop button. The problem I am running
into is that it seems like when the scheduler is in the DoServicing()
method (on its own thread), often times the application would switch
threads (back to the main UI thread). So whats happening is that the
file is not getting written. At other times, if there are 2
communication messages in DoServicing() its doing only 1 before
switching threads back to the main UI thread. Interestingly enough, if
I had no communication messages (just writing to file) everything works
perfectly according to the given schedule. As soon as I put the
communication messages in the DoServicing() method, it seems to switch
threads before going through the whole method.
In this sort of application, where the data must be collected and
written exactly at user specified intervals, obviously I need more
control on the thread on which the scheduler is running. What I would
like is that the scheduler should be continuously running according to
the given individual schedules (communicating to the panel and writing
the files). How do I achieve that?
Note: I have already tried lock around the entire body of the
DoServicing() method, as well as setting the thread priority of the
scheduler thread to highest. Neither of them worked.

Kevin,
Thanks for your input. The only problem that I have with your
suggestion is that
if I have a separate thread for each individual schedule, and I have
about 10 scehdules (since the user can freely set schedules) I am going
to end up with 10 threads + the main thread for the application! The
reason I put the scheduler in a separate thread and tried to do it by
polling is because then I would only have 2 threads and it would be
easier to manage (or so I thought!). Also, since the individual
schedules write to different files but use the same communication (same
ethernet ip/port) I would also have to synchronize the threads which I
would like to avoid if possible.

Since putting the original post, I have tried a few other things.
(1) Using the threadpool timed callbacks - problems: need 1 thread per
individual schedule, plus looking at the file log it didn't appear as
it
was doing the schedules at the right intervals (communication might be
slowing
it down as before)
(2) Doing away with threads altogether, and making the scheduler
subscribe to events that are generated. Works fine with no
communication, erratic results with communication.

It seems to me that I have no option but to put the communication in a
separate thread by itself. I am fine with that, as long as I can keep
the total # of threads to some finite manageable number, rather than
being at the mercy of the user in how many schedules he/she specifies.

Ultimately, the problem comes down to this: I have to collect data, and
I have to do it at an exact time. If this was a problem of just having
a long operation take place in the background, this would be no
problem. But since the whole point of the application is collecting
data at a specified interval, whatever thread the communication is
running on must be allowed to finish collecting the data and writing to
the file. Now since in a multi-threaded application, we really can't
control which thread is running, I am beginning to run out of ideas.

Any help would be greatly appreciated!

Kevin Spencer wrote:

An application doesn't switch threads. It runs in one thread. It can spawn
others. It sounds like you're running the debugger, which may be jumping
around from one thread to another, but the reason that the files are not
being written has nothing to do with the application switching threads.

It also seems to me that you're having the parent application do too much of
the work involved in these individual threads, such as telling them when to
run, managing a timer for each one, etc.

We have several services which run in one service. They run at different
intervals, but they all share a few things in common. Each must be able to
be started, keep track of its own timing, and perform at task at intervals.
So I created a base class that has a timer and a timer interval. The base
abstract class has a Start, Stop, and Run method. The Start method calls the
Run method. The Stop method safely stops the class from running. The class
raises events which can inform a listener (the Service in which they run)
what it is about to do, doing, or has just finished doing. The class is
configurable as to interval and a number of other things, and sets its own
interval using an app.config file (although it could be configured in other
ways). In other words, the class minds its own business. The abstract method
is the Run method. It defines exactly what the derived class is supposed to
do, and must be explicitly implemented in each inherited class. The Run
method runs in a separate thread. It is not accessible, except by the Start
and Stop methods.

So, I can host these classes in a Service or an application, any kind of app
in other words, and the app can manage the classes as much as it needs to.
It can start them, and stop them. It can configure them when they are
stopped. It can respond to the events they raise. Everything is neatly
encapsulated. And I can re-use the base class in any similar type of
application (generally designed for use in services). I can add as many of
these to a single service as I need. And it is simple to manage.

Threading is a tricky business. The simpler you can make it, the better.

--
HTH,

Kevin Spencer
Microsoft MVP
.Net Developer
Ambiguity has a certain quality to it.

Offhand, it sounds like your Scheduler is doing too much of the work that
these threads of yours should be doing for themselves.
<Ra*******@hotmail.com> wrote in message
news:11**********************@g43g2000cwa.googlegr oups.com...

Hi guys,
I would like some advice on thread programming using C#.

I am writing an application that communicates with a panel over
ethernet, collects data and writes it to a file. The way the data is
collected is that we have different schedules (so one set of data is
collected say every second, another set of data might be collected
every 30 seconds, and so on).
The way I am approaching this is that I have created a class called
Scheduler, which consists of an ArrayList which has the individual
schedules. Each individual schedule has a timer object which would fire
after a specified amount of time (again this would be different for
each different schedule). When it fires, it would set a flag. In the
scheduler I have a function called Run, which goes through each
individual schedule item and see if the flag is set. If it is, it would
call a method in the individual schedule (called DoServicing()) and
clear the flag. The DoServicing() method calls the communication
function and then writes the flag.
Now I am creating the scheduler on a separate thread when the user
presses a Start button, and the scheduler just keeps calling Run()
forever, until the user presses a stop button. The problem I am running
into is that it seems like when the scheduler is in the DoServicing()
method (on its own thread), often times the application would switch
threads (back to the main UI thread). So whats happening is that the
file is not getting written. At other times, if there are 2
communication messages in DoServicing() its doing only 1 before
switching threads back to the main UI thread. Interestingly enough, if
I had no communication messages (just writing to file) everything works
perfectly according to the given schedule. As soon as I put the
communication messages in the DoServicing() method, it seems to switch
threads before going through the whole method.
In this sort of application, where the data must be collected and
written exactly at user specified intervals, obviously I need more
control on the thread on which the scheduler is running. What I would
like is that the scheduler should be continuously running according to
the given individual schedules (communicating to the panel and writing
the files). How do I achieve that?
Note: I have already tried lock around the entire body of the
DoServicing() method, as well as setting the thread priority of the
scheduler thread to highest. Neither of them worked.

RahimAsif <Ra*******@gmail.com> wrote:

Thanks for your input. The only problem that I have with your
suggestion is that
if I have a separate thread for each individual schedule, and I have
about 10 scehdules (since the user can freely set schedules) I am going
to end up with 10 threads + the main thread for the application!
That's not a huge number.
The reason I put the scheduler in a separate thread and tried to do it by
polling is because then I would only have 2 threads and it would be
easier to manage (or so I thought!).
Sometimes putting all the work onto one thread will be easier, but
often it will be harder. It depends on the exact situation.
Also, since the individual
schedules write to different files but use the same communication (same
ethernet ip/port) I would also have to synchronize the threads which I
would like to avoid if possible.
Why? If you're worried about performance, you should really write some
code which is reliable first, and check the performance.

You *could* have one thread just for the communication, reading froma
queue which all the other threads write to.

<snip>
Ultimately, the problem comes down to this: I have to collect data, and
I have to do it at an exact time.

Depending on how exact you mean, this may well be infeasible. What
would you expect to do if multiple schedules said you had to collect
data at exactly the same time?

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

Hi Jon,
Thanks for the reply. 10 schedules might not be a huge number if it
were limited to 10. The limit really is on the user, so that number
could easily be really large. The other thing is that there will be 2
versions of this application, one that runs on PC and one that would
run on a CE device. On a CE device such a high number of threads would
really bog the system down.

As for your question:
"What would you expect to do if multiple schedules said you had to
collect
data at exactly the same time?"

In that case I would expect one schedule to collect the data and write
to file, then the next schedule to collect data and save to the file
and so on. You are right, depending on the # of previous schedules that
were running before, this could make a schedule go slightly off, but
that's fine. In other words, they should queue up and as soon as the
schedule is in front of the queue it should execute. But no schedule
should be missed. That's why my original approach was doing a separate
thread on which the scheduler runs and polls all the individual
schedules and services them one by one. But for some reason, with that
approach when communication was involved, some schedules were getting
missed. Here is the code:

public void RunForever()
{
while(this.bDone == false)
{
this.Run();
Thread.Sleep(50);
}
}

public void Run()
{
for(int i=0; i<schedules.Count; i++)
{
IndividualSchedule s = (IndividualSchedule) schedules[i];
if(s.bNeedsServicing) s.DoServicing();
}
}

Basically the main thread spawned a thread for the Scheduler which
started on the RunForever() function.
I had put the 50 ms sleep so that the Main thread remains responsive.
The DoServicing() method of the
individual schedule is the one that calls communication and file
writing functions. I was calling DoServincing()
from the scheduler instead of the individual schedules so that no two
schedules try to access communication at the
same time. Like I said before, everything works great without
communication (if I just try to write to files) but
putting in communication throws everything off: schedules are missed,
the scheduler thread hangs etc.


Jon wrote:

RahimAsif <Ra*******@gmail.com> wrote:

Thanks for your input. The only problem that I have with your
suggestion is that
if I have a separate thread for each individual schedule, and I have
about 10 scehdules (since the user can freely set schedules) I am going
to end up with 10 threads + the main thread for the application!

That's not a huge number.

The reason I put the scheduler in a separate thread and tried to do it by
polling is because then I would only have 2 threads and it would be
easier to manage (or so I thought!).

Sometimes putting all the work onto one thread will be easier, but
often it will be harder. It depends on the exact situation.

Also, since the individual
schedules write to different files but use the same communication (same
ethernet ip/port) I would also have to synchronize the threads which I
would like to avoid if possible.

Why? If you're worried about performance, you should really write some
code which is reliable first, and check the performance.

You *could* have one thread just for the communication, reading froma
queue which all the other threads write to.

<snip>

Ultimately, the problem comes down to this: I have to collect data, and
I have to do it at an exact time.

Depending on how exact you mean, this may well be infeasible. What
would you expect to do if multiple schedules said you had to collect
data at exactly the same time?

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

I'd like to suggest a radical change in the way the scheduler works.
Have you looked into using priority queues? The scheduler is simple
when using priority queues because you only need one thread. No timers
are required. A priority queue allows you store items (schedules) with
a certain priority (the next run time) and then the Dequeue operation
pulls out the item with the highest priority. For example:

public class Scheduler
{
public static void Run(ICollection schedules)
{
// Sadly, the framework does not have this builtin. You'll have to
// find or author your own.
PriorityQueue queue = new PriorityQueue();

// Load the queue using the next run times as the priority.
foreach (Schedule s in schedules)
{
s.NextRunTime = DateTime.Now + s.Interval;
queue.Enqueue(s.NextRunTime, s);
}

// Keep pulling out the schedule with the highest priority,
// execute it, and then add it back to the queue.
while (true)
{
Schedule s = (Schedule)queue.Dequeue();
TimeSpan span = s.NextRunTime - DateTime.Now;
Thread.Sleep(span.TotalMilliseconds);
s.Execute();
s.NextRunTime = DateTime.Now + s.Interval;
queue.Enqueue(s.NextRunTime, s);
}
}
}

This brief example can be modified in many ways to facilitate your own
requirements. You could put the call to Execute on the ThreadPool if
desired. Some changes may need to be made to unsure that the next run
time doesn't drift as well. Those are all easy changes using this
approach. The most difficult part is coding your own PriorityQueue.
Of course, there are plenty on the net to be found as well.

Brian
RahimAsif wrote:

Hi Jon,
Thanks for the reply. 10 schedules might not be a huge number if it
were limited to 10. The limit really is on the user, so that number
could easily be really large. The other thing is that there will be 2
versions of this application, one that runs on PC and one that would
run on a CE device. On a CE device such a high number of threads would
really bog the system down.

As for your question:
"What would you expect to do if multiple schedules said you had to
collect
data at exactly the same time?"

In that case I would expect one schedule to collect the data and write
to file, then the next schedule to collect data and save to the file
and so on. You are right, depending on the # of previous schedules that
were running before, this could make a schedule go slightly off, but
that's fine. In other words, they should queue up and as soon as the
schedule is in front of the queue it should execute. But no schedule
should be missed. That's why my original approach was doing a separate
thread on which the scheduler runs and polls all the individual
schedules and services them one by one. But for some reason, with that
approach when communication was involved, some schedules were getting
missed. Here is the code:

public void RunForever()
{
while(this.bDone == false)
{
this.Run();
Thread.Sleep(50);
}
}

public void Run()
{
for(int i=0; i<schedules.Count; i++)
{
IndividualSchedule s = (IndividualSchedule) schedules[i];
if(s.bNeedsServicing) s.DoServicing();
}
}

Basically the main thread spawned a thread for the Scheduler which
started on the RunForever() function.
I had put the 50 ms sleep so that the Main thread remains responsive.
The DoServicing() method of the
individual schedule is the one that calls communication and file
writing functions. I was calling DoServincing()
from the scheduler instead of the individual schedules so that no two
schedules try to access communication at the
same time. Like I said before, everything works great without
communication (if I just try to write to files) but
putting in communication throws everything off: schedules are missed,
the scheduler thread hangs etc.

Hi Rahim,

if I have a separate thread for each individual schedule, and I have
about 10 scehdules (since the user can freely set schedules) I am going
to end up with 10 threads + the main thread for the application!
There are limits to everything in the world of computers. An application is
limited by available processor, memory, file storage, etc. So, you may need
to examine the business requirement that the user may add as many schedules
as desired.

Still, you can create quite a few threads simultaneously, and 10, as Jon
pointed out, is not many. On the other hand, the more threads, the slower
the overall process. You should set some sort of limit on this.
Also, since the individual
schedules write to different files but use the same communication (same
ethernet ip/port) I would also have to synchronize the threads which I
would like to avoid if possible.
Synchronizing threads doesn't have to be too difficult. In the design I
explained to you, you may recall that each object is created and resides in
the main process, as a member of that class. Each object raises events at
different times during the Run() method, to indicate that certain things are
about to happen, happening, or have finished happening. Each object raises
an event when an Exception occurs.

The (in this case) service class that hosts these individual classes (as
private fields) acts as a manager for them. It may contain event handlers
for each of the objects it hosts, and may contain fields or properties that
are accessible to all the member classes in it. It may also call Start() and
Stop() on any of these objects at any time.

So, for example, I have an inheited class which performs a task of
tranferring data from one point to another, and creating and uploading that
same data as a file to an FTP server. It takes a variable length of time for
this class to perform its task. It also writes to a number of XML log files
to record the results of each transaction. Another inherited class performs
Reporting. The Run() method of the second class must read from the same
files that the first class writes to. So, it is important that the second
class does not attempt to read from a file while the first class is writing
to it. And each class runs on a configurable and different schedule.

So, the service manager has a field called "ClientRunning," which is set to
true when the first class starts its' Run() method (as a rsponse to the
"RunStart" event of the first class), and set to false when the first class
raises the "RunStop" event. So, when the second class's Run() method starts,
as a result of its timer, it first checks the service manager's
"ClientRunning" field. If it is true, the Run() thread sleeps for 1 second,
and checks again. When it reads false, the Run() thread continues on its
merry way.

Note that the only class which sets the "ClientRunning" member is the host
service. The hosted classes may only read from it.

In other words, I have a very simple synchronization going on between 2
class instances that don't even see each other.

--
HTH,

Kevin Spencer
Microsoft MVP
..Net Developer
Ambiguity has a certain quality to it.

"RahimAsif" <Ra*******@gmail.com> wrote in message
news:11**********************@z14g2000cwz.googlegr oups.com... Kevin,
Thanks for your input. The only problem that I have with your
suggestion is that
if I have a separate thread for each individual schedule, and I have
about 10 scehdules (since the user can freely set schedules) I am going
to end up with 10 threads + the main thread for the application! The
reason I put the scheduler in a separate thread and tried to do it by
polling is because then I would only have 2 threads and it would be
easier to manage (or so I thought!). Also, since the individual
schedules write to different files but use the same communication (same
ethernet ip/port) I would also have to synchronize the threads which I
would like to avoid if possible.

Since putting the original post, I have tried a few other things.
(1) Using the threadpool timed callbacks - problems: need 1 thread per
individual schedule, plus looking at the file log it didn't appear as
it
was doing the schedules at the right intervals (communication might be
slowing
it down as before)
(2) Doing away with threads altogether, and making the scheduler
subscribe to events that are generated. Works fine with no
communication, erratic results with communication.

It seems to me that I have no option but to put the communication in a
separate thread by itself. I am fine with that, as long as I can keep
the total # of threads to some finite manageable number, rather than
being at the mercy of the user in how many schedules he/she specifies.

Ultimately, the problem comes down to this: I have to collect data, and
I have to do it at an exact time. If this was a problem of just having
a long operation take place in the background, this would be no
problem. But since the whole point of the application is collecting
data at a specified interval, whatever thread the communication is
running on must be allowed to finish collecting the data and writing to
the file. Now since in a multi-threaded application, we really can't
control which thread is running, I am beginning to run out of ideas.

Any help would be greatly appreciated!

Kevin Spencer wrote:

An application doesn't switch threads. It runs in one thread. It can
spawn
others. It sounds like you're running the debugger, which may be jumping
around from one thread to another, but the reason that the files are not
being written has nothing to do with the application switching threads.

It also seems to me that you're having the parent application do too much
of
the work involved in these individual threads, such as telling them when
to
run, managing a timer for each one, etc.

We have several services which run in one service. They run at different
intervals, but they all share a few things in common. Each must be able
to
be started, keep track of its own timing, and perform at task at
intervals.
So I created a base class that has a timer and a timer interval. The base
abstract class has a Start, Stop, and Run method. The Start method calls
the
Run method. The Stop method safely stops the class from running. The
class
raises events which can inform a listener (the Service in which they run)
what it is about to do, doing, or has just finished doing. The class is
configurable as to interval and a number of other things, and sets its
own
interval using an app.config file (although it could be configured in
other
ways). In other words, the class minds its own business. The abstract
method
is the Run method. It defines exactly what the derived class is supposed
to
do, and must be explicitly implemented in each inherited class. The Run
method runs in a separate thread. It is not accessible, except by the
Start
and Stop methods.

So, I can host these classes in a Service or an application, any kind of
app
in other words, and the app can manage the classes as much as it needs
to.
It can start them, and stop them. It can configure them when they are
stopped. It can respond to the events they raise. Everything is neatly
encapsulated. And I can re-use the base class in any similar type of
application (generally designed for use in services). I can add as many
of
these to a single service as I need. And it is simple to manage.

Threading is a tricky business. The simpler you can make it, the better.

--
HTH,

Kevin Spencer
Microsoft MVP
.Net Developer
Ambiguity has a certain quality to it.

Offhand, it sounds like your Scheduler is doing too much of the work that
these threads of yours should be doing for themselves.
<Ra*******@hotmail.com> wrote in message
news:11**********************@g43g2000cwa.googlegr oups.com...

> Hi guys,
> I would like some advice on thread programming using C#.
>
> I am writing an application that communicates with a panel over
> ethernet, collects data and writes it to a file. The way the data is
> collected is that we have different schedules (so one set of data is
> collected say every second, another set of data might be collected
> every 30 seconds, and so on).
>
>
> The way I am approaching this is that I have created a class called
> Scheduler, which consists of an ArrayList which has the individual
> schedules. Each individual schedule has a timer object which would fire
> after a specified amount of time (again this would be different for
> each different schedule). When it fires, it would set a flag. In the
> scheduler I have a function called Run, which goes through each
> individual schedule item and see if the flag is set. If it is, it would
> call a method in the individual schedule (called DoServicing()) and
> clear the flag. The DoServicing() method calls the communication
> function and then writes the flag.
>
>
> Now I am creating the scheduler on a separate thread when the user
> presses a Start button, and the scheduler just keeps calling Run()
> forever, until the user presses a stop button. The problem I am running
> into is that it seems like when the scheduler is in the DoServicing()
> method (on its own thread), often times the application would switch
> threads (back to the main UI thread). So whats happening is that the
> file is not getting written. At other times, if there are 2
> communication messages in DoServicing() its doing only 1 before
> switching threads back to the main UI thread. Interestingly enough, if
> I had no communication messages (just writing to file) everything works
> perfectly according to the given schedule. As soon as I put the
> communication messages in the DoServicing() method, it seems to switch
> threads before going through the whole method.
>
>
> In this sort of application, where the data must be collected and
> written exactly at user specified intervals, obviously I need more
> control on the thread on which the scheduler is running. What I would
> like is that the scheduler should be continuously running according to
> the given individual schedules (communicating to the panel and writing
> the files). How do I achieve that?
>
>
> Note: I have already tried lock around the entire body of the
> DoServicing() method, as well as setting the thread priority of the
> scheduler thread to highest. Neither of them worked.
>