.Net Service - Random Code Execution (.Net Bug?)

[Celldss]

I have a .Net service (VB - 1.1 SP1) deployed in our environment that
is experiencing an odd issue. The service does a series of file &
registry based checks, logs to the event log and uploads the results to
an Oracle database. However, a few clients (a couple hundred or so out
of tens of thousands) are experiencing a rather odd issue.

The service is configure to fire it's checks every 48 hours (using a
timer), or whenever the service starts, whichever is sooner. Some
clients essentially "freak out" and execute the main block of code
several times per second for a period of time, resulting in hundreds of
thousands of records being added to the database in a 48 hour span of
time (instead of a couple). The event logs that are generated by the
service when this happens indicate that blocks of code are being
executed at random. Sometimes the same block is executed multiple
times. Here's what I've been able to ascertain:

- When the service encounters one of these "episodes", it lasts
anywhere from a couple hours to a couple days, but usually less than 48
hours, when it inexplicably stops and resumes normal functionality.

- Thousands of events are generated in the event log. This causes the
event log to wrap making it difficult for me investigate the source of
the problem. On the one machine where the event log had not wrapped,
there is nothing obvious either before the problem started or after it
stopped.

- Restarting the service does not seem to fix the issue. The problem
will persist until it stops of it's own accord.

- The issue is not tied to a specific build or make of machine.

- Based on the database records, a normal scan fires then terminates. A
few seconds elapse, and it enters this "loop".

- I could not reproduce the problem in test. The debugger ran normally.

I apologize, but I am not at liberty to post source code; I will do my
best to outline the flow of the program should any such questions
arise. Also, I am in the process of upgrading to .Net 2.0, but this
will take time.

Thank you very much. Any help is greatly appreciated.

[Chris Dunaway]

Celldss wrote:

I have a .Net service (VB - 1.1 SP1) deployed in our environment that
is experiencing an odd issue. The service does a series of file &
registry based checks, logs to the event log and uploads the results to
an Oracle database. However, a few clients (a couple hundred or so out
of tens of thousands) are experiencing a rather odd issue.

The service is configure to fire it's checks every 48 hours (using a
timer), or whenever the service starts, whichever is sooner. Some
clients essentially "freak out" and execute the main block of code
several times per second for a period of time, resulting in hundreds of
thousands of records being added to the database in a 48 hour span of
time (instead of a couple). The event logs that are generated by the
service when this happens indicate that blocks of code are being
executed at random. Sometimes the same block is executed multiple

Just a shot in the dark. Could there be something amiss with the
clocks on the affected machines? Is the timer being disabled when
entering the timer event? Can you tell if the timer is firing often
when the problem occurs? What method do you use for the timer? Does
it just tick periodically and you calculate when the 48 hours have
elapsed? Can you describe how you have implemented the timer?

One crude, but effective, debugging method would be to create a simple
logging mechanism that logs to a file. Periodically in the code have
it write out something like "Checkpoint A", "Checkpoint B", etc. so you
can follow the execution path. Have it output the values of pertinent
variables as well. You can then study the results and see if it helps
with the problem.

Good Luck