Windows Service written in c# Hangs

[mohitkumar]

Hello All,

We have 3 windows services written in C#. We use them to process files
from a folder into database and further to process the data into various SQL
Tables.

All the services hang atleast once in a week stopping the entire process. The
files are queued up in the folders waiting to be processed. The services in
the Service Control Manager seems to be running but they do not process any
files. But when the processes are restarted, they run just fine until any one
hangs again. The following error is written to Event Viewer by Service Control Manager:

The [Service Name] service terminated unexpectedly. It has done this XX
time(s). The following corrective action will be taken in 0 milliseconds: Restart Service.

Any help would be appreciated. (find code below)

using System;
using System.IO ;
using System.Threading ;
using System.Xml ;
 
namespace XXX.ABCD.Dls.Queue
{
    /// <summary>
    /// A comparison is done between the two Data objects, by individual
entities
    /// This means only entities that have changed will be updated
    /// this causes less SQL interaction and speeds the whole process up
    /// </summary>
    public class QueueProcess
    {
 
        // Two XML documents to hold the queue and previous data
        private XmlDocument xmlQueue = new XmlDocument();
        private XmlDocument xmlPrevious = new XmlDocument();
 
        // A data queue object to process information from the Queue
        private DataQueue oDataQueue = new DataQueue() ;
        // A data previous object to get the last XML processed from the database
        private DataPrevious oDataPrevious = new DataPrevious() ;
 
        public const int status_begin = 0 ;
        public const int status_end = 999 ;
 
        private bool stopped = false ;
 
        // Event argument instances
        private QueueProcess_Status_EventArgs eQueueProcess_Status = new 
QueueProcess_Status_EventArgs() ;
 
        #region Public
 
        // Constructor
        public QueueProcess()
        {
        }
 
        // Make the link between the delegate function and the place in code where
it is 
                               called
        public event QueueProcess_Status QueueProcess_Status_Event ;
 
        /// <summary>
        /// Begin polling the  folders in the path_incoming for ZIP files
        /// 
        /// </summary>
        public void Start()
        {
            Status_Event(status_begin, "Queue Thread Begin") ;
            stopped = false ;
 
            while (true)
            {
                Thread.Sleep(100) ;
                Start_QueueProcess() ;
                if (stopped) break ;    
            }
            Status_Event(status_end, "Queue Thread End") ;
 
        }
 
        public void Stop()
        {
            stopped = true ;
        }
 
        #endregion
 
        #region Private functions
 
        /// <summary>
        /// Method to Start processing QueueProcess
        /// </summary>
        private void Start_QueueProcess()
        {
            // Process the Queue
            try
            {
                oDataQueue.GetNext() ;
                // if it is valid process it
                if (oDataQueue.QueueID != 0)
                {
                    try
                    {
 
                        #region Load the XML from the Queue and create a ComputerNew object
                        // Get the info from the XML
                        try
                        {
                            xmlQueue.LoadXml(oDataQueue.INVData) ;
                        }
                        catch
                        {
                            xmlQueue.LoadXml("<XXXX/>") ;
                        }
                        ComputerNew oComputerNew = new ComputerNew(oDataQueue.AccountID , ref
xmlQueue);
                        #endregion
 
                        // If AccountID == 0. This means an error has occured loading computer
new
                        if (oComputerNew.AccountID != 0)
                        {
                            #region Get the XML from the previous data in the DB and Create a
ComputerPrevious object
                            oDataPrevious.Get(oComputerNew.AccountID , oComputerNew.NetBIOSName);
 
                            // Get the info from the XML
                            try
                            {
                                xmlPrevious.LoadXml(oDataPrevious.INVData) ;
                            }
                            catch
                            {
                                xmlPrevious.LoadXml("<XXXX/>") ;
                            }
 
                            ComputerNew oComputerPrevious = null ;
                            try
                            {
                                oComputerPrevious = new ComputerNew(0 , ref xmlPrevious) ;
                            }
                            catch
                            {
                                Status_Event(107,"Error : XML - Could not create computer previous
object") ;
                            }
                            #endregion
 
                            #region Compare the New and Previous Prepared XML and if different
build the collection of data
                            // If different then call prepare data on entity new to populate
collection
                            if (oComputerNew.customNew.PreparedXml != oComputerPrevious.customNew.
PreparedXml )
                                oComputerNew.customNew.PrepareData(ref xmlQueue) ; // This sets
updaterequired in entity
 
                            if (oComputerNew.summaryNew.PreparedXml != oComputerPrevious.
summaryNew.PreparedXml )
                                oComputerNew.summaryNew.PrepareData(ref xmlQueue) ; // This sets
updaterequired in entity
 
                            if (oComputerNew.displayNew.PreparedXml != oComputerPrevious.
displayNew.PreparedXml )
                                oComputerNew.displayNew.PrepareData(ref xmlQueue) ; // This sets
updaterequired in entity
 
                            if (oComputerNew.driveNew.PreparedXml != oComputerPrevious.driveNew.
PreparedXml )
                                oComputerNew.driveNew.PrepareData(ref xmlQueue) ; // This sets
updaterequired in entity
 
                            if (oComputerNew.networkNew.PreparedXml != oComputerPrevious.
networkNew.PreparedXml )
                                oComputerNew.networkNew.PrepareData(ref xmlQueue) ; // This sets
updaterequired in entity
 
                            if (oComputerNew.printerNew.PreparedXml != oComputerPrevious.
printerNew.PreparedXml )
                                oComputerNew.printerNew.PrepareData(ref xmlQueue) ; // This sets
updaterequired in entity
 
                            if (oComputerNew.shortcutNew.PreparedXml != oComputerPrevious.
shortcutNew.PreparedXml )
                                oComputerNew.shortcutNew.PrepareData(ref xmlQueue) ; // This sets
updaterequired in entity
 
                            if (oComputerNew.soeNew.PreparedXml != oComputerPrevious.soeNew.
PreparedXml )
                                oComputerNew.soeNew.PrepareData(ref xmlQueue) ; // This sets
updaterequired in entity
 
                            if (oComputerNew.softwareNew.PreparedXml != oComputerPrevious.
softwareNew.PreparedXml )
                                oComputerNew.softwareNew.PrepareData(ref xmlQueue) ; // This sets
updaterequired in entity
 
                            if (oComputerNew.usbNew.PreparedXml != oComputerPrevious.usbNew.
PreparedXml )
                                oComputerNew.usbNew.PrepareData(ref xmlQueue) ; // This sets
updaterequired in entity
 
                            #endregion
 
                            ComputerCurrent oComputerCurrent = new ComputerCurrent(oComputerNew);
 
                            if (oComputerCurrent.Compare())
                            {
                                try
                                {
                                    //Now update the Previous Data from the data queue object
                                    oDataPrevious.INVData = oDataQueue.INVData ;
                                    //Set the ID of the entity as the ComputerID
                                    oDataPrevious.ID = oComputerCurrent.ID;
                                    oDataPrevious.Insert();
                                }
                                catch
                                {
                                    Status_Event(108,"Error : Could not update previous data") ;
                                }
 
                                try
                                {
                                    oDataQueue.Delete();
                                }
                                catch
                                {
                                    Status_Event(101,"Error : Could not Delete DataQueue item") ;
                                }
 
                            }
                            else
                            {
                                try
                                {
                                    oDataQueue.Failed();
                                    Status_Event(102,"Error : Failed to process queue entry") ;
                                }
                                catch
                                {
                                    Status_Event(103,"Error : Could not Mark DataQueue Failed") ;
                                }
                            }
 
                        }
                        else // error creating computernew. No exception AccountID set to 0
                        {
                            try
                            {
                                oDataQueue.Failed();
                                Status_Event(105,"Error : XML - Could not create computer new object")
;
                            }
                            catch
                            {
                                Status_Event(106,"Error : Could not Mark DataQueue Failed") ;
                            }
                        }
 
                    }
                    catch
                    {
                        Status_Event(104,"Error : Could not create Computer New Object") ;
                    }
 
                }
 
            }
            catch (Exception ex)
            {
                Status_Event(100,"Error : Could not get next item in DataQueue") ;
                Status_Event(100,"Exception : " + ex.Message ) ;
            }
        }
 
        private void Status_Event(int code , string message)
        {
            int codeoffset = 2000 ;
            code += codeoffset ;
            eQueueProcess_Status.StatusCode = code ;
            eQueueProcess_Status.StatusMessage = message ;
            QueueProcess_Status_Event(this,eQueueProcess_Status) ;
        }
 
        #endregion
 
        #region Properties
 
        public bool Stopped
        {
            get
            {
                return stopped ;
            }
        }
 
        #endregion
 
        #region Event Classes and Delegates
 
        #region Status Event Class
 
        public delegate void QueueProcess_Status( object sender,
QueueProcess_Status_EventArgs e) ;
 
        public class QueueProcess_Status_EventArgs : EventArgs
        {
            #region Private Variables
 
            private int statuscode ;
            private string statusmessage ;
 
            #endregion
 
            #region Public Properties
 
            public int StatusCode 
            {
                get { return statuscode ; }
                set { statuscode = value ; }
            }
 
            public string StatusMessage 
            {
                get { return statusmessage ; }
                set { statusmessage = value ; }
            }
 
            #endregion
        }
 
        #endregion
 
        #endregion
 
    }
}
 

[PRR]

Firstly is this your service start function?

public void Start()
         {
             Status_Event(status_begin, "Queue Thread Begin") ;
             stopped = false ;
 
             while (true)
             {
                 Thread.Sleep(100) ;
                 Start_QueueProcess() ;
                 if (stopped) break ;    
             }
             ....
 

I don't see any timers being used... You can use system.timers.timer or threading timer for services...
You mentioned that you, "process files....form folder.. into database...further to process the data . " Can you explain more here... perhaps the exact procedure?
Problems that i can guess:
* Ownership of file can be a problem....for e.g. the service is trying access a file that is locked or being used by another process...
* You are also connecting to database? connection times out? DB not available...
* You should also check for Deadlocks....

This code is fairly wrong.... (especially from a service point of view )

 while (true)
             {
                 Thread.Sleep(100) ;
                 Start_QueueProcess() ;
                 if (stopped) break ;    
             }
 

Possible solutions:
* If you intent to call Start_QueueProcess(), periodically you need to use system.timers.timer, in case of service...
* Also you must consider using threads or better asynchronous programming so that you windows service doesn't hang....
* Make sure your service start returns immediately .... Create a separate thread if needed to execute any function....
* Lock your functions so that only one instance of function is executing....
* Periodically do ...

GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();
 

* Write exception to a log file/ Event log

[mohitkumar]

Hi Deepblue,
thanks for your reply and ideas..

Firstly is this your service start function?
This function starts our process..

Process:
We process our inventory format files that come into a designated folder(only used by this process). First process reads the file (xml format) and stores the content to a designated table.

Then second service picks up the xml string, reads data from nodes and further stores this data to several other tables.

how do i replace Thread.sleep() with System Timer ? does interval() method fit here? how about AutoReset?

thanks in advance for all your ideas..

[PRR]

Could you post OnStart(), OnStop() code? I would suggest you replace

  while (true)
              {
                  Thread.Sleep(100) ;
                  Start_QueueProcess() ;
                  if (stopped) break ;    
              }
 

with
System.Timers.Timer

"We process our inventory format files that come into a designated folder(only used by this process)."
As i understand periodically files are created on a folder "A"? and your service scans this folder?
"First process reads the file (xml format) "
and connects to DB...? are you sure the DB is on ?

[mohitkumar]

protected override void OnStart(string[] args)
{
#region This code goes into service start
oServiceLogger.Insert(1000,"DLS Start") ;
XXX.ABCD.Dal.VersionDetails  oVersion = new XXX.ABCD.Dal.VersionDetails() ;
oServiceLogger.Insert(1001,"DAL Version : " + oVersion.Get) ;
#region QueueThread
oQueueProcess = new QueueProcess() ;
// Add event handler
oQueueProcess.QueueProcess_Status_Event +=new XXX.ABCD.Dls.Queue.QueueProcess.QueueProcess_Status(oQueueProcess_QueueProcess_Status_Event);
// Create and execute the thread
Thread QueueThread = new Thread(new ThreadStart(oQueueProcess.Start )); 
QueueThread.Start() ; //Thread start
QueueThread = null ;
#endregion
#endregion
}
 
protected override void OnStop()
{
    #region This code goes into service stop
    Stop_Threads() ;
    oServiceLogger.Insert(1999,"DLS Stop") ;
    #endregion
}
 
private void Stop_Threads()
{
    try
    {
        if (!oQueueProcess.Stopped) oQueueProcess.Stop() ; 
    }
    catch
    {
    }
}
 
public void Stop()
{
    stopped = true ;
}
 
 

Yes, the service scans this folder for any incoming inventory format files continuosly. ( sleeps for 100 milliseconds, this idea may be wrong).

Yes, the DB is always on. But as you suggested, deadlocks are potential problem (should I use SQL Trace for checking deadlocks?) hw do I know there is a deadlock?

Thanks in advance for your ideas..

Regards,
Mohith

[PRR]

Briefly i will explain certain things which may be useful..
1. System.Timers.Timer:

//Declare the timer 
System.Timers.Timer myTimer = new System.Timers.Timer();
 
// initialize in service constructor...  
myTimer.Elapsed += new ElapsedEventHandler(OnTimedEvent);
            myTimer.Interval = 5000;
            myTimer.Enabled = true;
 

As the code suggest declare a system.timers.timer and initialize it ...

static bool lockTimer = true;
 
private  void OnTimedEvent(object source, ElapsedEventArgs e)
        {
            if (true)
            {
                lockTimer = false;
                CallFun2();
 
            }
        }
 
public void CallFun2()
        {
            try
            {
                //Do processing
                Console.WriteLine("Calling...");
 
            }
            catch (Exception ex)
            {
                //log exception
            }
            finally 
            {
                lockTimer = true;
                //Unlock timer function
            }
        }
 

This way your function will be executed asynchronously and only once instance will be called ... You can also use locking object( static) ...
CallFun2 will be your function that you call after a set interval...

[PRR]

Any tasks that require short processing and can be independently executed can be done by using ThreadPool, such as writing to logs etc...

ThreadPool.QueueUserWorkItem(new WaitCallback(func1));
 
static void func1(Object o)
        {            
            Console.WriteLine("Thread Pool");
        }
 

[PRR]

Last but most important what ever you do, in your service do it as series of asynchronous tasks...Look into Asynchronous Programming Overview

[mohitkumar]

Hats off to you DeepBlue..

Thanks a lot for your time and suggestions...you Rock!!


Regards,
Mohith

[PRR]

Welcome... do continue to post your queries on Bytes