471,353 Members | 1,781 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,353 software developers and data experts.

Update Thread

Hi there,

I'm currently writing a program in which I use the FileMonitor to monitor a
folder in which I store downloaded images. I know that I can't add much
coding in the filemonitor's event in risk of losing some new entries, so
I've deceided to create an update thread. This thread is created when the
program's start and should (for various reason) run not in sync with the
Filemonitor.

The Filemonitor event creates an entry in a database table (lets call it
TEMP) in which it stores the location and name of a new image. The update
thread scans this table and if it finds any entries, is does a serie of
operations (hash calculation, check for multiple entries etc). The thread
then creates a new entry in table [QUEUE] in which it stores the image
title, location, hashvalue etc.Then it gets the next entry in table TEMP. If
there aren't any records left, it goes to sleep. When waking up it scans
table TEMP again.

Here is my problem. When terminating my program, the update thread should
stop, but only after it finishes processing the current image it's working
on. It should not create any corrupted/broken records. How can I prevent
this from happening?

// Main Program
private void FileMonitor_Created(Object sender, FileSystemEventArgs e)
{
// Creates a new record in table TEMP
CreateNewTempEntry(e.FullPath);
}

// Update Thread
private void ScanTempTable()
{
OleDbCommand myCmd = new OleDBCommand("SELECT * FROM TEMP");
myCmd.Connection = ...;
OleDbDataReader myReader = myCmd.ExecuteReader();
string myFile = "", myHash = "";
bool bFound = false;
while(myReader.Read())
{
myFile = myReader.GetString(0);
myHash = CalcHash(myFile);
bFound = Scan4MultipleImages(myHash)

-----------------------------------------------------------
// Thread isn't allow to stop here
WriteNewImageRecord(myFile, myHash, bFound);
// Thread can stop now
------------------------------------------------------------
}
// Sleep for a while
}

thnx in advance

Peter
Jun 27 '08 #1
5 1625
On May 2, 10:14 am, "P.J.M. Beker" <pjmbe...@orange.nlwrote:

<snip>
Here is my problem. When terminating my program, the update thread should
stop, but only after it finishes processing the current image it's working
on. It should not create any corrupted/broken records. How can I prevent
this from happening?
Make the thread a foreground thread, which will mean the program keeps
running while the thread is alive.

Then add a way (a flag, for instance) of telling the thread to quit
after it's finished the current record. The "sleep for a while" part
should probably be something like Monitor.Wait so you can pulse a
monitor to make it respond quickly if it's idle when you want to quit.

Jon
Jun 27 '08 #2
On May 2, 5:14*am, "P.J.M. Beker" <pjmbe...@orange.nlwrote:
Hi there,

I'm currently writing a program in which I use the FileMonitor to monitor a
folder in which I store downloaded images. I know that I can't add much
coding in the filemonitor's event in risk of losing some new entries, so
I've deceided to create an update thread. This thread is created when the
program's start and should (for various reason) run not in sync with the
Filemonitor.
Pardon my ignorance , but what is FileMonitor?
AFAIk the only class in the framework that do that is
FileSystemWatcher

>
Here is my problem. When terminating my program, the update thread should
stop, but only after it finishes processing the current image it's working
on. It should not create any corrupted/broken records. How can I prevent
this from happening?
You could use a similar mechanisn than to indicate what file to
process. Using a flag in a table (of course you have to remove the
flag before ending)
Jun 27 '08 #3
I had the same problem about a year ago. The solution I came up with is the
abstract class below. Yes, this is a C# group and the class is in VB, but
the concept is the same. It uses a Linked List instead of a queue because
the linked list actually provides both better performance and memory usage
when the inputs are coming faster than they can be processed.

Derive a class from this and provide the "private StatusFlag ProcessValue(T
value)". Note the protected bool isBackground property that default to
"false" so all the items added to the queue will be processed before the
thread ends.

Mike.

======================
Option Explicit On
Option Strict On
Option Compare Text

Imports System.Collections.ObjectModel
Imports System.Threading

Public MustInherit Class ProcessingQueue(Of T)

<Flags()_
Public Enum StatusFlag
GetNextValue = 0
ExitThread = 1
RequeueValue = 2
End Enum

Private Queue As New QuickQueue(Of T)
Public MustOverride Function ProcessValue(ByVal value As T) As StatusFlag

' Thread control
Protected isBackground As Boolean = False
Protected Name As String = ""
Private th As New Thread(AddressOf ThreadProc)

' Statistical information
Private mTotalAdded As Integer = 0
Private syncTotalAdded As New SynchronizationContext

' NOTHING is legal - it simply forces the workerthreadproc to be started
Public Sub Add()
add(Nothing)
End Sub
Public Sub Add(ByVal value As T)
If value IsNot Nothing Then Queue.Enqueue(value)
SyncLock syncTotalAdded
If value IsNot Nothing Then mTotalAdded += 1

If Count = 0 Then Exit Sub
If th.IsAlive Then Exit Sub
th = New Thread(AddressOf ThreadProc)
th.IsBackground = isBackground
If Name <"" Then th.Name = Name
th.Start()
End SyncLock
End Sub

Protected Sub New()
End Sub

Public Sub Join()
On Error Resume Next
Do While Me.Count 0
th.Join()
Loop
End Sub

Private Sub ThreadProc()
Dim value As T = Queue.Dequeue()
Do While value IsNot Nothing
Try
Dim result As StatusFlag = ProcessValue(value)

If (result And ProcessingQueue(Of T).StatusFlag.RequeueValue) =
StatusFlag.RequeueValue Then Queue.Enqueue(value)
If (result And ProcessingQueue(Of T).StatusFlag.ExitThread) =
StatusFlag.ExitThread Then Exit Do

value = Queue.Dequeue()
Catch ex As Exception
WriteLog(ex)
End Try
Loop
End Sub

Public ReadOnly Property Count() As Integer
Get
Return Queue.Count
End Get
End Property

Public ReadOnly Property TotalAdded() As Integer
Get
SyncLock syncTotalAdded
Return mTotalAdded
End SyncLock
End Get
End Property

Private Class QuickQueue(Of QuickQueue_Element)
Private Queue As New LinkedList(Of QuickQueue_Element)
Private so As New SynchronizationContext

Public Sub Enqueue(ByVal value As QuickQueue_Element)
SyncLock so
Queue.AddLast(value)
End SyncLock
End Sub

Public Function Dequeue() As QuickQueue_Element
Dim r As QuickQueue_Element = Nothing
SyncLock so
Try
r = CType(Queue.First().Value, QuickQueue_Element)
Queue.RemoveFirst()
Catch
End Try
End SyncLock
Return r
End Function

Public ReadOnly Property Count() As Integer
Get
SyncLock so
Return Queue.Count
End SyncLock
End Get
End Property
End Class
End Class

======================

"P.J.M. Beker" <pj******@orange.nlwrote in message
news:48***********************@news.wanadoo.nl...
Hi there,

I'm currently writing a program in which I use the FileMonitor to monitor
a folder in which I store downloaded images. I know that I can't add much
coding in the filemonitor's event in risk of losing some new entries, so
I've deceided to create an update thread. This thread is created when the
program's start and should (for various reason) run not in sync with the
Filemonitor.

The Filemonitor event creates an entry in a database table (lets call it
TEMP) in which it stores the location and name of a new image. The update
thread scans this table and if it finds any entries, is does a serie of
operations (hash calculation, check for multiple entries etc). The thread
then creates a new entry in table [QUEUE] in which it stores the image
title, location, hashvalue etc.Then it gets the next entry in table TEMP.
If there aren't any records left, it goes to sleep. When waking up it
scans table TEMP again.

Here is my problem. When terminating my program, the update thread should
stop, but only after it finishes processing the current image it's working
on. It should not create any corrupted/broken records. How can I prevent
this from happening?

// Main Program
private void FileMonitor_Created(Object sender, FileSystemEventArgs e)
{
// Creates a new record in table TEMP
CreateNewTempEntry(e.FullPath);
}

// Update Thread
private void ScanTempTable()
{
OleDbCommand myCmd = new OleDBCommand("SELECT * FROM TEMP");
myCmd.Connection = ...;
OleDbDataReader myReader = myCmd.ExecuteReader();
string myFile = "", myHash = "";
bool bFound = false;
while(myReader.Read())
{
myFile = myReader.GetString(0);
myHash = CalcHash(myFile);
bFound = Scan4MultipleImages(myHash)

-----------------------------------------------------------
// Thread isn't allow to stop here
WriteNewImageRecord(myFile, myHash, bFound);
// Thread can stop now
------------------------------------------------------------
}
// Sleep for a while
}

thnx in advance

Peter



Jun 27 '08 #4
Hi Jon,

This is the first time I'm gonna to use multiple threads in a program, so my
experience is not great on this subject. My idea was to create some kind of
"Laid Back" thread (is there a job to do, then it will do the job. If it
can't finished the job on time, it will continue the next time the program
starts). It doesn't have to race the FileSystemWatcher events (sorry Ignacio
;-)), but I've have to make sure the "Laid Back" thread doesn't create
corrupt records.

Your solution the create a foreground thread could work, but does it
influence the main UI thread (as I said before; my experience with threads
is null)?

Peter

"Jon Skeet [C# MVP]" <sk***@pobox.comschreef in bericht
news:4a**********************************@m73g2000 hsh.googlegroups.com...
On May 2, 10:14 am, "P.J.M. Beker" <pjmbe...@orange.nlwrote:

<snip>
>Here is my problem. When terminating my program, the update thread should
stop, but only after it finishes processing the current image it's
working
on. It should not create any corrupted/broken records. How can I prevent
this from happening?

Make the thread a foreground thread, which will mean the program keeps
running while the thread is alive.

Then add a way (a flag, for instance) of telling the thread to quit
after it's finished the current record. The "sleep for a while" part
should probably be something like Monitor.Wait so you can pulse a
monitor to make it respond quickly if it's idle when you want to quit.

Jon

Jun 27 '08 #5
P.J.M. Beker <pj******@orange.nlwrote:
This is the first time I'm gonna to use multiple threads in a program, so my
experience is not great on this subject.
Best of luck :)
My idea was to create some kind of
"Laid Back" thread (is there a job to do, then it will do the job. If it
can't finished the job on time, it will continue the next time the program
starts). It doesn't have to race the FileSystemWatcher events (sorry Ignacio
;-)), but I've have to make sure the "Laid Back" thread doesn't create
corrupt records.
Well, in that case you could leave it as a background thread, but make
the final bit of the record creation bit atomic - e.g. create a
temporary file, then rename it at the very end. Could be more
complicated though.
Your solution the create a foreground thread could work, but does it
influence the main UI thread (as I said before; my experience with threads
is null)?
No, it shouldn't be a problem.

--
Jon Skeet - <sk***@pobox.com>
Web site: http://www.pobox.com/~skeet
Blog: http://www.msmvps.com/jon.skeet
C# in Depth: http://csharpindepth.com
Jun 27 '08 #6

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

reply views Thread by Fraser Hanson | last post: by
5 posts views Thread by ECathell | last post: by
2 posts views Thread by Dick Swager | last post: by
8 posts views Thread by =?Utf-8?B?R3JlZyBMYXJzZW4=?= | last post: by
3 posts views Thread by Billy Bob | last post: by
reply views Thread by XIAOLAOHU | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.