473,326 Members | 2,127 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

Timer + High CPU Usage

LBT
I have a window service written using VB.NET. This window service will scan
folders for file and grab the file content to be inserted to SQL Server on
file detection. There are altogether 18 folders to be watched. Each folder is
assigned with a timer for watching purpose. Hence there will be 18 timers and
each timer is set to elapse on every second.

Problem here, once the window service is installed and started, the CPU
usage is very high (take about 10% of overall CPU usage at a four 2GHz
processors server even when there is no file being sent in for processing).
This highly comsumption of CPU power is because of the timers elapsed event?
Any idea to improve it? Any advice and comment would be much appreciated.
Nov 21 '05 #1
7 4919
On 2004-12-01, LBT <LB*@discussions.microsoft.com> wrote:
I have a window service written using VB.NET. This window service will scan
folders for file and grab the file content to be inserted to SQL Server on
file detection. There are altogether 18 folders to be watched. Each folder is
assigned with a timer for watching purpose. Hence there will be 18 timers and
each timer is set to elapse on every second.

Problem here, once the window service is installed and started, the CPU
usage is very high (take about 10% of overall CPU usage at a four 2GHz
processors server even when there is no file being sent in for processing).
This highly comsumption of CPU power is because of the timers elapsed event?
More then likely what your doing in the event... How often are the
timers firing? Which timer class are you using?
Any idea to improve it? Any advice and comment would be much appreciated.


Have you looked at the FileSystemWatcher class in System.IO. You
shouldn't need timers and polling this way...

--
Tom Shelton [MVP]
Nov 21 '05 #2
LBT
Thanks for the reply.

I have used the Timer under "Components" tab located in the .NET IDE Toolbox.

I did use the FileSystemWatcher class before but the class somehow have
limitation. Whenever SQL Server is down or offline, the file that being
halfway processed or subsequent file sent in to the folder will not be
processed as FileSystemWatcher only provides method such as OnCreated,
OnModified, OnDeleted, etc. The file will be stuck inside the folder without
processed after all. Hence I have changed the mechanism by always scanning
folder for file using timer.

"Tom Shelton" wrote:
On 2004-12-01, LBT <LB*@discussions.microsoft.com> wrote:
I have a window service written using VB.NET. This window service will scan
folders for file and grab the file content to be inserted to SQL Server on
file detection. There are altogether 18 folders to be watched. Each folder is
assigned with a timer for watching purpose. Hence there will be 18 timers and
each timer is set to elapse on every second.

Problem here, once the window service is installed and started, the CPU
usage is very high (take about 10% of overall CPU usage at a four 2GHz
processors server even when there is no file being sent in for processing).
This highly comsumption of CPU power is because of the timers elapsed event?


More then likely what your doing in the event... How often are the
timers firing? Which timer class are you using?
Any idea to improve it? Any advice and comment would be much appreciated.


Have you looked at the FileSystemWatcher class in System.IO. You
shouldn't need timers and polling this way...

--
Tom Shelton [MVP]

Nov 21 '05 #3
In article <6F**********************************@microsoft.co m>, LBT wrote:
Thanks for the reply.

I have used the Timer under "Components" tab located in the .NET IDE Toolbox.

I did use the FileSystemWatcher class before but the class somehow have
limitation. Whenever SQL Server is down or offline, the file that being
halfway processed or subsequent file sent in to the folder will not be
processed as FileSystemWatcher only provides method such as OnCreated,
OnModified, OnDeleted, etc. The file will be stuck inside the folder without
processed after all. Hence I have changed the mechanism by always scanning
folder for file using timer.


You should still be able to use filesystemwatcher... You simply need to
queue failed requests - then you would only need one timer. That timer
would check periodically to see if there were files that needed to be
processed in the failed queue.

private sub watcher_Created (byval sender as object, byval e as
filesystemeventargs e)

try
attempt to process file

catch ex as exception
we couldn't process - so
failedqueue.add (file)
end try
end sub

private sub timer_elapsed (....)
try
attempt to process files in queue
catch ex as excpetion
add current file back to queue
failedqueue.add (file)
end try
end sub

make sense?

--
Tom Shelton [MVP]
OS Name: Microsoft Windows XP Professional
OS Version: 5.1.2600 Service Pack 2 Build 2600
System Up Time: 39 Days, 19 Hours, 23 Minutes, 47 Seconds
Nov 21 '05 #4
LBT
Appreciated for the suggestion :) But I still have doubt and wish that Tom
could provide me some valuable idea.

I think I will explain some logic flow of my window service in order to have
a better understanding to cope my hassle.

The window service needs to process files from 16 directories. Each folder
has its own file format. There will be no limitation on the number of files
that can be sent in to a folder at one time. But only 1 file is allowed to
process at one time (sequence of file based on creation datetime is very
important). I might need a queue table to stack the records (file info)
before each file get its turn to be processed. The role of the window service
is to obtain the file content and insert to a SQL table as a record. In the
SQL table, there is a INSERT trigger which will be invoked and execute a
stored procedure to carry out validation towards the record. At the last
portion of the stored procedure, it will update the queued record to
"FINISH". When WS detect for this "FINISH" status, it will move the file to
archive folder and processing for 1 file is considered finish.

Question here, if I use the FileSystemWatcher class, it will help me to put
the file info to the queue table with the help of OnCreated event. But I
might still need a timer to process the queue table to ensure that file is
processed in sequence and only one file is processed at one time.

Currently, I use a timer to scan directory. If no file is detected, the
process will stop and wait for next elapsed event. If file(s) detected, all
file info as grabbed at that time will be put to a queue table. All record(s)
inside the queue table should be finish processed before subsequent file
exists in a folder would be catered again. In order words, the timer elapsed
event will not scan directory for file until queue table is finish processed
(controlled using a boolean flag).

As what can be seen, timer should be there for first method
(FileSystemWatcher + Timer) and second method which I used for this moment.
Timer seem can't be escaped from using for both methods and the use of timer
would again increase the CPU usage when all timers elapsed at the same time.
For yuor information, timer interval is set to 1 second.

Any way I can improve so that the CPU usage will not reach the critical
level? What I've tested shows that the CPU usage is considered high when
there is no file being sent in to any of the folders. The CPU usage will
become higher when there is file sent in for processing. Any weakness in my
existing file processing mechanism? Any better workaround?

Thanks a lot


Nov 21 '05 #5
Uset the FileSystemWatcher to tell you when a new file has been created.
Then use the same scanning algorithm you currently use. By using the
FileSystemWatcher, you eliminate both excessive scans and delayed scans.
Once you have completed scanning, then restart start the FileSystemWatcher.

Mike Ober.

"LBT" <LB*@discussions.microsoft.com> wrote in message
news:78**********************************@microsof t.com...
Appreciated for the suggestion :) But I still have doubt and wish that Tom
could provide me some valuable idea.

I think I will explain some logic flow of my window service in order to have a better understanding to cope my hassle.

The window service needs to process files from 16 directories. Each folder
has its own file format. There will be no limitation on the number of files that can be sent in to a folder at one time. But only 1 file is allowed to
process at one time (sequence of file based on creation datetime is very
important). I might need a queue table to stack the records (file info)
before each file get its turn to be processed. The role of the window service is to obtain the file content and insert to a SQL table as a record. In the SQL table, there is a INSERT trigger which will be invoked and execute a
stored procedure to carry out validation towards the record. At the last
portion of the stored procedure, it will update the queued record to
"FINISH". When WS detect for this "FINISH" status, it will move the file to archive folder and processing for 1 file is considered finish.

Question here, if I use the FileSystemWatcher class, it will help me to put the file info to the queue table with the help of OnCreated event. But I
might still need a timer to process the queue table to ensure that file is
processed in sequence and only one file is processed at one time.

Currently, I use a timer to scan directory. If no file is detected, the
process will stop and wait for next elapsed event. If file(s) detected, all file info as grabbed at that time will be put to a queue table. All record(s) inside the queue table should be finish processed before subsequent file
exists in a folder would be catered again. In order words, the timer elapsed event will not scan directory for file until queue table is finish processed (controlled using a boolean flag).

As what can be seen, timer should be there for first method
(FileSystemWatcher + Timer) and second method which I used for this moment. Timer seem can't be escaped from using for both methods and the use of timer would again increase the CPU usage when all timers elapsed at the same time. For yuor information, timer interval is set to 1 second.

Any way I can improve so that the CPU usage will not reach the critical
level? What I've tested shows that the CPU usage is considered high when
there is no file being sent in to any of the folders. The CPU usage will
become higher when there is file sent in for processing. Any weakness in my existing file processing mechanism? Any better workaround?

Thanks a lot

Nov 21 '05 #6
LBT
Currently I used the System.IO.Directory.GetFiles method to scan directory.
Is it the method will cause lot of undesired overhead? The use of
FileSystemWatcher will be able to eliminate this?

Thanks

"Michael D. Ober" wrote:
Uset the FileSystemWatcher to tell you when a new file has been created.
Then use the same scanning algorithm you currently use. By using the
FileSystemWatcher, you eliminate both excessive scans and delayed scans.
Once you have completed scanning, then restart start the FileSystemWatcher.

Mike Ober.

"LBT" <LB*@discussions.microsoft.com> wrote in message
news:78**********************************@microsof t.com...
Appreciated for the suggestion :) But I still have doubt and wish that Tom
could provide me some valuable idea.

I think I will explain some logic flow of my window service in order to

have
a better understanding to cope my hassle.

The window service needs to process files from 16 directories. Each folder
has its own file format. There will be no limitation on the number of

files
that can be sent in to a folder at one time. But only 1 file is allowed to
process at one time (sequence of file based on creation datetime is very
important). I might need a queue table to stack the records (file info)
before each file get its turn to be processed. The role of the window

service
is to obtain the file content and insert to a SQL table as a record. In

the
SQL table, there is a INSERT trigger which will be invoked and execute a
stored procedure to carry out validation towards the record. At the last
portion of the stored procedure, it will update the queued record to
"FINISH". When WS detect for this "FINISH" status, it will move the file

to
archive folder and processing for 1 file is considered finish.

Question here, if I use the FileSystemWatcher class, it will help me to

put
the file info to the queue table with the help of OnCreated event. But I
might still need a timer to process the queue table to ensure that file is
processed in sequence and only one file is processed at one time.

Currently, I use a timer to scan directory. If no file is detected, the
process will stop and wait for next elapsed event. If file(s) detected,

all
file info as grabbed at that time will be put to a queue table. All

record(s)
inside the queue table should be finish processed before subsequent file
exists in a folder would be catered again. In order words, the timer

elapsed
event will not scan directory for file until queue table is finish

processed
(controlled using a boolean flag).

As what can be seen, timer should be there for first method
(FileSystemWatcher + Timer) and second method which I used for this

moment.
Timer seem can't be escaped from using for both methods and the use of

timer
would again increase the CPU usage when all timers elapsed at the same

time.
For yuor information, timer interval is set to 1 second.

Any way I can improve so that the CPU usage will not reach the critical
level? What I've tested shows that the CPU usage is considered high when
there is no file being sent in to any of the folders. The CPU usage will
become higher when there is file sent in for processing. Any weakness in

my
existing file processing mechanism? Any better workaround?

Thanks a lot


Nov 21 '05 #7
Hi,

I made a form, adding this code

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As
System.EventArgs) Handles Button1.Click
Dim oTimer(19) As MyTimer
For I As Integer = 0 To 19
oTimer(I) = New MyTimer
oTimer(I).Interval = 1000
oTimer(I).nID = I

AddHandler oTimer(I).Elapsed, AddressOf oTimer_Elapsed
oTimer(I).Start()
Next
End Sub

Public Sub oTimer_Elapsed(ByVal sender As Object, ByVal e As
System.Timers.ElapsedEventArgs)
Dim sArray() As String = System.IO.Directory.GetFiles("C:\T\" &
CType(sender, MyTimer).nID & "\", "*.*")
If Not sArray Is Nothing Then
If sArray.Length > 0 Then
Debug.WriteLine("Found file...")
End If
End If

End Sub

Public Class MyTimer
Inherits System.Timers.Timer

Private m_nID As Integer
Public Property nID() As Integer
Get
Return m_nID
End Get
Set(ByVal Value As Integer)
m_nID = Value
End Set
End Property
End Class

When running it (made 20 directories) it takes zero % processor power.

Also, a word of caution, a mistake that I made a while ago, if you create a
new timer and dont stop the old one (or run out of scope) it still elapes,
and when you have several thousand, the processor will start feeling it..
- Fredrik Melin
"LBT" <LB*@discussions.microsoft.com> wrote in message
news:E4**********************************@microsof t.com...
I have a window service written using VB.NET. This window service will scan
folders for file and grab the file content to be inserted to SQL Server on
file detection. There are altogether 18 folders to be watched. Each folder
is
assigned with a timer for watching purpose. Hence there will be 18 timers
and
each timer is set to elapse on every second.

Problem here, once the window service is installed and started, the CPU
usage is very high (take about 10% of overall CPU usage at a four 2GHz
processors server even when there is no file being sent in for
processing).
This highly comsumption of CPU power is because of the timers elapsed
event?
Any idea to improve it? Any advice and comment would be much appreciated.

Nov 21 '05 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Ian Taite | last post by:
Hello, I'm exploring why one of my C# .NET apps has "high" memory usage, and whether I can reduce the memory usage. I have an app that wakes up and processes text files into a database...
3
by: jam | last post by:
Dear all, I am wrtiing a console application and doing some test on timer, the below is my sample code I got from some site, but I cannot make it work... Error is 'System.Timers.Timer' does not...
2
by: Michael Evans | last post by:
First, we rely on a stable update rate so that our physics and dynamics calculations and integrations are based on a known interval and therefore are true-to-life. Second, the graphics positions...
4
by: Liverpool fan | last post by:
I have a windows application written using VB .NET that encompasses a countdown timer modal dialog. The timer is a System.Timers.Timer with an interval of 1 second. AutoReset is not set so accepts...
6
by: paresh | last post by:
Hi all, I need to set timer in C/linux like alram, such that i will get a timeout signal after specific timeout and my process remain executing as is it. I can use signal(SIGALRM, xyz) and then...
10
by: DaTurk | last post by:
Hi, I'm creating an application that will need to use a timer to do a static method call every second. My question, is what would be the best timer to use? I know there are several, but I'd...
9
by: =?Utf-8?B?TWlrZTk5MDA=?= | last post by:
Hello, I am wondering if it is a good idea to use GC.Collect() in a timer. For example, timer is fired every 5 minutes and calls GC.Collect(). In our server app the memory goes to 600MB and...
7
by: Andrew Wan | last post by:
I found this excellent High Speed Timer (in Pascal). I compiled it (using Turbo Pascal 7 and it runs fine): http://www.sorucevap.com/bilisimteknolojisi/programcilik/pascal/ders.asp?207995 and...
6
by: dantz | last post by:
HI everyone, I hope someone can help me on this. I have form application that has 3 Timers that does an animation (changing an image for every interval) Each image are loaded at start of...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.