473,836 Members | 1,563 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Looking for Advice on Thread.Sleep

I ran into a situation where my Window Service had to process 100,000+
files, when I first noticed I needed to tweak various routines.

Everything runs fine, but here's what I ran into:

In the routine that loops through the file buffer:

for (int i=0;i < _Buffer.length; i++)
{
// Code here
}

If I run it as is, it throttles CPU usage 100%.

If I toss in a simple statement:

Thread.Sleep(1) ;

The CPU usage throttles around 0-3% as it works through the list.

The difference in files processed is amazingly different though,
something to the tune of 9-10,000 files without the Sleep. 800-1000
files with. (Report dumped once a minute to Event Log).

Looking through various code in this Newsgroup, I've seen
Thread.Sleep(0) instead of Thread.Sleep(1) .

2 Questions I have are:

1) Why use 0 instead of 1 if you actually want the Thread to sleep?

2) Is there a better way to run a process that is I/O intensive, SQL
intensive and keep the CPU throttle within reason, other than
incorporating Thread.Sleep statements within the looping mechanisms ?
Nov 15 '05 #1
8 2797
Cider,

If you want to keep the CPU from being so overwhelmed, I would consider
using the threadpool. Basically, I would have a list of items that you need
to process. Once you have that, you place an item (or items) in the
threadpool and have them process one item at a time. When an item is done,
it would add a new item to the threadpool (if there were items in the list
to process).

If you do it this way, I assume performance is going to decline, but you
will not get the excessive CPU usage. You are going to see that generally,
by throttling the CPU usage, you will not be able to process that many items
at a time.

Hope this helps.
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard. caspershouse.co m

"Cider123" <ci******@hotma il.com> wrote in message
news:55******** *************** ***@posting.goo gle.com...
I ran into a situation where my Window Service had to process 100,000+
files, when I first noticed I needed to tweak various routines.

Everything runs fine, but here's what I ran into:

In the routine that loops through the file buffer:

for (int i=0;i < _Buffer.length; i++)
{
// Code here
}

If I run it as is, it throttles CPU usage 100%.

If I toss in a simple statement:

Thread.Sleep(1) ;

The CPU usage throttles around 0-3% as it works through the list.

The difference in files processed is amazingly different though,
something to the tune of 9-10,000 files without the Sleep. 800-1000
files with. (Report dumped once a minute to Event Log).

Looking through various code in this Newsgroup, I've seen
Thread.Sleep(0) instead of Thread.Sleep(1) .

2 Questions I have are:

1) Why use 0 instead of 1 if you actually want the Thread to sleep?

2) Is there a better way to run a process that is I/O intensive, SQL
intensive and keep the CPU throttle within reason, other than
incorporating Thread.Sleep statements within the looping mechanisms ?

Nov 15 '05 #2
You could have it sleep on every X number of loops. If this is commercial
ware to be packaged and run on different hardware, then you could use
PerformanceCoun ters to see how much CPU you are using, and dynamically
change how often or length of the sleep.

Thread.Sleep(0) will tell the processor to let other executing threads run.
This is probably the best (if there is no work needed, your proc will get
more CPU time, but if other load is heavy, it will give it up). Depending
on the length of each iteration, you might consider doing this every X
iterations.

And, of course, if you use less CPU for the same code, you're going to get
less work done :).

-mike
MVP

"Cider123" <ci******@hotma il.com> wrote in message
news:55******** *************** ***@posting.goo gle.com...
I ran into a situation where my Window Service had to process 100,000+
files, when I first noticed I needed to tweak various routines.

Everything runs fine, but here's what I ran into:

In the routine that loops through the file buffer:

for (int i=0;i < _Buffer.length; i++)
{
// Code here
}

If I run it as is, it throttles CPU usage 100%.

If I toss in a simple statement:

Thread.Sleep(1) ;

The CPU usage throttles around 0-3% as it works through the list.

The difference in files processed is amazingly different though,
something to the tune of 9-10,000 files without the Sleep. 800-1000
files with. (Report dumped once a minute to Event Log).

Looking through various code in this Newsgroup, I've seen
Thread.Sleep(0) instead of Thread.Sleep(1) .

2 Questions I have are:

1) Why use 0 instead of 1 if you actually want the Thread to sleep?

2) Is there a better way to run a process that is I/O intensive, SQL
intensive and keep the CPU throttle within reason, other than
incorporating Thread.Sleep statements within the looping mechanisms ?

Nov 15 '05 #3
Just reduce the priority of the service to something below Normal and call
Thread.Sleep(0) to give up your thread quota at the end of the loop.

Willy.

"Cider123" <ci******@hotma il.com> wrote in message
news:55******** *************** ***@posting.goo gle.com...
I ran into a situation where my Window Service had to process 100,000+
files, when I first noticed I needed to tweak various routines.

Everything runs fine, but here's what I ran into:

In the routine that loops through the file buffer:

for (int i=0;i < _Buffer.length; i++)
{
// Code here
}

If I run it as is, it throttles CPU usage 100%.

If I toss in a simple statement:

Thread.Sleep(1) ;

The CPU usage throttles around 0-3% as it works through the list.

The difference in files processed is amazingly different though,
something to the tune of 9-10,000 files without the Sleep. 800-1000
files with. (Report dumped once a minute to Event Log).

Looking through various code in this Newsgroup, I've seen
Thread.Sleep(0) instead of Thread.Sleep(1) .

2 Questions I have are:

1) Why use 0 instead of 1 if you actually want the Thread to sleep?

2) Is there a better way to run a process that is I/O intensive, SQL
intensive and keep the CPU throttle within reason, other than
incorporating Thread.Sleep statements within the looping mechanisms ?

Nov 15 '05 #4
One thing that I would consider is whether you need for this thread to
yield. Is this process a background task, or is this the server's primary
responsibility? Seems weird, but sometimes if the CPU is running at 100%,
it just means that you are operating efficiently. 2000/NT is a
preemptive-multitasking operating system, so it can interrupt (preempt) your
thread if it needs to. The operating system is going to share the CPU time,
and if possible, it's going to throttle the CPU to 100%. It wants to fully
utilize the CPU -- it's trying to hit 100%. Thing is, there usually isn't
enough going on to keep the CPU that busy.

Now, if you are finding that other, competing operations are occuring too
slowly, then maybe there is some tuning to do.

Said another way, I wouldn't be too concerned about the CPU hitting 100%,
unless it is having too heavy an impact on other applications on the server.
What you probably want is for your I/O process to get as much CPU time as it
can, and yet yield time to other applications as necessary.

If your thread is running at normal priority, it will be pre-empted if a
higher priority item comes up. It will share time with threads of equal
priority, and it will preempt threads of lower priority. If you are
experiencing a performance issue with other systems on the server, then your
thread may be competing too aggressively.

You might try playing with sleep(0), you might do it only x number of
iterations. That will cause your thread to yield to other threads. Now, if
you are doing a sleep(1) (or anything >0), you are really telling your
thread to yield to any other thread that is running, even the low-priority
ones. I'm not sure whether sleep(0) yields to lower priority threads or
not).

If your process is supposed to be strictly a background process (work as
hard as you can, unless something interrupts you), then you can just alter
its priority. If it's running at a slightly lower than normal priority, it
will consume 100% CPU if it can, but if any normal priority thread (and most
threads are normal) needs attention, your process thread will be preempted
by the operating system. The most notable example of this that I can think
of would be the grid computing community ware out there -- like that cancer
genome project, or the SETI downloadable that runs in the backgound. Those
processes have almost no impact on your throughput if your computer is doing
something, but they consume 100% cpu time if your machine goes idle -- they
do that by manipulating their thread priority.

The downside to changing priority is that your process can be starved by
another thread. If a normal priority thread gets busy for several minutes
(or an hour or so), and you have reduced the priority of your I/O process
thread, then your thread will be preempted (starved) while the higher
priority threads run.

Using Sleep() + normal priority your thread would not be starved by other
normal -priority threads. On the other hand (I know, too many hands <g>),
you don't want to overdo it with the sleep commands. Every time the CPU
stops executing one thread and picks up another, it has to do what's called
a "context switch". I don't know all of the details of what goes on during
a context switch -- I think that a bunch of state data has to be saved for
the sleeping thread and loaded for the upcoming thread. I do know that they
are pretty expensive, and if you sleep your thread too often (or if you get
too many threads running), you will actually be wasting CPU time performing
context switches.

I would advise that you don't get too hung up on CPU utlization, and instead
examine the performance of the server holistically. How are the other
applications performing with this process running? Then you can tune your
process thread with sleeps or priority settings to achieve a satisfactory
throughput.

You can test that stuff that I told by by doing a little while loop:
while (true) {} and checking system responsiveness.

"Cider123" <ci******@hotma il.com> wrote in message
news:55******** *************** ***@posting.goo gle.com...
I ran into a situation where my Window Service had to process 100,000+
files, when I first noticed I needed to tweak various routines.

Everything runs fine, but here's what I ran into:

In the routine that loops through the file buffer:

for (int i=0;i < _Buffer.length; i++)
{
// Code here
}

If I run it as is, it throttles CPU usage 100%.

If I toss in a simple statement:

Thread.Sleep(1) ;

The CPU usage throttles around 0-3% as it works through the list.

The difference in files processed is amazingly different though,
something to the tune of 9-10,000 files without the Sleep. 800-1000
files with. (Report dumped once a minute to Event Log).

Looking through various code in this Newsgroup, I've seen
Thread.Sleep(0) instead of Thread.Sleep(1) .

2 Questions I have are:

1) Why use 0 instead of 1 if you actually want the Thread to sleep?

2) Is there a better way to run a process that is I/O intensive, SQL
intensive and keep the CPU throttle within reason, other than
incorporating Thread.Sleep statements within the looping mechanisms ?


Nov 15 '05 #5
Another reason to be concerned about pegging the CPU is if you're writing
competitive software.

Review writers sometimes make an issue of it, and can convey to readers the
impression that your software is poorly written, because it maxes-out the
CPU resources. The implication is that your software is running at the
expense of other software on the same machine, and many users would consider
that a negative.

Sometimes it's not good enough just to write good software. It has to be
PERCEIVED to be good software. (If it's for in-house use only, then
obviously this isn't an issue at all.)

"J.Marsch" <je****@ctcdeve loper.com> wrote in message
news:%2******** **********@TK2M SFTNGP12.phx.gb l...
...
Said another way, I wouldn't be too concerned about the CPU hitting 100%,
unless it is having too heavy an impact on other applications on the server. What you probably want is for your I/O process to get as much CPU time as it can, and yet yield time to other applications as necessary.

Nov 15 '05 #6
Jerry:

That's a very good point. Unfortunate, but good. Yes, I can see how
pegging the CPU could be perceived as a bad thing, even though it might
really mean that everything is working just fine.

Perhaps a reasonable strategy might be to make the throttle tunable. Then,
the software could ship slightly detuned by default, so that it would not
use 100% CPU (and probably not run as fast). Then the people who have to
use the software could re-tune for performance.

Regards

Jeremy

"Jerry Houston" <JH******@AMSWo rld.com> wrote in message
news:e3******** ******@tk2msftn gp13.phx.gbl...
Another reason to be concerned about pegging the CPU is if you're writing
competitive software.

Review writers sometimes make an issue of it, and can convey to readers the impression that your software is poorly written, because it maxes-out the
CPU resources. The implication is that your software is running at the
expense of other software on the same machine, and many users would consider that a negative.

Sometimes it's not good enough just to write good software. It has to be
PERCEIVED to be good software. (If it's for in-house use only, then
obviously this isn't an issue at all.)

"J.Marsch" <je****@ctcdeve loper.com> wrote in message
news:%2******** **********@TK2M SFTNGP12.phx.gb l...
...
Said another way, I wouldn't be too concerned about the CPU hitting 100%, unless it is having too heavy an impact on other applications on the server.
What you probably want is for your I/O process to get as much CPU time

as it
can, and yet yield time to other applications as necessary.


Nov 15 '05 #7
Or have a "Are you a software reviewer?" checkbox on installation ;). Or
ship reviewers special copies with different setup files. Since they have
to write review after review, many times they don't really know how to
evaluate something (industry-specific reviewers are somewhat better though).

-mike
MVP

"J.Marsch" <je****@ctcdeve loper.com> wrote in message
news:ue******** ********@TK2MSF TNGP09.phx.gbl. ..
Jerry:

That's a very good point. Unfortunate, but good. Yes, I can see how
pegging the CPU could be perceived as a bad thing, even though it might
really mean that everything is working just fine.

Perhaps a reasonable strategy might be to make the throttle tunable. Then, the software could ship slightly detuned by default, so that it would not
use 100% CPU (and probably not run as fast). Then the people who have to
use the software could re-tune for performance.

Regards

Jeremy

"Jerry Houston" <JH******@AMSWo rld.com> wrote in message
news:e3******** ******@tk2msftn gp13.phx.gbl...
Another reason to be concerned about pegging the CPU is if you're writing
competitive software.

Review writers sometimes make an issue of it, and can convey to readers

the
impression that your software is poorly written, because it maxes-out the CPU resources. The implication is that your software is running at the
expense of other software on the same machine, and many users would

consider
that a negative.

Sometimes it's not good enough just to write good software. It has to be PERCEIVED to be good software. (If it's for in-house use only, then
obviously this isn't an issue at all.)

"J.Marsch" <je****@ctcdeve loper.com> wrote in message
news:%2******** **********@TK2M SFTNGP12.phx.gb l...
...
Said another way, I wouldn't be too concerned about the CPU hitting

100%, unless it is having too heavy an impact on other applications on the

server.
What you probably want is for your I/O process to get as much CPU time

as
it
can, and yet yield time to other applications as necessary.



Nov 15 '05 #8
You know, thinking back, I wonder if Microsoft did that with SQL Server. I
remember that with SQL Server 6.0, one of the first things that you had to
do after installing was to go in and increase the amount of memory that it
was allowed to use for caching, because it shipped with some ridiculously
low value (8MB?).
"Michael Giagnocavo [MVP]" <mg*******@Atre vido.net> wrote in message
news:%2******** ********@TK2MSF TNGP10.phx.gbl. ..
Or have a "Are you a software reviewer?" checkbox on installation ;). Or
ship reviewers special copies with different setup files. Since they have
to write review after review, many times they don't really know how to
evaluate something (industry-specific reviewers are somewhat better though).
-mike
MVP

"J.Marsch" <je****@ctcdeve loper.com> wrote in message
news:ue******** ********@TK2MSF TNGP09.phx.gbl. ..
Jerry:

That's a very good point. Unfortunate, but good. Yes, I can see how
pegging the CPU could be perceived as a bad thing, even though it might
really mean that everything is working just fine.

Perhaps a reasonable strategy might be to make the throttle tunable.

Then,
the software could ship slightly detuned by default, so that it would not
use 100% CPU (and probably not run as fast). Then the people who have to use the software could re-tune for performance.

Regards

Jeremy

"Jerry Houston" <JH******@AMSWo rld.com> wrote in message
news:e3******** ******@tk2msftn gp13.phx.gbl...
Another reason to be concerned about pegging the CPU is if you're writing competitive software.

Review writers sometimes make an issue of it, and can convey to readers the
impression that your software is poorly written, because it maxes-out the CPU resources. The implication is that your software is running at
the expense of other software on the same machine, and many users would

consider
that a negative.

Sometimes it's not good enough just to write good software. It has to be PERCEIVED to be good software. (If it's for in-house use only, then
obviously this isn't an issue at all.)

"J.Marsch" <je****@ctcdeve loper.com> wrote in message
news:%2******** **********@TK2M SFTNGP12.phx.gb l...
> ...
> Said another way, I wouldn't be too concerned about the CPU hitting

100%,
> unless it is having too heavy an impact on other applications on the
server.
> What you probably want is for your I/O process to get as much CPU

time as
it
> can, and yet yield time to other applications as necessary.



Nov 15 '05 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

38
2915
by: Anthony Baxter | last post by:
On behalf of the Python development team and the Python community, I'm happy to announce the release of Python 2.3.1 (final). Python 2.3.1 is a pure bug fix release of Python 2.3, released in late July. A number of obscure crash-causing bugs have been fixed, various memory leaks have been squished, but no new features have been added to the language or to the library. For more information on Python 2.3.1, including download links for...
4
5440
by: Matthew Groch | last post by:
Hi all, I've got a server that handles a relatively high number of concurrent transactions (on the magnitude of 1000's per second). Client applications establish socket connections with the server. Data is sent and received over these connections using the asynchronous model. The server is currently in beta testing. Sporadically over the course of the day, I'll observe the thread count on the process (via perfmon) start climbing....
9
3492
by: Chris Dunaway | last post by:
According to the docs, calling Thread.Sleep(0) causes the thread to be "suspended to allow other waiting threads to execute." What happens if I call Thread.Sleep(500)? Do other threads not get a chance to execute during this time? What is the difference between the two? I have code that runs in a loop like this: Dim dResetTime As DateTime = DateTime.Now
32
1877
by: John Salerno | last post by:
My first project when I started learning C# was to make a little timer to tell me when my laundry was done :) and I thought it would be fun to convert this to Python. Here's what I came up with after much struggling with the Timer class from the threading module -- as you can see, I abandoned it for the sleep() function from timer. Please let me know if this is a good (i.e. Pythonic) way of doing this, and if it can be improved (although...
6
2824
by: k.mellor | last post by:
Hi, I hope someone can help. I have written a simple form to demonstrate my problem/question. The code follows. The form starts a thread, which using delegates updates a label (Every second adds another dot to the label). This works great. However, when I put the GUI thread to sleep (Thread.Sleep), the thread seems to stall. At first I was expecting dots to still appear, but obviously as the GUI thread is asleep, they will not. ...
232
13404
by: robert maas, see http://tinyurl.com/uh3t | last post by:
I'm working on examples of programming in several languages, all (except PHP) running under CGI so that I can show both the source files and the actually running of the examples online. The first set of examples, after decoding the HTML FORM contents, merely verifies the text within a field to make sure it is a valid representation of an integer, without any junk thrown in, i.e. it must satisfy the regular expression: ^ *?+ *$ If the...
0
2702
by: Buckaroo Banzai | last post by:
Hello, newbie here... I'm writing this program but when I click the start button which should initiate either the Hare or the Tortoise, it does not, this is the first time I use threads, so the problem might be the way I'm passing the value. can someone please take a look at this code and maybe give me some guidance on what I'm doing wrong. Please note there are 3 separate files with the corresponding classes, I separated it with ===>...
3
1551
by: Peter Webb | last post by:
When I started my current extremely graphics intensive project, I ignored advice in this ng to use the Paint method, and used the alternate CreateGraphics approach. I thought there were some good reasons for that, which I won't go into. Anyway, there has always been one tiny bug that has annoyed me - the code that draws the initial graphic image into PictureBox1 will only work if wire it up to a button, it doesn't actually do anything if...
2
3377
by: Steve | last post by:
Hi All, I've been trying to come up with a good way to run a certain process at a timed interval (say every 5 mins) using the SLEEP command and a semaphore flag. The basic thread loop was always sitting in the sleep command and not able to be interrupted. When the time came to set the semaphore flag to false (stopping the thread), my program would have to wait up to the entire sleep time to break out of the loop. I have finally found...
0
9826
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9674
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10560
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10604
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9389
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5659
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
4466
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
4026
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
3116
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.