context switch

[Ryan Liu]

Hi,

I have a client/server application, using one thread/client approach.

I see very high context switch/sec. What are the ways to reduce it? Each
thread sleep longer in its endless loop if not reducing thread?

On the same machine, I also run a mysql server. I can see same amount thread
in mysqld, seems mysql is also using one thread/client. But its
context/swith is much lower. How can I get same performance?

Thanks a lot!
Ryan

[Peter Duniho]

On Sun, 01 Jun 2008 17:48:11 -0700, Ryan Liu <rl**@powercati .comwrote:

I have a client/server application, using one thread/client approach.

I see very high context switch/sec. What are the ways to reduce it? Each
thread sleep longer in its endless loop if not reducing thread?

On the same machine, I also run a mysql server. I can see same amount
thread
in mysqld, seems mysql is also using one thread/client. But its
context/swith is much lower. How can I get same performance?

The simple answer is: discard the "one thread/client" design. It doesn't
scale well on Windows.

Use the asynchronous API on the i/o classes (especially network i/o, which
I assume is what you're talking about), and that will take advantage of
the built-in mechanisms in Windows designed to prevent excessive context
switching and minimizing thread count (i/o completion ports). For
example, use Socket.BeginRec eive, Stream.BeginRea d, etc. (with matching
"End..." methods, of course).

I'm surprised to hear that MySql uses "one thread/client", and I'm curious
how you've confirmed this. Regardless, it's not possible to answer "how
can I get the same performance" without knowing the specific
implementation details for MySql, and of course knowing the specific
implementation details would tell you directly how to get the same
performance. :)

Finally, of course, you should ask yourself whether it's important. If
you only have a small number of clients and performance is acceptable,
then a high number of context switches may not be a problem. Yes, it
might mean the design could be more efficient, but if the code works and
performs up to your needs, complicating the design may not be the best
idea.

Pete

[Nicholas Paldino [.NET/C# MVP]]

Ryan,

Generally, you are going to have context switches when you have more
threads that are running than there are processors to handle the threads.

How are you allocating the threads? Generally, you should probably use
the ThreadPool to assign tasks to your threads, as the ThreadPool will take
into account the number of processors on the machine to determine how many
threads to keep in the pool (and minimize context switches).

Also, are you sure the context switches are causing a performance
impact? If you switch to less threads, how do you kno you will get the same
concurrency that you have now?
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard. caspershouse.co m

"Ryan Liu" <rl**@powercati .comwrote in message
news:%2******** ********@TK2MSF TNGP06.phx.gbl. ..

Hi,

I have a client/server application, using one thread/client approach.

I see very high context switch/sec. What are the ways to reduce it? Each
thread sleep longer in its endless loop if not reducing thread?

On the same machine, I also run a mysql server. I can see same amount
thread in mysqld, seems mysql is also using one thread/client. But its
context/swith is much lower. How can I get same performance?

Thanks a lot!
Ryan

[Peter Duniho]

On Sun, 01 Jun 2008 18:01:04 -0700, Nicholas Paldino [.NET/C# MVP]
<mv*@spam.guard .caspershouse.c omwrote:

Ryan,

Generally, you are going to have context switches when you have more
threads that are running than there are processors to handle the threads.

True. Which is why IOCP is a good way to address the issue. Windows
knows to keep a given IOCP thread busy if it can, rather than letting
another thread run. Threads waiting on a completion event for an IOCP
won't get the event if another thread is already ready to deal with it. A
perfectly balanced IOCP implementation will have exactly the right number
of IOCP threads, but going over a little will generally not cause addition
context switches, as a more naïve i/o implementation might.

How are you allocating the threads? Generally, you should probably
use the ThreadPool to assign tasks to your threads, as the ThreadPool
will take into account the number of processors on the machine to
determine how many threads to keep in the pool (and minimize context
switches).

Are you sure this is true? More specifically, while it's true that the
ThreadPool uses the CPU count to determine the number of threads, the
total thread count has always been much higher than the actual number of
CPUs. Off the top of my head, it used to be 25 threads per CPU, and now
it's 250 threads, or something like that.

I've never known the ThreadPool to be a particularly good manager of
context-switching. :)

Also, are you sure the context switches are causing a performance
impact? If you switch to less threads, how do you kno you will get the
same concurrency that you have now?

A very good question. :)

Pete

[Ryan Liu]

Nicholas , Thanks for your quick response.

I have 120 client machines connect to one server for the C/S application.
MySql runs on the same server machine. Each client keep one connection to
the db server as well. Clients and the server exchange very small data, but
very frequently.

I am using sync I/O to read from client, and use
ThreadPool.Queu eUserWorkItem to send response back.

I haven't use ThreadPool to read data from client, because 1: my current
approach is easy to write code, and 2: I am afraid if I handle to ThreadPool
and since server work load is high, will my receiving job fail to be queued
or wait in queue for too long time? Is ThreadPool suitable for C/S
application that constantly scan and trying to receive data from clients?

I think maybe I wait longer in each loop or use one thread for 10 clients.
In each loop, scan 10 clients one by one, and sleep less in each loop. So
for each client, interval maybe still same. Will that help a lot?

Now it sleep 100 ms in each loop. As said, I have 120 loops running.
Generally, is 100 ms a suitable amount?

Thanks!

"Nicholas Paldino [.NET/C# MVP]" <mv*@spam.guard .caspershouse.c om>
??????:uk****** ********@TK2MSF TNGP06.phx.gbl. ..

Ryan,

Generally, you are going to have context switches when you have more
threads that are running than there are processors to handle the threads.

How are you allocating the threads? Generally, you should probably use
the ThreadPool to assign tasks to your threads, as the ThreadPool will
take into account the number of processors on the machine to determine how
many threads to keep in the pool (and minimize context switches).

Also, are you sure the context switches are causing a performance
impact? If you switch to less threads, how do you kno you will get the
same concurrency that you have now?
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard. caspershouse.co m

"Ryan Liu" <rl**@powercati .comwrote in message
news:%2******** ********@TK2MSF TNGP06.phx.gbl. ..

>Hi,

I have a client/server application, using one thread/client approach.

I see very high context switch/sec. What are the ways to reduce it? Each
thread sleep longer in its endless loop if not reducing thread?

On the same machine, I also run a mysql server. I can see same amount
thread in mysqld, seems mysql is also using one thread/client. But its
context/swith is much lower. How can I get same performance?

Thanks a lot!
Ryan

[Nicholas Paldino [.NET/C# MVP]]

Peter,

Well, the statement about the thread pool is true, in the sense that it
takes the number of threads into account when figuring out how many threads
it should keep in the pool. I didn't say that it kept a number of threads
equal to the number of processors =)

The new parallel task library is better at this, I believe (the new
System.Threadin g.dll).
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard. caspershouse.co m

"Peter Duniho" <Np*********@nn owslpianmk.comw rote in message
news:op******** *******@petes-computer.local. ..

On Sun, 01 Jun 2008 18:01:04 -0700, Nicholas Paldino [.NET/C# MVP]
<mv*@spam.guard .caspershouse.c omwrote:

>Ryan,

Generally, you are going to have context switches when you have more
threads that are running than there are processors to handle the threads.

True. Which is why IOCP is a good way to address the issue. Windows
knows to keep a given IOCP thread busy if it can, rather than letting
another thread run. Threads waiting on a completion event for an IOCP
won't get the event if another thread is already ready to deal with it. A
perfectly balanced IOCP implementation will have exactly the right number
of IOCP threads, but going over a little will generally not cause addition
context switches, as a more naïve i/o implementation might.

> How are you allocating the threads? Generally, you should probably
use the ThreadPool to assign tasks to your threads, as the ThreadPool
will take into account the number of processors on the machine to
determine how many threads to keep in the pool (and minimize context
switches).

Are you sure this is true? More specifically, while it's true that the
ThreadPool uses the CPU count to determine the number of threads, the
total thread count has always been much higher than the actual number of
CPUs. Off the top of my head, it used to be 25 threads per CPU, and now
it's 250 threads, or something like that.

I've never known the ThreadPool to be a particularly good manager of
context-switching. :)

> Also, are you sure the context switches are causing a performance
impact? If you switch to less threads, how do you kno you will get the
same concurrency that you have now?

A very good question. :)

Pete

[Ryan Liu]

"Peter Duniho" <Np*********@nn owslpianmk.comÐ ´ÈëÏûÏ¢ÐÂÎÅ:op* **************@ petes-computer.local. ..

On Sun, 01 Jun 2008 17:48:11 -0700, Ryan Liu <rl**@powercati .comwrote:

>I have a client/server application, using one thread/client approach.

I see very high context switch/sec. What are the ways to reduce it? Each
thread sleep longer in its endless loop if not reducing thread?

On the same machine, I also run a mysql server. I can see same amount
thread
in mysqld, seems mysql is also using one thread/client. But its
context/swith is much lower. How can I get same performance?

The simple answer is: discard the "one thread/client" design. It doesn't
scale well on Windows.

Use the asynchronous API on the i/o classes (especially network i/o, which
I assume is what you're talking about), and that will take advantage of
the built-in mechanisms in Windows designed to prevent excessive context
switching and minimizing thread count (i/o completion ports). For
example, use Socket.BeginRec eive, Stream.BeginRea d, etc. (with matching
"End..." methods, of course).

I'm surprised to hear that MySql uses "one thread/client", and I'm curious
how you've confirmed this. Regardless, it's not possible to answer "how
can I get the same performance" without knowing the specific
implementation details for MySql, and of course knowing the specific
implementation details would tell you directly how to get the same
performance. :)

Finally, of course, you should ask yourself whether it's important. If
you only have a small number of clients and performance is acceptable,
then a high number of context switches may not be a problem. Yes, it
might mean the design could be more efficient, but if the code works and
performs up to your needs, complicating the design may not be the best
idea.

Pete

Thanks Pete!

I used asynchronous API before, but I was told by my customer, the server
stops itself after run for 15 minutes.

Instead write a while(!Stop) in each thread, when I use async approach, I
call BeginRead() again in each call back method of EndRead, to create an
endless loop.

Maybe wrong idea: I also worry will asynchronous read/write make the server
not response in timely manner?

I have 120 client machines connect to one server for the C/S application.
MySql runs on the same server machine. Each client keep one connection to
the db server as well. Clients and the server exchange very small data, but
very frequently.

I use mysql command "show full processlist", and I can see about 120
threads. But about 119 are sleeping.

The applications runs well for most of time, but once a while, server gets
very busy and slow.

Thanks again!

[Rene]

Hey Nicholas, you are back!!

Where were you hiding? Did you take a looooooooong vacation? :)

It’s great to have you back!

"Nicholas Paldino [.NET/C# MVP]" <mv*@spam.guard .caspershouse.c omwrote in
message news:uk******** ******@TK2MSFTN GP06.phx.gbl...

Ryan,

Generally, you are going to have context switches when you have more
threads that are running than there are processors to handle the threads.

How are you allocating the threads? Generally, you should probably use
the ThreadPool to assign tasks to your threads, as the ThreadPool will
take into account the number of processors on the machine to determine how
many threads to keep in the pool (and minimize context switches).

Also, are you sure the context switches are causing a performance
impact? If you switch to less threads, how do you kno you will get the
same concurrency that you have now?
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard. caspershouse.co m

"Ryan Liu" <rl**@powercati .comwrote in message
news:%2******** ********@TK2MSF TNGP06.phx.gbl. ..

>Hi,

I have a client/server application, using one thread/client approach.

I see very high context switch/sec. What are the ways to reduce it? Each
thread sleep longer in its endless loop if not reducing thread?

On the same machine, I also run a mysql server. I can see same amount
thread in mysqld, seems mysql is also using one thread/client. But its
context/swith is much lower. How can I get same performance?

Thanks a lot!
Ryan

[Peter Duniho]

On Sun, 01 Jun 2008 18:57:54 -0700, Ryan Liu <rl**@powercati .comwrote:

More info:

I use performance counter fount that:

Threads used by mysql server on the same machine, context/switches is
abut
5-15 per second.
Mine thread, very silly, is about 43,384,322 - 4,295,039,822

How CPUs do you have? I have a hard time imagining how you could possibly
have 4 billion context switches in a second. Even 43 million seems
implausible if your polling thread (assuming you have one...you haven't
confirmed that yet) sleeps for 10ms each loop.

>
And as for I/O data bytes/sec , mysql is 350,000 to 439,951. My server
application is 0.(Suprise, I need check this again.)

And my application % process time is lower than mysql as well at the
moment
I watch.

And whole syste ave disk queue length is vary greatly, from 1.5 to
15.845.

And system-threads average is 881,049, max is 29,934,599. So high!

What OS? On 32-bit Windows, I think the maximum number of threads,
assuming nothing else competing for resources, is something like 1000 per
process. Without an enormous number of processes, it's hard to see how
you could get up to 30 million threads (especially assuming that most
processes aren't themselves hosting an exorbitant number of threads).

On 64-bit Windows, my recollection is that the max is much higher of
course. In any case, I'd agree that even nearing 1 million threads seems
wrong. But again, how many CPUs is there on the hardware?

I reiterate: you really should move to the async API.

Pete

[Peter Duniho]

On Sun, 01 Jun 2008 18:42:44 -0700, Ryan Liu <rl**@powercati .comwrote:

[...]
I used asynchronous API before, but I was told by my customer, the
server
stops itself after run for 15 minutes.

I don't know that that means. Were you instructed by your customer to
make the server stop after 15 minutes of execution? Or did your customer
complain to you that the server would only run after 15 minutes of
execution? In either case, how is that relevant to the current topic.

Instead write a while(!Stop) in each thread, when I use async
approach, I
call BeginRead() again in each call back method of EndRead, to create an
endless loop.

Correct. You can abort a BeginRead() that hasn't completed yet by closing
the i/o object (e.g. call Stream.Close()) . Then the i/o will complete
with an exception thrown when you call EndRead(). In that way you can
detect the closure of the object and do whatever cleanup you need to,
rather than calling BeginRead() again.

Maybe wrong idea: I also worry will asynchronous read/write make the
server
not response in timely manner?

It should have the exact opposite effect. Using IOCP is the best way to
reduce CPU overhead and make the server run efficiently.

I have 120 client machines connect to one server for the C/S application.
MySql runs on the same server machine. Each client keep one connection
to
the db server as well. Clients and the server exchange very small data,
but
very frequently.

I use mysql command "show full processlist", and I can see about 120
threads. But about 119 are sleeping.

Well, the fact is that there are other things about your own
implementation that, at least without further clarification from you, seem
particularly troubling. It's certainly theoretically possible to manage
120 clients with a "one thread/client" implementation, but only if you do
it correctly.

If you are in fact not dedicating a single thread to a particular client's
read and write both, but rather (as it seems from the vague information
you've provided so far) polling for reads and queuing each write to the
thread pool individually, then that seems like that would in fact be one
of the worst ways to try to implement a "one thread/client" design.

The applications runs well for most of time, but once a while, server
gets
very busy and slow.

From what we've heard so far, that doesn't seem surprising to me. You
should at a minimum fix your design so that you have a dedicated thread
for each client (don't use the thread pool), where that dedicated thread
handles both reads and writes with the client. Preferably, you should
just go ahead and use the async API, to take advantage of the efficiency
that IOCP will get you.

Pete