Thread Pool versus Dedicated Threads

"gpderetta" <gp*******@gmai l.comwrote in message
news:5b******** *************** ***********@59g 2000hsb.googleg roups.com...

On Aug 16, 7:47 am, "Chris M. Thomasson" <n...@spam.inva lidwrote:

>"Ian Collins" <ian-n...@hotmail.co mwrote in message

news:6g******* ******@mid.indi vidual.net...

Chris M. Thomasson wrote:

>"Chris Becke" <chris.be...@gm ail.comwrote:

>>Microsoft Windows needs to allocate stack space for each thread
created. On the 32bit version of >the OS then, this means an
immediately scalibility problem :- with only 2Gb of address space per
process, this implies a hard limit of 2048 connections (threads) per
server. Even on a 64bit OS >the working set added to the process for
each thread means that phsyical hardware limits will be >reached that
much faster than a system that uses asynchronous IO to keep lots of
connections on >one thread.

>I have personally created IOCP servers on Windows which can handle
__well__ over 40,000 connections; want some tips?

But I'd bet several gallons for my favourite beer that you didn't
create
40,000 threads!

I only created around 2 * N threads for the IOCP treading pool, where N
is
the number of processors in the system. I did create a couple of more
threads whose only job was to perform some resource maintenance tasks...

The one thread per connection model simply isn't scalable beyond a
handful of threads per core.

Right. Well, I guess you could use one user-thread (e.g. fiber)
per-connection and implement your own scheduler. The question is why in
the
world would you do that on Windows when there is the wonderful and
scalable
IOCP mechanism to work with...

You can of course use user-threads on top of IOCP and get the best of
both worlds.

Sure. I guess you would use an IOCP thread as the actual scheduler for the
fibers within it. When an IO completeion is encountered, you extract the
fiber context from the completeion key and simply switch to that fiber. When
the fiber does its thing, it switches back to the IOCP thread. Something
like:

// pseudo-code
struct per_io {
OVERLAPPED ol;
char buf[1024];
DWORD bytes;
int action;
BOOL status;
};

struct per_socket {
SOCKET sck;
void* fiber_socket_co ntext;
void* fiber_iocp_cont ext;
struct per_io* active_io;
};
DWORD WINAPI iocp_entry(LPVO ID state) {
for (;;) {
struct per_io* pio = NULL;
struct per_socket* psck = NULL;
DWORD bytes = 0;
BOOL status = GQCS(...,
&bytes,
...,
(LPOVERLAPPED)& pio,
(PULONG_PTR)&ps ck,
INFINITE);
pio->status = status;
psck->active_io = pio;
SwitchToFiber(p sck->fiber_socket_c ontext);
}
return 0;
}
VOID WINAPI per_socket_entr y(LPVOID state) {
struct per_socket* const _this = state;
for (;;) {
struct per_io* const pio = _this->active_io;
switch (pio->action) {
case ACTION_RECV:
[...];
break;
case ACTION_SEND:
[...];

[whatever...];
}
}
}

BTW, a good reference on the topic of (web) server scalability:

http://www.kegel.com/c10k.html

(I guess many here know this page).

Indeed.

On Aug 17, 5:18 am, "Chris M. Thomasson" <n...@spam.inva lidwrote:

"gpderetta" <gpdere...@gmai l.comwrote in message

You can of course use user-threads on top of IOCP and get the best of
both worlds.

I jumped the gun here before actually working out a solution... Now that I
think about it some more, well, this scheme may not work after all. The
problem is that fibers are bound to specific threads for their lifetime.

Hum as far as I understand from the win32 documentation, fibers are
allowed to migrate from one thread to another:

"You can call SwitchToFiber with the address of a fiber created by a
different thread. To do this, you must have the address returned to
the other thread when it called CreateFiber and you must use proper
synchronization . "

(in fact I think you also need some appropriate compiler flags to
disable some TLS related optimizations)

Of course the problem is proper sinchronization :

However, IOCP completions can allow a socket to receive completions on
different threads. Think about it. If a socket issues two overlapped io
operations, well, those completions may come in on two different threads.
How would you use fibers in this scenario?

Hum don't do two overlapped operations linked to the same fiber
then :).

More seriously, the problem is making sure never to wake up a fiber if
it is already running and never go to sleep if there is any ready
operation not already acknowledged for.

For example, for every asynchronous operation posted, you could
increment a counter ; when an operation complete, you decrement the
counter and check if the fiber is not awake, if not, mark the fiber as
awake and run it (or put it in a ready queue); when you stop a fiber,
you first suspend it the atomically check for ready operations
operations pending and mark it as sleeping. If there are ready
operations, you abort the suspend and restart the fiber. Getting
things right without missing wakeups is not trivial.

You need to synchronize access to the counter and fiber state, of
course, but you would need to synchronize access to any state attached
to the socket anyway , so IMHO it doesn't make much of a difference.
In fact I'm sure you can come up with some scheme that actually
doesn't require locks.

The only way I can see it working
is if you created a IOCP handle for each io processing thread, which defeats
the purpose of IOCP in the first place. Therefore, I conclude that fibers
and IOCP will _not_ work well together as-is...

Even in the one IOCP per thread, you are no worse than unix select and
friends: IOCP is still a pretty good reactor and will still scale way
better than using the one thread per connection model; you lose the
benefit of having the OS control the optimal number of running
threads, but but you gain by not needing synchronization and better
cache locality.

--
gpd

"gpderetta" <gp*******@gmai l.comwrote in message
news:83******** *************** ***********@25g 2000hsx.googleg roups.com...

On Aug 17, 5:18 am, "Chris M. Thomasson" <n...@spam.inva lidwrote:

>"gpderetta" <gpdere...@gmai l.comwrote in message

You can of course use user-threads on top of IOCP and get the best of
both worlds.

I jumped the gun here before actually working out a solution... Now that
I
think about it some more, well, this scheme may not work after all. The
problem is that fibers are bound to specific threads for their lifetime.

Hum as far as I understand from the win32 documentation, fibers are
allowed to migrate from one thread to another:

"You can call SwitchToFiber with the address of a fiber created by a
different thread. To do this, you must have the address returned to
the other thread when it called CreateFiber and you must use proper
synchronization . "

[...]

Okay. I just thought that any thread which created fibers would end up
destroying said fibers when it gets terminated (e.g., returning from initial
thread function). This is where I got the term "fibers are bound to specific
threads". Please note that I never used fibers in any of my code in any way
shape or form! I love to learn new things!

You can read the following:

http://developer.amd.com/documentati...031200677.aspx

And tell me where I am going wrong.

:^)

On Aug 24, 12:17 pm, "Chris M. Thomasson" <n...@spam.inva lidwrote:

"gpderetta" <gpdere...@gmai l.comwrote in message

news:83******** *************** ***********@25g 2000hsx.googleg roups.com...

On Aug 17, 5:18 am, "Chris M. Thomasson" <n...@spam.inva lidwrote:

"gpderetta" <gpdere...@gmai l.comwrote in message

You can of course use user-threads on top of IOCP and get the best of
both worlds.

I jumped the gun here before actually working out a solution... Now that
I
think about it some more, well, this scheme may not work after all. The
problem is that fibers are bound to specific threads for their lifetime.

Hum as far as I understand from the win32 documentation, fibers are
allowed to migrate from one thread to another:

"You can call SwitchToFiber with the address of a fiber created by a
different thread. To do this, you must have the address returned to
the other thread when it called CreateFiber and you must use proper
synchronization . "

[...]

Okay. I just thought that any thread which created fibers would end up
destroying said fibers when it gets terminated (e.g., returning from initial
thread function). This is where I got the term "fibers are bound to specific
threads". Please note that I never used fibers in any of my code in any way
shape or form! I love to learn new things!

You can read the following:

http://developer.amd.com/documentati...031200677.aspx

As far as I can tell, the article only says that the fiber currently
running on a specific thread, is destroyed when the tread terminates
(which is the behavior I would expect).
Thinking of possible implementations of fibers, I see no reason to
kill all fibers generated by a specific thread when that thread exit
(in fact it would be quite expensive and require a lot of book-keeping
to do).

In the event you are right with fibers getting destroyed at
inconvenient time, it is always possible to make your own 'fiber-like'
abstraction without such a limitation (and you know ASM well enough to
do it :) )

To keep this more C++ related: the (tentative) boost.coroutine library
uses fibers internally on win32, but for portability reasons, it
doesn't allow thread migration. You can look there for a possible
custom implementation of fibers (or threadlets or whatever you want to
call them).

--
gpd