Hi all,
Recently I had a new coworker. There is some dispute between us.
The last company he worked for has a special networking programming
model. They split the business logic into different modules, and have
a dedicated thread for the each module. Modules exchanged info
through a in-memory message queue.
In my opinion, such a model means very complicated asynchronous
interaction between module. A simple function call between modules
would require a timer to avoid waiting for answer forever.
And if a module was blocked by IO (such as db query), other modules
depends on would have to wait for it.
For example, if module A want to query db, it would
1. save states in a list
2 .sending a message to db-adapter module (a thread dedicated for db
operation)
3. start a timer
4. if response message arrived on time, retrieve states from the
list, and go on
5. if timer fires, log an error message and cancel the operation ——
send an error notify to user……
My new coworker had written 300,000 lines of code in this model and
claimed this is the most simple way to write a network application.
He said we could implement a message queue in half-a day and message
would make interface much more clear.
I think if module interact with each other through function calls and
a thread/process pool model would be more easier, in which each
thread/
process has no dedicated job but handle whatever the master thread
give it.
But as I don't have much experience in this area, I am not quite
sure.
What do u think about it? Is there any successful projects that could
prove which model is **right**?
Aug 14 '08
23  4292 
>
"Chris Becke" <ch*********@gm ail.comwrote in message
news:12******** *******@vasbyt. isdsl.net...
>>This sort of design can ultimately be far better tuned to keep
the CPU cores as busy as possible, while minimising needless
context switches from having an "active" thread count far in
excess of CPU availability.
>>Do you have actual measurements from a real application to
support this claim. I doubt that it's true for most
application s.
>Microsoft Windows needs to allocate stack space for each thread created. On
the 32bit version of >the OS then, this means an immediately scalibility
problem :- with only 2Gb of address space per >process, this implies a hard
limit of 2048 connections (threads) per server. Even on a 64bit OS >the
working set added to the process for each thread means that phsyical
hardware limits will be >reached that much faster than a system that uses
asynchronous IO to keep lots of connections on >one thread.
I have personally created IOCP servers on Windows which can handle __well__
over 40,000 connections; want some tips?
Can anyone explain what is thread pool and dedicated pool ?
I have read billion of materials still not very understand.
Chris M. Thomasson wrote:
>>
"Chris Becke" <ch*********@gm ail.comwrote:
>Microsoft Windows needs to allocate stack space for each thread
created. On the 32bit version of >the OS then, this means an
immediately scalibility problem :- with only 2Gb of address space per
>process, this implies a hard limit of 2048 connections (threads) per
server. Even on a 64bit OS >the working set added to the process for
each thread means that phsyical hardware limits will be >reached that
much faster than a system that uses asynchronous IO to keep lots of
connections on >one thread.
I have personally created IOCP servers on Windows which can handle
__well__ over 40,000 connections; want some tips?
But I'd bet several gallons for my favourite beer that you didn't create
40,000 threads!
The one thread per connection model simply isn't scalable beyond a
handful of threads per core.
--
Ian Collins.
"Ian Collins" <ia******@hotma il.comwrote in message
news:6g******** *****@mid.indiv idual.net...
Chris M. Thomasson wrote:
>>>
"Chris Becke" <ch*********@gm ail.comwrote:
>>Microsoft Windows needs to allocate stack space for each thread
created. On the 32bit version of >the OS then, this means an
immediately scalibility problem :- with only 2Gb of address space per
process, this implies a hard limit of 2048 connections (threads) per
server. Even on a 64bit OS >the working set added to the process for
each thread means that phsyical hardware limits will be >reached that
much faster than a system that uses asynchronous IO to keep lots of
connections on >one thread.
I have personally created IOCP servers on Windows which can handle
__well__ over 40,000 connections; want some tips?
But I'd bet several gallons for my favourite beer that you didn't create
40,000 threads!
I only created around 2 * N threads for the IOCP treading pool, where N is
the number of processors in the system. I did create a couple of more
threads whose only job was to perform some resource maintenance tasks...
The one thread per connection model simply isn't scalable beyond a
handful of threads per core.
Right. Well, I guess you could use one user-thread (e.g. fiber)
per-connection and implement your own scheduler. The question is why in the
world would you do that on Windows when there is the wonderful and scalable
IOCP mechanism to work with...
On Aug 16, 6:08 am, Ian Collins <ian-n...@hotmail.co mwrote:
Chris M. Thomasson wrote:
"Chris Becke" <chris.be...@gm ail.comwrote:
Microsoft Windows needs to allocate stack space for each thread
created. On the 32bit version of >the OS then, this means an
immediately scalibility problem :- with only 2Gb of address space per
process, this implies a hard limit of 2048 connections (threads) per
server. Even on a 64bit OS >the working set added to the process for
each thread means that phsyical hardware limits will be >reached that
much faster than a system that uses asynchronous IO to keep lots of
connections on >one thread.
I have personally created IOCP servers on Windows which can handle
__well__ over 40,000 connections; want some tips?
But I'd bet several gallons for my favourite beer that you
didn't create 40,000 threads!
The one thread per connection model simply isn't scalable beyond a
handful of threads per core.
It depends. We're not at 40,000 connections yet, but we've
certainly more than a handful per core. And there's no problem
with the one thread per connection model for our application; in
fact, it would work better with two threads per connection
(one for push, and the other for pull). I've done a few tests
on Solaris, and there's no problem with thousands of threads.
It depends on what each connection is doing, and how long they
stay connected. (In our case, connections tend to last anywhere
between four and twelve hours. And of course, most of that
time, they are quiescent.)
--
James Kanze (GABI Software) email:ja******* **@gmail.com
Conseils en informatique orientée objet/
Beratung in objektorientier ter Datenverarbeitu ng
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
James Kanze wrote:
On Aug 16, 6:08 am, Ian Collins <ian-n...@hotmail.co mwrote:
>The one thread per connection model simply isn't scalable beyond a
handful of threads per core.
It depends. We're not at 40,000 connections yet, but we've
certainly more than a handful per core. And there's no problem
with the one thread per connection model for our application; in
fact, it would work better with two threads per connection
(one for push, and the other for pull). I've done a few tests
on Solaris, and there's no problem with thousands of threads.
That depends what they are doing, I've been hit by a thundering heard
with just 100 or so.
It depends on what each connection is doing, and how long they
stay connected. (In our case, connections tend to last anywhere
between four and twelve hours. And of course, most of that
time, they are quiescent.)
Ah, that explains it. I guess very few are blocking on the same
resource and you will have a very low rate of context switches. The
problems begin when the thread lifetime is short, the classic example
being a web server.
--
Ian Collins.
On Aug 16, 7:47 am, "Chris M. Thomasson" <n...@spam.inva lidwrote:
"Ian Collins" <ian-n...@hotmail.co mwrote in message
news:6g******** *****@mid.indiv idual.net...
Chris M. Thomasson wrote:
"Chris Becke" <chris.be...@gm ail.comwrote:
>Microsoft Windows needs to allocate stack space for each thread
created. On the 32bit version of >the OS then, this means an
immediately scalibility problem :- with only 2Gb of address space per
process, this implies a hard limit of 2048 connections (threads) per
server. Even on a 64bit OS >the working set added to the process for
each thread means that phsyical hardware limits will be >reached that
much faster than a system that uses asynchronous IO to keep lots of
connections on >one thread.
I have personally created IOCP servers on Windows which can handle
__well__ over 40,000 connections; want some tips?
But I'd bet several gallons for my favourite beer that you didn't create
40,000 threads!
I only created around 2 * N threads for the IOCP treading pool, where N is
the number of processors in the system. I did create a couple of more
threads whose only job was to perform some resource maintenance tasks...
The one thread per connection model simply isn't scalable beyond a
handful of threads per core.
Right. Well, I guess you could use one user-thread (e.g. fiber)
per-connection and implement your own scheduler. The question is why in the
world would you do that on Windows when there is the wonderful and scalable
IOCP mechanism to work with...
You can of course use user-threads on top of IOCP and get the best of
both worlds.
BTW, a good reference on the topic of (web) server scalability: http://www.kegel.com/c10k.html
(I guess many here know this page).
HTH,
--
gpd
On Aug 16, 12:47 pm, Ian Collins <ian-n...@hotmail.co mwrote:
James Kanze wrote:
On Aug 16, 6:08 am, Ian Collins <ian-n...@hotmail.co mwrote:
The one thread per connection model simply isn't scalable beyond a
handful of threads per core.
It depends. We're not at 40,000 connections yet, but we've
certainly more than a handful per core. And there's no problem
with the one thread per connection model for our application; in
fact, it would work better with two threads per connection
(one for push, and the other for pull). I've done a few tests
on Solaris, and there's no problem with thousands of threads.
That depends what they are doing, I've been hit by a
thundering heard with just 100 or so.
Exactly. If you're using threads to parallelize operations,
then too many will be counter-productive. If you're using them
to separate various concerns, it depends.
It depends on what each connection is doing, and how long they
stay connected. (In our case, connections tend to last anywhere
between four and twelve hours. And of course, most of that
time, they are quiescent.)
Ah, that explains it. I guess very few are blocking on the
same resource and you will have a very low rate of context
switches.
Probably. In our case, clients have to remain connected,
because we use both push and pull. And there is client
(connection) specific state, related to things like privileges.
There are many ways of handling this: I'm pretty sure that the
entire application could have been written in a single thread
without too many problems; alternatively, we could have used two
threads per connection (one for the push, and one for the pull).
Or any number of mixtures of this (a thread per connection for
the pull, but a single thread for the push).
The problems begin when the thread lifetime is short, the
classic example being a web server.
HTTP tends to be an example where a new thread per connection is
NOT a good idea. But there are a lot of other protocols out
there, and a lot of other client/server architectures. And I
suspect that writing C++ code to handle HTTP connections is
fairly rare: I would expect that most people would use existing
server (Apache, WebSphere, etc.) software, with JSP or something
similar for the dynamically generated contents. The C++ parts
would be the back-end engines, where the connections wouldn't
necessarily (but could) reflect the incoming connections.
--
James Kanze (GABI Software) email:ja******* **@gmail.com
Conseils en informatique orientée objet/
Beratung in objektorientier ter Datenverarbeitu ng
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
"Chris M. Thomasson" <no@spam.invali dwrote in message
news:lA******** *********@newsf e01.iad...
"gpderetta" <gp*******@gmai l.comwrote in message
news:5b******** *************** ***********@59g 2000hsb.googleg roups.com...
>On Aug 16, 7:47 am, "Chris M. Thomasson" <n...@spam.inva lidwrote:
>>"Ian Collins" <ian-n...@hotmail.co mwrote in message
news:6g****** *******@mid.ind ividual.net...
Chris M. Thomasson wrote:
"Chris Becke" <chris.be...@gm ail.comwrote:
Microsoft Windows needs to allocate stack space for each thread
created. On the 32bit version of >the OS then, this means an
immediatel y scalibility problem :- with only 2Gb of address space
per
process, this implies a hard limit of 2048 connections (threads)
per
server. Even on a 64bit OS >the working set added to the process for
each thread means that phsyical hardware limits will be >reached
that
much faster than a system that uses asynchronous IO to keep lots of
connection s on >one thread.
I have personally created IOCP servers on Windows which can handle
__well__ over 40,000 connections; want some tips?
But I'd bet several gallons for my favourite beer that you didn't
create
40,000 threads!
I only created around 2 * N threads for the IOCP treading pool, where N
is
the number of processors in the system. I did create a couple of more
threads whose only job was to perform some resource maintenance tasks...
The one thread per connection model simply isn't scalable beyond a
handful of threads per core.
Right. Well, I guess you could use one user-thread (e.g. fiber)
per-connection and implement your own scheduler. The question is why in
the
world would you do that on Windows when there is the wonderful and
scalable
IOCP mechanism to work with...
You can of course use user-threads on top of IOCP and get the best of
both worlds.
Sure. I guess you would use an IOCP thread as the actual scheduler for the
fibers within it. When an IO completeion is encountered, you extract the
fiber context from the completeion key and simply switch to that fiber.
When the fiber does its thing, it switches back to the IOCP thread.
Something like:
WHOOPS! I accidentally sent this to early! Retarded keypress... Anyway, I
needed to allow the per_socket fiber to switch back to the iocp fiber!!!
>
// pseudo-code
struct per_io {
OVERLAPPED ol;
char buf[1024];
DWORD bytes;
int action;
BOOL status;
};
struct per_socket {
SOCKET sck;
void* fiber_socket_co ntext;
void* fiber_iocp_cont ext;
struct per_io* active_io;
};
DWORD WINAPI iocp_entry(LPVO ID state) {
for (;;) {
struct per_io* pio = NULL;
struct per_socket* psck = NULL;
DWORD bytes = 0;
BOOL status = GQCS(...,
&bytes,
...,
(LPOVERLAPPED)& pio,
(PULONG_PTR)&ps ck,
INFINITE);
pio->status = status;
psck->active_io = pio;
psck->fiber_iocp_con text = state;
SwitchToFiber(p sck->fiber_socket_c ontext);
}
return 0;
}
VOID WINAPI per_socket_entr y(LPVOID state) {
struct per_socket* const _this = state;
for (;;) {
struct per_io* const pio = _this->active_io;
switch (pio->action) {
case ACTION_RECV:
[...];
break;
case ACTION_SEND:
[...];
[whatever...];
}
SwitchToFiber(_ this->fiber_iocp_con text);
}
}
>BTW, a good reference on the topic of (web) server scalability:
http://www.kegel.com/c10k.html
(I guess many here know this page).
Indeed.
"gpderetta" <gp*******@gmai l.comwrote in message
news:5b******** *************** ***********@59g 2000hsb.googleg roups.com...
On Aug 16, 7:47 am, "Chris M. Thomasson" <n...@spam.inva lidwrote:
>"Ian Collins" <ian-n...@hotmail.co mwrote in message
news:6g******* ******@mid.indi vidual.net...
Chris M. Thomasson wrote:
>"Chris Becke" <chris.be...@gm ail.comwrote:
>>Microsoft Windows needs to allocate stack space for each thread
created. On the 32bit version of >the OS then, this means an
immediately scalibility problem :- with only 2Gb of address space per
process, this implies a hard limit of 2048 connections (threads) per
server. Even on a 64bit OS >the working set added to the process for
each thread means that phsyical hardware limits will be >reached that
much faster than a system that uses asynchronous IO to keep lots of
connections on >one thread.
>I have personally created IOCP servers on Windows which can handle
__well__ over 40,000 connections; want some tips?
But I'd bet several gallons for my favourite beer that you didn't
create
40,000 threads!
I only created around 2 * N threads for the IOCP treading pool, where N
is
the number of processors in the system. I did create a couple of more
threads whose only job was to perform some resource maintenance tasks...
The one thread per connection model simply isn't scalable beyond a
handful of threads per core.
Right. Well, I guess you could use one user-thread (e.g. fiber)
per-connection and implement your own scheduler. The question is why in
the
world would you do that on Windows when there is the wonderful and
scalable
IOCP mechanism to work with...
You can of course use user-threads on top of IOCP and get the best of
both worlds.
I jumped the gun here before actually working out a solution... Now that I
think about it some more, well, this scheme may not work after all. The
problem is that fibers are bound to specific threads for their lifetime.
However, IOCP completions can allow a socket to receive completions on
different threads. Think about it. If a socket issues two overlapped io
operations, well, those completions may come in on two different threads.
How would you use fibers in this scenario? The only way I can see it working
is if you created a IOCP handle for each io processing thread, which defeats
the purpose of IOCP in the first place. Therefore, I conclude that fibers
and IOCP will _not_ work well together as-is...
What am I missing?
[...] This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: David Sworder |
last post by:
This message was already cross-posted to C# and ADO.NET, but I forgot to
post to this "general" group... sorry about that. It just occured to me
after my first post that the "general" group readers might have some
thoughts on this perplexing .NET blocking issue.
(see below)
=====
Hi,
|
by: John Doe |
last post by:
Hi all,
I know the standard doesn't care about threads (wrongly :-)
But in current compilers implementation, is the "list" which holds count
of the occupied and free heap addresses SHARED among various threads or not?
I don't know if I was clear:
Case1: the list which holds count of what is free and what is occupied
is SHARED among all the threads. In this case each
|
by: David Sworder |
last post by:
Hi,
I'm developing an application that will support several thousand
simultaneous connections on the server-side. I'm trying to maximize
throughput. The client (WinForms) and server communicate via a socket
connection (no remoting, no ASP.NET). The client sends a message to the
server that contains some instructions and the server responds in an
asynchronous fashion. In other words, the client doesn't block while waiting
for the...
|
by: Droopy Toon |
last post by:
Hi,
I am using asynchronous socket (BeginAccept for example).
I tried to name each thread I am using but threads created by
asynchronous Socket functions (like BeginAccept) creates "anonymous"
threads.
I named the thread (see sample code below) in Accept callback started by
BeginAccept but all connections accepted run in a thread that has the
same name !
So, even a new thread is not started by BeginAccept, even my naming is
|
by: admin |
last post by:
ok This is my main. Pretty much it goes through each category and
starts up 4 worker threads that then ask for groups to gether from. My
problem is that when the thread gets done it keeps the mysql
connections open so I end up with quite a few at the end. Is there a
different way that I should do this?
class Program
{
static string categories = { "emulation" , "audio" ,
"console" , "anime" , "xxx" , "tv" , "pictures" , "video" };
| | |
by: =?Utf-8?B?cmFuZHkxMjAw?= |
last post by:
I have an application with several BackgroundWorker threads. I hoped I'd be
able to just type backgroundworker1.Name = "bw1"; but I don't see a name
property.
Any thoughts on how to name a backgroundworker thread?
Thanks,
Randy
|
by: Creativ |
last post by:
Why does Thread class not support IDisposable? It's creating quite
some problem. Namely, it can exhaust the resource and you have not
control over it.
|
by: Curious |
last post by:
On Jun 10, 3:32 am, <s...@dailycoding.comwrote:
Thanks! I'll use thread pool.
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
| | |
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
|
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |
