Connecting using socket class to server: Scalability/Performance

[Bruce]

Hello
I am building a C# app that creates anywhere from 10 to 100
connections to a specified server and sends 1000s of TCP requests and
processes the responses. (it is a stress tool)
I planned to create a Socket object for each connection and do a
BeginReceive on each socket to handle the responses. (it is clean and
I can avoid the hassles of managing my own threads)

My question is, will this scale well? Or is there a more efficient way
to achieve this? (like having a single dedicated thread reading from
all socket streams and queueing the data to be processed by another
thread)

my code:

for(int i = 0; i < numberOfConnections; i++) {
Socket s = new Socket(.....);
s.Connect(serverEndpoint);
GetResponse(s);
}

GetResponse(Socket s) {
s.BeginReceive(...new AsyncCallback(OnReceived)...);
}

OnReceived(IAsyncResult ar) {
s.EndReceive();
GetResponse(s);
}

Send() {
<randomly chosen so\cket>.Send(...);
}

Thanks
Bruce

[Peter Duniho]

In article
<11**********************@e34g2000pro.googlegroups .com>Bruce
<br*************@gmail.comwrote:

Hello
I am building a C# app that creates anywhere from 10 to 100
connections to a specified server and sends 1000s of TCP requests
andprocesses the responses. (it is a stress tool)
I planned to create a Socket object for each connection and do a
BeginReceive on each socket to handle the responses. (it is clean
andI can avoid the hassles of managing my own threads)

My question is, will this scale well?

It should scale very well. The async Socket methods use IOCP when
supported, which is the most scalable i/o technique for sockets.
Chris Mullins has a great write-up on his observations and experience
using this method for his high-volume server code; if i recall
correctly, in that implementation hundreds of thousands of clients are
served with high efficiency.

I'd provide the link, but I don't have it handy. You may be able to
find it by searching the newsgroup, or he may see this thread and post
it himself. In any case, the basic technique of using the async
methods is a good one.

Pete

--
I'm trying a new usenet client for Mac, Nemo OS X.
You can download it at http://www.malcom-mac.com/nemo

[Bruce]

On Oct 28, 11:29 am, Peter Duniho <[e-mail]>
wrote:

In article
<[e-mail] .com>Bruce

<[e-mail]:

Hello
I am building a C# app that creates anywhere from 10 to 100
connections to a specified server and sends 1000s of TCP requests
andprocesses the responses. (it is a stress tool)
I planned to create a Socket object for each connection and do a
BeginReceive on each socket to handle the responses. (it is clean
andI can avoid the hassles of managing my own threads)
My question is, will this scale well?

It should scale very well. The async Socket methods use IOCP when
supported, which is the most scalable i/o technique for sockets.
Chris Mullins has a great write-up on his observations and experience
using this method for his high-volume server code; if i recall
correctly, in that implementation hundreds of thousands of clients are
served with high efficiency.

I'd provide the link, but I don't have it handy. You may be able to
find it by searching the newsgroup, or he may see this thread and post
it himself. In any case, the basic technique of using the async
methods is a good one.

Pete

--
I'm trying a new usenet client for Mac, Nemo OS X.
You can download it athttp://www.malcom-mac.com/nemo

<I posted this before,I don't know why it has not appeared after more
than 3 hours.>

I think I got Chris Dullins's thread ->
http://groups.google.com/group/micro...9268bf8635ea65

My main concern with this async socket approach is this - When I have
a large number of connections and a large number of receives/sends
happening, everytime an async receive/send is done, .NET has to find a
free thread from the threadpool to do the job for every asyncCallback.
at high rates, there should be a large number of context switches
across many threads (though the threads themselves do not do strenuous
jobs). Won't this lead to higher CPU and thereby be less efficient?
(In contrast, using a dedicated thread does not have this issue)

Thanks
Bruce

[Peter Duniho]

On 2007-10-28 18:57:16 -0700, Bruce <br*************@gmail.comsaid:

I think I got Chris Dullins's thread ->
http://groups.google.com/group/micro...9268bf8635ea65

That's

>

not it. (And it's Chris Mullins). The post I'm thinking of was some
time ago (more than a year) and contained a link to his write-up.

My main concern with this async socket approach is this - When I have
a large number of connections and a large number of receives/sends
happening, everytime an async receive/send is done, .NET has to find a
free thread from the threadpool to do the job for every asyncCallback.
at high rates, there should be a large number of context switches
across many threads (though the threads themselves do not do strenuous
jobs). Won't this lead to higher CPU and thereby be less efficient?
(In contrast, using a dedicated thread does not have this issue)

No. The whole point of IOCP is to deal with exactly that issue. The
underlying network i/o driver signals via the completion port that some
i/o has finished. Some specific number of threads are assigned to
dequeue i/o completions via the completion port. Completed i/o
operations are preferentially handled by a thread that's already
running, avoiding context switches if possible.

This number of threads is chosen to balance the needs of
responsiveness, the number of available CPUs and other factors (for
example, other i/o bandwidth issues to ensure the threads don't compete
with each other for disk i/o, etc.) On the whole, however, having some
relatively small number of threads, slightly more than the number of
CPUs, is a nice number.

One of the biggest problems with having a single thread handling all of
the i/o is, how to do decide which socket the i/o happened on? IOCP is
the way to most efficiently manage this, as well as the threading
issues, and that's exactly how the async methods on the Socket class
are implemented.

There's a reason they use IOCP for those methods. It wasn't just an
arbitrary decision.

Pete

[Bruce]

On Oct 28, 7:10 pm, Peter Duniho <[e-mail]:

On 2007-10-28 18:57:16 -0700, Bruce <[e-mail]:

I think I got Chris Dullins's thread ->
http://groups.google.com/group/micro...languages.csha...

That's

not it. (And it's Chris Mullins). The post I'm thinking of was some
time ago (more than a year) and contained a link to his write-up.

My main concern with this async socket approach is this - When I have
a large number of connections and a large number of receives/sends
happening, everytime an async receive/send is done, .NET has to find a
free thread from the threadpool to do the job for every asyncCallback.
at high rates, there should be a large number of context switches
across many threads (though the threads themselves do not do strenuous
jobs). Won't this lead to higher CPU and thereby be less efficient?
(In contrast, using a dedicated thread does not have this issue)

No. The whole point of IOCP is to deal with exactly that issue. The
underlying network i/o driver signals via the completion port that some
i/o has finished. Some specific number of threads are assigned to
dequeue i/o completions via the completion port. Completed i/o
operations are preferentially handled by a thread that's already
running, avoiding context switches if possible.

This number of threads is chosen to balance the needs of
responsiveness, the number of available CPUs and other factors (for
example, other i/o bandwidth issues to ensure the threads don't compete
with each other for disk i/o, etc.) On the whole, however, having some
relatively small number of threads, slightly more than the number of
CPUs, is a nice number.

One of the biggest problems with having a single thread handling all of
the i/o is, how to do decide which socket the i/o happened on? IOCP is
the way to most efficiently manage this, as well as the threading
issues, and that's exactly how the async methods on the Socket class
are implemented.

There's a reason they use IOCP for those methods. It wasn't just an
arbitrary decision.

Pete

That was a lucid explanation. Thanks Pete. I think I should read up
more on IOCP and how they exactly work. With a single thread, I
thought of looping through all the sockets to read from the network
stream, but this definitely seems like a cleaner and easier approach.
My only concern was performance, but as you mentioned, I guess it is
not a hit after all. I will incorporate this in my tool and hope it
scales well!

Bruce

[Bruce]

On Oct 28, 11:29 am, Peter Duniho <[e-mail]>
wrote:

In article
<[e-mail] .com>Bruce

<[e-mail]:

Hello
I am building a C# app that creates anywhere from 10 to 100
connections to a specified server and sends 1000s of TCP requests
andprocesses the responses. (it is a stress tool)
I planned to create a Socket object for each connection and do a
BeginReceive on each socket to handle the responses. (it is clean
andI can avoid the hassles of managing my own threads)
My question is, will this scale well?

It should scale very well. The async Socket methods use IOCP when
supported, which is the most scalable i/o technique for sockets.
Chris Mullins has a great write-up on his observations and experience
using this method for his high-volume server code; if i recall
correctly, in that implementation hundreds of thousands of clients are
served with high efficiency.

I'd provide the link, but I don't have it handy. You may be able to
find it by searching the newsgroup, or he may see this thread and post
it himself. In any case, the basic technique of using the async
methods is a good one.

Pete

--
I'm trying a new usenet client for Mac, Nemo OS X.
You can download it athttp://www.malcom-mac.com/nemo

Thanks for the response. I think i found Chris Mullins's link (this
one? http://developers.coversant.net/Down...3/Default.aspx)
My main concern with this async socket style for a lot of connections
is thread contention. For every action, BeginAccept/BeginReceive,
callbacks etc, .NET needs to find a free thread in the threadpool, use
the thread to execute the action, put the thread back in the
threadpool. Given that there are a lot of sends and receives this
should lead to a lot of context switches between the threads (though
the work they do is not strenuous), leading to higher CPU. In
contrast, having a single dedicated thread will prevent this overhead.

Bruce