Building a Scalable (100K+ User) Socket Server in .Net

[Chris Mullins]

One of the things I've spent the last several years working on is a highly
scalable socket server written in C#.

The SoapBox Server has recently been tested to well over 100k simultanous
users. This means it's handling 100k TCP connections on a Windows Platform
without any trouble at all.

I finally got around to writing a blog post that describes the different
architectures we've gone through, and the pros and cons of each. All of
these architectures scale fairly well, but there have been some surprising
results. These architectures have included using the system threadpool,
managing our own thread pool, doing work on the IOCP threads, and a few
other things thrown in for good luck.

I've seen so many people asking for guidance in this area, and I hope this
serves people as a good starting point.

http://makeashorterlink.com/?R2EE5246D

All feedback is welcome, either here or on the blog...

--
Chris Mullins
Coversant, Inc.

[Carl Daniel [VC++ MVP]]

Chris Mullins wrote:

One of the things I've spent the last several years working on is a
highly scalable socket server written in C#.

The SoapBox Server has recently been tested to well over 100k
simultanous users. This means it's handling 100k TCP connections on a
Windows Platform without any trouble at all.

I finally got around to writing a blog post that describes the
different architectures we've gone through, and the pros and cons of
each. All of these architectures scale fairly well, but there have
been some surprising results. These architectures have included using
the system threadpool, managing our own thread pool, doing work on
the IOCP threads, and a few other things thrown in for good luck.

I've seen so many people asking for guidance in this area, and I hope
this serves people as a good starting point.

http://makeashorterlink.com/?R2EE5246D

All feedback is welcome, either here or on the blog...

Thanks for sharing - great article!

-cd

[Chris Mullins]

"Carl Daniel [VC++ MVP]" wrote

> [Writing a Scalable Socket Server]
http://makeashorterlink.com/?R2EE5246D

Thanks for sharing - great article!

Any suggestions for how to improve things? The great thing about UseNet is
the wealth of knowledge out here, people wise. Getting hold of that
knowledge can be tough, but please, if you have suggestions fire away!

--
Chris Mullins
http://www.coversant.net/blogs/cmullins

[Carl Daniel [VC++ MVP]]

"Chris Mullins" <cm******@yahoo.comwrote in message
news:Og**************@TK2MSFTNGP03.phx.gbl...

"Carl Daniel [VC++ MVP]" wrote

>> [Writing a Scalable Socket Server]
http://makeashorterlink.com/?R2EE5246D

>Thanks for sharing - great article!

Any suggestions for how to improve things? The great thing about UseNet is
the wealth of knowledge out here, people wise. Getting hold of that
knowledge can be tough, but please, if you have suggestions fire away!

I think you're to the (really) hard part now - do as SQL Server does (as
discussed in a previous thread). You need to keep your threads from being
blocked and avoid context switches like the plague. Unfortunately, ADO.NET
doesn't help you out a lot there. It'd be interesting to see (as you've
already commented) how much more you'd get out at the high end just by using
lockfree queues on high-end systems while sticking with the locking queue on
low(er) end systems.

-cd

[Chris Mullins]

"Carl Daniel [VC++ MVP]" wrote:

It'd be interesting to see (as you've already commented) how much more
you'd get out at the high end just by using lockfree queues on high-end
systems while sticking with the locking queue on low(er) end systems.

My next topic will probably be just that. I wrote a mean little program that
is contention bound, and uses all the different types of locks. I ran this
on single x86, dualx86, single x64, dual x64, dual x64 with hyperthreading,
dual itanium2, quad itanium2, and 16 processor itanium2.

The resulting graphs are.... interesting.

Now I just need to find time to write that up...

--
Chris Mullins
http://www.coversant.net/blogs/cmullins

[Jon Skeet [C# MVP]]

Chris Mullins <cm******@yahoo.comwrote:

"Carl Daniel [VC++ MVP]" wrote:

It'd be interesting to see (as you've already commented) how much more
you'd get out at the high end just by using lockfree queues on high-end
systems while sticking with the locking queue on low(er) end systems.

My next topic will probably be just that. I wrote a mean little program that
is contention bound, and uses all the different types of locks. I ran this
on single x86, dualx86, single x64, dual x64, dual x64 with hyperthreading,
dual itanium2, quad itanium2, and 16 processor itanium2.

The resulting graphs are.... interesting.

Now I just need to find time to write that up...

If you have the time to run the tests with my "feature-rich" locks, I'd
be really interested in seeing the results. See
http://www.pobox.com/~skeet/csharp/m...e/locking.html for
usage, download link etc.

It sounds like they wouldn't be appropriate for your situation, but it
would be good to get more diverse stats on the penalty for using my
code over the "plain" locking.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Chris Mullins]

"Jon Skeet [C# MVP]" <sk***@pobox.comwrote

>
If you have the time to run the tests with my "feature-rich" locks, I'd
be really interested in seeing the results. See
http://www.pobox.com/~skeet/csharp/m...e/locking.html for
usage, download link etc.

I recently (just this week) took a long and deep look through Jeff Richters
"Power Threading" library. It's online at:
http://www.wintellect.com/Resources.aspx
(signing up for an account is free, then you can download the source).

As somebody who takes quite a bit of pride in my knowledge of multi-threaded
code, I have to say I was humbled.

I learned more about locking in 4 hours this past tuesday evening than I've
leared in a long, long time - possibly ever. Some of his source code
commends were... eye opening.
// NOTE: Just reading here (as compared to repeatedly calling Exchange)
// improves performance because writing forces all CPUs to update this value

// Don't do variable = value because reordered by compiler and/or CPU

There was stuff like this throughout - as well as stuff that I wish I could
wrap my head arond.

I quite liked his abstract ResourceLock implementation, and the concrete
variations on it.

Unfortunatly I can't use this library, as the licensing of it prevents it
from being run with Mono. I'll have to send him an email and see if I can
get a relaxed license version.

--
Chris Mullins

[Carl Daniel [VC++ MVP]]

Chris Mullins wrote:

"Jon Skeet [C# MVP]" <sk***@pobox.comwrote

>>
If you have the time to run the tests with my "feature-rich" locks,
I'd be really interested in seeing the results. See
http://www.pobox.com/~skeet/csharp/m...e/locking.html for
usage, download link etc.

I recently (just this week) took a long and deep look through Jeff
Richters "Power Threading" library. It's online at:
http://www.wintellect.com/Resources.aspx
(signing up for an account is free, then you can download the source).

As somebody who takes quite a bit of pride in my knowledge of
multi-threaded code, I have to say I was humbled.

I learned more about locking in 4 hours this past tuesday evening
than I've leared in a long, long time - possibly ever. Some of his
source code commends were... eye opening.
// NOTE: Just reading here (as compared to repeatedly calling
Exchange) // improves performance because writing forces all CPUs to
update this value
// Don't do variable = value because reordered by compiler and/or CPU

There was stuff like this throughout - as well as stuff that I wish I
could wrap my head arond.

I quite liked his abstract ResourceLock implementation, and the
concrete variations on it.

Unfortunatly I can't use this library, as the licensing of it
prevents it from being run with Mono. I'll have to send him an email
and see if I can get a relaxed license version.

Cool library - definitely some good stuff in there. Kinda short on
documentation though :) Makes for more interesting reading that way, I
guess.

-cd

[Barry Kelly]

"Chris Mullins" <cm******@yahoo.comwrote:

I recently (just this week) took a long and deep look through Jeff Richters
"Power Threading" library. It's online at:
http://www.wintellect.com/Resources.aspx
(signing up for an account is free, then you can download the source).

As somebody who takes quite a bit of pride in my knowledge of multi-threaded
code, I have to say I was humbled.

Yes, memory models are mind-bending - especially when you bring compiler
optimizations into mind as well. Are we really certain that the JIT
compiler respects Thread.MemoryBarrier(), or does it just affect the
CPU? :) It's certainly enough to scare you away from playing with
sophisticated lock-free programming unless you've got a lot of time and
hardware to play with to prove and test correctness.

-- Barry

--
http://barrkel.blogspot.com/