Multi-threaded TCP/IP Server without .NET Socket Class

[kodart]

Introduction

Performance is the main concern to most server application developers. That’s why many of them anticipate using .NET platform to develop high performance server application regardless of the security features it provides.

Microsoft Windows provides a high performance model that uses I/O completion port (IOCP) to process network events. IOCP provides best performance, but difficult to use due to lack of good code samples and obligatory use of C/C++ language. From the other side, if server’s business logic is not trivial then it becomes difficult to support and test it. Finding a bug in C++ code may become a serious challenge for the developers.

What is the solution?

Ideally we would like to have a server application that has a performance comparable to C++ server and well designed, bugs free, unit tested business logic written in C#. Yeah, that would be great!

Let’s look at what .NET platforms offers for the network operations.

Socket class which supports asynchronous operations should be great for the purpose of writing a scalable high perfomance server. According to the documentation asynchronous network operation methods (BeginConnect, BeginReceive, BeginSend) support I/O completion port, but simple benchmarking showed that the performance is still very low. Socket class is very resource consuming when used asynchronously.

In addition to design and implementation issues[i] of several .NET classes, server written completely in .NET has a performance lack due to lots of managed/unmanaged code transitions and data marshaling which happen each time when an underlying system call is made (network I/O, threads synchronization, etc.). The only solution is to minimize those transitions. This approach has been used in XF.Server component which was recently released for community preview and available for public download (http://www.kodart.com/)

In XF.Server component managed/unmanaged code transition happens only when a message should be passed between business logic and transport logic layer. XF.Server handles all low level network I/O providing a client with an interface to work with network operations.

Now let’s write a high performance simple web server using C#, so that we could compare the performance with another web server, for example IIS or Apache (written in C/C++)

[i] http://msdn.microsoft.com/msdnmag/is...g/Default.aspx

What will our web server do?

Simple web server that we create will accept client connections, read http request and reply with a response containing the client’s request. As soon as a request is served the connection is closed. Web server response will be formatted in the following way:

private static readonly string responseFormat = 
            "HTTP/1.1 200 OK\r\n"+
            "Server: XF.HTTP/0.1\r\n" +
            "Content-Type: text/plain\r\n" +
            "Content-Length: {0}\r\n"+
            "Connection: Close\r\n\r\n{1}";

Please note that XF.HTTP is a planned open source project of the Kodart Technologies that may leave the competitor behind if it includes the features of the other products without degradation in the performance. Actually this article contains the source code of XF.HTTP server prototype, which was gladly provided by Kodart Technologies.
Code review

Ok, let’s return to the code. XF.Server component was developed in attention to support unit testing of the server’s business logic. That’s why server and connection objects are passed using the interfaces, which you can easily mock to test the logic.

internal static IServer server;
static void Main()
{
   server = new Server();
   StartServer();
   Console.WriteLine("Web Server is Ready. Press any key to exit.");
   Console.ReadKey(true);
   StopServer();
}
 
internal static void StartServer()
{
   server.OnConnect += OnClientConnect;
   /* Start the server listening on port 80 */
   server.Start(80);
}
 
internal static void StopServer()
{
   server.Stop();
}

Because it is only a demo http server, we don’t expect a request to be longer than 1024 bytes. If it is longer then it will be truncated.

internal static void OnClientConnect(IConnection conn)
{
   /* Allocate a buffer for read operation per client */
   byte[] buffer = new byte[1024];
   conn.ReadAsync(buffer, buffer.Length, OnReadComplete, null);
}

Completion routine mechanism in XF.Server has a great optimization; if you recall how asynchronous network programming model works in .NET you will remember that in completion routine you would repeat the asynchronous call. But that would again involve many operations with buffers locking, getting function pointer for callback delegate, etc.

In XF.Server you just say that you do/don’t want to repeat the operation using the Repeat flag in the OperarionArgs object.

internal static void OnReadComplete(OperationArgs args)
{
   /* If the connection is not closed */
   if (args.Bytes != 0)
   {
      string content = Encoding.ASCII.GetString(args.Buffer).Substring(0, args.Bytes);
      string response = String.Format(responseFormat, content.Length, content);
      args.Connection.WriteAsync(Encoding.ASCII.GetBytes(response), response.Length, OnWriteComplete, null);
      args.Repeat = false;
   }
}
 
internal static void OnWriteComplete(OperationArgs args)
{
   args.Connection.Close();
}

That's it. Just a few lines of code and a web server is ready.

Performance results

Our web server application and IIS/6.0 were tested using the same platform (Intel Core2 Duo, 2GB RAM) with Microsoft Web Application Stress Tool. Client and server were on different machines connected using 100MB/S LAN.

In order to make the performance testing fair, IIS/6.0 was requested to return a static content, short text file. IIS is supposed to cache frequently used content, that’s why we don’t consider disk I/O overhead. If IIS does not cache, then it is a big question to IIS developers ;)

The test has been run during one minute and the following results were received:

IIS/6.0 Results:
Requests/Second: 1338.14 
 
Socket Errors
Connect:0
Send:1
Recv:3
Timeout:0
 
----
 
 XF.HTTP Results:
Requests/Second: 3841.92
 
Socket Errors
Connect:0
Send:0
Recv:0
Timeout:0

The results are very surprising. Please check also CPU usage during the tests and you will notice that IIS’s CPU usage is not stable and always jumps between 20-80%.
In contrast XF.HTTP’s CPU usage is very stable and changes between 70-80% that denotes a better server design.

Conclusion

Regardless that XF.Server is still in development and has been only CTP released, you should consider it if you are writing server application or want to improve the existing .NET server application.
Using XF.Server component and several lines of code we managed to write a simple web server that can handle thousands of concurrent connections effectively.

XF.Server component provides performance that no one has offered before. All socket components that offer 300-500 requests/sec are far behind even with the CTP release of XF.Server.

Please post your questions and comments if you know a component that can provide better performance.
Please do the benchmarking and comment here if you have different results.

[mkhazbak]

It looks great with few exceptions:
1- Connection class is very poor in features. You cannot configure which IP to listen on, the backlog size, etc..
2- Source code is not yet available. Enterprise license is not available on kodart website

[kodart]

It looks great with few exceptions:
1- Connection class is very poor in features. You cannot configure which IP to listen on, the backlog size, etc..
2- Source code is not yet available. Enterprise license is not available on kodart website

New version of XF.Server has more features. Now you can configure listener.