By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
460,022 Members | 1,354 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 460,022 IT Pros & Developers. It's quick & easy.

Problem with BackLog (TCP Queue)..

P: n/a
I'm building a web server and having some issues with the
TCPListener.Start(BackLog). It doesn't seem to do as expected. I'm using
MS Web Stress Tool to test against my web server and when I see 200
connections at once to it, and I'm seeing 2/3 of the sockets as socket
errors.. They never get into my code, so it has to be failing on the
connection side.. Backlog is the only thing I can think could be the
problem. this.svrListener.Start(1024); should be well over enough, but
with no param passed in Start or with 4096, it seems to be exactly the same.

My code goes as follows...
-------------------------------------
//declare listener
private TcpListener svrListener;
//declare port
private Int32 m_lPort = 80; // Select any free port you wish.
Default to 80
//declare host
private String m_szHost = ""; // Select any ip you wish
//declare max kb
private Int32 m_lMaxKBPerSec = 50; // Select max KB per second. Default is
50KB.

//signal.
private static ManualResetEvent clientConnected = new
ManualResetEvent(false);

//Constructor
public CListen(string szHost, Int32 lPort, Int32 lMaxKBPerSec)
{
this.m_lPort = lPort;
this.m_szHost = szHost;
this.m_lMaxKBPerSec = lMaxKBPerSec;
}

//start listener
public void ProcessListener()
{
IPAddress inputDNS_IP;

//setup IP to listen on
if (this.m_szHost.Length == 0)
{
inputDNS_IP = IPAddress.Any;
this.m_szHost = "Any";
}
else
inputDNS_IP = Dns.GetHostEntry(this.m_szHost).AddressList[0];

//log it

try
{
//set to listen on specific IP:port
this.svrListener = new TcpListener(inputDNS_IP, this.m_lPort);
//setup timeouts..
this.svrListener.Server.ReceiveTimeout = 30000;
this.svrListener.Server.SendTimeout = 30000;
//start with a backlog "TCP/IP Queue" of 1024
this.svrListener.Start(1024);
}
catch (SocketException e)
{
//log it
}

while (true)
{
// Set the event to nonsignaled state.
clientConnected.Reset();
// Accept the connection.
// BeginAcceptSocket() creates the accepted socket.
this.svrListener.BeginAcceptSocket(new
AsyncCallback(DoAcceptSocketCallback), this.svrListener);

//log it

// Wait until a connection is made and processed before
// continuing.
clientConnected.WaitOne();

//verify service isn't asking for this DLL to stop.
if (MyWebServer.StopStatus)
break;
}

//this will allow a browser to finish downloading what
//files they are downloading, while not allowing any
//new connections.
while (MyWebServer.CurrentConnection 0)
Thread.Sleep(1000);
}

// Process the client connection.
public void DoAcceptSocketCallback(IAsyncResult ar)
{
//log client connecting

//Get the listener that handles the client request.
TcpListener listener = (TcpListener)ar.AsyncState;

//End the operation and display the received data on the
//console.
Socket sckConnection = listener.EndAcceptSocket(ar);

//Signal the calling thread to continue, connection info received..
clientConnected.Set();

//increment counter, if fail then too many connections
if (!MyWebServer.AddConnection())
{
//log it

//send message to browser, too many connections
return;
}

//log it accepting

//set vars for Socket Class
CSocket socket = new CSocket(sckConnection, this.m_szHost, this.m_lPort,
this.m_lMaxKBPerSec);

//start the thread
Thread th = new Thread(new ThreadStart(socket.Process));
th.Start();

//log thread created
}
Jan 7 '08 #1
Share this Question
Share on Google+
8 Replies


P: n/a
On Mon, 07 Jan 2008 00:06:09 -0800, Chizl <Ch***@NoShitMail.comwrote:
I'm building a web server and having some issues with the
TCPListener.Start(BackLog). It doesn't seem to do as expected. I'm
using
MS Web Stress Tool to test against my web server and when I see 200
connections at once to it, and I'm seeing 2/3 of the sockets as socket
errors.. They never get into my code, so it has to be failing on the
connection side.. Backlog is the only thing I can think could be the
problem. this.svrListener.Start(1024); should be well over enough, but
with no param passed in Start or with 4096, it seems to be exactly the
same.
Just off the top of my head...

My recollection is that the operation system has its own maximum backlog
value. Trying to set the backlog above this value will result in either
no effect, or setting to the maximum value. In neither case will you get
the backlog you're asking for.

I'm not sure it's reasonable to expect that your code will always be able
to handle some arbitrarily high number of simultaneous connection requests
anyway.

One suggestion: in the code you posted, rather than having a thread sit
and loop calling BeginAcceptSocket() over and over, just have the
EndAcceptSocket() callback call it (and do so right away). The way the
code is written now, you are guaranteed a thread switch between accepting
one socket and being ready to accept another, and then of course another
thread switch to accept the next one again. This is going to
unnecessarily inhibit your throughput, artificially reducing the number of
simultaneous connection requests you can process.

You have other performance-hindering design choices there as well:

* your accept callback really should be doing very little. If you
have some heavyweight processing you want to do when accepting a client,
you should handle that elsewhere if handling a large number of
simultaneous connections is a priority for you. Otherwise, you get stuck
with a thread that's doing something other than accepting connections.

* You should not be creating a new thread for each connection. Use
the async i/o methods for the Socket class instead (e.g. BeginReceive).
Using a thread for each connection you will unnecessarily limit the
maximum number of connections you can handle -- the number of threads any
given process can create is much lower than the number of sockets a
process could theoretically handle -- and at the same hurt performance
because of the constant thread context switches that will be required to
deal with multiple active connections.

Pete
Jan 7 '08 #2

P: n/a
First of all thanks for taking the time to respond.. I have some comments
below.

"Peter Duniho" <Np*********@nnowslpianmk.comwrote in message
news:op***************@petes-computer.local...
My recollection is that the operation system has its own maximum backlog
value. Trying to set the backlog above this value will result in either
no effect, or setting to the maximum value. In neither case will you get
the backlog you're asking for.
Doesn't that defeat the purpose of the call? I assume this is like
Affinity.. The OS has one setting, but you can override that with an API
for a specific application. The API doens't change the settings permanent,
but it does for the life of the application and that application only.
One suggestion: in the code you posted, rather than having a thread sit
and loop calling BeginAcceptSocket() over and over, just have the
EndAcceptSocket() callback call it (and do so right away). The way the
code is written now, you are guaranteed a thread switch between accepting
one socket and being ready to accept another, and then of course another
thread switch to accept the next one again. This is going to
unnecessarily inhibit your throughput, artificially reducing the number of
simultaneous connection requests you can process.
I'll look deeper into this, but I got this info from MS.
http://msdn2.microsoft.com/en-us/lib...eptsocket.aspx
You have other performance-hindering design choices there as well:

* your accept callback really should be doing very little. If you
have some heavyweight processing you want to do when accepting a client,
you should handle that elsewhere if handling a large number of
simultaneous connections is a priority for you. Otherwise, you get stuck
with a thread that's doing something other than accepting connections.
I'm assuming you mean my MyWebServer.AddConnection() call. That's doing
nothing but checking current connections < max connection, then incriment a
counter.
* You should not be creating a new thread for each connection. Use
the async i/o methods for the Socket class instead (e.g. BeginReceive).
Using a thread for each connection you will unnecessarily limit the
maximum number of connections you can handle -- the number of threads any
given process can create is much lower than the number of sockets a
process could theoretically handle -- and at the same hurt performance
because of the constant thread context switches that will be required to
deal with multiple active connections.
The thread I'm creating is after I've released the callback. In VC++ I've
tested a spawn of over 2000 threads, your saying in C# that isn't possible?
Based on the examples I've seen with using BeginReceive() it's holding the
receive open longer than I currently am. But I'll look deeper and see if I
can find some better examples.
Jan 8 '08 #3

P: n/a
FYI, I changed the registry settings.. I have zero socket errors now..

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Servic es\Tcpip\Parameters]
"TcpNumConnections"=dword:00fffffe
"MaxFreeTcbs"=dword:000007d0
"MaxHashTableSize"=dword:00000200
"TcpTimedWaitDelay"=dword:000000f0

=============================
Stats..
=============================
Number of hits: 13094
Requests per Second: 109.11

Socket Statistics
--------------------------------------------------------------------------------
Socket Connects: 13099
Total Bytes Sent (in KB): 5966.29
Bytes Sent Rate (in KB/s): 49.72
Total Bytes Recv (in KB): 97789.26
Bytes Recv Rate (in KB/s): 814.86

Socket Errors
--------------------------------------------------------------------------------
Connect: 0
Send: 0
Recv: 0
Timeouts: 0
RDS Results
--------------------------------------------------------------------------------
Successful Queries: 0

Script Settings
=============================
Server: localhost
Number of threads: 200
Test length: 00:02:00

"Chizl" <Ch***@NoShitMail.comwrote in message
news:ex**************@TK2MSFTNGP04.phx.gbl...
First of all thanks for taking the time to respond.. I have some
comments below.

"Peter Duniho" <Np*********@nnowslpianmk.comwrote in message
news:op***************@petes-computer.local...
>My recollection is that the operation system has its own maximum backlog
value. Trying to set the backlog above this value will result in either
no effect, or setting to the maximum value. In neither case will you get
the backlog you're asking for.

Doesn't that defeat the purpose of the call? I assume this is like
Affinity.. The OS has one setting, but you can override that with an API
for a specific application. The API doens't change the settings
permanent, but it does for the life of the application and that
application only.
>One suggestion: in the code you posted, rather than having a thread sit
and loop calling BeginAcceptSocket() over and over, just have the
EndAcceptSocket() callback call it (and do so right away). The way the
code is written now, you are guaranteed a thread switch between accepting
one socket and being ready to accept another, and then of course another
thread switch to accept the next one again. This is going to
unnecessarily inhibit your throughput, artificially reducing the number
of simultaneous connection requests you can process.

I'll look deeper into this, but I got this info from MS.
http://msdn2.microsoft.com/en-us/lib...eptsocket.aspx
>You have other performance-hindering design choices there as well:

* your accept callback really should be doing very little. If you
have some heavyweight processing you want to do when accepting a client,
you should handle that elsewhere if handling a large number of
simultaneous connections is a priority for you. Otherwise, you get stuck
with a thread that's doing something other than accepting connections.

I'm assuming you mean my MyWebServer.AddConnection() call. That's doing
nothing but checking current connections < max connection, then incriment
a counter.
> * You should not be creating a new thread for each connection. Use
the async i/o methods for the Socket class instead (e.g. BeginReceive).
Using a thread for each connection you will unnecessarily limit the
maximum number of connections you can handle -- the number of threads any
given process can create is much lower than the number of sockets a
process could theoretically handle -- and at the same hurt performance
because of the constant thread context switches that will be required to
deal with multiple active connections.

The thread I'm creating is after I've released the callback. In VC++
I've tested a spawn of over 2000 threads, your saying in C# that isn't
possible? Based on the examples I've seen with using BeginReceive() it's
holding the receive open longer than I currently am. But I'll look
deeper and see if I can find some better examples.

Jan 8 '08 #4

P: n/a
I want to preface this reply by noting that it's been a few years since
I've done any significant network code. My working knowledge is limited
these days, and lots of details are fuzzy. You'll get much better advice
in a Winsock- and/or TCP/IP-specific newsgroup for questions like this.
That said, here's what I have to offer...

On Mon, 07 Jan 2008 16:47:07 -0800, Chizl <Ch***@NoShitMail.comwrote:
First of all thanks for taking the time to respond.. I have some
comments
below.

"Peter Duniho" <Np*********@nnowslpianmk.comwrote in message
news:op***************@petes-computer.local...
>My recollection is that the operation system has its own maximum backlog
value. Trying to set the backlog above this value will result in either
no effect, or setting to the maximum value. In neither case will you
get
the backlog you're asking for.

Doesn't that defeat the purpose of the call?
I don't see how. Just because there's a maximum, that doesn't make being
able to set the value a useless operation.

You'll probably find it useful to read the Winsock doc page for the
listen() function:
http://msdn2.microsoft.com/en-us/lib...68(VS.85).aspx

Note the comment near the bottom:

The backlog parameter is limited (silently) to a reasonable
value as determined by the underlying service provider. Illegal
values are replaced by the nearest legal value. There is no
standard provision to find out the actual backlog value

Note also that it says that if you request a "reasonable maximum" for the
backlog on a normal TCP socket, it will use "several hundred".
Unfortunately, it's not more specific than that, so I have no way to know
whether 1024 is going to be respected or not. I suspect not though.

Unfortunately, I don't have any recent working knowledge of this. All I
can tell you is what my recollection is from having used the API in the
past.

Basically: if you're only trying to connect up to a couple hundred clients
at once and not all of them succeed, that suggests to me that either
you're congesting the network or the backlog is not getting set to 1024
(or both could be true at the same time, I suppose).

Of course, there's also the possibility that you're not running into a
backlog problem, and that you're getting errors just due to not running
your accept frequently enough. If that's the case, then no matter what
you set the backlog to, it won't fix the problem. I don't have a good
enough recollection off the top of my head with regards to exactly the
interchange between the endpoints when a connection request winds up in
the backlog but not yet processed to know whether this applies or not.
>One suggestion: in the code you posted, rather than having a thread sit
and loop calling BeginAcceptSocket() over and over, just have the
EndAcceptSocket() callback call it (and do so right away). [...]

I'll look deeper into this, but I got this info from MS.
http://msdn2.microsoft.com/en-us/lib...eptsocket.aspx
For better or worse, the MSDN doc samples are not always the best. Also,
note that the sample you're looking at does not actually handle multiple
connection requests. In other words, that sample doesn't really even
claim to be illustrating how to write code that asynchronously handles
multiple connection requests. It's just a very basic illustration as to
how the async API can be used.
>You have other performance-hindering design choices there as well:

* your accept callback really should be doing very little. If you
have some heavyweight processing you want to do when accepting a client,
you should handle that elsewhere if handling a large number of
simultaneous connections is a priority for you. Otherwise, you get
stuck
with a thread that's doing something other than accepting connections.

I'm assuming you mean my MyWebServer.AddConnection() call. That's doing
nothing but checking current connections < max connection, then
incriment a
counter.
Okay...it's good that's not expensive, but the rest of the method
definitely is expensive. Creating a thread and starting it isn't cheap.
Hopefully your CSocket constructor is low-cost, but even so...doing all of
this stuff can add up.
> * You should not be creating a new thread for each connection. Use
the async i/o methods for the Socket class instead (e.g. BeginReceive).
Using a thread for each connection you will unnecessarily limit the
maximum number of connections you can handle -- the number of threads
any
given process can create is much lower than the number of sockets a
process could theoretically handle -- and at the same hurt performance
because of the constant thread context switches that will be required to
deal with multiple active connections.

The thread I'm creating is after I've released the callback. In
VC++ I've
tested a spawn of over 2000 threads, your saying in C# that isn't
possible?
I'm surprised you got over 2000 threads even in unmanaged code. The
theoretical maximum, with 1MB stacks for each thread, on 32-bit Windows is
2048 just based on the address space alone, and that assumes that
_nothing_ else is consuming any of that address space. Obviously, any
application that does anything interesting is going to allow fewer threads
than that.

On 64-bit Windows, it's completely different. The theoretical limit will
be bounded more by your disk space than anything else, but even there
you're going to run into performance issues first.

Finally, in the context of a server, 2000 connections is a drop in the
bucket. A properly written server can handle hundreds of thousands of
active connections, and you just are not going to reach that scale using
one thread per connection.

The bottom line: creating lots of threads is both unnecessary and
inefficient. If the number of active connections you expect to manage is
in the hundreds, then you can get away with that design. If it's only
dozens, then it might even work well. But once you get into thousands of
connections or more, the cost of the threads really starts affecting your
throughput and scalability.

Also, even if you weren't creating a new thread for each connection,
having two different threads managing the accept logic is going to slow
your code down significantly. But adding a new thread for each connection
just makes it worse, because not only will Windows have to context switch
from the thread calling EndAcceptSocket() back to the one calling
BeginAcceptSocket(), it _also_ has to context switch to the thread you
just created as well. In other words, the one-thread-per-connection
design is not only inherently inefficient, it synergistically worsens the
problems with the other part of your design.

Either problem independently is something worth fixing. But together,
they can really hurt.

Now, is all of this the reason for the behavior you're seeing? I've no
idea. It's not really possible to say without more information about what
exactly is causing the connection failures. But it's certainly a
possibility.

Pete
Jan 8 '08 #5

P: n/a
On Mon, 07 Jan 2008 17:09:00 -0800, Chizl <Ch***@NoShitMail.comwrote:
FYI, I changed the registry settings.. I have zero socket errors now..
Well, that's useful information. But it's not a solution. Messing with
the network registry settings is practically never a solution. The
Windows developers choose and manage those settings with a certain degree
of care to balance the needs of the network code, both for multiple
network clients as well as with other operating system components.

Pete
Jan 8 '08 #6

P: n/a
"Peter Duniho" <Np*********@nnowslpianmk.comwrote in message
news:op***************@petes-computer.local...
On Mon, 07 Jan 2008 17:09:00 -0800, Chizl <Ch***@NoShitMail.comwrote:

Well, that's useful information. But it's not a solution. Messing with
the network registry settings is practically never a solution. The
Windows developers choose and manage those settings with a certain degree
of care to balance the needs of the network code, both for multiple
network clients as well as with other operating system components.
I agree, but based on your first and second posting of Start(backlog) isn't
really doing what I was expecting it to do, what other solution is there to
increase the TCP queue? No matter what my code does, the queue would have
to be increased since the default is 5. And if backlog isn't working
there's no other way to increase the queue outside of the registry.
Jan 8 '08 #7

P: n/a
On Mon, 07 Jan 2008 18:10:36 -0800, Chizl <Ch***@NoShitMail.comwrote:
I agree, but based on your first and second posting of Start(backlog)
isn't
really doing what I was expecting it to do, what other solution is there
to
increase the TCP queue? No matter what my code does, the queue would
have
to be increased since the default is 5. And if backlog isn't working
there's no other way to increase the queue outside of the registry.
You may in fact want to increase the backlog. But don't expect doing so
to resolve your connection problems. It may well alleviate them, but you
still need your code to operate efficiently. Make sure your code works
efficiently and _then_ worry about the backlog. And at some point accept
that there's a limit to the number of connections you'll be able to deal
with at any given moment. No computing resource is infinite.

Pete
Jan 8 '08 #8

P: n/a
"Chizl" <Ch***@NoShitMail.comwrote in message
news:uL*************@TK2MSFTNGP04.phx.gbl...
FYI, I changed the registry settings.. I have zero socket errors now..

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Servic es\Tcpip\Parameters]
"TcpNumConnections"=dword:00fffffe
"MaxFreeTcbs"=dword:000007d0
"MaxHashTableSize"=dword:00000200
"TcpTimedWaitDelay"=dword:000000f0
What OS are you running this on?
Seems like you are using a "Client OS" (non Vista), not a "Server OS". The
TCP stack on client OS'ses is non "optimized" for this kind of stress
testing, you simply increased the default parameters to the default values
for a server OS.
Also, it looks like you are running the stress test tool on the same box as
the "web server" (please correct me if I'm wrong), this makes little or no
sense, the results of your tests are quite meaningless. The reason for this
is that the client requires so many resources that you are negatively
influencing the behavior of the "web server". In your case , the number of
threads (one thread per connected client at the server side plus a thread at
the client side) in the system will rocket sky-high, consuming an insane
amount of memory (1MB per thread) and CPU resources (thread switches). Also
by increasing the MaxFreeTcbs, you are consuming more memory from the page
pool which is smaller on a "Client OS" then on a "Server OS".
This will negatively influence the results of the test
When (seriously) stressing TCP/IP servers you need to run the server on a
server box loaded with a server OS and the clients need to run on separate
boxes interconnected over a LAN.

Willy.

Jan 8 '08 #9

This discussion thread is closed

Replies have been disabled for this discussion.