473,386 Members | 1,712 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,386 software developers and data experts.

A Scalable TCP Server

After reading the newsgroups and various .Net web sites, I have come to a
conclusion that BeginReceive and BeginSend are the way to go as they use
IOCompletion Ports(I have no clue what they mean but most C++ programmers
seem to prefer them, based on my readings). I have decided to use this
sample from MSDN as a starting point to build my TCP server.

http://msdn.microsoft.com/library/de...rverSocket.asp

The server would eventually communicate with around 1500 to 2500 clients.
These clients try to keep the connection open as long as possible(they are
third party - over GPRS - and I have no control of it). So in theory I could
have all the clients connected at same time. Is the above suggested approach
good for this type of situation ? The server will parse the incoming data in
the readCallback function and perform database read/writes based on that and
send back data to the client.

Thanks in advance for any info

Srinivas

Nov 16 '05 #1
4 9115
Also see these refs.
http://www.devhood.com/tutorials/tut...utorial_id=709
http://msdn.microsoft.com/library/de...ketexample.asp

Async servers are good for many clients. They are harder to program
however. If your around <= ~1000 clients, then one or more threaded
server(s) may also be an option for you and much easier to program as
program flow is natural. If you truly will have 1500-2500 *active clients
really pounding, then you have to ask yourself if one machine will handle
it. Async is good, but if you really have that many active clients, your
machine will still need to share cpu between all the callbacks going on. If
not done right, you can also hit the wall on the thread pool as the IOCP on
done on iocp threadpool, but the callbacks are run on thread pool threads,
so you need to make sure you don't have a bunch of overlapped callbacks
going on (just something to watch out for.) Moreover, because the callback
are run on a thread pool thread, you want those methods as quick as
possible. There is another point where a threaded server can be better, as
one thread will (normally) not effect the other clients (each in its own
thread) and the os scheduler handles fairness between threads.

--
William Stacey, MVP

"SRLoka" <ls******@hotmail.com> wrote in message
news:e0*************@TK2MSFTNGP11.phx.gbl...
After reading the newsgroups and various .Net web sites, I have come to a
conclusion that BeginReceive and BeginSend are the way to go as they use
IOCompletion Ports(I have no clue what they mean but most C++ programmers
seem to prefer them, based on my readings). I have decided to use this
sample from MSDN as a starting point to build my TCP server.

http://msdn.microsoft.com/library/de...rverSocket.asp
The server would eventually communicate with around 1500 to 2500 clients.
These clients try to keep the connection open as long as possible(they are
third party - over GPRS - and I have no control of it). So in theory I could have all the clients connected at same time. Is the above suggested approach good for this type of situation ? The server will parse the incoming data in the readCallback function and perform database read/writes based on that and send back data to the client.

Thanks in advance for any info

Srinivas


Nov 16 '05 #2
1) In fact the link I send is same code as your MSDN link. The one I send
has additional description it.
2) "If your around <= ~1000 clients, then one or more threaded
server(s) may also be an option for you and much easier to program as
program flow is natural."
I have actually a working server of this type. Right now only 5 test
clients. I got this code sample and idea on one of the C# sites too. Here a
single thread accepts incoming connections and passes each client to a
separate processing thread. To avoid too many threads and context switching,
this programmer(I cant remember his name) adds all connected sockets to a Q
and processes them in order. But my clients send data only every 3 to 10
minutes(configurable) but just stay connected. So looping through the Q is
an overkill.
3) I have gone through Asad Aziz's article(first link). Its a more refined
version of MSDN sample. I do not want to disconnect any socket even if there
is no data for a predefined time(I think he does that). I want to close it
only when the client disconnects. As per the docs, the BeginReceive will
throw an exception if the client disconnects and I want to catch this
exception and release that connection.

So is the server as written by Asad Aziz more scalable ? Like I said, the
clients send data very infrequently but they just stay connected. So does
that mean each BeginReceive will hold a IOCP thread ? What is the limit to
such threads ? Does it make a difference if its a beefy server ?

4)"but the callbacks are run on thread pool threads"
Are these thread pool limits per application or system wide ? If it is
per application, I can put the incoming data in MSMQ and another application
handle the parsing so that the Call Backs finish quickly.

Thank You

Srinivas

"William Stacey [MVP]" <st***********@mvps.org> wrote in message
news:uK**************@TK2MSFTNGP15.phx.gbl...
Also see these refs.
http://www.devhood.com/tutorials/tut...utorial_id=709
http://msdn.microsoft.com/library/de...ketexample.asp

Async servers are good for many clients. They are harder to program
however. If your around <= ~1000 clients, then one or more threaded
server(s) may also be an option for you and much easier to program as
program flow is natural. If you truly will have 1500-2500 *active clients
really pounding, then you have to ask yourself if one machine will handle
it. Async is good, but if you really have that many active clients, your
machine will still need to share cpu between all the callbacks going on.
If
not done right, you can also hit the wall on the thread pool as the IOCP
on
done on iocp threadpool, but the callbacks are run on thread pool threads,
so you need to make sure you don't have a bunch of overlapped callbacks
going on (just something to watch out for.) Moreover, because the
callback
are run on a thread pool thread, you want those methods as quick as
possible. There is another point where a threaded server can be better,
as
one thread will (normally) not effect the other clients (each in its own
thread) and the os scheduler handles fairness between threads.

--
William Stacey, MVP

"SRLoka" <ls******@hotmail.com> wrote in message
news:e0*************@TK2MSFTNGP11.phx.gbl...
After reading the newsgroups and various .Net web sites, I have come to a
conclusion that BeginReceive and BeginSend are the way to go as they use
IOCompletion Ports(I have no clue what they mean but most C++ programmers
seem to prefer them, based on my readings). I have decided to use this
sample from MSDN as a starting point to build my TCP server.

http://msdn.microsoft.com/library/de...rverSocket.asp

The server would eventually communicate with around 1500 to 2500 clients.
These clients try to keep the connection open as long as possible(they
are
third party - over GPRS - and I have no control of it). So in theory I

could
have all the clients connected at same time. Is the above suggested

approach
good for this type of situation ? The server will parse the incoming data

in
the readCallback function and perform database read/writes based on that

and
send back data to the client.

Thanks in advance for any info

Srinivas

Nov 16 '05 #3
> I have actually a working server of this type. Right now only 5 test
clients. I got this code sample and idea on one of the C# sites too. Here a single thread accepts incoming connections and passes each client to a
separate processing thread. To avoid too many threads and context switching, this programmer(I cant remember his name) adds all connected sockets to a Q and processes them in order.
Could you find the link for that please? I would be interested in looking
at that one. TIA.
3) I have gone through Asad Aziz's article(first link). Its a more refined
version of MSDN sample. I do not want to disconnect any socket even if there is no data for a predefined time(I think he does that). I want to close it
only when the client disconnects. As per the docs, the BeginReceive will
throw an exception if the client disconnects and I want to catch this
exception and release that connection.
You could catch that with a blocking Receive also if you go the threaded
route.
So is the server as written by Asad Aziz more scalable ?
Not as far as I can tell. Similar to the MSDN one.
Like I said, the
clients send data very infrequently but they just stay connected. So does
that mean each BeginReceive will hold a IOCP thread ? What is the limit to
such threads ?
I think the IOCP thread pool is max 1000 threads. The default .net thread
pool that calls your callbacks is 25. This will queue your callbacks in
order. If one thread blocks, another thread is released from the pool up to
25 max. So keep your callbacks fast.
Are these thread pool limits per application or system wide ?
One per process.
per application, I can put the incoming data in MSMQ and another application handle the parsing so that the Call Backs finish quickly.


Not sure about your app, but I don't think you need MSMQ for this and will
probably just get in the way.

--
William Stacey, MVP
Nov 16 '05 #4
> Could you find the link for that please? I would be interested in looking
at that one. TIA. I found it after going through my IE history
http://www.codeproject.com/dotnet/dotnettcp.asp
If you look at the source code, I think I used the server3 code.
See also the discussion under it - particularly the post by Anonymous
towards the end. This is when I started to research on IOCP and ended up
with the MSDN sample.
I think the IOCP thread pool is max 1000 threads.

Does that mean, if I have 1000 connections(some - actually most - may be
idle), further requests will be denied by the OS ?

Another approach I have seen is (just a description - I could not find any
samples) to add all the connected sockets to an IList and use the Select
Method.
Has anyone used this approach ?

So it looks like if the client is done sending data, it should disconnect or
the server should disconnect it after a timeout. Is that a correct
assumption ?

Thanks Again

Srinivas


"William Stacey [MVP]" <st***********@mvps.org> wrote in message
news:es**************@tk2msftngp13.phx.gbl...
I have actually a working server of this type. Right now only 5 test
clients. I got this code sample and idea on one of the C# sites too. Here

a
single thread accepts incoming connections and passes each client to a
separate processing thread. To avoid too many threads and context

switching,
this programmer(I cant remember his name) adds all connected sockets to a

Q
and processes them in order.


Could you find the link for that please? I would be interested in looking
at that one. TIA.
3) I have gone through Asad Aziz's article(first link). Its a more
refined
version of MSDN sample. I do not want to disconnect any socket even if

there
is no data for a predefined time(I think he does that). I want to close
it
only when the client disconnects. As per the docs, the BeginReceive will
throw an exception if the client disconnects and I want to catch this
exception and release that connection.


You could catch that with a blocking Receive also if you go the threaded
route.
So is the server as written by Asad Aziz more scalable ?


Not as far as I can tell. Similar to the MSDN one.
Like I said, the
clients send data very infrequently but they just stay connected. So does
that mean each BeginReceive will hold a IOCP thread ? What is the limit
to
such threads ?


I think the IOCP thread pool is max 1000 threads. The default .net thread
pool that calls your callbacks is 25. This will queue your callbacks in
order. If one thread blocks, another thread is released from the pool up
to
25 max. So keep your callbacks fast.
Are these thread pool limits per application or system wide ?


One per process.
per application, I can put the incoming data in MSMQ and another

application
handle the parsing so that the Call Backs finish quickly.


Not sure about your app, but I don't think you need MSMQ for this and will
probably just get in the way.

--
William Stacey, MVP

Nov 16 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Robert J. O'Hara | last post by:
For some time I've struggled with the problem of displaying simple captioned figures on webpages in a way that is robust and scalable. I tend to make "boring" (um, I mean "conservatively elegant")...
4
by: Dean J Garrett | last post by:
Hello, We're beginning to build a new .NET application that must be very scalable, i.e. to address future load requirements, we'd like to be able to simply add additional web servers. How do you...
1
by: Dean J Garrett | last post by:
We need to determine the proper architecture for a new .NET application which must be scalable, i.e. we must be able to add additional web servers as the need arises in order to accommodate...
0
by: Jonas Hei | last post by:
I need to develop a scalable server which has to receive and send UDP messages. It is required to process hundreds of messages (coming from different remote computers) per second (possibly even...
1
by: John Grandy | last post by:
Could someone point me in the direction of good discussions on scalable state management solutions? Specifically, pros and cons of following strategies: Strategy 1 : temporary business-objects...
3
by: Terry Holland | last post by:
Ive read that to build scalable web apps it is not recommended that state be stored in session variables. My understanding of this is that, with many users using the application concurrently,...
8
by: Chris Mullins | last post by:
One of the things I've spent the last several years working on is a highly scalable socket server written in C#. The SoapBox Server has recently been tested to well over 100k simultanous users....
2
by: Simon Shaw | last post by:
I am that rare point in a developers career where I am starting a new web based project and I have a 4-8 week slot to train myself in the most appropriate technology for this task. As a non-web...
1
by: hyperboreean | last post by:
Hi, I am writing the application server for a three-tier architecture and sending the client's response in xml. My question is: is there a way to build the xml dom in a more scalable way and faster...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.