473,574 Members | 3,159 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Asynchronous I/O or multiple threads?

Hi all,

I'm designing a fairly simple service that will run on W2K/SP4 and W2K3
servers. It's job is to be a very specialized database server that listens
on a given IP address / TCP port and handles multiple connections. Client
programs will make a connection and pass text strings to the service, which
will then return a value for each of the strings. It's unlikely the service
will ever have more than 100 simultaneous connections and most of what it
will do is a lookup in a table for each string. We've considered MSDE for
this and for a number of reasons it's impractical for us to use it.

In researching various ways of handling simultaneous connections I've seen
examples using multithreading and others using async I/O. I'd like the
program to scale well but I don't want to overwhelm the server. I'm looking
for guidance on choosing between the two methods.

Alternately, I'd be happy to use a commercial database server if I could
find one that can be distributed easily and royalty free and which doesn't
cost too much. Something like the VistaDB database engine would be ideal but
it can't run as a service.

Thoughts / suggestions?

Nov 16 '05 #1
6 4427
Quiet Man <qu******@maili nator.com> wrote:
I'm designing a fairly simple service that will run on W2K/SP4 and W2K3
servers. It's job is to be a very specialized database server that listens
on a given IP address / TCP port and handles multiple connections. Client
programs will make a connection and pass text strings to the service, which
will then return a value for each of the strings. It's unlikely the service
will ever have more than 100 simultaneous connections and most of what it
will do is a lookup in a table for each string. We've considered MSDE for
this and for a number of reasons it's impractical for us to use it.

In researching various ways of handling simultaneous connections I've seen
examples using multithreading and others using async I/O. I'd like the
program to scale well but I don't want to overwhelm the server. I'm looking
for guidance on choosing between the two methods.

Alternately, I'd be happy to use a commercial database server if I could
find one that can be distributed easily and royalty free and which doesn't
cost too much. Something like the VistaDB database engine would be ideal but
it can't run as a service.

Thoughts / suggestions?


Given that MSDE is the most obvious choice IMO, it would help if you
could state the reasons you have for not using it.

In terms of handling simultaneous connections, asynchronous IO is
likely to scale better, but be warned that you'll end up using system
threadpool threads automatically - you need to be aware of this, as if
you start doing other things which require the threadpool within
threads that are already running in the pool, you can easily deadlock
(by having all the threadpool threads waiting until another threadpool
thread is available before they can do any more work).

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too
Nov 16 '05 #2
"Jon Skeet [C# MVP]" <sk***@pobox.co m> wrote in message
news:MP******** *************** *@msnews.micros oft.com...
Quiet Man <qu******@maili nator.com> wrote:
I'm designing a fairly simple service that will run on W2K/SP4 and W2K3
servers. It's job is to be a very specialized database server that listens
on a given IP address / TCP port and handles multiple connections. Client
programs will make a connection and pass text strings to the service, which will then return a value for each of the strings. It's unlikely the service will ever have more than 100 simultaneous connections and most of what it
will do is a lookup in a table for each string. We've considered MSDE for
this and for a number of reasons it's impractical for us to use it.

In researching various ways of handling simultaneous connections I've seen
examples using multithreading and others using async I/O. I'd like the
program to scale well but I don't want to overwhelm the server. I'm looking for guidance on choosing between the two methods.

Alternately, I'd be happy to use a commercial database server if I could
find one that can be distributed easily and royalty free and which doesn't
cost too much. Something like the VistaDB database engine would be ideal but it can't run as a service.

Thoughts / suggestions?


Given that MSDE is the most obvious choice IMO, it would help if you
could state the reasons you have for not using it.

In terms of handling simultaneous connections, asynchronous IO is
likely to scale better, but be warned that you'll end up using system
threadpool threads automatically - you need to be aware of this, as if
you start doing other things which require the threadpool within
threads that are already running in the pool, you can easily deadlock
(by having all the threadpool threads waiting until another threadpool
thread is available before they can do any more work).

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

----------------

Thanks for the reply Jon.

Regarding MSDE - one big reason is the complexity of deployment. This app
will be installed on servers that may or may not already have MSDE 1.0, MSDE
2000, MSDE 2000A, SQL 7 or one of the SQL 2000 engines (personal, developer,
standard, enterprise) installed already for use by other apps - this makes
the deployment process / installation program a bear. Another reason is
size - the rest of the app is looking like about 4 MB of code and the MSDE
is 45MB or so. Lessor reasons include the throughput uncertainty that comes
with the workgroup governor (which has bitten us in the past) and resistance
on the part of some network / server admins due to past security issues with
the SQL 2000 engine. Still lessor reasons are the thought that MSDE is
overkill and that a specialized app will have a smaller memory footprint and
possibly be quicker.

All that being said I do like the MSDE and have used it successfully in the
past.

Thanks for the async I/O scalability note. I also appreciate your pointing
out the threadpool consideration. I'm hoping it won't be a problem in that
99% of what this service will be doing is looking up values in a global
table in read-only mode. The table will peak at about 300,000 string / value
pairs, average string length of 8 characters. The data will be refreshed
once per day or less. I may have to look into limiting the number of
connections that the service will accept.

Quiet Man

Nov 16 '05 #3
Hi,

I may have missed the point here, but why do you need a database at all for
such a simple task ?

Cheers

Doug Forster

"Quiet Man" <qu******@maili nator.com> wrote in message
news:eQ******** ******@TK2MSFTN GP10.phx.gbl...
"Jon Skeet [C# MVP]" <sk***@pobox.co m> wrote in message
news:MP******** *************** *@msnews.micros oft.com...
Quiet Man <qu******@maili nator.com> wrote:
I'm designing a fairly simple service that will run on W2K/SP4 and W2K3
servers. It's job is to be a very specialized database server that
listens
on a given IP address / TCP port and handles multiple connections. Client
programs will make a connection and pass text strings to the service,

which
will then return a value for each of the strings. It's unlikely the

service
will ever have more than 100 simultaneous connections and most of what it
will do is a lookup in a table for each string. We've considered MSDE for
this and for a number of reasons it's impractical for us to use it.

In researching various ways of handling simultaneous connections I've
seen
examples using multithreading and others using async I/O. I'd like the
program to scale well but I don't want to overwhelm the server. I'm

looking
for guidance on choosing between the two methods.

Alternately, I'd be happy to use a commercial database server if I could
find one that can be distributed easily and royalty free and which
doesn't
cost too much. Something like the VistaDB database engine would be ideal

but
it can't run as a service.

Thoughts / suggestions?


Given that MSDE is the most obvious choice IMO, it would help if you
could state the reasons you have for not using it.

In terms of handling simultaneous connections, asynchronous IO is
likely to scale better, but be warned that you'll end up using system
threadpool threads automatically - you need to be aware of this, as if
you start doing other things which require the threadpool within
threads that are already running in the pool, you can easily deadlock
(by having all the threadpool threads waiting until another threadpool
thread is available before they can do any more work).

--
Jon Skeet - <sk***@pobox.co m>
http://www.pobox.com/~skeet
If replying to the group, please do not mail me too

----------------

Thanks for the reply Jon.

Regarding MSDE - one big reason is the complexity of deployment. This app
will be installed on servers that may or may not already have MSDE 1.0,
MSDE
2000, MSDE 2000A, SQL 7 or one of the SQL 2000 engines (personal,
developer,
standard, enterprise) installed already for use by other apps - this makes
the deployment process / installation program a bear. Another reason is
size - the rest of the app is looking like about 4 MB of code and the MSDE
is 45MB or so. Lessor reasons include the throughput uncertainty that
comes
with the workgroup governor (which has bitten us in the past) and
resistance
on the part of some network / server admins due to past security issues
with
the SQL 2000 engine. Still lessor reasons are the thought that MSDE is
overkill and that a specialized app will have a smaller memory footprint
and
possibly be quicker.

All that being said I do like the MSDE and have used it successfully in
the
past.

Thanks for the async I/O scalability note. I also appreciate your pointing
out the threadpool consideration. I'm hoping it won't be a problem in that
99% of what this service will be doing is looking up values in a global
table in read-only mode. The table will peak at about 300,000 string /
value
pairs, average string length of 8 characters. The data will be refreshed
once per day or less. I may have to look into limiting the number of
connections that the service will accept.

Quiet Man

Nov 16 '05 #4
100 threads should be not problem. Some use <= 1000 threads without issue
(however 1000 is pushing it imo.) You could fire up new threads upto to
some max, say 500, then block new requests with server busy reply. You
could also use your own thread pool (many existing example out there such as
Codeproject) with some max and have ready threads ready to go. These
servers are much easier to create and reason about. Async servers may scale
better, but are harder to code and get right and have the potential thread
pool blocking issue. If you have like 10,000 active requests, then async is
the option, but if you really have that many then the server will melt
anyway or be so slow that almost nothing gets done. So your back to
factoring out load between other servers - which you could do with either
approach. Therefore, the threaded server may start to look better again.
If your not blocking alot in your processing piece (i.e. just doing cached
table lookups) then potentially you could service all requests with *one*
thread using a queued approach. Queue up all requests and service the queue
with one tight server loop and send the reply. Your replies could be put
into yet another queue that is serviced by one-to-N threads (say your own
thread pool or the system thread pool) so that one reply can't block the
server. For that matter, your receiver side could be 1-N threads also that
waits for and builds the complete requests - so one slow client does not do
a DoS on your server. Each Client-Listener thread would put completed
request into the server IN-Q. The one server thread reads from IN-Q,
processes, and writes reply to OUT-Q. The Out thread pool handles the
socket sends and socket closing (UDP may work better for this single
request/reply deal). Even with TCP, this would work well for simple
request/reply where a reply signals the end of the session. I tend to favor
the threaded server or queued server approach (others may vary.) hth

--
William Stacey, MVP
http://mvp.support.microsoft.com

"Quiet Man" <qu******@maili nator.com> wrote in message
news:en******** ******@TK2MSFTN GP14.phx.gbl...
Hi all,

I'm designing a fairly simple service that will run on W2K/SP4 and W2K3
servers. It's job is to be a very specialized database server that listens
on a given IP address / TCP port and handles multiple connections. Client
programs will make a connection and pass text strings to the service, which will then return a value for each of the strings. It's unlikely the service will ever have more than 100 simultaneous connections and most of what it
will do is a lookup in a table for each string. We've considered MSDE for
this and for a number of reasons it's impractical for us to use it.

In researching various ways of handling simultaneous connections I've seen
examples using multithreading and others using async I/O. I'd like the
program to scale well but I don't want to overwhelm the server. I'm looking for guidance on choosing between the two methods.

Alternately, I'd be happy to use a commercial database server if I could
find one that can be distributed easily and royalty free and which doesn't
cost too much. Something like the VistaDB database engine would be ideal but it can't run as a service.

Thoughts / suggestions?


Nov 16 '05 #5
Thanks for your thoughts William. It helps to have some guidelines regarding
what a reasonable number of threads is, etc. I obviously haven't done a
great deal of multithreaded programming.

I'm comfortable with limiting the service to accepting 100 simultaneous
connections. A typical load on the server will be less than 1000 connections
per hour; each connection will pass (in one big lump) an average of 200 text
strings that will be looked up. If I go with a multithreaded approach my
current thought is that each thread will take one of these connections,
lookup values for each of the text strings, return all the values and at
that point terminate the connection (so yes, the reply indicates the end of
the session). The thread would then be put back into the thread pool for
reuse. All the table data (probably 300,000 rows) will stay in RAM so
lookups should be extremely quick.

I might be able to pull this off with a single threaded app but it seems
better to plan for future growth now. I haven't done any performance testing
yet but I can't imagine that the actual lookup process will take much time -
I'm guessing that most of the time will be occupied by making the connection
and transferring the data between the client and the server.

I'll have to study up on the response queue idea.

Again, thanks. You've given me plenty to think about.

------------------------------

"William Stacey [MVP]" <st***********@ mvps.org> wrote in message
news:#R******** *****@TK2MSFTNG P12.phx.gbl...
100 threads should be not problem. Some use <= 1000 threads without issue
(however 1000 is pushing it imo.) You could fire up new threads upto to
some max, say 500, then block new requests with server busy reply. You
could also use your own thread pool (many existing example out there such as
Codeproject) with some max and have ready threads ready to go. These
servers are much easier to create and reason about. Async servers may scale
better, but are harder to code and get right and have the potential thread
pool blocking issue. If you have like 10,000 active requests, then async is
the option, but if you really have that many then the server will melt
anyway or be so slow that almost nothing gets done. So your back to
factoring out load between other servers - which you could do with either
approach. Therefore, the threaded server may start to look better again.
If your not blocking alot in your processing piece (i.e. just doing cached
table lookups) then potentially you could service all requests with *one*
thread using a queued approach. Queue up all requests and service the queue
with one tight server loop and send the reply. Your replies could be put
into yet another queue that is serviced by one-to-N threads (say your own
thread pool or the system thread pool) so that one reply can't block the
server. For that matter, your receiver side could be 1-N threads also that
waits for and builds the complete requests - so one slow client does not do
a DoS on your server. Each Client-Listener thread would put completed
request into the server IN-Q. The one server thread reads from IN-Q,
processes, and writes reply to OUT-Q. The Out thread pool handles the
socket sends and socket closing (UDP may work better for this single
request/reply deal). Even with TCP, this would work well for simple
request/reply where a reply signals the end of the session. I tend to favor
the threaded server or queued server approach (others may vary.) hth

--
William Stacey, MVP
http://mvp.support.microsoft.com

"Quiet Man" <qu******@maili nator.com> wrote in message
news:en******** ******@TK2MSFTN GP14.phx.gbl...
Hi all,

I'm designing a fairly simple service that will run on W2K/SP4 and W2K3
servers. It's job is to be a very specialized database server that listens
on a given IP address / TCP port and handles multiple connections. Client
programs will make a connection and pass text strings to the service, which will then return a value for each of the strings. It's unlikely the service will ever have more than 100 simultaneous connections and most of what it
will do is a lookup in a table for each string. We've considered MSDE for
this and for a number of reasons it's impractical for us to use it.

In researching various ways of handling simultaneous connections I've seen
examples using multithreading and others using async I/O. I'd like the
program to scale well but I don't want to overwhelm the server. I'm looking for guidance on choosing between the two methods.

Alternately, I'd be happy to use a commercial database server if I could
find one that can be distributed easily and royalty free and which doesn't
cost too much. Something like the VistaDB database engine would be ideal but it can't run as a service.

Thoughts / suggestions?


Nov 16 '05 #6
Just got a chance to check the group again. Many thanks! I'll be examining
the code and trying it out as well tomorrow.

"William Stacey [MVP]" <st***********@ mvps.org> wrote in message
news:e9******** ******@TK2MSFTN GP10.phx.gbl...
Sound good. Here is a little tcp server and client (see attached zip of
both projects) I just worked up for a simple example. I use XmlSerializer
to serialize a type to send and receive. This is overkill if you just need
to send one string and return one string, but is helpful if you needs are
more then just passing one string. Just got it to the point of working
without a lot of debugging or error handling, but should be a good start.
The main server loop is below:

private void ProcessStart()
{
listener = new TcpListener(lep );
listener.Start( );
Socket s = null;

while( IsStarted )
{
try
{
s = listener.Accept Socket();
}
catch(System.Ne t.Sockets.Socke tException sx)
{
if ( ! IsStarted )
break; // We Stopped server.
Console.WriteLi ne("Server Error:"+sx.Mess age);
isStarted = false;
}
MyWorker newWorker = new MyWorker(s, this);
newWorker.Start ();
Console.WriteLi ne("Server received new request.");
}
}

--
William Stacey, MVP
http://mvp.support.microsoft.com

"Quiet Man" <qu******@maili nator.com> wrote in message
news:OO******** *****@TK2MSFTNG P11.phx.gbl...
Thanks for your thoughts William. It helps to have some guidelines regarding what a reasonable number of threads is, etc. I obviously haven't done a
great deal of multithreaded programming.

....

Nov 16 '05 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
2926
by: Zunbeltz Izaola | last post by:
Hi, I have the following problem. I'm developing a GUI program (wxPython). This program has to comunicate (TCP) whit other program that controls a laboratory machine to do a measurement. I have a dialog box, wiht two buttoms "Start measurement" and "Stop". "Start" executes a function that do the measurement in the following way.
48
5400
by: Steve - DND | last post by:
I'm trying to determine if I need to make my application multi-threaded, or if it can be done with asynchronous programming. I realize that asynch calls do create a new thread in the background, but when they come back, they return to the thread that launched them, is that correct? If I go multi-threaded, then I just need to spawn a new thread...
1
1543
by: Julian Hershel | last post by:
Reading about asynchronous programming (ms-help://MS.NETFrameworkSDK/cpguidenf/html/cpconasynchronousdesignpatterno verview.htm) I could not clarify some doubts. Hope you can help me. 1) Are asynchronous programming and multithreaded programming two different pictures? As I read the help topic above it is not clear to me if the design...
4
3804
by: taskswap | last post by:
I have a legacy application written in C that I'm trying to convert to C#. It processes a very large amount of data from many clients (actually, upstream servers - this is a mux) simultaneously. I've read through what must be dozens of ways to do socket communication in C#, and it seems they all devolve into three basic options -...
2
2174
by: Ronodev.Sen | last post by:
the way my program needs to go is -- 1) open a socket and listen on it 2) moment a client connects to the socket - process some data (by sending it to another machine), get the result and send it back to the SAME socket - the data isnt a large value (measured in bytes rather than MB or GB) i TRIED thinking of this in the Asynchronous...
0
1180
by: MalamisuraE | last post by:
I need to write a class that will process a qued list of items, each item could take a long or short time based on the items. I want to use Asynchronous Callback Methods to do this instead of creating my own threads as I am convinced this is the way to go from researching the material. For threads you can create an array of threads, but for...
4
3594
by: Engineerik | last post by:
I am trying to create a socket server which will listen for connections from multiple clients and call subroutines in a Fortran DLL and pass the results back to the client. The asynchronous socket client and asynchronous socket server example code provided in the .NET framework developers guide is a great start but I have not dealt with...
4
4069
by: Morgan Cheng | last post by:
Since ASP.NET 2.0, asynchronous web service client can be implemented with event-based pattern, instead of original BeginXXX/EndXXX pattern. However, I didn't find any material about event-based server side asynchronous web service. So, we can only implement asynchronous webmethod with BeginXXX/EndXXX pattern, right? I don't why ASP.NET 2.0...
0
1262
by: alan | last post by:
Hello all, I'd like to ask recommendations about a "good" generic asynchronous I/O library for C++. By generic I mean, something I can use even on files, stdin/stdout, and sockets. I've seen boost::asio, and also ioxxx, but most of the examples I've seen with them are about sockets. Can they be used for general i/o (files, terminal)? Can...
0
7841
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
7760
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
8101
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
0
8137
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
1
5654
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
5335
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3774
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
1
2273
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1369
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.