TCP data loss

[Sharon]

I have encountered with a most disturbing TCP problem:

I have cases (too many of them) that result in data loss.
Some inforamation on my test configuration:
I have to PC's which are the same (Dell Dimension 2400).
PC1 is the tcp server, PC2 is the tcp client, they two configured to 65535
bytes tco buffer/window, and working in synchronous manner while for each
client the server start a dedicated thread.
The client send the data size in the first block and this way the server
knows when to stop processing the data from this client by accumulating the
retuned valued from the Socket.Receive( ..) till it gets to the size received
in the first block.

When the server is handling a single client, no data is lost. But when I
switch the PC's so PC2 is the server and PC1 is the client, data is lost
causing the server to keep on waiting on the Socket.Receive( ..) while the
client is done.

When the server (on PC1) is handling several clients (multiple threads
generated in PC2) again data is lost and some of the clients data is missing
in the server side again causing the server to keep on waiting on the
Socket.Receive( ..) while all clients are done sending all their data.

I'm really don't know how fix that or what am I doing wrong.

Any advice will be more then welcome.
--------
Thanks
Sharon

[laimis]

Sounds like you have some miscalculations or mistakes of that sort in
your application. I think it would help if you posted some more
important parts of your code or equivalent logic on how you are handling
multiple clients and things of that sort.

[Sharon]

Ok, here is the code. I hope I didn't make any mistakes coping and cleaning
the code when posting it:

//////////////////////////////////////////////////////////////////////////
/// The TCP Client
//////////////////////////////////////////////////////////////////////////
int blockSize = 65535;
byte [] rowData = new byte[1048576];// 1 MByte
int blockCount = rowData.Length/blockSize + ((rowData.Lengt h % blockSize ==
0) ? 0 : 1);
IPEndPoint ep = new IPEndPoint(IPAd dress.Parse("Se rver IP"), 31001);
clientSocket = new Socket(AddressF amily.InterNetw ork, SocketType.Stre am,
ProtocolType.Tc p);
clientSocket.Se tSocketOption(S ocketOptionLeve l.Socket,
SocketOptionNam e.SendBuffer, blockSize);
clientSocket.Co nnect(ep);
byte [] info;
byte [] initBuf = new byte[8];
// Setting the block size into first 4 bytes.
info = BitConverter.Ge tBytes(blockSiz e);
int i = 0;
for( ; i<info.Length; ++i )
{
initBuf[i] = info[i];
}
// Setting the overall data size into bytes 4 to 7.
info = BitConverter.Ge tBytes(rowData. Length);
for( i=0; i<info.Length; ++i )
{
initBuf[4+i] = info[i];
}
clientSocket.Se nd( initBuf );

// Sending the data in a 65535 block size.
int length, sentSize;
for( i=0; i < blockCount ; ++i )
{
if( i < (blockCount - 1) )
{
length = blockSize;
}
else
{
length = rowData.Length - (i * blockSize);
}
sentSize = clientSocket.Se nd(rowData, i*blockSize, length,
System.Net.Sock ets.SocketFlags .None);
}
clientSocket.Cl ose();

//////////////////////////////////////////////////////////////////////////
/// The TCP Server
//////////////////////////////////////////////////////////////////////////
IPEndPoint ipNport = new IPEndPoint(IPAd dress.Parse("Se rver IP"), 31001)
TcpListener m_server = new TcpListener(ipN port);
ArrayList m_socketListene rsList = new ArrayList();
// Start the Server and start the thread to listen to client requests.
m_server.Start( );
m_serverThread = new Thread( new ThreadStart(Ser verThreadStart) );
m_serverThread. Start();

private void ServerThreadSta rt()
{
// Client Socket variable;
Socket clientSocket = null;
TCPSocketListen er socketListener = null;
while( true )
{
// Wait for any client requests and if there is any
// request from any client accept it (Wait indefinitely).
clientSocket = m_server.Accept Socket();
socketListener = new TCPSocketListen er(clientSocket );
// Add the socket listener to an array list in a thread safe fashion.
lock( m_socketListene rsList )
{
m_socketListene rsList.Add(clie ntSocket);
}
// Start a communicating with the client in a different thread.
socketListener. StartSocketList ener();
}
}

public class TCPSocketListen er
{
private int m_ID;
private int m_DataSize = 0;
private int m_TotalByteRcvd = 0;
private bool m_IsfirstBlock = true;
private Socket m_clientSocket = null;

public TCPSocketListen er(Socket clientSocket)
{
m_clientSocket = clientSocket;
}
public void StartSocketList ener()
{
m_clientListene rThread = new Thread(new
ThreadStart(Soc ketListenerThre adStart));
m_clientListene rThread.Start() ;
}
private void SocketListenerT hreadStart()
{
int size = 0;
byte [] byteBuffer = new byte[131072];// 128 Kilobyte.

while( true )
{
size = m_clientSocket. Receive(byteBuf fer);
if( size > 0 )
{
if( m_IsfirstBlock )
{
m_TotalByteRcvd = 0;
blocksSize = BitConverter.To Int32(rowdata, 4);
m_DataSize = BitConverter.To Int32(rowdata, 8);
m_Buffer = new byte[dataSize];
m_clientSocket. SetSocketOption (SocketOptionLe vel.Socket,
SocketOptionNam e.ReceiveBuffer , blocksSize);
}
else
{
m_TotalByteRcvd += size;
if( m_TotalByteRcvd >= m_DataSize )
{
break;
}
}
}
}
m_clientSocket. Close();
}
}
/////////////////////////////////////////////////////////////

I hope it's clear enough.

Any Idea will highly appreciated.

---------------------
Thanks & Regards
Sharon

[laimis]

I assume you have tried debugging both client and server in problem
situations. Did you notice anything different? I find it very strange
that when you replace the computer roles (client/server), the program
stops working.

For me just by looking at the code is really hard to say what could be
wrong. I took a look and nothing seemed out of order there anyways.

[Sami Vaaraniemi]

First off, it would be helpful if you could post a complete sample (one that
anyone can compile and run without modifications). But anyway, comments
inline:

"Sharon" <Sh****@discuss ions.microsoft. com> wrote in message
news:ED******** *************** ***********@mic rosoft.com...

Ok, here is the code. I hope I didn't make any mistakes coping and
cleaning
the code when posting it:

//////////////////////////////////////////////////////////////////////////
/// The TCP Client
//////////////////////////////////////////////////////////////////////////
int blockSize = 65535;
byte [] rowData = new byte[1048576];// 1 MByte
int blockCount = rowData.Length/blockSize + ((rowData.Lengt h % blockSize
==
0) ? 0 : 1);
IPEndPoint ep = new IPEndPoint(IPAd dress.Parse("Se rver IP"), 31001);
clientSocket = new Socket(AddressF amily.InterNetw ork, SocketType.Stre am,
ProtocolType.Tc p);
clientSocket.Se tSocketOption(S ocketOptionLeve l.Socket,
SocketOptionNam e.SendBuffer, blockSize);
clientSocket.Co nnect(ep);
byte [] info;
byte [] initBuf = new byte[8];
// Setting the block size into first 4 bytes.
info = BitConverter.Ge tBytes(blockSiz e);
int i = 0;
for( ; i<info.Length; ++i )
{
initBuf[i] = info[i];
}
// Setting the overall data size into bytes 4 to 7.
info = BitConverter.Ge tBytes(rowData. Length);
for( i=0; i<info.Length; ++i )
{
initBuf[4+i] = info[i];
}
clientSocket.Se nd( initBuf );

// Sending the data in a 65535 block size.
int length, sentSize;
for( i=0; i < blockCount ; ++i )
{
if( i < (blockCount - 1) )
{
length = blockSize;
}
else
{
length = rowData.Length - (i * blockSize);
}
sentSize = clientSocket.Se nd(rowData, i*blockSize, length,
System.Net.Sock ets.SocketFlags .None);
}
clientSocket.Cl ose();

//////////////////////////////////////////////////////////////////////////
/// The TCP Server
//////////////////////////////////////////////////////////////////////////
IPEndPoint ipNport = new IPEndPoint(IPAd dress.Parse("Se rver IP"), 31001)
TcpListener m_server = new TcpListener(ipN port);
ArrayList m_socketListene rsList = new ArrayList();
// Start the Server and start the thread to listen to client requests.
m_server.Start( );
m_serverThread = new Thread( new ThreadStart(Ser verThreadStart) );
m_serverThread. Start();

private void ServerThreadSta rt()
{
// Client Socket variable;
Socket clientSocket = null;
TCPSocketListen er socketListener = null;
while( true )
{
// Wait for any client requests and if there is any
// request from any client accept it (Wait indefinitely).
clientSocket = m_server.Accept Socket();
socketListener = new TCPSocketListen er(clientSocket );
// Add the socket listener to an array list in a thread safe fashion.
lock( m_socketListene rsList )
{
m_socketListene rsList.Add(clie ntSocket);
}
// Start a communicating with the client in a different thread.
socketListener. StartSocketList ener();
}
}

public class TCPSocketListen er
{
private int m_ID;
private int m_DataSize = 0;
private int m_TotalByteRcvd = 0;
private bool m_IsfirstBlock = true;
private Socket m_clientSocket = null;

public TCPSocketListen er(Socket clientSocket)
{
m_clientSocket = clientSocket;
}
public void StartSocketList ener()
{
m_clientListene rThread = new Thread(new
ThreadStart(Soc ketListenerThre adStart));
m_clientListene rThread.Start() ;
}
private void SocketListenerT hreadStart()
{
int size = 0;
byte [] byteBuffer = new byte[131072];// 128 Kilobyte.

while( true )
{
size = m_clientSocket. Receive(byteBuf fer);
if( size > 0 )
{
At this point, it appears you assume you have received exactly 8 bytes but
that is not necessarily the case. 'size' could be anything from 1 to number
of bytes in the TCP receive buffer. Most notably, if the number of bytes is
greater than 8, then you're missing out the extra bytes here as the next
iteration won't pick them up.
if( m_IsfirstBlock )
{
m_TotalByteRcvd = 0;
blocksSize = BitConverter.To Int32(rowdata, 4);
m_DataSize = BitConverter.To Int32(rowdata, 8);
m_Buffer = new byte[dataSize];
m_clientSocket. SetSocketOption (SocketOptionLe vel.Socket,
SocketOptionNam e.ReceiveBuffer , blocksSize);
}
else
{
m_TotalByteRcvd += size;
if( m_TotalByteRcvd >= m_DataSize )
{
break;
}
}
}
}
m_clientSocket. Close();
}
}
/////////////////////////////////////////////////////////////

So the lesson is that never assume anything about the amount of data you
receive with TCP. If the sender calls Send two times trying to send e.g.,
16-bytes each time, the receiver might receive all of it in one 16-byte
chunk, or in two 8-byte chunks, or... It is the receiver's responsibility to
handle all possible cases correctly.

HTH,
Sami

[Sharon]

Hi Sami,

Thanks a lot for your answer, your observation is the problem.

The client is sending 8 byte buffer in the first block, therefore I assumed
at the server side that this is the first buffer size I should receive, BUT
there are cases when this first block is 73008 bytes and not 8 bytes, causing
me to lose 73000 bytes.

So now, after I fixed it, it works fine even if I run 20 fast clients and a
server PC that is also running Outlook and Internet Radio.

So now it all good.

Again, thanks a lot.

Sharon