Async Socket: How to tell when there is no more data.

[mscirri]

The code below is what I am using to asynchronously get data from a
PocketPC device. The data comes in fine in blocks of 1024 bytes but
even when I send no data from the PocketPC constant blocks of 1024 with
all values set to Null arrive. Other than examine a block of 1024 to
see if the entire block is null, is there any other way to determine if
, say a chat message "Hi Charlie" has been received completely?


private ArrayList aryIncomingData = new ArrayList(); // pretty much a
byte[], easier to manage an ArrayList

public void WaitForData()
{
if ( pfnCallBack == null )
pfnCallBack = new AsyncCallback (OnDataReceived);

// now start to listen for any data...
m_asynResult =
sckInteractive.BeginReceive
(m_DataBuffer,0,m_DataBuffer.Length,SocketFlags.No ne,pfnCallBack,null);
}

public void OnDataReceived(IAsyncResult asyn)
{
//end receive...
int iRx = 0 ;
iRx = sckInteractive.EndReceive (asyn);

char[] chars = new char[iRx + 1];
System.Text.Decoder d = System.Text.Encoding.UTF8.GetDecoder();
int charLen = d.GetChars(m_DataBuffer, 0, iRx, chars, 0);
System.String szData = new System.String(chars);

System.Text.ASCIIEncoding asc = new ASCIIEncoding();
byte[] dataBytes = asc.GetBytes(szData);
LoadIncomingData(this.m_DataBuffer);
WaitForData();
}

private bool LoadIncomingData(byte[] data)
{
foreach (byte b in data)
{
aryIncomingData.Add(b);
}
return true;

}

[DalePres]

You could create a unique terminator string and pass it at the end of each
message. Perhaps something like "<::EOM::>". To tell the difference
between that string as part of a message and that string as the end of the
message, test for the string followed by null.

HTH

DalePres
MCAD, MCDBA, MCSE

<ms*****@osmose.com> wrote in message
news:11**********************@o13g2000cwo.googlegr oups.com...

The code below is what I am using to asynchronously get data from a
PocketPC device. The data comes in fine in blocks of 1024 bytes but
even when I send no data from the PocketPC constant blocks of 1024 with
all values set to Null arrive. Other than examine a block of 1024 to
see if the entire block is null, is there any other way to determine if
, say a chat message "Hi Charlie" has been received completely?


private ArrayList aryIncomingData = new ArrayList(); // pretty much a
byte[], easier to manage an ArrayList

public void WaitForData()
{
if ( pfnCallBack == null )
pfnCallBack = new AsyncCallback (OnDataReceived);

// now start to listen for any data...
m_asynResult =
sckInteractive.BeginReceive
(m_DataBuffer,0,m_DataBuffer.Length,SocketFlags.No ne,pfnCallBack,null);
}

public void OnDataReceived(IAsyncResult asyn)
{
//end receive...
int iRx = 0 ;
iRx = sckInteractive.EndReceive (asyn);

char[] chars = new char[iRx + 1];
System.Text.Decoder d = System.Text.Encoding.UTF8.GetDecoder();
int charLen = d.GetChars(m_DataBuffer, 0, iRx, chars, 0);
System.String szData = new System.String(chars);

System.Text.ASCIIEncoding asc = new ASCIIEncoding();
byte[] dataBytes = asc.GetBytes(szData);
LoadIncomingData(this.m_DataBuffer);
WaitForData();
}

private bool LoadIncomingData(byte[] data)
{
foreach (byte b in data)
{
aryIncomingData.Add(b);
}
return true;

}

[Mainak Sarcar]

Very Correct... But using a terminator like that is ONLY possible if you
have control over what the server is sending..
What if I am using real world servers like POP3 or SMTP which have no
terminators to denote that all data has been received... I have tried using
socket.Available property but it is unreliable... because at times it gives 0
bytes available, but if you wait for 1-2 secs and check the property again
.... it has some bytes available for reading.. How would you get to know when
to stop reading and when to call "beginReceive" method again ????

I found this crappy code from MSDN.. which is ABSOLUTELY WRONG... the
program never reaches the "else" part of the code...while the socket is
connected.
private static void ReceiveCallback( IAsyncResult ar )
{
try
{
StateObject state = (StateObject) ar.AsyncState;
Socket client = state.workSocket;
int bytesRead = client.EndReceive(ar);

if (bytesRead > 0)
{
//Get more data....

state.sb.Append(Encoding.ASCII.GetString(state.buf fer,0,bytesRead));
client.BeginReceive(state.buffer,0,StateObject.Buf ferSize,0, new
AsyncCallback(ReceiveCallback), state);
}
else
{
// All the data has arrived; put it in response.
if (state.sb.Length > 1)
{
response = state.sb.ToString();
}
// Signal that all bytes have been received.
receiveDone.Set();
}
} catch (Exception e) {
Console.WriteLine(e.ToString());
}
}

[David Levine]

When a sender closes the socket the receiver is notified when the received
byte count is zero - this indicates that a graceful shutdown of the socket
was initiated from the other end, which means that you should close the
socket on your end to complete the shutdown. This is NOT the same as reading
the value of the Available property. Perhaps this is what the sample code
was intending.

I use a two-phase method for determining how much data to read. The 1st
phase sends a fixed size header block, with fields like this...

struct header
{
int version; // and other header stuff to detect options and/or corruption
in the sender
int size; // size of the variable length portion of the data
}

You can put anything you like in the header but you must at least put the
size there. Once you've read in the fixed size header, read the length of
the data that follows. An advantage of this mechanism is that you always
know when you've read in the entire message without needing special EOF or
formatting characters. Also, if you ever receive bad values in the header
you know that the sender has become corrupted and you should terminate the
connection.

Your code then implements a state machine, either at step 1 (reading header)
or step 2 (reading data). It's quite simple and effective.

Also, if you are using TCP then you should know that your call to receive
may not get all the data in one call...you need a buffering scheme that
reads data and adds it to a temporary buffer until the entire message has
been received by your application. In other words, the code you posted may
get partial strings because message boundaries are not respected. I use code
like this (not aysnc, I use blocking calls and separate threads per
client)...

public static void Receive(Socket s,byte[] buffer)
{
int total = 0;
int size = buffer.Length;
int dataLeft = size;
int received = 0;
int loops = 0;
while ( total < size )
{
received = s.Receive(buffer,total,dataLeft,SocketFlags.None);
if ( received == 0 )
return; // socket was closed by client - disconnect.
total += received;
dataLeft -= received;
loops++; // diagnostics only - this can be greater then 1
}
} // Receive

If you are using UDP then message boundaries are respected and the above
does not apply, but I don't think that is the case because then you would
not even need framing characters and you would not need to "know" the size
of the incoming message.

Hope this helps,
"Mainak Sarcar" <Ma**********@discussions.microsoft.com> wrote in message
news:78**********************************@microsof t.com...

Very Correct... But using a terminator like that is ONLY possible if you
have control over what the server is sending..
What if I am using real world servers like POP3 or SMTP which have no
terminators to denote that all data has been received... I have tried
using
socket.Available property but it is unreliable... because at times it
gives 0
bytes available, but if you wait for 1-2 secs and check the property again
... it has some bytes available for reading.. How would you get to know
when
to stop reading and when to call "beginReceive" method again ????

I found this crappy code from MSDN.. which is ABSOLUTELY WRONG... the
program never reaches the "else" part of the code...while the socket is
connected.
private static void ReceiveCallback( IAsyncResult ar )
{
try
{
StateObject state = (StateObject) ar.AsyncState;
Socket client = state.workSocket;
int bytesRead = client.EndReceive(ar);

if (bytesRead > 0)
{
//Get more data....

state.sb.Append(Encoding.ASCII.GetString(state.buf fer,0,bytesRead));
client.BeginReceive(state.buffer,0,StateObject.Buf ferSize,0,
new
AsyncCallback(ReceiveCallback), state);
}
else
{
// All the data has arrived; put it in response.
if (state.sb.Length > 1)
{
response = state.sb.ToString();
}
// Signal that all bytes have been received.
receiveDone.Set();
}
} catch (Exception e) {
Console.WriteLine(e.ToString());
}
}

[Mainak Sarcar]

No David, your answer did not help me much..... Firstly the code that you
have written is a blocking type of code (synchronous). In a real world
application I can suspend the user interface in an indefinite loop till the
server sending data... Am I right. The correct approach is an Asynchronous
connection.

Your first part of the answer is correct... If the server disconnects the
socket then the client gets a received byte count of 0. But what if the
server does not disconnect the socket ... you as a client have to send some
command to the server to do that.

This is where my question arises. In the Receive Callback method how do I
get to know how much data to read. Data may be in a few bytes or it may be in
MBs..

Regards,
Mainak

"David Levine" wrote:

When a sender closes the socket the receiver is notified when the received
byte count is zero - this indicates that a graceful shutdown of the socket
was initiated from the other end, which means that you should close the
socket on your end to complete the shutdown. This is NOT the same as reading
the value of the Available property. Perhaps this is what the sample code
was intending.

I use a two-phase method for determining how much data to read. The 1st
phase sends a fixed size header block, with fields like this...

struct header
{
int version; // and other header stuff to detect options and/or corruption
in the sender
int size; // size of the variable length portion of the data
}

You can put anything you like in the header but you must at least put the
size there. Once you've read in the fixed size header, read the length of
the data that follows. An advantage of this mechanism is that you always
know when you've read in the entire message without needing special EOF or
formatting characters. Also, if you ever receive bad values in the header
you know that the sender has become corrupted and you should terminate the
connection.

Your code then implements a state machine, either at step 1 (reading header)
or step 2 (reading data). It's quite simple and effective.

Also, if you are using TCP then you should know that your call to receive
may not get all the data in one call...you need a buffering scheme that
reads data and adds it to a temporary buffer until the entire message has
been received by your application. In other words, the code you posted may
get partial strings because message boundaries are not respected. I use code
like this (not aysnc, I use blocking calls and separate threads per
client)...

public static void Receive(Socket s,byte[] buffer)
{
int total = 0;
int size = buffer.Length;
int dataLeft = size;
int received = 0;
int loops = 0;
while ( total < size )
{
received = s.Receive(buffer,total,dataLeft,SocketFlags.None);
if ( received == 0 )
return; // socket was closed by client - disconnect.
total += received;
dataLeft -= received;
loops++; // diagnostics only - this can be greater then 1
}
} // Receive

If you are using UDP then message boundaries are respected and the above
does not apply, but I don't think that is the case because then you would
not even need framing characters and you would not need to "know" the size
of the incoming message.

Hope this helps,
"Mainak Sarcar" <Ma**********@discussions.microsoft.com> wrote in message
news:78**********************************@microsof t.com...

Very Correct... But using a terminator like that is ONLY possible if you
have control over what the server is sending..
What if I am using real world servers like POP3 or SMTP which have no
terminators to denote that all data has been received... I have tried
using
socket.Available property but it is unreliable... because at times it
gives 0
bytes available, but if you wait for 1-2 secs and check the property again
... it has some bytes available for reading.. How would you get to know
when
to stop reading and when to call "beginReceive" method again ????

I found this crappy code from MSDN.. which is ABSOLUTELY WRONG... the
program never reaches the "else" part of the code...while the socket is
connected.
private static void ReceiveCallback( IAsyncResult ar )
{
try
{
StateObject state = (StateObject) ar.AsyncState;
Socket client = state.workSocket;
int bytesRead = client.EndReceive(ar);

if (bytesRead > 0)
{
//Get more data....

state.sb.Append(Encoding.ASCII.GetString(state.buf fer,0,bytesRead));
client.BeginReceive(state.buffer,0,StateObject.Buf ferSize,0,
new
AsyncCallback(ReceiveCallback), state);
}
else
{
// All the data has arrived; put it in response.
if (state.sb.Length > 1)
{
response = state.sb.ToString();
}
// Signal that all bytes have been received.
receiveDone.Set();
}
} catch (Exception e) {
Console.WriteLine(e.ToString());
}
}

[David Levine]

"Mainak Sarcar" <Ma**********@discussions.microsoft.com> wrote in message
news:76**********************************@microsof t.com...

No David, your answer did not help me much..... Firstly the code that you
have written is a blocking type of code (synchronous). In a real world
application I can suspend the user interface in an indefinite loop till
the
server sending data... Am I right. The correct approach is an Asynchronous
connection.
As I said, the thread it is running on is synchronous and the socket is in
blocking mode, but it is a different thread from the main or UI thread. It
is actually far more efficient to do it that way and is more responsive to
UI events. When the thread is blocked waiting on a socket event it does not
consume any CPU cycles at all, and there is no overhead of the callback
mechanism. Internally the runtime does much the same thing - it uses a
separate thread (I think from the threadpool) on which it does its socket
processing. All you need to do is launch another thread and have it use the
socket that you created on your main or listening thread.

You should buy a book on multi-threaded socket programming. There are many
ways of achieving parallel processing. I recommned Richard Blum's book C#
Network Programming.

Your first part of the answer is correct... If the server disconnects the
socket then the client gets a received byte count of 0. But what if the
server does not disconnect the socket ... you as a client have to send
some
command to the server to do that.
All you need to do is close the socket. You can optionally shutdown the
socket first and then close the socket. There are several techniques for
closing the socket.


This is where my question arises. In the Receive Callback method how do I
get to know how much data to read. Data may be in a few bytes or it may be
in
MBs..
Read my message again about the two-phase approach. 1st read a fixed block
header that tells you how many additional bytes you need to read to get the
entire message.

Regards,
Mainak

"David Levine" wrote:

When a sender closes the socket the receiver is notified when the
received
byte count is zero - this indicates that a graceful shutdown of the
socket
was initiated from the other end, which means that you should close the
socket on your end to complete the shutdown. This is NOT the same as
reading
the value of the Available property. Perhaps this is what the sample code
was intending.

I use a two-phase method for determining how much data to read. The 1st
phase sends a fixed size header block, with fields like this...

struct header
{
int version; // and other header stuff to detect options and/or
corruption
in the sender
int size; // size of the variable length portion of the data
}

You can put anything you like in the header but you must at least put the
size there. Once you've read in the fixed size header, read the length of
the data that follows. An advantage of this mechanism is that you always
know when you've read in the entire message without needing special EOF
or
formatting characters. Also, if you ever receive bad values in the header
you know that the sender has become corrupted and you should terminate
the
connection.

Your code then implements a state machine, either at step 1 (reading
header)
or step 2 (reading data). It's quite simple and effective.

Also, if you are using TCP then you should know that your call to receive
may not get all the data in one call...you need a buffering scheme that
reads data and adds it to a temporary buffer until the entire message has
been received by your application. In other words, the code you posted
may
get partial strings because message boundaries are not respected. I use
code
like this (not aysnc, I use blocking calls and separate threads per
client)...

public static void Receive(Socket s,byte[] buffer)
{
int total = 0;
int size = buffer.Length;
int dataLeft = size;
int received = 0;
int loops = 0;
while ( total < size )
{
received = s.Receive(buffer,total,dataLeft,SocketFlags.None);
if ( received == 0 )
return; // socket was closed by client - disconnect.
total += received;
dataLeft -= received;
loops++; // diagnostics only - this can be greater then 1
}
} // Receive

If you are using UDP then message boundaries are respected and the above
does not apply, but I don't think that is the case because then you would
not even need framing characters and you would not need to "know" the
size
of the incoming message.

Hope this helps,
"Mainak Sarcar" <Ma**********@discussions.microsoft.com> wrote in message
news:78**********************************@microsof t.com...

> Very Correct... But using a terminator like that is ONLY possible if
> you
> have control over what the server is sending..
> What if I am using real world servers like POP3 or SMTP which have no
> terminators to denote that all data has been received... I have tried
> using
> socket.Available property but it is unreliable... because at times it
> gives 0
> bytes available, but if you wait for 1-2 secs and check the property
> again
> ... it has some bytes available for reading.. How would you get to
> know
> when
> to stop reading and when to call "beginReceive" method again ????
>
> I found this crappy code from MSDN.. which is ABSOLUTELY WRONG... the
> program never reaches the "else" part of the code...while the socket is
> connected.
> private static void ReceiveCallback( IAsyncResult ar )
> {
> try
> {
> StateObject state = (StateObject) ar.AsyncState;
> Socket client = state.workSocket;
> int bytesRead = client.EndReceive(ar);
>
> if (bytesRead > 0)
> {
> //Get more data....
>
> state.sb.Append(Encoding.ASCII.GetString(state.buf fer,0,bytesRead));
> client.BeginReceive(state.buffer,0,StateObject.Buf ferSize,0,
> new
> AsyncCallback(ReceiveCallback), state);
> }
> else
> {
> // All the data has arrived; put it in response.
> if (state.sb.Length > 1)
> {
> response = state.sb.ToString();
> }
> // Signal that all bytes have been received.
> receiveDone.Set();
> }
> } catch (Exception e) {
> Console.WriteLine(e.ToString());
> }
> }
>