long streams

[Yair Cohen]

hello

some background on my framework:
win XP, 512 DDR, P4 2GHz,60GB,Realtech.
VB.NET.
i wrote a program (based on the multi-user chat program from msdn) that
handles tcpip connection between client and server.
i extended this class to be able to send and recieve XML data. i use xml to
send:
"<Command>" - command to operate in the server.
or
"<Request>" - request data from the server.
or
"<Response>" - the response of the server.
the server itself take responsablity of the DB connections.
the problem is, my basis (the chat app) built to work on one line of data
(ended with vbCrLf). but my app need to handle much larger amount of data.
my problem was, because the network stream sends only 1024 bytes, how to
know where the command/request ends and a new command/request start.
I solved it by aggregating the data into stringbuilder any time it get, it
checks to see 'chr(127)' (telnet breakcode).
when he see it, it handle the request (or command) by sending the entire
string of the stringbuilder to a Client object. the rest of the string
(after the chr(127)) is used to create a new stringbuilder object.
it works, for the most of the times. and this is the problem,
some times it just "regard" the breakchar, and the data send to the client
object is incorrect (xml with 2 root elements...) this cause my app to fail
sometimes.
my question is how to handle this amount of data to correctly send the write
data to each Client object to handle.

2. how do i do priodicaly check to catch lost connections ? (client that did
not disconnect properly leave its identify on the server and then will not
be able to connect).

[Nak]

Hi there,

I know this one, I used to program control systems for various pieces of
hardware so I know how to implement reliable protocols. Try to follow these
guidelines,

*Make things as simple as possible
*Make the protocol as small as possible
*Make the protocol easily documentable

I'll give you an example of a protocol. Firstly you will need to pick
an "STX" (Start of packet) and "ETX" (End of packet) value. This should be
a value that is NOT in the ASCII range, i.e. something like 2 for "STX" and
4 for "ETX". Then pick 2 letter acronyms for your headers, for example

CMD = Command
REQ = Request
RES = Response

Then pick a character to separate the header from the rest of the
packet, such as ":". Then you should simply specify a value for the index
of the Command, Request or Response, separated with the same character as
the header ":". The data can then follow. For example...

*Remember that the "STX" and "ETX" should NOT be in the ASCII range. To
get the correct character for character code 2 and 4, use chr()

chr(2) & "CMD:1:THIS IS MY PACKET DATA" & chr(4)

This may seem very simply but it can be very versatile. The packet data
could even contain comma delimited parameters that you can easily split at
the other end using the String.Split() method. If you want to send binary
data you could use Base64 encoding.....

--------------------------------------------------------
The following has been untested, but the principal remains the same in
testing*

-----------------
Use the following function to turn a file into a byte array
-----------------
Dim pBytFileBytes As Byte() = fileToByteArray("D:\Personal\Development\VB
dotnet\nikvue.net\Documents\slideshow v2\test\test.jpg")
Dim pStrBase64 as String = System.Convert.ToBase64String(pBytFileBytes)

***
chr(2) & "CMD:5:" & pStrBase64 & chr(4)
***

-----------------
Use the following function to turn a file into a byte array
-----------------

Private Function fileToByteArray(ByVal iPath As String) As Byte()
Dim pFESStream As FileStream
Try
pFESStream = New FileStream(iPath, FileMode.Open, FileAccess.Read)
Dim pBytFileBytes(pFESStream.Length - 1) As Byte
Dim pIntBytesRead As Integer = pFESStream.Read(pBytFileBytes, 0,
pFESStream.Length)
Call pFESStream.Close()
pFESStream = Nothing
Return (pBytFileBytes)
Catch
Return (Nothing)
Finally
If (Not pFESStream Is Nothing) Then
Call pFESStream.Close()
pFESStream = Nothing
End If
End Try
End Function

--------------------------------------------------------

Anyway, implementing what you desire is very simple to do. Most people
/ companies make extemely stupid protocols, Sony being one of them. I even
knew of a protocol of theirs to control a projector, it replicated remote
control commands and even had facility for presure sensitivity of button
presses!! WHY? It was a damn projector not a Playstation, oh well!!!

My main advice is keep it simple and document it well :-) The simpler
and more compact your protocol is the easier it will be to debug, maintain
and update (Just like code really).

Also one way to check to see if the remote host is still alive is to do
something called "polling", this is done by sending a simple packet which
requires a simple response (Just like a ping). If no response is recieved
within a set period the host is presumed dead and the connection is
dropped....

chr(2) & "REQ:1:Are you alive?" & chr(4)

I hope all this helps :-)

Nick.

*For the benefit it picky buggers... Jack you know who you are!

--
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
"No matter. Whatever the outcome, you are changed."

Fergus - September 5th 2003
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

[Yair Cohen]

Thanks !
u realy help me.
i thuat on something in thiss direction, u realy help in this.
thanks !
"Nak" <[e-mail]> wrote in message
news:O4**************@TK2MSFTNGP10.phx.gbl...

Hi there,

I know this one, I used to program control systems for various pieces of hardware so I know how to implement reliable protocols. Try to follow these guidelines,

*Make things as simple as possible
*Make the protocol as small as possible
*Make the protocol easily documentable

I'll give you an example of a protocol. Firstly you will need to pick
an "STX" (Start of packet) and "ETX" (End of packet) value. This should be a value that is NOT in the ASCII range, i.e. something like 2 for "STX" and 4 for "ETX". Then pick 2 letter acronyms for your headers, for example

CMD = Command
REQ = Request
RES = Response

Then pick a character to separate the header from the rest of the
packet, such as ":". Then you should simply specify a value for the index
of the Command, Request or Response, separated with the same character as
the header ":". The data can then follow. For example...

*Remember that the "STX" and "ETX" should NOT be in the ASCII range. To get the correct character for character code 2 and 4, use chr()

chr(2) & "CMD:1:THIS IS MY PACKET DATA" & chr(4)

This may seem very simply but it can be very versatile. The packet data could even contain comma delimited parameters that you can easily split at
the other end using the String.Split() method. If you want to send binary
data you could use Base64 encoding.....

--------------------------------------------------------
The following has been untested, but the principal remains the same in
testing*

-----------------
Use the following function to turn a file into a byte array
-----------------
Dim pBytFileBytes As Byte() = fileToByteArray("D:\Personal\Development\VB
dotnet\nikvue.net\Documents\slideshow v2\test\test.jpg")
Dim pStrBase64 as String = System.Convert.ToBase64String(pBytFileBytes)

***
chr(2) & "CMD:5:" & pStrBase64 & chr(4)
***

-----------------
Use the following function to turn a file into a byte array
-----------------

Private Function fileToByteArray(ByVal iPath As String) As Byte()
Dim pFESStream As FileStream
Try
pFESStream = New FileStream(iPath, FileMode.Open, FileAccess.Read)
Dim pBytFileBytes(pFESStream.Length - 1) As Byte
Dim pIntBytesRead As Integer = pFESStream.Read(pBytFileBytes, 0,
pFESStream.Length)
Call pFESStream.Close()
pFESStream = Nothing
Return (pBytFileBytes)
Catch
Return (Nothing)
Finally
If (Not pFESStream Is Nothing) Then
Call pFESStream.Close()
pFESStream = Nothing
End If
End Try
End Function

--------------------------------------------------------

Anyway, implementing what you desire is very simple to do. Most people / companies make extemely stupid protocols, Sony being one of them. I even knew of a protocol of theirs to control a projector, it replicated remote
control commands and even had facility for presure sensitivity of button
presses!! WHY? It was a damn projector not a Playstation, oh well!!!

My main advice is keep it simple and document it well :-) The simpler
and more compact your protocol is the easier it will be to debug, maintain
and update (Just like code really).

Also one way to check to see if the remote host is still alive is to do something called "polling", this is done by sending a simple packet which
requires a simple response (Just like a ping). If no response is recieved
within a set period the host is presumed dead and the connection is
dropped....

chr(2) & "REQ:1:Are you alive?" & chr(4)

I hope all this helps :-)

Nick.

*For the benefit it picky buggers... Jack you know who you are!

--
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ "No matter. Whatever the outcome, you are changed."

Fergus - September 5th 2003
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

[Nak]

> Thanks !

u realy help me.
i thuat on something in thiss direction, u realy help in this.
thanks !

My pleasure, seriously, if it is done simply you will be able to implement
anything! I managed to make a chat program once that had a drawin board and
shared internet browser done with a very simply protocol. Glad I could
help.

Nick.

--
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
"No matter. Whatever the outcome, you are changed."

Fergus - September 5th 2003
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

[Cor]

Nick
And then you discovered VB.net languages and stopped the project?
A special newsgroup to talk in all languages Very Busy over the Net.
Where is Fergus?
:-))))))))
Cor
ps. I have already read why you stopped it, this was just for fun

[Nak]

> Where is Fergus?
Hi Cor,

I think he might be busy job hunting...

orrrr downloading "rare material" ;-) ;-)

Nick.

--
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
"No matter. Whatever the outcome, you are changed."

Fergus - September 5th 2003
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\