473,326 Members | 2,196 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

HELP! - .NET sequences segmented packets before data available in NetworkStream?

Hi,

I am confused with how NetworkStream works.

My application needs to handle heavy requests sent through TCP socket
connection. I use NetworkStream.Read method to get the stream data. The
data are sent from the client servers, black boxes for my project team. But
we have an agreement to the client vendor that we exchange only fixed-length
data, say, 200 bytes per data set. A data set is a bombination of data
fields, like 56 bytes of character, followed by 4 bytes of integer, followed
by 2 bytes of characters, and so on.

I notice that the lower layer of TCP splits some packets into smaller
packets, as expected. I assumed before stream data are loaded into
NetworkStream, the smaller packets will be combined together by their
sequence number, so programmers don't need to worry about the segmentation
things of TCP stream socket. However, it seems the smaller packets are put
into the NetworkStream as soon as they are received. This causes disaster
because the only way we retrieve data sets is by the fixed length. If an
incompleted data set is read from NetworkStream, the following data sets
cannot be understood by our application.

Is it true that .NET doesn't sequence the smaller packets before they are
loaded into NetworkStream? Or do i miss something? Losts of sample codes
available on MSND and Google search, but can't find any article discussing
about this.

What could I do to sovle this problem?

Any suggestion or hint is highly appreciated!



Ryan


Nov 20 '05 #1
6 3293
"Ryan" <ry**@cradle.com.tw> wrote:
Is it true that .NET doesn't sequence the smaller packets before they are
loaded into NetworkStream?


Yes. It is called Network*Stream* because you receive a stream of bytes, there
is no 'record' or 'packet' as you are thinking of them.

In the Free Stuff folder on my website at www.abderaware.com you will find
several examples of socket programming. A couple of them use a class called
TcpPacketizer which takes care of sending and receiving records (packets) over a
socket's natural stream of bytes. You could adapt it to use NetworkStream, or
simply use the Socket class as I did in the sample projects.
--
Abderaware
Fine Components For .NET
Turn on, tune in, download.
zane a@t abderaware.com
Nov 20 '05 #2
Hi, Zane,

Thank you for your reply.

I have downloaded the codes and read through TcpPacketizer class, it seems
the codes deal with simalar things that networkStream does, in asynchronous
way. But I can't find the lines dealing with fragmented packets. If the
codes have taken that into consideration, could you please show me which
block of the class handles the job related to combining fragmented packets?

If the codes don't mean to deal with that, do you have any suggestion for us
to tackle down the problem?

BTW, interesting site, i showed it to my colleages.

Ryan
"Zane Thomas [.NET/C# MVP]" <za**@abderaware.com> wrote in message
news:3f***************@news.microsoft.com...
"Ryan" <ry**@cradle.com.tw> wrote:
Is it true that .NET doesn't sequence the smaller packets before they are
loaded into NetworkStream?
Yes. It is called Network*Stream* because you receive a stream of bytes,

there is no 'record' or 'packet' as you are thinking of them.

In the Free Stuff folder on my website at www.abderaware.com you will find
several examples of socket programming. A couple of them use a class called TcpPacketizer which takes care of sending and receiving records (packets) over a socket's natural stream of bytes. You could adapt it to use NetworkStream, or simply use the Socket class as I did in the sample projects.
--
Abderaware
Fine Components For .NET
Turn on, tune in, download.
zane a@t abderaware.com

Nov 20 '05 #3
Ryan,
But I can't find the lines dealing with fragmented packets. If the
codes have taken that into consideration, could you please show me which
block of the class handles the job related to combining fragmented packets?


The code can be somewhat confusing because it's all asynchronous. Maybe I
should write the blocking methods too. :-)

Before a 'packet' is written by the sender a four-byte length is written out,
and then the data itself. On the receive side it goes like this:

1. An instance is created and initialized with a socket to use.

2. It invokes Socket.BeginReceive passing OnReceiveLength as the method to
invoke when some data arrives.

3. OnReceiveLength reads upto four bytes (no more than four!), if there are not
yet four bytes goto 2 (basically, the code is inline in OnReceiveLength).

4. Once four bytes have been received the packet length is known and the
BeginReceive is invoked again, this time with OnReceiveMessage as the callback.

5. In OnReceiveMessage as many bytes as possible, upto but not exceeded the
packet length from step 3, are read and saved.

6. If the packet is not yet complete BeginReceive is invoked with
OnReceiveMessage as the callback, again. If the packet is complete the
TcpPacketizer user's callback is invoked and the code (inline again) returns to
step 2.
Blocking this would be:

1. Keep reading until you have four bytes. That's the length.
2. Keep reading until you have received a length's worth of bytes.

That's all.

--
Abderaware
Fine Components For .NET
Turn on, tune in, download.
zane a@t abderaware.com
Nov 20 '05 #4
Zane,

I can't thank you enough for your patient explanation.

I start to believe what we are dealing with is a little bit different. We
are doing ok with retrieving the stream data of a single packet from
networkstream, since the data sender agrees to send a fixed-length data set.
However, if a data set (or a message package) is splited into two fragmented
packets, and if these two fragmented packets are separated by other
packet(s) in NETWORKSTREAM ( i have no idea if this could happen or not?!)
where we will retrive the stream data, the programmer will have a nightmare
to put them back together.
If this is not very clear to you, drop me a message. I will draw a picture
to explain that and send it to your email account.
Ryan


"Zane Thomas [.NET/C# MVP]" <za**@abderaware.com> wrote in message
news:3f***************@news.microsoft.com...
Ryan,
But I can't find the lines dealing with fragmented packets. If the
codes have taken that into consideration, could you please show me which
block of the class handles the job related to combining fragmented
packets?
The code can be somewhat confusing because it's all asynchronous. Maybe I
should write the blocking methods too. :-)

Before a 'packet' is written by the sender a four-byte length is written out, and then the data itself. On the receive side it goes like this:

1. An instance is created and initialized with a socket to use.

2. It invokes Socket.BeginReceive passing OnReceiveLength as the method to invoke when some data arrives.

3. OnReceiveLength reads upto four bytes (no more than four!), if there are not yet four bytes goto 2 (basically, the code is inline in OnReceiveLength).

4. Once four bytes have been received the packet length is known and the
BeginReceive is invoked again, this time with OnReceiveMessage as the callback.
5. In OnReceiveMessage as many bytes as possible, upto but not exceeded the packet length from step 3, are read and saved.

6. If the packet is not yet complete BeginReceive is invoked with
OnReceiveMessage as the callback, again. If the packet is complete the
TcpPacketizer user's callback is invoked and the code (inline again) returns to step 2.
Blocking this would be:

1. Keep reading until you have four bytes. That's the length.
2. Keep reading until you have received a length's worth of bytes.

That's all.

--
Abderaware
Fine Components For .NET
Turn on, tune in, download.
zane a@t abderaware.com

Nov 20 '05 #5
Ryan,
I start to believe what we are dealing with is a little bit different. We
are doing ok with retrieving the stream data of a single packet from
networkstream, since the data sender agrees to send a fixed-length data set.
However, if a data set (or a message package) is splited into two fragmented
packets, and if these two fragmented packets are separated by other
packet(s) in NETWORKSTREAM ( i have no idea if this could happen or not?!)
where we will retrive the stream data, the programmer will have a nightmare
to put them back together.


Ok, I think I understand the situation. The first thing you should know is that
if the sender sends you two records A and B, it is true that both A and B may
arrive either in one Receive or in multiple Receives ... but no data for B will
be Received until all the data for A has been Received. TCP takes care of that
for you.

What TCP _stream_ sockets do not do is guarantee that either A or B will be
received with one invocation of Receive. You might get all of A, and then B in
three parts, or A in two parts and B in one part. There is no predicting, it
depends upon network and socket stack conditions.

So ... if you're sending Records then you and the sender must agree on what a
record is. In the common internet protocols (smtp for instance) a record is a
string of characters followed by "crlf". But that's not a big help if you're
sending binary data. If you're sending arbitrary binary data that has no
recognizable record-start or record-end delimeters then you must precede the
binary data with a byte count. TcpPacketizer assumes the worst case and sends a
byte count before each record.

You and the data sender must cooperate in order to correctly send and receive
packets.
--
Abderaware
Fine Components For .NET
Turn on, tune in, download.
zane a@t abderaware.com
Nov 20 '05 #6
Zane,

Now I am totally there.

Thank you so much. I appreciate your assistance.

Ryan
"Zane Thomas [.NET/C# MVP]" <za**@abderaware.com> wrote in message
news:3f***************@news.microsoft.com...
Ryan,
I start to believe what we are dealing with is a little bit different. Weare doing ok with retrieving the stream data of a single packet from
networkstream, since the data sender agrees to send a fixed-length data set.However, if a data set (or a message package) is splited into two fragmentedpackets, and if these two fragmented packets are separated by other
packet(s) in NETWORKSTREAM ( i have no idea if this could happen or not?!)where we will retrive the stream data, the programmer will have a nightmareto put them back together.
Ok, I think I understand the situation. The first thing you should know

is that if the sender sends you two records A and B, it is true that both A and B may arrive either in one Receive or in multiple Receives ... but no data for B will be Received until all the data for A has been Received. TCP takes care of that for you.

What TCP _stream_ sockets do not do is guarantee that either A or B will be received with one invocation of Receive. You might get all of A, and then B in three parts, or A in two parts and B in one part. There is no predicting, it depends upon network and socket stack conditions.

So ... if you're sending Records then you and the sender must agree on what a record is. In the common internet protocols (smtp for instance) a record is a string of characters followed by "crlf". But that's not a big help if you're sending binary data. If you're sending arbitrary binary data that has no
recognizable record-start or record-end delimeters then you must precede the binary data with a byte count. TcpPacketizer assumes the worst case and sends a byte count before each record.

You and the data sender must cooperate in order to correctly send and receive packets.
--
Abderaware
Fine Components For .NET
Turn on, tune in, download.
zane a@t abderaware.com

Nov 20 '05 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

9
by: Tom | last post by:
A question for gui application programmers. . . I 've got some GUI programs, written in Python/wxPython, and I've got a help button and a help menu item. Also, I've got a compiled file made with...
4
by: Sarir Khamsi | last post by:
Is there a way to get help the way you get it from the Python interpreter (eg, 'help(dir)' gives help on the 'dir' command) in the module cmd.Cmd? I know how to add commands and help text to...
2
by: Sudheer Kareem | last post by:
Dear All Please tell me how to assosiate help files with my Vb.net Project. Regards Sudheer
6
by: wukexin | last post by:
Help me, good men. I find mang books that introduce bit "mang header files",they talk too bit,in fact it is my too fool, I don't learn it, I have do a test program, but I have no correct doing...
6
by: d.warnermurray | last post by:
I am doing a project for school that involves creating help files for a html authoring tool. If you could help me with answers to some questions it would really help. 1. What tasks do you expect...
0
by: tbatwork828 | last post by:
If you were like me trying to figure out how to launch context sensitive help topic by the context id, here is the link: http://weblogs.asp.net/kencox/archive/2004/09/12/228349.aspx and if...
8
by: Mark | last post by:
I have loaded Visual Studio .net on my home computer and my laptop, but my home computer has an abbreviated help screen not 2% of the help on my laptop. All the settings look the same on both...
10
by: JonathanOrlev | last post by:
Hello everybody, I wrote this comment in another message of mine, but decided to post it again as a standalone message. I think that Microsoft's Office 2003 help system is horrible, probably...
1
by: trunxnirvana007 | last post by:
'UPGRADE_WARNING: Array has a new behavior. Click for more: 'ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?keyword="9B7D5ADD-D8FE-4819-A36C-6DEDAF088CC7"' 'UPGRADE_WARNING: Couldn't resolve...
0
by: hitencontractor | last post by:
I am working on .NET Version 2003 making an SDI application that calls MS Excel 2003. I added a menu item called "MyApp Help" in the end of the menu bar to show Help-> About. The application...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.