Hi,
I am confused with how NetworkStream works.
My application needs to handle heavy requests sent through TCP socket
connection. I use NetworkStream.Read method to get the stream data. The
data are sent from the client servers, black boxes for my project team. But
we have an agreement to the client vendor that we exchange only fixed-length
data, say, 200 bytes per data set. A data set is a bombination of data
fields, like 56 bytes of character, followed by 4 bytes of integer, followed
by 2 bytes of characters, and so on.
I notice that the lower layer of TCP splits some packets into smaller
packets, as expected. I assumed before stream data are loaded into
NetworkStream, the smaller packets will be combined together by their
sequence number, so programmers don't need to worry about the segmentation
things of TCP stream socket. However, it seems the smaller packets are put
into the NetworkStream as soon as they are received. This causes disaster
because the only way we retrieve data sets is by the fixed length. If an
incompleted data set is read from NetworkStream, the following data sets
cannot be understood by our application.
Is it true that .NET doesn't sequence the smaller packets before they are
loaded into NetworkStream? Or do i miss something? Losts of sample codes
available on MSND and Google search, but can't find any article discussing
about this.
What could I do to sovle this problem?
Any suggestion or hint is highly appreciated!
Ryan 6 3293
"Ryan" <ry**@cradle.com.tw> wrote: Is it true that .NET doesn't sequence the smaller packets before they are loaded into NetworkStream?
Yes. It is called Network*Stream* because you receive a stream of bytes, there
is no 'record' or 'packet' as you are thinking of them.
In the Free Stuff folder on my website at www.abderaware.com you will find
several examples of socket programming. A couple of them use a class called
TcpPacketizer which takes care of sending and receiving records (packets) over a
socket's natural stream of bytes. You could adapt it to use NetworkStream, or
simply use the Socket class as I did in the sample projects.
--
Abderaware
Fine Components For .NET
Turn on, tune in, download.
zane a@t abderaware.com
Hi, Zane,
Thank you for your reply.
I have downloaded the codes and read through TcpPacketizer class, it seems
the codes deal with simalar things that networkStream does, in asynchronous
way. But I can't find the lines dealing with fragmented packets. If the
codes have taken that into consideration, could you please show me which
block of the class handles the job related to combining fragmented packets?
If the codes don't mean to deal with that, do you have any suggestion for us
to tackle down the problem?
BTW, interesting site, i showed it to my colleages.
Ryan
"Zane Thomas [.NET/C# MVP]" <za**@abderaware.com> wrote in message
news:3f***************@news.microsoft.com... "Ryan" <ry**@cradle.com.tw> wrote:
Is it true that .NET doesn't sequence the smaller packets before they are loaded into NetworkStream? Yes. It is called Network*Stream* because you receive a stream of bytes,
there is no 'record' or 'packet' as you are thinking of them.
In the Free Stuff folder on my website at www.abderaware.com you will find several examples of socket programming. A couple of them use a class
called TcpPacketizer which takes care of sending and receiving records (packets)
over a socket's natural stream of bytes. You could adapt it to use
NetworkStream, or simply use the Socket class as I did in the sample projects.
-- Abderaware Fine Components For .NET Turn on, tune in, download. zane a@t abderaware.com
Ryan, But I can't find the lines dealing with fragmented packets. If the codes have taken that into consideration, could you please show me which block of the class handles the job related to combining fragmented packets?
The code can be somewhat confusing because it's all asynchronous. Maybe I
should write the blocking methods too. :-)
Before a 'packet' is written by the sender a four-byte length is written out,
and then the data itself. On the receive side it goes like this:
1. An instance is created and initialized with a socket to use.
2. It invokes Socket.BeginReceive passing OnReceiveLength as the method to
invoke when some data arrives.
3. OnReceiveLength reads upto four bytes (no more than four!), if there are not
yet four bytes goto 2 (basically, the code is inline in OnReceiveLength).
4. Once four bytes have been received the packet length is known and the
BeginReceive is invoked again, this time with OnReceiveMessage as the callback.
5. In OnReceiveMessage as many bytes as possible, upto but not exceeded the
packet length from step 3, are read and saved.
6. If the packet is not yet complete BeginReceive is invoked with
OnReceiveMessage as the callback, again. If the packet is complete the
TcpPacketizer user's callback is invoked and the code (inline again) returns to
step 2.
Blocking this would be:
1. Keep reading until you have four bytes. That's the length.
2. Keep reading until you have received a length's worth of bytes.
That's all.
--
Abderaware
Fine Components For .NET
Turn on, tune in, download.
zane a@t abderaware.com
Zane,
I can't thank you enough for your patient explanation.
I start to believe what we are dealing with is a little bit different. We
are doing ok with retrieving the stream data of a single packet from
networkstream, since the data sender agrees to send a fixed-length data set.
However, if a data set (or a message package) is splited into two fragmented
packets, and if these two fragmented packets are separated by other
packet(s) in NETWORKSTREAM ( i have no idea if this could happen or not?!)
where we will retrive the stream data, the programmer will have a nightmare
to put them back together.
If this is not very clear to you, drop me a message. I will draw a picture
to explain that and send it to your email account.
Ryan
"Zane Thomas [.NET/C# MVP]" <za**@abderaware.com> wrote in message
news:3f***************@news.microsoft.com... Ryan,
But I can't find the lines dealing with fragmented packets. If the codes have taken that into consideration, could you please show me which block of the class handles the job related to combining fragmented
packets? The code can be somewhat confusing because it's all asynchronous. Maybe I should write the blocking methods too. :-)
Before a 'packet' is written by the sender a four-byte length is written
out, and then the data itself. On the receive side it goes like this:
1. An instance is created and initialized with a socket to use.
2. It invokes Socket.BeginReceive passing OnReceiveLength as the method
to invoke when some data arrives.
3. OnReceiveLength reads upto four bytes (no more than four!), if there
are not yet four bytes goto 2 (basically, the code is inline in OnReceiveLength).
4. Once four bytes have been received the packet length is known and the BeginReceive is invoked again, this time with OnReceiveMessage as the
callback. 5. In OnReceiveMessage as many bytes as possible, upto but not exceeded
the packet length from step 3, are read and saved.
6. If the packet is not yet complete BeginReceive is invoked with OnReceiveMessage as the callback, again. If the packet is complete the TcpPacketizer user's callback is invoked and the code (inline again)
returns to step 2.
Blocking this would be:
1. Keep reading until you have four bytes. That's the length. 2. Keep reading until you have received a length's worth of bytes.
That's all. -- Abderaware Fine Components For .NET Turn on, tune in, download. zane a@t abderaware.com
Ryan, I start to believe what we are dealing with is a little bit different. We are doing ok with retrieving the stream data of a single packet from networkstream, since the data sender agrees to send a fixed-length data set. However, if a data set (or a message package) is splited into two fragmented packets, and if these two fragmented packets are separated by other packet(s) in NETWORKSTREAM ( i have no idea if this could happen or not?!) where we will retrive the stream data, the programmer will have a nightmare to put them back together.
Ok, I think I understand the situation. The first thing you should know is that
if the sender sends you two records A and B, it is true that both A and B may
arrive either in one Receive or in multiple Receives ... but no data for B will
be Received until all the data for A has been Received. TCP takes care of that
for you.
What TCP _stream_ sockets do not do is guarantee that either A or B will be
received with one invocation of Receive. You might get all of A, and then B in
three parts, or A in two parts and B in one part. There is no predicting, it
depends upon network and socket stack conditions.
So ... if you're sending Records then you and the sender must agree on what a
record is. In the common internet protocols (smtp for instance) a record is a
string of characters followed by "crlf". But that's not a big help if you're
sending binary data. If you're sending arbitrary binary data that has no
recognizable record-start or record-end delimeters then you must precede the
binary data with a byte count. TcpPacketizer assumes the worst case and sends a
byte count before each record.
You and the data sender must cooperate in order to correctly send and receive
packets.
--
Abderaware
Fine Components For .NET
Turn on, tune in, download.
zane a@t abderaware.com
Zane,
Now I am totally there.
Thank you so much. I appreciate your assistance.
Ryan
"Zane Thomas [.NET/C# MVP]" <za**@abderaware.com> wrote in message
news:3f***************@news.microsoft.com... Ryan,
I start to believe what we are dealing with is a little bit different.
Weare doing ok with retrieving the stream data of a single packet from networkstream, since the data sender agrees to send a fixed-length data
set.However, if a data set (or a message package) is splited into two
fragmentedpackets, and if these two fragmented packets are separated by other packet(s) in NETWORKSTREAM ( i have no idea if this could happen or
not?!)where we will retrive the stream data, the programmer will have a
nightmareto put them back together. Ok, I think I understand the situation. The first thing you should know
is that if the sender sends you two records A and B, it is true that both A and B
may arrive either in one Receive or in multiple Receives ... but no data for B
will be Received until all the data for A has been Received. TCP takes care of
that for you.
What TCP _stream_ sockets do not do is guarantee that either A or B will
be received with one invocation of Receive. You might get all of A, and then
B in three parts, or A in two parts and B in one part. There is no predicting,
it depends upon network and socket stack conditions.
So ... if you're sending Records then you and the sender must agree on
what a record is. In the common internet protocols (smtp for instance) a record
is a string of characters followed by "crlf". But that's not a big help if
you're sending binary data. If you're sending arbitrary binary data that has no recognizable record-start or record-end delimeters then you must precede
the binary data with a byte count. TcpPacketizer assumes the worst case and
sends a byte count before each record.
You and the data sender must cooperate in order to correctly send and
receive packets.
-- Abderaware Fine Components For .NET Turn on, tune in, download. zane a@t abderaware.com This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Tom |
last post by:
A question for gui application programmers. . .
I 've got some GUI programs, written in Python/wxPython, and I've got
a help button and a help menu item. Also, I've got a compiled file
made with...
|
by: Sarir Khamsi |
last post by:
Is there a way to get help the way you get it from the Python
interpreter (eg, 'help(dir)' gives help on the 'dir' command) in the
module cmd.Cmd? I know how to add commands and help text to...
|
by: Sudheer Kareem |
last post by:
Dear All
Please tell me how to assosiate help files with my Vb.net
Project.
Regards
Sudheer
|
by: wukexin |
last post by:
Help me, good men. I find mang books that introduce bit "mang header
files",they talk too bit,in fact it is my too fool, I don't learn it, I have
do a test program, but I have no correct doing...
|
by: d.warnermurray |
last post by:
I am doing a project for school that involves creating help files for a html
authoring tool.
If you could help me with answers to some questions it would really help.
1. What tasks do you expect...
|
by: tbatwork828 |
last post by:
If you were like me trying to figure out how to launch context
sensitive help topic by the context id, here is the link:
http://weblogs.asp.net/kencox/archive/2004/09/12/228349.aspx
and if...
|
by: Mark |
last post by:
I have loaded Visual Studio .net on my home computer and my laptop, but my
home computer has an abbreviated help screen not 2% of the help on my laptop.
All the settings look the same on both...
|
by: JonathanOrlev |
last post by:
Hello everybody,
I wrote this comment in another message of mine, but decided to post it
again as a standalone message.
I think that Microsoft's Office 2003 help system is horrible, probably...
|
by: trunxnirvana007 |
last post by:
'UPGRADE_WARNING: Array has a new behavior. Click for more: 'ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?keyword="9B7D5ADD-D8FE-4819-A36C-6DEDAF088CC7"'
'UPGRADE_WARNING: Couldn't resolve...
|
by: hitencontractor |
last post by:
I am working on .NET Version 2003 making an SDI application that calls MS Excel 2003.
I added a menu item called "MyApp Help" in the end of the menu bar to show Help-> About.
The application...
|
by: DolphinDB |
last post by:
Tired of spending countless mintues downsampling your data? Look no further!
In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
|
by: isladogs |
last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM).
In this month's session, we are pleased to welcome back...
|
by: Vimpel783 |
last post by:
Hello!
Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
|
by: jfyes |
last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
|
by: Defcon1945 |
last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
|
by: Shællîpôpï 09 |
last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
|
by: af34tf |
last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
|
by: Faith0G |
last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome former...
| |