469,358 Members | 1,610 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,358 developers. It's quick & easy.

Asynchronous sockets and buffer size

been using asynchronous sockets with AsyncCallback and
System.Net.Sockets as per the excellent MSDN example.

everything has been working great, except when i want to receive more
total bytes than my buffer can hold.

i open a connection with an 8k buffer on the socket. if the server
sends me 5k and closes connection it works fine. if the server sends me
10k, my BeginReceive callback (which does loop) gives me 8k of the data,
after which EndReceive returns 0 and so i close my socket.

however, if i set the buffer to 32k i can get all the data.

is it supposed to work like that?
May 26 '07 #1
16 8270
Matthew, I am not quite sure which 'excellent MSDN example' you were
following. Normally TCP/IP servers (I assume you do TCP/IP) do not imitate
disconnect - clients do. On the server side do not terminate client
connection until the client disconnects - BeginReceive() on the server side
returns 0, and you will not loose data.

Michael

"Matthew Geyer" <no****@geyer.matt.gmail.comwrote in message
news:ui**************@TK2MSFTNGP05.phx.gbl...
been using asynchronous sockets with AsyncCallback and System.Net.Sockets
as per the excellent MSDN example.

everything has been working great, except when i want to receive more
total bytes than my buffer can hold.

i open a connection with an 8k buffer on the socket. if the server sends
me 5k and closes connection it works fine. if the server sends me 10k, my
BeginReceive callback (which does loop) gives me 8k of the data, after
which EndReceive returns 0 and so i close my socket.

however, if i set the buffer to 32k i can get all the data.

is it supposed to work like that?

May 26 '07 #2
Sorry, I don't know what I expected with an explanation like that.

here's what my code is like with all the fluff cut out:

private void OnReceive(IAsyncResult asyn)
{
ObjectThing Job = (ObjectThing)asyn.AsyncState;
long BytesReceived = Job.s.EndReceive(asyn);

if (BytesReceived 0)
{
ProcessData(Job);
Job.s.BeginReceive(Job.buffer, 0, Job.buffer.Length, 0, new
AsyncCallback(this.OnReceive), Job);
}

else
{
Job.s.Close();
Job.s = null;
}

}

private void OnSend(IAsyncResult asyn)
{
ObjectThing Job = (ObjectThing)asyn.AsyncState;
Job.s.EndSend(asyn);
Job.s.BeginReceive(Job.buffer, 0, Job.buffer.Length, 0, new
AsyncCallback(this.OnReceive), Job);
}

private void OnConnect(IAsyncResult asyn)
{

ObjectThing Job = (ObjectThing)asyn.AsyncState;
Job.s.EndConnect(asyn);
Job.s.BeginSend(Encoding.ASCII.GetBytes(Job.data + "\r\n"), 0,
Encoding.ASCII.GetBytes(Job.data + "\r\n").Length, 0, new
AsyncCallback(this.OnSend), Job);

}
Job.s is the socket and i call Job.s.BeginConnect(...) to start.
Michael Rubinstein wrote:
Matthew, I am not quite sure which 'excellent MSDN example' you were
following. Normally TCP/IP servers (I assume you do TCP/IP) do not imitate
disconnect - clients do. On the server side do not terminate client
connection until the client disconnects - BeginReceive() on the server side
returns 0, and you will not loose data.

Michael
May 27 '07 #3
On Sat, 26 May 2007 22:07:52 -0700, Matthew Geyer
<no****@geyer.matt.gmail.comwrote:
Sorry, I don't know what I expected with an explanation like that.

here's what my code is like with all the fluff cut out: [snip]
Well, I see at least one problem: you don't check the byte count for
Socket.EndSend(). From the documentation:

If you are using a connection-oriented protocol, EndSend will block
until
some of the buffer was sent. If the return value from EndSend indicates
that the buffer was not completely sent, call the BeginSend method
again,
modifying the buffer to hold the unsent data.

Assuming you have the same error in your server, then it's not clear to me
that the server is actually sending all of the data you expect it to send.

The other thing I note is that you are using Socket.Close() to terminate
the connection. However, that will reset the connection abortively. You
should use Socket.Shutdown() to ensure a graceful closure. Again, if the
server has the same error, it could be that it's aborting the socket
connection before the client has a chance to receive all of the data. If
the client and server use Socket.Shutdown(), then the connection will
remain half-connected until the client has actually received all of the
data that has been sent.

Since you didn't actually post all of the code involved in your question,
it's hard to say for sure what's going wrong. Suffice to say that it's
not normal for one end to not be able to receive all of the data the other
end sent. If you are having that problem, there's a bug in your code
somewhere. Possibly it's related to the above.

Pete
May 27 '07 #4
Thanks for the tips. I wasn't encountering a problem with EndSend yet
because I was lucky enough to be sending all the data each time. Now I
check the byte count.

I also added Socket.Shutdown(SocketShutdown.Both) before I close the
socket. Still having the same problem.

That's pretty much all of my network code except for the try catch
blocks where i catch SocketExceptions, the part where i create the
socket, and the part where it begins with Socket.BeginConnect. (Though
the BeginConnect is called by a BackgroundWorker)

The buffer only fills up once, after which Socket.EndReceive returns 0.

If I ignore the 0 that EndReceive returns and try another BeginReceive
for the hell of it nothing happens.
May 27 '07 #5
Matthew, from your original message:
>i open a connection with an 8k buffer on the socket. if the server sends
me 5k and closes connection it works fine.
The code you posted seems to be for the client side. You call BeginReceive()
from two methods - OnReceive() and OnSend(). Calling BeginReceive() after
EndSend() is odd, to say the least. You call BeginSend() from OnConnect(),
which is fine if you do so to tell the server 'send my some data'. If it is
not the case and you send the very data that you loose on the other side of
the connection, then you call things wrong names. Server is the socket that
'listens' and 'accepts'. Client is the socket that 'connects'. Both can
receive and send. If your client socket sends data and then closes the
connection upon EndSend() on the client side, then you should first call
ShutDown(SocketShutdown.Both), and not right after EndSend(). Normally you
should close client socket connection either after EndReceive() returned 0
bytes and the application does not need that connection any more, or there
was no receive data for some time (timer) and the application code can
'safely assume' you are done, or by calls from GUI controls (buttons, menu,
etc), including application exit. I don't recall ever calling
Socket.ShutDown()/Socket.Close() from within OnSend() callback that did not
generate socket errors.I assume you trap socket errors in the 'fluffed'
code? What do you have on the server side? Is it also your code? By the way,
I keep my socket buffers at 4096 and don't loose data.

Michael

"Matthew Geyer" <no****@geyer.matt.gmail.comwrote in message
news:ex**************@TK2MSFTNGP02.phx.gbl...
Sorry, I don't know what I expected with an explanation like that.

here's what my code is like with all the fluff cut out:

private void OnReceive(IAsyncResult asyn)
{
ObjectThing Job = (ObjectThing)asyn.AsyncState;
long BytesReceived = Job.s.EndReceive(asyn);

if (BytesReceived 0)
{
ProcessData(Job);
Job.s.BeginReceive(Job.buffer, 0, Job.buffer.Length, 0, new
AsyncCallback(this.OnReceive), Job);
}

else
{
Job.s.Close();
Job.s = null;
}

}

private void OnSend(IAsyncResult asyn)
{
ObjectThing Job = (ObjectThing)asyn.AsyncState;
Job.s.EndSend(asyn);
Job.s.BeginReceive(Job.buffer, 0, Job.buffer.Length, 0, new
AsyncCallback(this.OnReceive), Job);
}

private void OnConnect(IAsyncResult asyn)
{

ObjectThing Job = (ObjectThing)asyn.AsyncState;
Job.s.EndConnect(asyn);
Job.s.BeginSend(Encoding.ASCII.GetBytes(Job.data + "\r\n"), 0,
Encoding.ASCII.GetBytes(Job.data + "\r\n").Length, 0, new
AsyncCallback(this.OnSend), Job);

}
Job.s is the socket and i call Job.s.BeginConnect(...) to start.
Michael Rubinstein wrote:
> Matthew, I am not quite sure which 'excellent MSDN example' you were
following. Normally TCP/IP servers (I assume you do TCP/IP) do not
imitate disconnect - clients do. On the server side do not terminate
client connection until the client disconnects - BeginReceive() on the
server side returns 0, and you will not loose data.

Michael

May 27 '07 #6
On Sun, 27 May 2007 02:37:12 -0700, Matthew Geyer
<no****@geyer.matt.gmail.comwrote:
[...]
That's pretty much all of my network code except for the try catch
blocks where i catch SocketExceptions, the part where i create the
socket, and the part where it begins with Socket.BeginConnect. (Though
the BeginConnect is called by a BackgroundWorker)
Well, it obviously is not all of the network code, since you didn't post
anything at all from the server side.
The buffer only fills up once, after which Socket.EndReceive returns 0.
EndReceive() isn't going to return 0 for the byte count unless the
connection has been closed. After it returns 0, of course you will not
have any more successful reads from the socket.

Either you have a bug in the receiving code, in which you actually receive
more data than you have accounted for somehow, or you have a bug in the
sending code, in which the connection is closed before the client has had
a chance to receive all of the data. Without a complete-but-concise
sample code that reliably reproduces the problem to look at, it would be
impossible for anyone reading your posts to know what's actually
happening. But it pretty much has to be one of those two things.

Pete
May 27 '07 #7
On Sun, 27 May 2007 09:15:52 -0700, Michael Rubinstein
<mSPAM_REMOVEr@mĀ®ubinstein.comwrote:
The code you posted seems to be for the client side. You call
BeginReceive()
from two methods - OnReceive() and OnSend(). Calling BeginReceive() after
EndSend() is odd, to say the least. You call BeginSend() from OnConnect()
It is unusual, but not a fundamental problem, I think. All it means is
that he has no code ready to receive data until he's completed his first
send. In this case, it appears that his server won't send data until the
client's send has fully completed, so it's unlikely there will be any
noticeable difference.

In other situations, it is true that problems could occur. For example,
if the remote end is trying to send data, but reaches a point where no
more data can be queued because the local end isn't receiving it, while at
the same time the local end is trying to send data but reaches a point
where no more data can be queued because the remote end is busy trying to
send data and hasn't started receiving yet. Network deadlock.

Normally, a network application should post the first receive right away
and always be ready to handle incoming data. But it doesn't seem that
that's a problem here, since the remote end is only sending data in
response to the data sent by the local end, and so won't even start trying
to send data until the local end has finished sending.
[...] If your client socket sends data and then closes the
connection upon EndSend() on the client side, then you should first call
ShutDown(SocketShutdown.Both), and not right after EndSend().
As far as I know, the code he posted is for the client side, and he surely
didn't post any code that closes the socket right after sending. He only
closes the socket once he gets the 0 byte receive.
[...] By the way,
I keep my socket buffers at 4096 and don't loose data.
You can have socket buffers only 1 byte long with TCP, and you will still
not lose data. Using TCP, the buffer size is mainly related to
efficiency. The protocol is reliable, and you're assured of getting all
of your data, assuming you use the protocol correctly.

Pete
May 27 '07 #8
Pete,
It is unusual, but not a fundamental problem, I think. All it means is
that he has no code ready to receive data until he's completed his first
send. In this case, it appears that his server won't send data until the
client's send has fully completed, so it's unlikely there will be any
noticeable difference.
Well, if the client socket disconnects after completing the first send and
server socket did not send any data prior to it, then client will receive no
data at all.

The Socket.Close() in OnSend() callback draw my attention because of the
ambiguity - which side looses data? If the client did not receive all data
because the server 'brutally' terminated the connection, then what does
Socket.Close() after Socket.EndSend() on the client side do? If the server
socket missed data, then Matthew needs posting code for the server socket
side, provided it his code.

Michael

"Peter Duniho" <Np*********@nnowslpianmk.comwrote in message
news:op***************@petes-computer.local...
On Sun, 27 May 2007 09:15:52 -0700, Michael Rubinstein
<mSPAM_REMOVEr@m®ubinstein.comwrote:
>The code you posted seems to be for the client side. You call
BeginReceive()
from two methods - OnReceive() and OnSend(). Calling BeginReceive() after
EndSend() is odd, to say the least. You call BeginSend() from OnConnect()

It is unusual, but not a fundamental problem, I think. All it means is
that he has no code ready to receive data until he's completed his first
send. In this case, it appears that his server won't send data until the
client's send has fully completed, so it's unlikely there will be any
noticeable difference.

In other situations, it is true that problems could occur. For example,
if the remote end is trying to send data, but reaches a point where no
more data can be queued because the local end isn't receiving it, while at
the same time the local end is trying to send data but reaches a point
where no more data can be queued because the remote end is busy trying to
send data and hasn't started receiving yet. Network deadlock.

Normally, a network application should post the first receive right away
and always be ready to handle incoming data. But it doesn't seem that
that's a problem here, since the remote end is only sending data in
response to the data sent by the local end, and so won't even start trying
to send data until the local end has finished sending.
>[...] If your client socket sends data and then closes the
connection upon EndSend() on the client side, then you should first call
ShutDown(SocketShutdown.Both), and not right after EndSend().

As far as I know, the code he posted is for the client side, and he surely
didn't post any code that closes the socket right after sending. He only
closes the socket once he gets the 0 byte receive.
>[...] By the way,
I keep my socket buffers at 4096 and don't loose data.

You can have socket buffers only 1 byte long with TCP, and you will still
not lose data. Using TCP, the buffer size is mainly related to
efficiency. The protocol is reliable, and you're assured of getting all
of your data, assuming you use the protocol correctly.

Pete

May 27 '07 #9
On Sun, 27 May 2007 15:46:19 -0700, Michael Rubinstein
<mSPAM_REMOVEr@mĀ®ubinstein.comwrote:
Well, if the client socket disconnects after completing the first send
and
server socket did not send any data prior to it, then client will
receive no
data at all.
I see no indication the code in question does that though.
The Socket.Close() in OnSend() callback draw my attention because of the
ambiguity
What Socket.Close() in OnSend()?
- which side looses data? If the client did not receive all data
because the server 'brutally' terminated the connection, then what does
Socket.Close() after Socket.EndSend() on the client side do? If the
server
socket missed data, then Matthew needs posting code for the server socket
side, provided it his code.
I really have no idea what you're talking about. It's as if you're
referring to some code other than that posted in this article:

http://groups.google.com/group/micro...a?dmode=source

Pete

(p.s. the word is "loses", not "looses")
May 27 '07 #10
What Socket.Close() in OnSend()?
Disregard, there is no Close() in OnSend() in Matthew's code. Sorry about
the confusion.

Michael

"Peter Duniho" <Np*********@nnowslpianmk.comwrote in message
news:op***************@petes-computer.local...
On Sun, 27 May 2007 15:46:19 -0700, Michael Rubinstein
<mSPAM_REMOVEr@m®ubinstein.comwrote:
>Well, if the client socket disconnects after completing the first send
and
server socket did not send any data prior to it, then client will
receive no
data at all.

I see no indication the code in question does that though.
>The Socket.Close() in OnSend() callback draw my attention because of the
ambiguity

What Socket.Close() in OnSend()?
>- which side looses data? If the client did not receive all data
because the server 'brutally' terminated the connection, then what does
Socket.Close() after Socket.EndSend() on the client side do? If the
server
socket missed data, then Matthew needs posting code for the server socket
side, provided it his code.

I really have no idea what you're talking about. It's as if you're
referring to some code other than that posted in this article:

http://groups.google.com/group/micro...a?dmode=source

Pete

(p.s. the word is "loses", not "looses")

May 28 '07 #11
well, i finally fixed my problem. i had a couple bugs in how i was
processing the received data. for some retarded reason i was dumping the
contents of the whole buffer, not just the portion of the buffer that
was filled...thus giving me strange behavior on decoding it from UTF8.

the network code was mostly okay, luckily i hadn't encountered the Send
bug yet, now i check how many bytes were sent.

i also properly close the connection now with Socket.Shutdown, though i
did spend a few minutes trying to figure out why Socket.BeginDisconnect
throws a bad argument exception instead of a platform not supported
exception.

i also took Michael's advice and setup my Socket.BeginReceive sooner.
it actually wasn't a problem as far as this application goes, but i do
admit it was a little strange looking.

thank you both (Peter & Michael) for helping me, and especially Peter
for pointing out the Socket.EndSend bug that i probably never would have
noticed.

kind of embarrassed i had issues with such simple stuff, this time i'll
blame the tonsilitis + fever i'm experiencing ;)
May 28 '07 #12
Matthew,
i also took Michael's advice and setup my Socket.BeginReceive sooner. it
actually wasn't a problem as far as this application goes, but i do admit
it was a little strange looking.
I am actually embarrassed by the advice I gave you. Nothing wrong with that
part of your code.

Good luck, Michael

"Matthew Geyer" <no****@geyer.matt.gmail.comwrote in message
news:Oe**************@TK2MSFTNGP05.phx.gbl...
well, i finally fixed my problem. i had a couple bugs in how i was
processing the received data. for some retarded reason i was dumping the
contents of the whole buffer, not just the portion of the buffer that was
filled...thus giving me strange behavior on decoding it from UTF8.

the network code was mostly okay, luckily i hadn't encountered the Send
bug yet, now i check how many bytes were sent.

i also properly close the connection now with Socket.Shutdown, though i
did spend a few minutes trying to figure out why Socket.BeginDisconnect
throws a bad argument exception instead of a platform not supported
exception.

i also took Michael's advice and setup my Socket.BeginReceive sooner. it
actually wasn't a problem as far as this application goes, but i do admit
it was a little strange looking.

thank you both (Peter & Michael) for helping me, and especially Peter for
pointing out the Socket.EndSend bug that i probably never would have
noticed.

kind of embarrassed i had issues with such simple stuff, this time i'll
blame the tonsilitis + fever i'm experiencing ;)

May 28 '07 #13
On Mon, 28 May 2007 03:45:21 -0700, Matthew Geyer
<no****@geyer.matt.gmail.comwrote:
[...]
thank you both (Peter & Michael) for helping me, and especially Peter
for pointing out the Socket.EndSend bug that i probably never would have
noticed.
You're most welcome...glad to hear it's all working now.
kind of embarrassed i had issues with such simple stuff, this time i'll
blame the tonsilitis + fever i'm experiencing ;)
No need to be embarassed, especially if you're relatively new to the
socket API. It's a simple enough API, but there are some subtle implied
behaviors that unless you've read every single word in the documentation
(and I admit, most people don't) you wouldn't know about. Once you've
done it enough, it will become second-nature to notice those things. :)

Pete
May 28 '07 #14
Pete,
No need to be embarassed, especially if you're relatively new to the
socket API.
I am not new to socket API.

Michael

"Peter Duniho" <Np*********@nnowslpianmk.comwrote in message
news:op***************@petes-computer.local...
On Mon, 28 May 2007 03:45:21 -0700, Matthew Geyer
<no****@geyer.matt.gmail.comwrote:
>[...]
thank you both (Peter & Michael) for helping me, and especially Peter
for pointing out the Socket.EndSend bug that i probably never would have
noticed.

You're most welcome...glad to hear it's all working now.
>kind of embarrassed i had issues with such simple stuff, this time i'll
blame the tonsilitis + fever i'm experiencing ;)

No need to be embarassed, especially if you're relatively new to the
socket API. It's a simple enough API, but there are some subtle implied
behaviors that unless you've read every single word in the documentation
(and I admit, most people don't) you wouldn't know about. Once you've
done it enough, it will become second-nature to notice those things. :)

Pete

May 29 '07 #15
On Mon, 28 May 2007 18:58:45 -0700, Michael Rubinstein
<mSPAM_REMOVEr@mĀ®ubinstein.comwrote:
>No need to be embarassed, especially if you're relatively new to the
socket API.
I am not new to socket API.
And I wasn't replying to you.

You *should* be embarassed. :) :) :)
May 29 '07 #16
I am.
"Peter Duniho" <Np*********@nnowslpianmk.comwrote in message
news:op***************@petes-computer.local...
On Mon, 28 May 2007 18:58:45 -0700, Michael Rubinstein
<mSPAM_REMOVEr@m®ubinstein.comwrote:
>>No need to be embarassed, especially if you're relatively new to the
socket API.
I am not new to socket API.

And I wasn't replying to you.

You *should* be embarassed. :) :) :)

May 29 '07 #17

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

reply views Thread by Stuart Norris | last post: by
7 posts views Thread by Colin | last post: by
4 posts views Thread by taskswap | last post: by
reply views Thread by Macca | last post: by
5 posts views Thread by Dan Ritchie | last post: by
4 posts views Thread by Engineerik | last post: by
1 post views Thread by CARIGAR | last post: by
reply views Thread by zhoujie | last post: by
reply views Thread by suresh191 | last post: by
1 post views Thread by Marylou17 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.