473,856 Members | 1,724 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

socket buffer flush question

Hello Everybody!
I am writing a networking application in python for a small piece of
hardware, in which there could sometimes be timeouts. I am using
sockets to communicate to this device. Data that is sent to the device
is instructions for that piece of hardware, and data recieved is just
responses from that hardware.
When a timeout happens, for some reason extra data is stored inside
the buffer, so when the timeout is over, that extra data (remember this
data is instructions) is executed by the hardware, which I don't want.
For timeout purposes, I want the socket to be nonblocking, but at the
same time this means that I can't flush the buffer, which I think would
solve the problem.

I realize this is probably quite unclear, so below is a tcpdump which
will explain what I mean.

15:10:37.667963 aurl06f2.39979 192.168.128.161 .46667: P 481:493(12)
ack 481 win 5840 <nop,nop,timest amp 157458905 528016(DF)
4500 0040 090c 4000 4006 599f 8fd1 06f2
c0a8 80a1 9c2b b64b 30c7 9289 a552 55df
8018 16d0 c3ec 0000 0101 080a 0962 a1d9
0008 0e90 0000 0001 f800 0110 0000 0000
15:10:51.107769 aurl06f2.39979 192.168.128.161 .46667: P 481:493(12)
ack 481 win 5840 <nop,nop,timest amp 157460249 528016(DF)
4500 0040 090d 4000 4006 599e 8fd1 06f2
c0a8 80a1 9c2b b64b 30c7 9289 a552 55df
8018 16d0 beac 0000 0101 080a 0962 a719
0008 0e90 0000 0001 f800 0110 0000 0000
15:10:51.108259 192.168.128.161 .46667 aurl06f2.39979: P 481:493(12)
ack 493 win 5792 <nop,nop,timest amp 533605 157460249(DF)
4500 0040 5580 4000 3f06 0e2b c0a8 80a1
8fd1 06f2 b64b 9c2b a552 55df 30c7 9295
8018 16a0 dbf5 0000 0101 080a 0008 2465
0962 a719 8000 0001 f800 0110 0000 4d05
15:10:51.108283 aurl06f2.39979 192.168.128.161 .46667: P 493:565(72)
ack 493 win 5840 <nop,nop,timest amp 157460249 533605(DF)
4500 007c 090e 4000 4006 5961 8fd1 06f2
c0a8 80a1 9c2b b64b 30c7 9295 a552 55eb
8018 16d0 cb29 0000 0101 080a 0962 a719
0008 2465 0000 0001 f800 0110 0000 0000
0000 0001 f800 0110 0000 0000 0000 0001
f800
************* NOTICE THE EXTRA DATA *************** **
15:10:51.108731 192.168.128.161 .46667 aurl06f2.39979: P 493:505(12)
ack 565 win 5792 <nop,nop,timest amp 533605 157460249(DF)
4500 0040 5581 4000 3f06 0e2a c0a8 80a1
8fd1 06f2 b64b 9c2b a552 55eb 30c7 92dd
8018 16a0 db99 0000 0101 080a 0008 2465
0962 a719 8000 0001 f800 0110 0000 4d0d
15:10:51.111760 aurl06f2.39979 192.168.128.161 .46667: P 565:577(12)
ack 505 win 5840 <nop,nop,timest amp 157460249 533605(DF)
4500 0040 090f 4000 4006 599c 8fd1 06f2
c0a8 80a1 9c2b b64b 30c7 92dd a552 55f7
8018 16d0 a867 0000 0101 080a 0962 a719
0008 2465 0000 0001 f800 0114 0000 0000

The last 3 words (32 bits/word) of the hex printout shows the
condition. The first of the last 3 words is the instruction, in this
case a read instruction: 0000_0001. The second is the address:
f800_0110 or f800_0114. The third is the data recieved.
As you can see, the first 2 printouts are a timeout (it is much longer
than this, but i didnt want to post all of that) because one machine is
communicating to the other without a response. The third printout is
when the machine recovers from the timeout, and it works fine.
However, the printout after the machine recovers has some extra data at
the end. This is the data I want to flush out of the buffer, but I
cant really figure out how to do so.

Sorry for the long post and thanks a lot for your help

Jul 5 '06 #1
2 6810
Kiran i'rta:
Hello Everybody!
I am writing a networking application in python for a small piece of
hardware, in which there could sometimes be timeouts. I am using
sockets to communicate to this device. Data that is sent to the device
is instructions for that piece of hardware, and data recieved is just
responses from that hardware.
When a timeout happens, for some reason extra data is stored inside
the buffer, so when the timeout is over, that extra data (remember this
data is instructions) is executed by the hardware, which I don't want.
Did you try setsockopt and TCP_NODELAY? That can prevent buffering in
some cases.

Laszlo

Jul 5 '06 #2
In article <11************ **********@b68g 2000cwa.googleg roups.com>,
Kiran <Ki*********@gm ail.comwrote:
>Hello Everybody!
I am writing a networking application in python for a small piece of
hardware, in which there could sometimes be timeouts. I am using
sockets to communicate to this device. Data that is sent to the device
is instructions for that piece of hardware, and data recieved is just
responses from that hardware.
When a timeout happens, for some reason extra data is stored inside
the buffer, so when the timeout is over, that extra data (remember this
data is instructions) is executed by the hardware, which I don't want.
For timeout purposes, I want the socket to be nonblocking, but at the
same time this means that I can't flush the buffer, which I think would
solve the problem.

I realize this is probably quite unclear, so below is a tcpdump which
will explain what I mean.

15:10:37.66796 3 aurl06f2.39979 192.168.128.161 .46667: P 481:493(12)
ack 481 win 5840 <nop,nop,timest amp 157458905 528016(DF)
4500 0040 090c 4000 4006 599f 8fd1 06f2
c0a8 80a1 9c2b b64b 30c7 9289 a552 55df
8018 16d0 c3ec 0000 0101 080a 0962 a1d9
0008 0e90 0000 0001 f800 0110 0000 0000
You sent a read f8000110 command to the device 12 bytes, sequence
481..493 in this connection. You've received 480 bytes from the device
>15:10:51.10776 9 aurl06f2.39979 192.168.128.161 .46667: P 481:493(12)
ack 481 win 5840 <nop,nop,timest amp 157460249 528016(DF)
4500 0040 090d 4000 4006 599e 8fd1 06f2
c0a8 80a1 9c2b b64b 30c7 9289 a552 55df
8018 16d0 beac 0000 0101 080a 0962 a719
0008 0e90 0000 0001 f800 0110 0000 0000
You sent the command again
>15:10:51.10825 9 192.168.128.161 .46667 aurl06f2.39979: P 481:493(12)
ack 493 win 5792 <nop,nop,timest amp 533605 157460249(DF)
4500 0040 5580 4000 3f06 0e2b c0a8 80a1
8fd1 06f2 b64b 9c2b a552 55df 30c7 9295
8018 16a0 dbf5 0000 0101 080a 0008 2465
0962 a719 8000 0001 f800 0110 0000 4d05
the device responds to your command with 12 bytes, which also appear
to be sequence 481..493. The device confirms it has received 492 bytes
from your program. I assume every command gets an equal length
return, so that when things are running correctly, the received
sequence numbers will match the last sent command.

>15:10:51.10828 3 aurl06f2.39979 192.168.128.161 .46667: P 493:565(72)
ack 493 win 5840 <nop,nop,timest amp 157460249 533605(DF)
4500 007c 090e 4000 4006 5961 8fd1 06f2
c0a8 80a1 9c2b b64b 30c7 9295 a552 55eb
8018 16d0 cb29 0000 0101 080a 0962 a719
0008 2465 0000 0001 f800 0110 0000 0000
0000 0001 f800 0110 0000 0000 0000 0001
f800
It looks to me like you've sent 6 more commands during the timeout period,
so you should expect 6 responses back, the first being sequence
493..505, then another for 505..517, etc.
************* NOTICE THE EXTRA DATA *************** **
15:10:51.10873 1 192.168.128.161 .46667 aurl06f2.39979: P 493:505(12)
ack 565 win 5792 <nop,nop,timest amp 533605 157460249(DF)
4500 0040 5581 4000 3f06 0e2a c0a8 80a1
8fd1 06f2 b64b 9c2b a552 55eb 30c7 92dd
8018 16a0 db99 0000 0101 080a 0008 2465
0962 a719 8000 0001 f800 0110 0000 4d0d
Here the device confirms that it has received 564 bytes from your
program and sends the first 12 bytes of responses.

>15:10:51.11176 0 aurl06f2.39979 192.168.128.161 .46667: P 565:577(12)
ack 505 win 5840 <nop,nop,timest amp 157460249 533605(DF)
4500 0040 090f 4000 4006 599c 8fd1 06f2
c0a8 80a1 9c2b b64b 30c7 92dd a552 55f7
8018 16d0 a867 0000 0101 080a 0962 a719
0008 2465 0000 0001 f800 0114 0000 0000

you send another command. At this point you've sent 576 bytes out (48
commands) and received back 504 bytes (42 responses). The last
response shows that 47 of the commands have made it to the device and
are (presumably) awaiting processing.
>The last 3 words (32 bits/word) of the hex printout shows the
condition. The first of the last 3 words is the instruction, in this
case a read instruction: 0000_0001. The second is the address:
f800_0110 or f800_0114. The third is the data recieved.
As you can see, the first 2 printouts are a timeout (it is much longer
than this, but i didnt want to post all of that) because one machine is
communicatin g to the other without a response. The third printout is
when the machine recovers from the timeout, and it works fine.
However, the printout after the machine recovers has some extra data at
the end. This is the data I want to flush out of the buffer, but I
cant really figure out how to do so.
What I see is that you continued sending new commands while not
getting responses from the device. The device has received them, since
it acknowledges that it has seen up to byte 565 of your output. This
is not an unflushed buffer problem - the data has reached the TCP/IP
stack of the device and what happens to it next depends on the
application layer in the device. If it is reading and discarding
commands, then you have a problem that you have sent several commands
which won't receive replies. If the device's application works
correctly, it will continue to dequeue commands 12 bytes at a time and
send replies. If it reads whatever is available and only processes 12
bytes from the read, then the device end application is sub-optimal
and you will have to ensure you don't issue any new commands until the
previous command has been processed. Once you transfer data to your
machine's network stack, there's no way to stop it being sent to the
remote end - in fact there's no practical way to determine if it's
been put in a packet, sent to the remote end and lost or put in a
packet, sent to the remote end and is awaiting processing, or is
waiting to be put into a packet and hasn't left your machine at all.

What happens if your application simply stops transmitting new
commands after a timeout? Do you eventually get all the responses in?

--
Jim Segrave (je*@jes-2.demon.nl)

Jul 5 '06 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
11786
by: Bryan Olson | last post by:
Here's the problem: Suppose we use: import socket f = some_socket.makefile() Then: f.read() is efficient, but verbose, and incorrect (or at least does not play will with others);
4
41970
by: Stephan Steiner | last post by:
Hi I have a networking application that periodically needs to go into sleep mode (from an application point of view, I'm simply suspending the receiver thread until it's time to start listening to incoming packets again). The packets I'm interested in are UDP broadcasts, so they are simply dropped when nobody application is listening at the specified port. However, when I initialize my receiver, there's a socket listening at the...
4
2750
by: Qingdong Z. | last post by:
I have an asynchronous Server Socket to push data to client (Socket.BeginSend) when data is available, Meanwhile, the client socket use Synchronous Client Socket to receive the data. I have two questions on this 1. When client socket does not receive data as fast as server, does server socket queues all the data, or just waits? If it queues, this may kill server 2. I declare socket receive buffer big enough, when I call Socket.Receive...
0
4704
by: Macca | last post by:
Hi, I am writing an asychronous socket server to handle 20+ simulataneous connections. I have used the example in MSDN as a base. The code is shown at end of question. Each connection has a number of different types of data coming in. I have a databuffer for each type of data coming in.
15
1561
by: googlegroups | last post by:
I am learning vb.net and trying to create a simple program to watch port 25 and receive email. I do not need any advanced features. I just need to connect, watch and grab the emails as they come. This seems like it should be a pretty simple but I can not seem to find any examples in vb.net 8. Any help is appreciated. TIA
6
39406
by: ppuniversal | last post by:
Can anyone tell me how to flush a socket stream in C++. I am making an Client Server application in C++ and use the send() and recv() functions. i want to flush the socket buffer as i am getting unwanted data when i use recv(). A part of my server code is like this: int bytesRecv = SOCKET_ERROR; char recvbuf=""; int ch ; bytesRecv = recv( clientSocket, recvbuf, 1, 0 );
4
12099
by: david | last post by:
hello, I have a client/server application. the server capture picture from webcam and send it to every client connected to it.the network part works good and the capture from webcam too. I associate an event when a capture is done, then every frame of the webcam should be sent to the client. but I cannot find a way to send bitmap throught network, of course I found many method from internet, some doen't work, some work but it's never...
4
7593
by: Andrew Jackson | last post by:
I am writing a newsgroup client. I have the protocol figured out. But I get slow transfer speeds off any of the network objects read the data from For example one of the commands for a news client to use is "XOVER articlenumber-" This return string after string of all the news articles from article number on.... Another newsclient, i wont name names, pulls data down just fine. Using a
4
5520
by: The Doctor | last post by:
Hey people, I have two applications: the server, which creates a server socket, waits for a real-time signal, and if it receives one, it creates a client socket. The client, which will connect to the server. This will generate a real- time signal at the server, who will create a client socket for this client.
0
9904
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
11051
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
1
10773
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10378
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
9527
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
7929
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5956
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
2
4170
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
3196
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.