473,657 Members | 2,515 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Data structure for storing serial port data in firmware

6 New Member
Hi all,

I am sending data from a linux application through serial port to an embedded device.

In the current implementation a byte circular buffer is used in the firmware. (Nothing but an array with a read and write pointer)
As the bytes come in, it is written to the circular bufffer.

Now the PC application appears to be sending the data too fast for the firmware to handle. Bytes are missed resulting in the firmware returning WRONG_INPUT too mant times.

I think baud rate (115200) is not the issue. A more efficient data structure at the firmware side might help. Any suggestions on choice of data structure?

Thanks
Jun 13 '09 #1
6 4051
Banfa
9,065 Recognized Expert Moderator Expert
My experience (of which I have quite a lot) of receiving data on the serial port on embedded devices is that missing data is either the baud rate is too fast for the device or the device is not servicing it's data receive interrupt in a timely fashion.

It is actually quite hard to tell the difference between these 2 faults, you have missed data did it arrive too fast or did you read it too slowly? That's 2 sides of the same coin. So you have to assume the second and if you are still loosing data once you have done everything you can then you try a lower baud rate.

One of the assumptions is normally that the data will not be a constant stream, that is there will be gaps in the data during which time embeded application can perform processing on the received data. The problem is quite different if the data stream is constant.

If you are not already using an interrupt driven receive on the UART then you should do this, using an interrupt helps to ensure that when data is received by the UART the firmware then actually reads the UART.

So this leads to the following ways they software could cause the loss of data.
  1. Failure to service the interrupt routine. An interrupt routine can be stopped from running either by another interrupt routine (on a platform that does not allow nested interrupts which is a large portion of them). For instance in the last year I was using a platform that had a particularly unless OS on it, the task scheduler for the OS took longer to run than the time to receive a character and ran in an interrupt routine which resulted in the loss of data when receiving large quantities of data in a block until I realised the problem.
  2. Buffers too small. You are using a circular buffer which is a common and fine thing to do but by no means the only option. However whatever buffer option you use if you buffer is too small for your largest use case that is the largest amount of data you are likely to receive in a block then you are going to loose characters when you run out of data because unless what you have to do is with the data is extremely simple or you have an extremely powerful processor you will spend more time processing the data than it takes to receive it. That is you are normally reliant on the gaps in the data stream to give the embeded processor time to catch up with processing the received data you have to be able to buffer enough data to span the largest expect burst of data between these gaps. As a rule of thumb I try to guestimate this size and then double it.
  3. Program logic error. The code written has a logic error in it that results in it loosing data or state.

It is unlikely that a circular buffer written correctly would cause a problem in itself as long as it is large enough for the job in hand.

It is important that you don't try to perform all your data processing in your data receive routine, you are unlikely to have time you need to buffer the data for later processing.

Checking that your interrupt routine is being executed in a timely fashion is quite hard to do. IIRC last time I had to do this I set a GPIO line high at the start of the interrupt and low at the end and then used an oscilloscope to follow the value of the pin. Which lead to the discovery of the processing gap and then doing the same for the other interrupts in turn lead me to the interrupt controling task scheduling.

Remember interrupt routines should be short and simple, if they are more than a page or 2 of code or contain complex control structures (switches or loops) then you should start being suspicious.
Jun 13 '09 #2
JosAH
11,448 Recognized Expert MVP
Assuming that the UART can receive bytes at the correct baud rate (115200 bps) then the processor has to deal with the received byte every 8/115200 seconds. Depending on what it has to do with that byte you can calculate (given the clock speed of your CPU on the device) if it can handle everything fast enough.

A circular buffer is fast enough but can the data be copied fast enough to elsewhere?

kind regards,

Jos
Jun 13 '09 #3
donbock
2,426 Recognized Expert Top Contributor
Some serial chips interrupt on each character; others have an onboard fifo and interrupt on every block. The advantages of a block interrupt is that it reduces the number of interrupts -- interrupt context switches are more expensive that subroutine calls. Disadvantages of a block interrupt are (a) you may need to empty the entire block before the next character is received; and (b) if the last byte of the message doesn't fill the block then you may be faced with a long wait for the interrupt that tells you to fetch the last bytes of the message.

Can you tell if the overflow is occurring at the serial chip or if it is occurring in the driver's FIFO?
Jun 13 '09 #4
james457
6 New Member
@donbock
If the fifo is full, as in the block is yet to be read and new data is already been sent, will that data be lost? or will the sender wait?
Jun 15 '09 #5
james457
6 New Member
@Banfa
Thanks a lot for the patient answer.

I am using your and others inputs and looking into it. I will get back when I make some headway.
Jun 15 '09 #6
donbock
2,426 Recognized Expert Top Contributor
@james457
How would the sender even know that the fifo is full? There are techniques for making this known to the sender:
  • "modem control" signals CTS and RTS can be used to force the sender hardware to wait
  • software protocols like X-ON and X-OFF can be used to ask the sender software to wait
However, it is common for no technique to be in place to make the sender wait. In that case a byte is lost (either the most recently received byte or the oldest entry in the fifo is discarded) and an overflow status bit is set.
Jun 15 '09 #7

Sign in to post your reply or Sign up for a free account.

Similar topics

5
2621
by: Loui Mercieca | last post by:
Hi, In my design i have a data structure used to store large amount of numbers ( in the range of lots of thousands ). Each element contains 3 items and the no of elements are dynamic.. of the form:
1
2546
by: kiran | last post by:
Hi all, I have a problem to communicate with serial port(COM3:). I am able to open the handle but cannot send any data. 1. I connected my motoroala handset to PC through datacable. 2. I installed valid driver to detec the motorla(V400) handset 3. I tested the handset through Hypertermal and succssfully sending any data. 4. I opened the Control Panel -> Phone and Model window and clicked the modem tab and I saw that the Motorola...
36
3193
by: Chuck Faranda | last post by:
I'm trying to debug my first C program (firmware for PIC MCU). The problem is getting serial data back from my device. My get commands have to be sent twice for the PIC to respond properly with the needed data. Any ideas? Here's the code in question, see any reason why a command would not trigger the 'kbhit' the first time a serial command is sent?: Thanks! Chuck **************************************************** while(1) //...
13
6189
by: Rob | last post by:
Hi all, I am fairly new to python, but not programming and embedded. I am having an issue which I believe is related to the hardware, triggered by the software read I am doing in pySerial. I am sending a short message to a group of embedded boxes daisy chained via the serial port. When I send a 'global' message, all the connected units should reply with their Id and Ack in this format '0 Ack' To be certain that I didn't miss a...
2
6177
by: Lou | last post by:
I have a class that creates an instance of the seril Port. Every thing works fine except whenever I receive data I cannot display the recieved data. I get no errors but the recived data seems to just go no where. I can see the recived data in my serial receive function but when I either raise an event with it or try to display it in a text box nothing happens. I do use beginInvoke on the text box. If I trace the code through it all...
0
596
by: pauland80 | last post by:
<snip> <snip> Late thanks for your both answers! (Please excuse me for that) The problem was a bug in the device firmware. But before finding this, I dugg lightly in the pyserial source and found (to take with care!) :
3
34013
by: newbie1 | last post by:
Dear All, I am new to python. I need to write ctrl-z to a serial port to signal the end of input. This is required by the message format. I use: ser = serial.Serial(port=COMPORT,baudrate=BAUDRATE,bytesize=8,parity='N',stopbits=1,timeout=3,xonxoff=0,rtscts=0,) ser.open() ser.write("testing\x1A") But the ctrl-z (ascii 26) doesn't seem to be written to the serial port. The firmware still waiting for message input because it can't see the...
1
4431
by: asadjahangir | last post by:
I need ur help regarding Serial communication in Win32. The problem, i m having is quite strange. It is related to fParity member of DCB structure After setting the fparity=True with SetCommState(), when i get DCB structure with GetCommState(), i always find fparity=FALSE. I have tested it on 3 machine all running windows xp sp2 and standard COM driver but having the same problem. Its looks as SetCommState is unable to set fparity...
1
15067
by: Rich | last post by:
Hello, I am working on a python library for sending and receiving data from a Subaru's ECU (the fuel injection computer) via the OBD-II port and an OBD to USB cable, with the Subaru Select Monitor protocol. There are a few open source programs that do this already (http://romraider.com/ , http://jdash.sourceforge.net/ , http://tari.co.za/downloads/software/source/ ), but they are written in Java or C++. I have never done anything with serial...
0
8844
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8742
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8518
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
7354
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6177
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5643
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4173
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4330
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
2743
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.