473,323 Members | 1,537 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,323 software developers and data experts.

Data structure for storing serial port data in firmware

Hi all,

I am sending data from a linux application through serial port to an embedded device.

In the current implementation a byte circular buffer is used in the firmware. (Nothing but an array with a read and write pointer)
As the bytes come in, it is written to the circular bufffer.

Now the PC application appears to be sending the data too fast for the firmware to handle. Bytes are missed resulting in the firmware returning WRONG_INPUT too mant times.

I think baud rate (115200) is not the issue. A more efficient data structure at the firmware side might help. Any suggestions on choice of data structure?

Thanks
Jun 13 '09 #1
6 4033
Banfa
9,065 Expert Mod 8TB
My experience (of which I have quite a lot) of receiving data on the serial port on embedded devices is that missing data is either the baud rate is too fast for the device or the device is not servicing it's data receive interrupt in a timely fashion.

It is actually quite hard to tell the difference between these 2 faults, you have missed data did it arrive too fast or did you read it too slowly? That's 2 sides of the same coin. So you have to assume the second and if you are still loosing data once you have done everything you can then you try a lower baud rate.

One of the assumptions is normally that the data will not be a constant stream, that is there will be gaps in the data during which time embeded application can perform processing on the received data. The problem is quite different if the data stream is constant.

If you are not already using an interrupt driven receive on the UART then you should do this, using an interrupt helps to ensure that when data is received by the UART the firmware then actually reads the UART.

So this leads to the following ways they software could cause the loss of data.
  1. Failure to service the interrupt routine. An interrupt routine can be stopped from running either by another interrupt routine (on a platform that does not allow nested interrupts which is a large portion of them). For instance in the last year I was using a platform that had a particularly unless OS on it, the task scheduler for the OS took longer to run than the time to receive a character and ran in an interrupt routine which resulted in the loss of data when receiving large quantities of data in a block until I realised the problem.
  2. Buffers too small. You are using a circular buffer which is a common and fine thing to do but by no means the only option. However whatever buffer option you use if you buffer is too small for your largest use case that is the largest amount of data you are likely to receive in a block then you are going to loose characters when you run out of data because unless what you have to do is with the data is extremely simple or you have an extremely powerful processor you will spend more time processing the data than it takes to receive it. That is you are normally reliant on the gaps in the data stream to give the embeded processor time to catch up with processing the received data you have to be able to buffer enough data to span the largest expect burst of data between these gaps. As a rule of thumb I try to guestimate this size and then double it.
  3. Program logic error. The code written has a logic error in it that results in it loosing data or state.

It is unlikely that a circular buffer written correctly would cause a problem in itself as long as it is large enough for the job in hand.

It is important that you don't try to perform all your data processing in your data receive routine, you are unlikely to have time you need to buffer the data for later processing.

Checking that your interrupt routine is being executed in a timely fashion is quite hard to do. IIRC last time I had to do this I set a GPIO line high at the start of the interrupt and low at the end and then used an oscilloscope to follow the value of the pin. Which lead to the discovery of the processing gap and then doing the same for the other interrupts in turn lead me to the interrupt controling task scheduling.

Remember interrupt routines should be short and simple, if they are more than a page or 2 of code or contain complex control structures (switches or loops) then you should start being suspicious.
Jun 13 '09 #2
JosAH
11,448 Expert 8TB
Assuming that the UART can receive bytes at the correct baud rate (115200 bps) then the processor has to deal with the received byte every 8/115200 seconds. Depending on what it has to do with that byte you can calculate (given the clock speed of your CPU on the device) if it can handle everything fast enough.

A circular buffer is fast enough but can the data be copied fast enough to elsewhere?

kind regards,

Jos
Jun 13 '09 #3
donbock
2,426 Expert 2GB
Some serial chips interrupt on each character; others have an onboard fifo and interrupt on every block. The advantages of a block interrupt is that it reduces the number of interrupts -- interrupt context switches are more expensive that subroutine calls. Disadvantages of a block interrupt are (a) you may need to empty the entire block before the next character is received; and (b) if the last byte of the message doesn't fill the block then you may be faced with a long wait for the interrupt that tells you to fetch the last bytes of the message.

Can you tell if the overflow is occurring at the serial chip or if it is occurring in the driver's FIFO?
Jun 13 '09 #4
@donbock
If the fifo is full, as in the block is yet to be read and new data is already been sent, will that data be lost? or will the sender wait?
Jun 15 '09 #5
@Banfa
Thanks a lot for the patient answer.

I am using your and others inputs and looking into it. I will get back when I make some headway.
Jun 15 '09 #6
donbock
2,426 Expert 2GB
@james457
How would the sender even know that the fifo is full? There are techniques for making this known to the sender:
  • "modem control" signals CTS and RTS can be used to force the sender hardware to wait
  • software protocols like X-ON and X-OFF can be used to ask the sender software to wait
However, it is common for no technique to be in place to make the sender wait. In that case a byte is lost (either the most recently received byte or the oldest entry in the fifo is discarded) and an overflow status bit is set.
Jun 15 '09 #7

Sign in to post your reply or Sign up for a free account.

Similar topics

5
by: Loui Mercieca | last post by:
Hi, In my design i have a data structure used to store large amount of numbers ( in the range of lots of thousands ). Each element contains 3 items and the no of elements are dynamic.. of the...
1
by: kiran | last post by:
Hi all, I have a problem to communicate with serial port(COM3:). I am able to open the handle but cannot send any data. 1. I connected my motoroala handset to PC through datacable. 2. I...
36
by: Chuck Faranda | last post by:
I'm trying to debug my first C program (firmware for PIC MCU). The problem is getting serial data back from my device. My get commands have to be sent twice for the PIC to respond properly with...
13
by: Rob | last post by:
Hi all, I am fairly new to python, but not programming and embedded. I am having an issue which I believe is related to the hardware, triggered by the software read I am doing in pySerial. I...
2
by: Lou | last post by:
I have a class that creates an instance of the seril Port. Every thing works fine except whenever I receive data I cannot display the recieved data. I get no errors but the recived data seems to...
0
by: pauland80 | last post by:
<snip> <snip> Late thanks for your both answers! (Please excuse me for that) The problem was a bug in the device firmware. But before finding this, I dugg lightly in the pyserial source...
3
by: newbie1 | last post by:
Dear All, I am new to python. I need to write ctrl-z to a serial port to signal the end of input. This is required by the message format. I use: ser =...
1
by: asadjahangir | last post by:
I need ur help regarding Serial communication in Win32. The problem, i m having is quite strange. It is related to fParity member of DCB structure After setting the fparity=True with...
1
by: Rich | last post by:
Hello, I am working on a python library for sending and receiving data from a Subaru's ECU (the fuel injection computer) via the OBD-II port and an OBD to USB cable, with the Subaru Select Monitor...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.