Connecting Tech Pros Worldwide Forums | Help | Site Map

COM bytes missing//jumbled when receiveing

Newbie
 
Join Date: Nov 2008
Location: Seattle, WA
Posts: 7
#1: Nov 7 '08
Thanks for at least reading this. Quick overview: I am new to object oriented programming (go FORTRAN) and have jumped into developing a C#.NET application that allows a user to input some values in a GUI, confirm it, and then transmit over a RF Modem while displaying received data. I am using two Xbee-PRO USB modems from Digi in a pseudo null modem test. I am using C# Express 2008 on a Dell Laptop with XP OS.

My current problem (that I have been Googleing and reading forums for two days now) is there is and issue with receiving the data. By adding in some IO, I dumped a copy of my Xmit structure into some log.txt and it was perfect every time with the entire 128 byte structure. On the other end I dumped the Incoming buffer into some log.txt and there would only be 110-124 bytes on a single structure. I thought this could just be chopping off the end but after multiple packets the entire receive structure gets jumbled. The Int16 Sync word starts in mid structure of a prior transmission structure. I should note that the 128 byte structure only has ~70 useful bytes in it with byte[70:128] = 0.

My current efforts on resolving this issue is to look at data control. IE I think I am not controlling my memory and the errors are induced by how I handle SerialPortDataReceived events. There is some error checking and locating that I am doing with the TestSync() function but this locks on to the first two 85 sync bytes everytime. I have deleted some other erroneous junk that I added to try and fix this problem. I think I have disabled FIFO on the modems so that it writes data as it comes (90% sure).

BreakupIncomingAttitude() is the main point where the Class "STB" takes the 128 byte[] structure and populates the encapsulated properties. Don't stone me but to pass data between Forms I made the STB Class public. This was how I first got it working and have not messed with it, yet.


serialPort1 Properties:
Baud 11520
Byte Threshold 128
No flow control - Another beast I have not tried to tackle but probably next

Expand|Select|Wrap|Line Numbers
  1.  
  2. public Main()
  3. {
  4. serialPort1.DataReceived +=new System.IO.Ports.SerialDataReceivedEventHandler(serialPort1_DataReceived);
  5. }
  6.  
  7. void serialPort1_DataReceived(object sender, System.IO.Ports.SerialDataReceivedEventArgs e)
  8. {
  9.       serialPort1.Read(buff_RecivedData, 0, serialPort1.BytesToRead);
  10.  
  11.        if (Array.Exists(buff_RecivedData, TestSync))
  12.        {
  13.            int PossibleSync = Array.FindIndex(buff_RecivedData, TestSync);
  14.            Buffer.BlockCopy(buff_RecivedData, PossibleSync, adjust_RecivedData, 0, 128);
  15.  
  16.             this.STB1.BreakupIncomingAttitude(adjust_RecivedData); // Main Data
  17.  
  18.             UpdateIN(adjust_RecivedData); // File IO
  19.  
  20.             var d = new Update_Text(update_TEL); // GUI Display
  21.             this.BeginInvoke(d);
  22.  
  23.         }
  24. }
  25.  
  26.  
You|Help|Please

Sam T.

Plater's Avatar
Moderator
 
Join Date: Apr 2007
Location: New England
Posts: 7,161
#2: Nov 7 '08

re: COM bytes missing//jumbled when receiveing


I can tell you that I have had very little luck with doing processing in the DataReceived event on the SerialPort objects. So be sure to try and do as little as possible in it (it can fire again before you've left the function and you could get overlap)

A good way to try things out is to seperate things apart.
For some of my tests (with an XBee as a matter of fact) I made a button that just SENT data out on the SerialPort(that was connected to the XBee).
I also had a button that would read in everything that had come in on the SerialPort. (SerialPort buffer defaults to 4048 bytes I think). Ignoring the DataReceived event, and just reading what was there.
That way I could click the SEND button and then I could be sure that I was waiting a "long enough time" for anything to be sent back to me.


The reason I say this is that you claimed to be "missing" bytes on the receiving side right? And you are doing a lot of processing on a function that could be called as often as every byte received.
So If you moved that processing to a button click (as a test) you could wait and be sure all data that was going to arrive, had arrived and then see if you are still missing bytes
Newbie
 
Join Date: Nov 2008
Location: Seattle, WA
Posts: 7
#3: Nov 7 '08

re: COM bytes missing//jumbled when receiveing


Well, by sending a single packet I would think that I am waiting long enough. It just never gets all of the 128 bytes in the SerialPort1.Buffer to trigger the DataRecived event and start this processing. I can not wait for 4k bytes and think shrinking the buffer size is a bad idea. Eventually I would like to get this in the sub 100 millisecond range but hey everyone can dream.

What do you think about using a timer to read the Port buffer every Nth second and dump it into the app's buffer. Then put an Int16 terminator at the end of the 128 bytes to trigger the processing? This has some extra overhead but I think this would transfer data control to my program and not the SerialPort class.

I am going to try separating the function and processing to see what I can learn. However I still need an automated process that will not only read some of the structure.

Oh and rereading the Serial port properties Baud rate is 115200.
Plater's Avatar
Moderator
 
Join Date: Apr 2007
Location: New England
Posts: 7,161
#4: Nov 7 '08

re: COM bytes missing//jumbled when receiveing


I don't mean wait for 4k bytes.
I mean you are having a bunch of code run EVERY time the event triggered.
I'm saying move all that code to a seperate function and call the function on a button click. Ignore the datareceived event entirely.
The data is still there, you won't "miss it" by not reading it durring the event. Its just to let you know something came in.
Newbie
 
Join Date: Nov 2008
Location: Seattle, WA
Posts: 7
#5: Nov 7 '08

re: COM bytes missing//jumbled when receiveing


Ok, I ripped out the heavy stuff. I think I might need to use a delegate to call ProcessData(). ReadThePort_Tick is running every 0.1s after the com port is opened. It will keep dumping information into the buffer until it has enough then purge.

Still having issues with mixed up structures. The first one works but then it fails seemingly randomly. It is much more accurate though so there is hope. There has to be something with the timing because there are more errors, the faster I send it. How do I protect the data?

Expand|Select|Wrap|Line Numbers
  1.     private void ReadThePort_Tick(object sender, EventArgs e)
  2.     {
  3.         int tempLeng = serialPort1.BytesToRead;
  4.         if (tempLeng >= 1)
  5.         {
  6.             serialPort1.Read(buff_RecivedData, buff_position, tempLeng);
  7.             buff_position += tempLeng;
  8.  
  9.             if (Array.Exists(buff_RecivedData, TestSync))
  10.             {
  11.                 int PossibleSync = Array.FindIndex(buff_RecivedData, TestSync);
  12.                 if (buff_RecivedData[PossibleSync + 1] == 85)
  13.                 {
  14.                     if (buff_position - PossibleSync >= 70)
  15.                     {
  16.                         ProcessData(buff_RecivedData, PossibleSync);
  17.                         buff_RecivedData = new byte[512];
  18.                         buff_position = 0;
  19.                     }
  20.                 }
  21.             }
  22.         }
  23.     }
  24.  

Samples from the file Logging
Xmit Log:
UU bÃ(%=ääR8p=@€
À 
UU bÃ(%=ääR8p=@€
À 
UU bÃ(%=ääR8p=@€
À 
UU bÃ(%=ääR8p=@€
À 
UU bÃ(%=ääR8p=@€
À 
UU bÃ(%=ääR8p=@€
À 
UU bÃ(%=ääR8p=@€
À 

Receive Log:
UU bÃ(%=ääR8p=@€
À 
UU %
UU bÃ(%=ääR8p=@€
À 
UU bÃ(%=ääR8p=@€
À 
UU bÃ(%=ääR8p=@€
À 
UU bÃ(%=ääR8p=@€
À 
UU bÃ(%=ääR8p=@€
À 
Newbie
 
Join Date: Nov 2008
Location: Seattle, WA
Posts: 7
#6: Nov 11 '08

re: COM bytes missing//jumbled when receiveing


Found an answer with some other tricks. If you are reading this start looking at the lock functions.

lock()
{

}
Plater's Avatar
Moderator
 
Join Date: Apr 2007
Location: New England
Posts: 7,161
#7: Nov 11 '08

re: COM bytes missing//jumbled when receiveing


Well using the lock() would mutex out all your calls to the datareceived event.

I wasn't aware you were doing this multithreaded
balabaster's Avatar
Moderator
 
Join Date: Mar 2007
Location: Canada
Posts: 757
#8: Nov 12 '08

re: COM bytes missing//jumbled when receiveing


In my experience when receiving data over any type of stream, be it tcp/ip, serial, radio, whatever I've had very little luck with typically suggested solutions.

What I've always done that's been successful is in the event that receives the data, all I do is receive the data and spin off a new thread to process the data. Usually I have a thread to process the data to a queue and then another thread that loops to process the queue. When the queue gets to empty a flag is set that stops the process thread... and then when more data is added to the queue, the thread that's adding the items to the queue reset the flag so that the process thread can continue.

In summary:
Receive Event: Grab the data and pass it to a Queuing Thread.

Queuing Thread: Put the data into a queue that the process thread will process and set the flag so the Process Queue knows it has data to process

Process Thread: Wait until the flag is set and then process all items in the queue. When the queue is empty reset the flag and wait...

This type of pattern will allow for the data to be processed as rapidly as possible without the receive event missing any of the data. The receive event should do as little processing as possible - only the bear minimum to get the raw data from the receive buffer.
Newbie
 
Join Date: Nov 2008
Location: Seattle, WA
Posts: 7
#9: Nov 21 '08

re: COM bytes missing//jumbled when receiveing


Turns out that Digi XBee-Pro USB modems need two stop bits for syncing. The program was solid and in then end became a better tool by tearing it apart.
Reply