Serial I/O problem with pywin32 ?

[Xavier]

Hi,

I try to access to a Bluetooth GPS data-logger with Python. I use
pySerial.

Sending and receiving little messages (~100 char) works fine. However,
when I ask the GPS to dump the trails, it returns some Mbytes and here
is the problem : in the stream of bytes, I randomly losts chunks of
~100bytes.

I tried USPP and pyBlueZ instead of pySerial : same problem.

It doesn't like it is a bufferoverun bug from me because :
- the GPS seems to wait when I do not read the stream,
- there is never more than 200 inWainting() characters,
- my code to test it is quite simple :

seriallink = serial.Serial("COM40")
fileName = "data.dump"
out = open(fileName, 'w')
while 1:
c = seriallink.read()
out.write(" %0.2X" % ord(c))
print "*",
out.close()

(with pyBluez :
sock=BluetoothSocket( RFCOMM )
sock.connect(("00:0A:...", 1)))

I tried my program on two different PC with two Bluetooth dongle,
(with pySerial, USPP, pyBluez).
The same things are :
- windows
- the GPS (which works fine with the dumper program of the
constructor)
- pyWin32

I've found another guy with a similar problem :
http://www.generation-nt.com/reponse...de-142097.html
He says:
- no problem with little messages
- lost of one byte every hour when receiving a lot of data

Any known problems with pywin32 and serial I/O when downloading a big
stream of data ?

[Xavier]

I tried under Linux... same problem.
So... it may comes from my little lines of code... or from my GPS.

[Diez B. Roggisch]

Xavier schrieb:

Hi,

I try to access to a Bluetooth GPS data-logger with Python. I use
pySerial.

Sending and receiving little messages (~100 char) works fine. However,
when I ask the GPS to dump the trails, it returns some Mbytes and here
is the problem : in the stream of bytes, I randomly losts chunks of
~100bytes.

I tried USPP and pyBlueZ instead of pySerial : same problem.

It doesn't like it is a bufferoverun bug from me because :
- the GPS seems to wait when I do not read the stream,
- there is never more than 200 inWainting() characters,
- my code to test it is quite simple :

seriallink = serial.Serial("COM40")
fileName = "data.dump"
out = open(fileName, 'w')
while 1:
c = seriallink.read()
out.write(" %0.2X" % ord(c))
print "*",
out.close()

(with pyBluez :
sock=BluetoothSocket( RFCOMM )
sock.connect(("00:0A:...", 1)))

I tried my program on two different PC with two Bluetooth dongle,
(with pySerial, USPP, pyBluez).
The same things are :
- windows
- the GPS (which works fine with the dumper program of the
constructor)
- pyWin32

I've found another guy with a similar problem :
http://www.generation-nt.com/reponse...de-142097.html
He says:
- no problem with little messages
- lost of one byte every hour when receiving a lot of data

Any known problems with pywin32 and serial I/O when downloading a big
stream of data ?

From my experience with serial lines I'd say: that's the culprit. They
tend to be rather unreliable, as there is no real protocol ensuring data
integrity.

You might consider downloading chunks of data if that's somehow
possible, and re-try failed chunks.

Diez

[Impotent Verse]

On 3 Sep, 16:33, "Diez B. Roggisch" <[e-mail]:

Xavier schrieb:

Hi,

I try to access to a Bluetooth GPS data-logger with Python. I use
pySerial.

Sending and receiving little messages (~100 char) works fine. However,
when I ask the GPS to dump the trails, it returns some Mbytes and here
is the problem : in the stream of bytes, I randomly losts chunks of
~100bytes.

I tried USPP and pyBlueZ instead of pySerial : same problem.

It doesn't like it is a bufferoverun bug from me because :
*- the GPS seems to wait when I do not read the stream,
*- there is never more than 200 inWainting() characters,
*- my code to test it is quite simple :

* seriallink = serial.Serial("COM40")
* fileName = "data.dump"
* out = open(fileName, 'w')
* while 1:
* * c = seriallink.read()
* * out.write(" %0.2X" % ord(c))
* * print "*",
* out.close()

(with pyBluez :
sock=BluetoothSocket( RFCOMM )
sock.connect(("00:0A:...", 1)))

I tried my program on two different PC with two Bluetooth dongle,
(with pySerial, USPP, pyBluez).
The same things are :
*- windows
*- the GPS (which works fine with the dumper program of the
constructor)
*- pyWin32

I've found another guy with a similar problem :
http://www.generation-nt.com/reponse...de-142097.html
He says:
*- no problem with little messages
*- lost of one byte every hour when receiving a lot of data

Any known problems with pywin32 and serial I/O when downloading a big
stream of data ?

*From my experience with serial lines I'd say: that's the culprit. They
tend to be rather unreliable, as there is no real protocol ensuring data
integrity.

You might consider downloading chunks of data if that's somehow
possible, and re-try failed chunks.

Diez

Could be hardware flow control. See this sometimes on the bluetooth
connections that are using Serial Port Protocol and the hardware flow
control hasn't been physically implemented.

Do you lose data after exactly the same amount of data has been
received?

[Xavier]

Hi

I have resolved my problem by checking paquets.
It seems that it is a problem of the GPS (it's a very cheap GPS
Datalogger).

Could be hardware flow control. See this sometimes on the bluetooth
connections that are using Serial Port Protocol and the hardware flow
control hasn't been physically implemented.

It seems it is the problem.
The policy seems to be :
- ask the GPS for the data
- touch wood
- retry with the missing chunks
Even the official driver is doing this.

Do you lose data after exactly the same amount of data has
been received?

Not. The lost are randomized but it's chunks, ex :
300 consecutive bytes ok
30 consecutive bytes lost
250 bytes ok
40 bytes lost
800 bytes ok
50 bytes lost
....