question about binary and serial info

[nephish]

i have an interesting project at work going on. here is the challenge.
i am using the serial module to read data from a serial input.
it comes in as a hex. i need to make it a binary and compare it bit by
bit to another byte. They have some weird way they set this up that i
have to compare these things with AND. in other words, if bit 1 is 1
AND bit 1 is 1 then the real value is 1...

long story short. is there a good way to compare bytes, bit by bit with
one of the modules of python. i want to know so i dont get halfway into
developing this and find that there is a much better way to do this
than by hand.

thanks for any suggestions.
sk <><

[Jordan Rastrick]

Sounds like you want the bitwise and operator, &

2 & 3 2 32 & 16 0 31 & 12

12

etc.

[e-mail] inquired: i have an interesting project at work going on. here is the challenge.
i am using the serial module to read data from a serial input.
it comes in as a hex. i need to make it a binary and compare it bit by
bit to another byte. They have some weird way they set this up that i
have to compare these things with AND. in other words, if bit 1 is 1
AND bit 1 is 1 then the real value is 1...

long story short. is there a good way to compare bytes, bit by bit with
one of the modules of python. i want to know so i dont get halfway into
developing this and find that there is a much better way to do this
than by hand.

thanks for any suggestions.
sk <><

[nephish]

yeah, i think i got that down, i need help with getting the hex to
binary, then splitting the byte up to compare each bit against the bit
in another byte.
unless i am not understanding this stuff with the bitwise right. there
wasn't a lot in the python library reference about it.
thanks

[mensanator]

ne*****@xit.net wrote:

yeah, i think i got that down, i need help with getting the hex to
binary, then splitting the byte up to compare each bit against the bit
in another byte.
The & operator does all 8 comparisons simultaneously. So if
the serial port byte is A, the reference byte is B then

AB = A & B

has only 1 bits where both A and B had 1's in their respective
positions. Now, you can test AB for a particular bit position
(say bit 3) by

testbit3 = AB & 2**3

If testbit3 > 0 then the bit was a 1.
unless i am not understanding this stuff with the bitwise right. there
wasn't a lot in the python library reference about it.
The GMPY module has some interesting bit functions.

Popcount can tell you how many of the AB bits are 1 without
specifying which ones:

for i in range(16): print gmpy.popcount(i),

0 1 1 2 1 2 2 3 1 2 2 3 2 3 3 4

Hamming distance tells you how many bits differ between
two numbers (again, without telling you which ones)
for i in range(16): print gmpy.hamdist(i,7),

3 2 2 1 2 1 1 0 4 3 3 2 3 2 2 1

If one operand is 0, then Hamming distance is the same
as popcount.

And then there's scan1 which will tell you the bit
bit position of the first 1 bit.
A = 48
B = 255
AB = A & B
print gmpy.scan1(AB)

4

So the first 1 bit is bit 4, which means bits 0, 1, 2 and 3
are all 0.
thanks

[nephish]

this is exactly what i need.
i have not previously had a need for this kind of thing, but i sure
need some
documentation for it now. is there a resource out there to find out how
to convert decimal number to a byte, a byte to a decimal and more about
the & operator?
i really appreciate this info. this is exactly what i was looking for.
shawn

[nephish]

oh wait, i found it. i found how to do the conversions and thanks to
you the operators.
appreciate everything,
shawn

[Dennis Lee Bieber]

On 17 Aug 2005 19:43:52 -0700, ne*****@xit.net declaimed the following
in comp.lang.python:

long story short. is there a good way to compare bytes, bit by bit with
one of the modules of python. i want to know so i dont get halfway into
developing this and find that there is a much better way to do this
than by hand.
Do the bitwise operators do it for you?

5 & 6

4

(in binary: 0101 & 0110 => 0100)
-- ================================================== ============ <
wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
wu******@dm.net | Bestiaria Support Staff <
================================================== ============ <
Home Page: <http://www.dm.net/~wulfraed/> <
Overflow Page: <http://wlfraed.home.netcom.com/> <

[nephish]

i got the bitwise part, i just cant seem to convert the incomming ascii
into hex, binary, integer, or decimal.
how do i do this, my serial port bytes just come in all weird looking
ascii characters
thanks

[Grant Edwards]

On 2005-08-18, ne*****@xit.net <ne*****@xit.net> wrote:

i got the bitwise part, i just cant seem to convert the
incomming ascii into hex, binary, integer, or decimal.
So you've got an ASCII string in one format and you want to
convert into an ASCII string in a different format?

For example you want to convert the string "FF" into the string "255"?
how do i do this, my serial port bytes just come in all weird
looking ascii characters

--
Grant Edwards grante Yow! It's OKAY -- I'm an
at INTELLECTUAL, too.
visi.com

[nephish]

i have an ascii string comming in the serial port and i need to convert
it to something else, like an integer, or binary, or even a hex so i
can use the bitwise comparison on it.
thanks

[Grant Edwards]

On 2005-08-18, ne*****@xit.net <ne*****@xit.net> wrote:

i have an ascii string comming in the serial port and i need to convert
it to something else, like an integer, or binary, or even a hex so i
can use the bitwise comparison on it.

But what do you mean by "integer", "binary", and "hex"?

Decimal, hex, and binary are all representations of integers.

On your computer all integers are 2's compliment binary (we're
going to ignore BCD for the moment).

The only thing that can be "decimal" or "hex" are _string_
representations of integer values.

If you want something upon which you can perform the bitwise
boolean operations &, ^, |, then you most probably just want an
integer object.

To convert a string to an integer, use the int() builtin
passing it the string an an optional base:

int("1234") 1234 int("1234",10) 1234 int("0x100",16) 256 int("100",16) 256 int("100",2)

4

Others have already shown you how to use the bitwise boolean
operators.

--
Grant Edwards grante Yow! Look DEEP into the
at OPENINGS!! Do you see any
visi.com ELVES or EDSELS... or a
HIGHBALL??...

[nephish]

>>> import serial

ser = serial.Serial('/dev/ttyS0', 2400, timeout= 10, bytesize=8, stopbits=1)
a = ser.read(1)
print a ^ ser.close() type(a) <type 'str'>
int(a, 16) Traceback (innermost last):
File "<stdin>", line 1, in ?
ValueError: invalid literal for int(): ^

so i run it again the same way, only reading 4 bytes this time.
ser = serial.Serial('/dev/ttyS0', 2400, timeout= 10, bytesize=8, stopbits=1)
a = ser.read(1)
print a ^AÜÀ ser.close() type(a)

<type 'str'>
int(a, 16)
Traceback (innermost last):
File "<stdin>", line 1, in ?
ValueError: invalid literal for int(): ^AÜÀ

i dont understand what i am missing here.
the string character represents a hex character.

[Grant Edwards]

On 2005-08-18, ne*****@xit.net <ne*****@xit.net> wrote:

import serial
ser = serial.Serial('/dev/ttyS0', 2400, timeout= 10, bytesize=8, stopbits=1)
a = ser.read(1)
print a ^
That's not a hex number. Hex numbers are composed of '0-9A-F'

0F48A is a hex number. ^ is not a hex number.

ser.close() type(a)<type 'str'>
int(a, 16) Traceback (innermost last):
File "<stdin>", line 1, in ?
ValueError: invalid literal for int(): ^

No big surprise there. We've already seen that a is bound to
the string '^', and that isn't a hex number.
so i run it again the same way, only reading 4 bytes this time.
It doesn't matter how many bytes you read. If what you're
reading isn't a hex number, more of it still isn't a hex number.

ser = serial.Serial('/dev/ttyS0', 2400, timeout= 10, bytesize=8, stopbits=1)
a = ser.read(1)
print a ^AÜÀ ser.close() type(a)

<type 'str'>
int(a, 16)
Traceback (innermost last):
File "<stdin>", line 1, in ?
ValueError: invalid literal for int(): ^AÜÀ

Again, not a hex number. Hex numbers consist only of 0-9A-F.
Hex numbers do not contain ^ Ü or À.
i dont understand what i am missing here. the string character
represents a hex character.

No, it doesn't. "Hex" is a base-16 string representation
consisting of the digits 0-9 and A-F[1]. The strings you're
reading are clearing not hex, since they consist of characters
other than 0-9 and A-F. You appear to be reading binary data
of some sort.

If you want to convert a string of 4 8-bit bytes (which is what
you get when you do whatever.read(4)) integer, then you need to use the
struct module. You're going to need to know whether the data
are being transmitted least significant byte first or most
significant byte first.

http://www.python.org/doc/current/li...le-struct.html

If all you want is to convert a single character (what you get
when you call whatever.read(1)), then all you need is the ord()
builtin:

http://www.python.org/doc/current/ref/types.html#l2h-51
[1] Unless you're got base-16 hardware (which you don't), then
the native hardware representation is hex.

--
Grant Edwards grante Yow! .. My pants just went
at on a wild rampage through a
visi.com Long Island Bowling Alley!!

[Peter Hansen]

ne*****@xit.net wrote:

import serial
ser = serial.Serial('/dev/ttyS0', 2400, timeout= 10, bytesize=8, stopbits=1)
a = ser.read(1)
print a
It sounds like you want to convert characters into their corresponding
integer values. To do this, use the ord() builtin function.
ord('^')

94

Then you can do the bitwise operations you want.

(Of course, you might be looking for the struct module (see the docs on
that) instead of just ord(). That would be the case if you want to
convert more than one character at a time, to some numeric value.)

If this is what you want, part of the confusion was your calling the
incoming data a "string" when it's really a series of characters, which
you will be dealing with individually.

Another part of the confusion was referring to ASCII. From the looks of
things, your data is just bytes, not ASCII. ASCII refers not to the
bytes themselves, but to the meaning assigned to those bytes for certain
purposes. For example, the bytes 70, 111, and 111 are just three bytes,
with no particular meaning to anyone. If you want to treat them as
ASCII, however, they represent these three characters: "Foo". Your data
looks like chunk when treated as ASCII, so it's probably just bytes.

-Peter

[Peter Hansen]

Peter Hansen wrote:

ASCII, however, they represent these three characters: "Foo". Your data
looks like chunk when treated as ASCII, so it's probably just bytes.

Weird. I think I meant "junk" (not "chunk"), but obviously was writing
verbally, not visually...

-Peter

[nephish]

all apologies, gentlemen, i feel like an idiot.
the ord() is what is returning what i need.
the manufacturer of the unit i am reading from
told me that it puts out a "string that represents a hex"
been thumping my head.
sorry for the confusion,
and thanks for your help
shawn

[Bengt Richter]

On 18 Aug 2005 10:42:32 -0700, ne*****@xit.net wrote:

import serial
ser =3D serial.Serial('/dev/ttyS0', 2400, timeout=3D 10, bytesize=3D8, =stopbits=3D1) a =3D ser.read(1)
print a^

In general,

print a

is not a good way to investigate what a is, because print uses str(a)
as a conversion, and when the original a is type str, that doesn't
change the data being fed to the presentation process (which BTW will
encode the characters depending on the coding scheme for your display output,
or other destination encoding info).

print repr(a)

will get you a string representation of a that is guaranteed to be printable,
so that if a is a str type containing unprintable characters, you will see
them all represented, e.g.,

a = '\x01\x02\x20\x41\x42\x43\x20\x30\x31\x32'
print a ?? ABC 012 print repr(a) '\x01\x02 ABC 012'

or, interactively, just
a '\x01\x02 ABC 012'

although for convenience None is not printed when evaluated interactively:
b = None
b
print repr(b) None

(interactively, it would be a nuisance when invoking functions that don't return results,
since they in actuality do return None by default).

Sometime it is hard to see the separate characters with printables and \-escaped characters
all joined together. list(a) is an easy way to separate them:
a '\x01\x02 ABC 012' print list(a) ['\x01', '\x02', ' ', 'A', 'B', 'C', ' ', '0', '1', '2']

or just
list(a)

['\x01', '\x02', ' ', 'A', 'B', 'C', ' ', '0', '1', '2']

The trick is to keep in mind that there is some abstraction that is
being represented in various ways, and make allowances for people's
saying "hex" when they mean the binary that the hex (characters
matching [0-9A-Fa-f]) representation is representing, and similarly
with other abstractions (e.g. binary ;-) and their representations (e.g.,
string representations of binary, or DRAM state representations of binary, etc.)

HTH

Regards,
Bengt Richter

[nephish]

thanks for your time.
i have started just doing a type(a) from idle a lot just so i can make
sure of what i am dealing with ( i do this a lot) before i build the
..py file. got tired of 'cannot concatonate str and init' stuff all the
time.
this has been a wild project. nothing like getting in way over your
head to crash learn something. i really dig python. just was going
outta my onion over this one.

thanks again
shawn

[John Machin]

Grant Edwards wrote:

On 2005-08-18, ne*****@xit.net <ne*****@xit.net> wrote:

i have an ascii string comming in the serial port and i need to convert
it to something else, like an integer, or binary, or even a hex so i
can use the bitwise comparison on it.
Nephish, *WHY* do want to use "the bitwise comparison" on it? What do
you understand a "bitwise comparison" to be?

But what do you mean by "integer", "binary", and "hex"?

Decimal, hex, and binary are all representations of integers.

On your computer all integers are 2's compliment binary
"all"???
"are"???

Try: sometimes, integers are represented in 2's complEment binary form;
sometimes they are represented in other forms such as unsigned binary.
For example an 8-bit byte can be use to represent integers in
range(-128, 128) using (signed) twos complement binary, or integers in
range(0, 256) using signed binary, or integers in the range (0, 10)
using decimal ASCII codes, or integers in range (0, 100) using BCD with
no sign nibble, or ...

(we're
going to ignore BCD for the moment).

The only thing that can be "decimal" or "hex" are _string_
representations of integer values.

If you want something upon which you can perform the bitwise
boolean operations &, ^, |, then you most probably just want an
integer object.

To convert a string to an integer, use the int() builtin
passing it the string an an optional base:

int("1234")

Grant:
The OP is reading raw output from a serial port. He's already said he's
getting "funny ASCII characters". One gets the impression he thinks he
needs to do bit-twiddling on *each* byte. Looks like he needs ord(), or
(better) struct.unpack()

Nephish:
(1) Please tell us what docs you have on what the serial gadget is
outputting, in what format.
(2) Please show us what code you are using to read the serial port.
(3) Please show us the results of
print repr(some_results_of_reading_from_the_serial_port)
(4) Please tell us what you like to transform (3) into, or what you need
to "bitwise" compare it to.
Then we might be able to give you some coherent advice.

[Grant Edwards]

On 2005-08-19, John Machin <sj******@lexicon.net> wrote:

The OP is reading raw output from a serial port. He's already said he's
getting "funny ASCII characters". One gets the impression he thinks he
needs to do bit-twiddling on *each* byte. Looks like he needs ord(), or
(better) struct.unpack()

Yes, I finally reached that conclusion. The OP kept insisting
he was reading hex strings -- an impression given him by the
device's vendor, apparently.

--
Grant Edwards grante Yow! Jesuit priests are
at DATING CAREER DIPLOMATS!!
visi.com

[nephish]

Sorry i am late getting back on this.
ord() is finally what is giving me what i wanted.
the vendor told me that what was comming in was an ascii string
representing hex characters. So i expected when i used the serial
module
that what i would be getting was something along the lines of 4A, 3D,
etc..
but i got weird characters. So then (with the help of this very thread)
discovered that i am reading raw bytes. i am supposed to validate the
bytes as they come in because the message comes in four at a time. and
i validate each byte
like this
var = (validation number)
a = ser.read(1)
x = ord(a)
then to validate : x = (x & var)

I am really sorry about the confusion. i am really new at this.
but i am getting what i am supposed to now.

i dont have the details on the script, because it is at work.
but i will post it monday.
thanks for everything.
i would still be banging my head if not for this thread

[Steve Holden]

ne*****@xit.net wrote:

Sorry i am late getting back on this.
ord() is finally what is giving me what i wanted.
the vendor told me that what was comming in was an ascii string
representing hex characters. So i expected when i used the serial
module
that what i would be getting was something along the lines of 4A, 3D,
etc..
but i got weird characters. So then (with the help of this very thread)
discovered that i am reading raw bytes. i am supposed to validate the
bytes as they come in because the message comes in four at a time. and
i validate each byte
like this
var = (validation number)
a = ser.read(1)
x = ord(a)
then to validate : x = (x & var)

I am really sorry about the confusion. i am really new at this.
but i am getting what i am supposed to now.

i dont have the details on the script, because it is at work.
but i will post it monday.
thanks for everything.
i would still be banging my head if not for this thread

Nice of you to provide some feedback, and great that your program in
understanding the data.

You'll find c.l.py can save you large amounts of time, and before you
know it you'll be answering other people's questions.

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC http://www.holdenweb.com/