I have some binary data read from a file that is arranged like
<3-byte int> <3-byte int> <3-byte int> etc.
The "ints" are big-endian and there are 169 of them. Is there any clever
way to convert these to regular Python ints other than (struct) unpack'ing
them one at a time and doing the math?
Thanks!
Bob 4 5098
On 27/06/2006 9:36 AM, Bob Greschke wrote: I have some binary data read from a file that is arranged like
<3-byte int> <3-byte int> <3-byte int> etc.
The "ints" are big-endian and there are 169 of them. Is there any clever way to convert these to regular Python ints other than (struct) unpack'ing them one at a time and doing the math?
I'd call that "arithmetic", not "math" :-)
Here's another way, not touted as "clever":
|>> guff = "\x00\x00\x01\x00\x02\x01\x03\x00\x00\x00\x00\ xFF"
|>> expected = [1, 513, 3 * 65536, 255]
|>> expected
[1, 513, 196608, 255]
|>> import array
|>> b = array.array('B', guff)
|>> b
array('B', [0, 0, 1, 0, 2, 1, 3, 0, 0, 0, 0, 255])
|>> actual = [b[x]*65536 + b[x+1]*256 + b[x+2] for x in range(0, len(b), 3)]
|>> actual
[1, 513, 196608, 255]
|>> actual == expected
True
Cheers,
John
Bob Greschke wrote: I have some binary data read from a file that is arranged like <3-byte int> <3-byte int> <3-byte int> etc. The "ints" are big-endian and there are 169 of them. Is there any clever way to convert these to regular Python ints other than (struct) unpack'ing them one at a time and doing the math?
Best way is with scipy (or Numeric or numarray), but for vanilla Python:
import array, itertools
ubytes = array.array('B')
sbytes = array.array('b')
ubytes.fromfile(binarysource, 169 * 3)
sbytes.fromstring(ubytes[::3].tostring())
result = array.array('i', (msb * 256 + mid) * 256 + lsb
for msb, mid, lsb
in itertools.izip(sbytes,
ubytes[1::3], ubytes[2::3])))
If you want to get fancy, you can replace the last statement with:
del ubytes[::3]
uhalf = array.array('H')
uhalf.fromstring(ubytes.tostring())
if array.array('H', [1]) != '\x00\x01': uhalf.byteswap()
result = array.array('i', (msb * 65536 + low for msb, low
in itertools.izip(sbytes, uhalf)))
That was fun.
--Scott David Daniels sc***********@acm.org
On 27/06/2006 9:59 AM, John Machin wrote: On 27/06/2006 9:36 AM, Bob Greschke wrote: I have some binary data read from a file that is arranged like
<3-byte int> <3-byte int> <3-byte int> etc.
The "ints" are big-endian and there are 169 of them. Is there any clever way to convert these to regular Python ints other than (struct) unpack'ing them one at a time and doing the math?
I'd call that "arithmetic", not "math" :-)
Here's another way, not touted as "clever":
|>> guff = "\x00\x00\x01\x00\x02\x01\x03\x00\x00\x00\x00\ xFF" |>> import array |>> b = array.array('B', guff) |>> actual = [b[x]*65536 + b[x+1]*256 + b[x+2] for x in range(0, len(b), 3)]
Two further points:
(1) If using struct.unpack, it's not necessary to unpack 3 bytes at a
time; one could substitute b = struct.unpack('507B', guff) in the above.
(2) The OP didn't specify whether the ints were signed or unsigned. The
above is for unsigned. To convert an unsigned X to signed, here's the
"math":
if X >= 0x800000: X -= 0x1000000
Cheers,
John
>I have some binary data read from a file that is arranged like <3-byte int> <3-byte int> <3-byte int> etc.
The "ints" are big-endian and there are 169 of them. Is there any clever way to convert these to regular Python ints other than (struct) unpack'ing them one at a time and doing the math?
Thanks guys! I was looking for a 'one liner' of some kind, but I guess
there isn't one (too bad you can't tell array or struct.unpack how many
bytes you want an integer to be). Yeah, these are signed ints so I'll have
to convert them.
To me balancing my checkbook involves higher "math". :)
Bob This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Nazgul |
last post by:
Hi!
I want to implement a small tool in Python for distributing "patches" and I
need Your advice. This application should be able to package all files
chosen by a user into a self-extracting.exe...
|
by: Calvin FONG |
last post by:
Dear all,
Are there any utility that can be call by python to create self
extracting zip file. I'm now using the powerarchiever. But the command
line options aren't flexible enough. Basically, I...
|
by: Avi |
last post by:
hi,
Can anyone tell me what the problem is and how to solve it
The following piece of code resides on an asp page on the server and
is used to download files from the server to the machine...
|
by: Michael Hill |
last post by:
Hi, folks.
I am writing a Javascript program that accepts (x, y) data pairs from
a text box and then analyzes that data in various ways. This is my
first time using text area boxes; in the past,...
|
by: Cognizance |
last post by:
Hi gang,
I'm an ASP developer by trade, but I've had to create client side
scripts with JavaScript many times in the past. Simple things, like
validating form elements and such.
Now I've been...
|
by: v0lcan0 |
last post by:
Any help on extracting the time part from the datetime field in SQL
database.
even though i had entered only the time part in the database when i
extract the field it gives me only the date...
|
by: Chris Belcher |
last post by:
First some background...
The database tracks Action Items assigned to a group of 20 or so
managers. Once the assignment is created it is then emailed to each of
the managers that are included in...
|
by: k_nil |
last post by:
I have a link on my web page for a self extracting executable file placed on
the server. When the link is clicked,
1) i could see dialog box with open or save options
2) when open clicked, self...
|
by: bjm |
last post by:
I created a self extracting zip file with about 9000 files in it. I
extracted it manually from the command line without a problem.
However, when I tried to do the same extraction at the same...
|
by: Werner |
last post by:
Hi,
I try to read (and extract) some "self extracting" zipefiles on a
Windows system. The standard module zipefile seems not to be able to
handle this.
False
Is there a wrapper or has...
|
by: taylorcarr |
last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
|
by: ryjfgjl |
last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
|
by: ryjfgjl |
last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
| |