473,549 Members | 2,723 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Unpacking Binary Data - not using struct module

I am working on a file conversion project that reads data from a one
file format, reformats in and writes in out to another. The data is
records of informations - names address, account number,statisti cs.

The numeric values in the original file are stored in what appears to
be a "packed" data format,using a structure that does not use any of
the more standard "C" formats, so I can't use the "struct" module.

As an example, the number "1130" is store using 3 bytes. The HEX
values of the 3 bytes would be 0x01,0x13,0x0F. In other words, the
hex value value "01130f" is unpacked to become "1130".

I believe the original program is written in a version of "Basic" that
dates from the early 90's if that helps.

Currently, I have no problem reading the data. I read the three bytes
and use the binascii.hexili fy() function convert the hex values to
their corresponding ASCII values, and then slice off the trailing "f".
In other words;

unpackednumber = int(binascii.he xlify(0x1130f)[:-1])

My questions ...
1) Does anyone recognize this numeric format ?
2) Is there a more efficient module/function I can use, or is the
above the best method?
3) I think that the trailing "f" is sometimes used to indicate the
sign of the number and perhaps even the number of decimal places ...
but am not sure.

Any advice/guidance would be appreciated.
Jul 18 '05 #1
3 4556
Geoffrey wrote:
My questions ...
1) Does anyone recognize this numeric format ?
Seem to be some decimal coded binary, where you only use hexa from 0 to
9, coding corresponding decimal digits.
The trailing hexa F may be used to indicate the end of the number (if
number can have variable length) as it is normally not used as a digit
in this coding.
3) I think that the trailing "f" is sometimes used to indicate the
sign of the number and perhaps even the number of decimal places ...
but am not sure.


Maybe... but have you an example of a negative number ?

A+

Laurent.
Jul 18 '05 #2
Geoffrey wrote:
I am working on a file conversion project that reads data from a one
file format, reformats in and writes in out to another. The data is
records of informations - names address, account number,statisti cs.

The numeric values in the original file are stored in what appears to
be a "packed" data format,using a structure that does not use any of
the more standard "C" formats, so I can't use the "struct" module.

As an example, the number "1130" is store using 3 bytes. The HEX
values of the 3 bytes would be 0x01,0x13,0x0F. In other words, the
hex value value "01130f" is unpacked to become "1130".

I believe the original program is written in a version of "Basic" that
dates from the early 90's if that helps.

Currently, I have no problem reading the data. I read the three bytes
and use the binascii.hexili fy() function convert the hex values to
their corresponding ASCII values, and then slice off the trailing "f".
In other words;

unpackednumber = int(binascii.he xlify(0x1130f)[:-1])

My questions ...
1) Does anyone recognize this numeric format ?
2) Is there a more efficient module/function I can use, or is the
above the best method?
3) I think that the trailing "f" is sometimes used to indicate the
sign of the number and perhaps even the number of decimal places ...
but am not sure.

Any advice/guidance would be appreciated.


It might be some form of floating point. If you can encode numbers like
11300, that might help (also throw in some negative numbers and some
small numbers). I'd like 0 and +/- versions of 2**n for n in range(16)
and +/- versions of 10**n for n in range(1,7).

--Scott David Daniels
Sc***********@A cm.Org
Jul 18 '05 #3

"Geoffrey" <ge********@hot mail.com> wrote in message
news:6b******** *************** ***@posting.goo gle.com...
1) Does anyone recognize this numeric format ?


Google :)
http://www.room42.com/store/computer..._decimal.shtml
Jul 18 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

17
2997
by: Pascal | last post by:
Hello, I've a binary file with data in it. This file come from an old ms dos application (multilog ~ 1980). In this application, a field is declared as a 'decimal' (999 999 999.99). I put 0.00 in the field and save the record to the file. When I look in the binary file (with python or an hex editor), the field is stored on 8 bytes:...
5
9768
by: Andreas Røsdal | last post by:
Hello, I want to parse a binary file in python. Does python have some built in methods for doing this easily? Any links to example code would be nice.. Thanks Andreas R.
2
1755
by: george young | last post by:
I came across an cool python feature that I've not seen discussed. This may be *implied* by the language reference manual http://docs.python.org/ref/assignment.html], but it was never obvious to me: An assignment statement can unpack a sequence, binding it's values to explicit names, e.g.: r,g,b = color_tuple and most of us use this...
3
3701
by: Albert Tu | last post by:
Dear there, We have an x-ray CT system. The acquisition computer acquires x-ray projections and outputs multiple data files in binary format (2-byte unsigned integer) such as projection0.raw, projection1.raw, projection2.raw ... up to projection500.raw. Each file is 2*1024*768-byte big. I would like to read those files and convert to...
6
4373
by: David M | last post by:
OK so here is my task. I want to get at the data stored in /var/account/pacct, which stores process accounting data, so that I can make it into a more human understandable format then what the program sa can do. The thing is, its in a binary format and an example program that reads some data from the file is done in C using a struct defined...
3
2181
by: Steve | last post by:
I want to ready binary data from a udp socket effeciently as possible in python. I know of the struct package but do people have any tips when dealing with binary data in python? Is there a library or api that is faster when dealing with binary data. I am looking for a any one with experience or ideas on the subject. Pointers any one? ...
4
1485
by: Mastastealth | last post by:
I'm trying to create a program to read a certain binary format. I have the format's spec which goes something like: First 6 bytes: String Next 4 bytes: 3 digit number and a blank byte --- Next byte: Height (Number up to 255) Next byte: Width (Number up to 255) Next byte: Number 0 - 5 Every 2 bytes after that: Supposedly a number 0000 -...
8
1932
by: Bryan.Fodness | last post by:
Hello, I am having trouble writing the code to read a binary string. I would like to extract the values for use in a calculation. Any help would be great. Here is my function that takes in a string. def parseSequence(data, start):
3
11636
by: Andrew Lentvorski | last post by:
Basically, I'd like to use the ctypes module as a much more descriptive "struct" module. Is there a way to take a ctypes.Structure-based class and convert it to/from a binary string? Thanks, -a
0
7520
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
7957
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
1
7470
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
6043
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5368
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
3500
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
0
3481
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1941
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1059
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.