Reading data from a simple binary file

[Chuck Anderson]

I've read some discussions, searched Google, tried pack, unpack,
bin2hex, explode, implode, and I just can't figure out how to convert
this data. I am trying to read a file with the following format (two
examples).

41 6D 74 72 61 6B 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 75 05 1A 00

41 6D 74 72 61 6B 20 4F 6E 6C 69 6E 65 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 8F 05 2B 00

The first 28 bytes are a 00 terminated character string (in this case
"Amtrak" and "Amtrak online")
Next are the location (offset) of an entry in another file (2 bytes),
followed by that entry's size in the file (the next 2 bytes). The above
examples are key entries in an index file pointing to variable sized
entries in a data file.

I am using fread to put each field into a string variable (28 bytes, 2
bytes, and 2 bytes). The binary values are "Little Endian" (right?). In
other words 75 05 is really x0575 and 1A 00 is really x001A. I am having
the most trouble with these (I've already used bin2hex, but I don't know
how to convert the little endian to decimal.)

1. I want to strip the 00's from the string.

2. I want to convert the location (offset) into an integer variable.

3. I want to convert the size into an integer variable.

I am going to use location and size to fseek and fread the entries in
the data file into another string.

My actual goal is to transfer this data (the string and it's associated,
variable sized text field) into a MySQL database.

My guess is (was) that this is easy, but I just can't figure it out.

Any help would be very much appreciated.

TIA.

--
*************** **************
Chuck Anderson • Boulder, CO
http://www.CycleTourist.com
Integrity is obvious.
The lack of it is common.
*************** **************

[Pedro Graca]

Chuck Anderson wrote:

41 6D 74 72 61 6B 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 75 05 1A 00

41 6D 74 72 61 6B 20 4F 6E 6C 69 6E 65 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 8F 05 2B 00

The first 28 bytes are a 00 terminated character string (in this case
"Amtrak" and "Amtrak online")
Next are the location (offset) of an entry in another file (2 bytes),
followed by that entry's size in the file (the next 2 bytes). The above
examples are key entries in an index file pointing to variable sized
entries in a data file. I am going to use location and size to fseek and fread the entries in
the data file into another string.

Something like this?

<?php
$in1 = "\x41\x6d\x74\x 72\x61\x6b\x00\ x00\x00\x00\x00 \x00\x00\x00\x0 0\x00" .
"\x00\x00\x00\x 00\x00\x00\x00\ x00\x00\x00\x00 \x00\x75\x05\x1 a\x00";
$in2 = "\x41\x6d\x74\x 72\x61\x6b\x20\ x4f\x6e\x6c\x69 \x6e\x65\x00\x0 0\x00" .
"\x00\x00\x00\x 00\x00\x00\x00\ x00\x00\x00\x00 \x00\x8f\x05\x2 b\x00";

$arr = array();
$arr[] = unpack('a28name/vlocation/vsize', $in1);
$arr[] = unpack('a28name/vlocation/vsize', $in2);

var_dump($arr);
echo "\n";
?>
Output is (reformatted):
array(2) {
[0] => array(3) {
["name"] => string(6) "Amtrak"
["location"] => int(1397)
["size"] => int(26)
}
[1] => array(3) {
["name"] => string(13) "Amtrak Online"
["location"] => int(1423)
["size"] => int(43)
}
}
HTH
--
--= my mail box only accepts =--
--= Content-Type: text/plain =--
--= Size below 10001 bytes =--

[Chuck Anderson]

Pedro Graca wrote:

Chuck Anderson wrote:

41 6D 74 72 61 6B 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 75 05 1A 00

41 6D 74 72 61 6B 20 4F 6E 6C 69 6E 65 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 8F 05 2B 00

The first 28 bytes are a 00 terminated character string (in this case
"Amtrak" and "Amtrak online")
Next are the location (offset) of an entry in another file (2 bytes),
followed by that entry's size in the file (the next 2 bytes). The above
examples are key entries in an index file pointing to variable sized
entries in a data file.

I am going to use location and size to fseek and fread the entries in
the data file into another string.

Something like this?

<?php
$in1 = "\x41\x6d\x74\x 72\x61\x6b\x00\ x00\x00\x00\x00 \x00\x00\x00\x0 0\x00" .
"\x00\x00\x00\x 00\x00\x00\x00\ x00\x00\x00\x00 \x00\x75\x05\x1 a\x00";
$in2 = "\x41\x6d\x74\x 72\x61\x6b\x20\ x4f\x6e\x6c\x69 \x6e\x65\x00\x0 0\x00" .
"\x00\x00\x00\x 00\x00\x00\x00\ x00\x00\x00\x00 \x00\x8f\x05\x2 b\x00";

$arr = array();
$arr[] = unpack('a28name/vlocation/vsize', $in1);
$arr[] = unpack('a28name/vlocation/vsize', $in2);

var_dump($arr) ;
echo "\n";
?>
Output is (reformatted):
array(2) {
[0] => array(3) {
["name"] => string(6) "Amtrak"
["location"] => int(1397)
["size"] => int(26)
}
[1] => array(3) {
["name"] => string(13) "Amtrak Online"
["location"] => int(1423)
["size"] => int(43)
}
}
HTH

THVM - That helps very much.

Thank you. I could not understand how to use unpack, but your example
explains it very well.

I only need to add the MySQL functions to load the data into a new
database (which I understand well)

You've saved me hours more of fumbling around ignorantly and the pain of
banging my head on the table while doing it. Thank you very, very much.
(And my head thanks you, too ҿج)

This should be a new thread, but I have one more question I'm guessing
is simple.
One of the key names has chars, then a NULL (x00), and then more chars
(an error got in there somehow.)

It looks like this
52 75 74 68 00 61 6E 00 00 00 00 00 00 00 00 00 - R u t h . a n
00 00 00 00 00 00 00 00 00 00 00 00 C7 53 88 00

So I get Ruth?an ("?" is actually a square - the "no char" symbol).
The "an" does not belong there.

In and of itself, I don't really care. Only 2 entries (of 220 total)
have this problem, but it makes me wonder; how do I make a string
terminate at the first NULL character? I tried sprintf("%s", $string),
but that doesn't do it.

How do you remove all bytes (chars) after the first NULL character?

If you like I could repost this as a new thread, but you seem very
knowledgeable.

And thanks again (very much) for the above!

--
*************** **************
Chuck Anderson • Boulder, CO
http://www.CycleTourist.com
Integrity is obvious.
The lack of it is common.
*************** **************

[Pedro Graca]

Chuck Anderson wrote:

It looks like this
52 75 74 68 00 61 6E 00 00 00 00 00 00 00 00 00 - R u t h . a n
00 00 00 00 00 00 00 00 00 00 00 00 C7 53 88 00

So I get Ruth?an ("?" is actually a square - the "no char" symbol).
The "an" does not belong there. How do you remove all bytes (chars) after the first NULL character?

I don't think PHP has a single function to do that. But it's easy to
make your own :)

<?php
function C_string($strin g) {
$zero = strpos($string, "\x00");
if ($zero === false) return $string;
else return substr($string, 0, $zero);
}

### example usage
$name = "Ruth\x00an\x00 \x00";
$name = C_string($name) ;
?>

--
--= my mail box only accepts =--
--= Content-Type: text/plain =--
--= Size below 10001 bytes =--

[Chuck Anderson]

Pedro Graca wrote:

Chuck Anderson wrote:

It looks like this
52 75 74 68 00 61 6E 00 00 00 00 00 00 00 00 00 - R u t h . a n
00 00 00 00 00 00 00 00 00 00 00 00 C7 53 88 00

So I get Ruth?an ("?" is actually a square - the "no char" symbol).
The "an" does not belong there.

How do you remove all bytes (chars) after the first NULL character?

I don't think PHP has a single function to do that. But it's easy to
make your own :)

<?php
function C_string($strin g) {
$zero = strpos($string, "\x00");
if ($zero === false) return $string;
else return substr($string, 0, $zero);
}

### example usage
$name = "Ruth\x00an\x00 \x00";
$name = C_string($name) ;
?>

Thanks once again. I always wonder if I should write little loops like
that, but I hate to resort to them only to find out that there is
already a function.

Thanks for helping to clear out the cobwebs of a C++ programmer who
hasn't written any programs in over 12 years. (All I've done is use Php,
now, for almost 2 years.)

BTW, I've got my database migrated and have almost completed the user
interface to it. Thanks a ton. There was a lot of important data in there.

--
*************** **************
Chuck Anderson • Boulder, CO
http://www.CycleTourist.com
Integrity is obvious.
The lack of it is common.
*************** **************