finding byte order

[biner]

Hello,

I am using a program that has to read binary data from files coming
from different machines. The file are always written with big endian.
I am using the struct module to read the data and it is fine because I
can specify in the format if the data are to be read with big or small
endian convention.

I would like to use the array module instead of struct because it is
suppose to be faster for big arrays. However, this module does not
provide a format specifier to say if the data are writtent with big or
small endian. The result is that it works on big-endian machine and
not on small-endian machine.

I would like to have a test to tell me if the current machine is
using big or small endian, this way I could use the array module in
the first case and the *slower* struct module on the second. I looked
but did not find. Is there a python function to know that?

Thanks!

[Scott David Daniels]

biner wrote:

I would like to have a test to tell me if the current machine is
using big or small endian, this way I could use the array module in
the first case and the *slower* struct module on the second. I looked
but did not find. Is there a python function to know that?

Thanks!

How about sys.byteorder?

At least if you are using array.array, note the "byteswap" method:

import array
v = array.array('h',range(8))
v array('h', [0, 1, 2, 3, 4, 5, 6, 7]) v.byteswap()
v

array('h', [0, 256, 512, 768, 1024, 1280, 1536, 1792])

--Scott David Daniels
Sc***********@Acm.Org

[Diez B. Roggisch]

> How about sys.byteorder?

This doesn't help, as he wants to read files from varying endianess - what
the _current_ endianess is doesn't matter here.

--
Regards,

Diez B. Roggisch

[Diez B. Roggisch]

> I would like to have a test to tell me if the current machine is

using big or small endian, this way I could use the array module in
the first case and the *slower* struct module on the second. I looked
but did not find. Is there a python function to know that?

There is no such test, as it's domain-specific if there _can_ be such a test
or not. If your data is composed in a way that you can infer the endianess
by reading e.g. a header at the beginning that has well-known fields, then
it might work. But only you can do that.

For a pure binary file, there is absolutely no way of telling the endianess.
--
Regards,

Diez B. Roggisch

[Scott David Daniels]

biner wrote:

I am using a program that has to read binary data from files coming
from different machines. The file are always written with big endian.

Diez B. Roggisch wrote:
[Scott David Daniels wrote]

How about sys.byteorder?

This doesn't help, as he wants to read files from varying endianess - what
the _current_ endianess is doesn't matter here.

But, in fact, he says the files are always big endian. So, code like
the following should address his problem. Note I use type 'h' as an
example so I can easily read samples.

import sys, array
f =open('huge.dat')
v = array.array('h') # Or whatever data type
v.fromfile(f, 4096)
f.close()
if sys.byteorder == 'little':
v.byteswap()

--Scott David Daniels
Sc***********@Acm.Org

[Diez B. Roggisch]

> But, in fact, he says the files are always big endian. So, code like

the following should address his problem. Note I use type 'h' as an
example so I can easily read samples.

I'm sorry, I confused that he asked for machine endianess. Then of course
you are right.

--
Regards,

Diez B. Roggisch

[biner.sebastien]

Scott David Daniels wrote:

biner wrote:

> I am using a program that has to read binary data from files coming

> from different machines. The file are always written with big endian.

Diez B. Roggisch wrote:
[Scott David Daniels wrote]How about sys.byteorder?

This doesn't help, as he wants to read files from varying endianess

- what the _current_ endianess is doesn't matter here.

But, in fact, he says the files are always big endian. So, code like
the following should address his problem. Note I use type 'h' as an
example so I can easily read samples.

import sys, array
f =open('huge.dat')
v = array.array('h') # Or whatever data type
v.fromfile(f, 4096)
f.close()
if sys.byteorder == 'little':
v.byteswap()

--Scott David Daniels
Sc***********@Acm.Org

This seems to do the what I want. I did not know about array.byteswap
and sys.byteorder.
Thanks for taking the time to answer my silly question.

Ciao!