How to Read Bytes from a file

[gregpinero]

It seems like this would be easy but I'm drawing a blank.

What I want to do is be able to open any file in binary mode, and read
in one byte (8 bits) at a time and then count the number of 1 bits in
that byte.

I got as far as this but it is giving me strings and I'm not sure how
to accurately get to the byte/bit level.

f1=file('somefi le','rb')
while 1:
abyte=f1.read(1 )

Thanks in advance for any help.

-Greg

[Alex Martelli]

gr********@gmai l.com <gr********@gma il.comwrote:

It seems like this would be easy but I'm drawing a blank.

What I want to do is be able to open any file in binary mode, and read
in one byte (8 bits) at a time and then count the number of 1 bits in
that byte.

I got as far as this but it is giving me strings and I'm not sure how
to accurately get to the byte/bit level.

f1=file('somefi le','rb')
while 1:
abyte=f1.read(1 )

You should probaby prepare before the loop a mapping from char to number
of 1 bits in that char:

m = {}
for c in range(256):
m[c] = countones(c)

and then sum up the values of m[abyte] into a running total (break from
the loop when 'not abyte', i.e. you're reading 0 bytes even though
asking for 1 -- that tells you the fine is finished, remember to close
it).

A trivial way to do the countones function:

def countones(x):
assert x>=0
c = 0
while x:
c += (x&1)
x >>= 1
return c

you just don't want to call it too often, whence the previous advice to
call it just 256 times to prep a mapping.

If you download and install gmpy you can use gmpy.popcount as a fast
implementation of countones:-).
Alex

[Leif K-Brooks]

Alex Martelli wrote:

You should probaby prepare before the loop a mapping from char to number
of 1 bits in that char:

m = {}
for c in range(256):
m[c] = countones(c)

Wouldn't a list be more efficient?

m = [countones(c) for c in xrange(256)]

[Bart Ogryczak]

On Mar 1, 7:52 am, "gregpin...@gma il.com" <gregpin...@gma il.com>
wrote:

It seems like this would be easy but I'm drawing a blank.

What I want to do is be able to open any file in binary mode, and read
in one byte (8 bits) at a time and then count the number of 1 bits in
that byte.

I got as far as this but it is giving me strings and I'm not sure how
to accurately get to the byte/bit level.

f1=file('somefi le','rb')
while 1:
abyte=f1.read(1 )

import struct
buf = open('somefile' ,'rb').read()
count1 = lambda x: (x&1)+(x&2>0)+( x&4>0)+(x&8>0)+ (x&16>0)+(x&32> 0)+
(x&64>0)+(x&128 >0)
byteOnes = map(count1,stru ct.unpack('B'*l en(buf),buf))

byteOnes[n] is number is number of ones in byte n.

[Jussi Salmela]

Bart Ogryczak kirjoitti:

On Mar 1, 7:52 am, "gregpin...@gma il.com" <gregpin...@gma il.com>
wrote:

>It seems like this would be easy but I'm drawing a blank.

What I want to do is be able to open any file in binary mode, and read
in one byte (8 bits) at a time and then count the number of 1 bits in
that byte.

I got as far as this but it is giving me strings and I'm not sure how
to accurately get to the byte/bit level.

f1=file('somef ile','rb')
while 1:
abyte=f1.read(1 )

import struct
buf = open('somefile' ,'rb').read()
count1 = lambda x: (x&1)+(x&2>0)+( x&4>0)+(x&8>0)+ (x&16>0)+(x&32> 0)+
(x&64>0)+(x&128 >0)
byteOnes = map(count1,stru ct.unpack('B'*l en(buf),buf))

byteOnes[n] is number is number of ones in byte n.

I guess struct.unpack is not necessary, because:

byteOnes2 = map(count1, (ord(ch) for ch in buf))

seems to do the trick also.

Cheers,
Jussi

[Alex Martelli]

Leif K-Brooks <eu*****@ecritt ers.bizwrote:

Alex Martelli wrote:

You should probaby prepare before the loop a mapping from char to number
of 1 bits in that char:

m = {}
for c in range(256):
m[c] = countones(c)

Wouldn't a list be more efficient?

m = [countones(c) for c in xrange(256)]

Yes, or an array.array -- actually I meant to use m[chr(c)] above (so
you could use the character you're reading directly to index m, rather
than calling ord(byte) a bazillion times for each byte you're reading),
but if you're using the numbers (as I did before) a list or array is
better.
Alex

[gregpinero]

On Mar 1, 8:53 am, "Bart Ogryczak" <B.Ogryc...@gma il.comwrote:

On Mar 1, 7:52 am, "gregpin...@gma il.com" <gregpin...@gma il.com>
wrote:

It seems like this would be easy but I'm drawing a blank.

What I want to do is be able to open any file in binary mode, and read
in one byte (8 bits) at a time and then count the number of 1 bits in
that byte.

I got as far as this but it is giving me strings and I'm not sure how
to accurately get to the byte/bit level.

f1=file('somefi le','rb')
while 1:
abyte=f1.read(1 )

import struct
buf = open('somefile' ,'rb').read()
count1 = lambda x: (x&1)+(x&2>0)+( x&4>0)+(x&8>0)+ (x&16>0)+(x&32> 0)+
(x&64>0)+(x&128 >0)
byteOnes = map(count1,stru ct.unpack('B'*l en(buf),buf))

byteOnes[n] is number is number of ones in byte n.

This solution looks nice, but how does it work? I'm guessing
struct.unpack will provide me with 8 bit bytes (will this work on any
system?)

How does count1 work exactly?

Thanks for the help.

-Greg

[John Machin]

On Mar 2, 12:53 am, "Bart Ogryczak" <B.Ogryc...@gma il.comwrote:

>
import struct
buf = open('somefile' ,'rb').read()
count1 = lambda x: (x&1)+(x&2>0)+( x&4>0)+(x&8>0)+ (x&16>0)+(x&32> 0)+
(x&64>0)+(x&128 >0)
byteOnes = map(count1,stru ct.unpack('B'*l en(buf),buf))

byteOnes = map(count1,stru ct.unpack('%dB' %len(buf),buf))

[Bart Ogryczak]

On Mar 1, 4:58 pm, "gregpin...@gma il.com" <gregpin...@gma il.com>
wrote:

On Mar 1, 8:53 am, "Bart Ogryczak" <B.Ogryc...@gma il.comwrote:

On Mar 1, 7:52 am, "gregpin...@gma il.com" <gregpin...@gma il.com>
wrote:

It seems like this would be easy but I'm drawing a blank.

What I want to do is be able to open any file in binary mode, and read
in one byte (8 bits) at a time and then count the number of 1 bits in
that byte.

I got as far as this but it is giving me strings and I'm not sure how
to accurately get to the byte/bit level.

f1=file('somefi le','rb')
while 1:
abyte=f1.read(1 )

import struct
buf = open('somefile' ,'rb').read()
count1 = lambda x: (x&1)+(x&2>0)+( x&4>0)+(x&8>0)+ (x&16>0)+(x&32> 0)+
(x&64>0)+(x&128 >0)
byteOnes = map(count1,stru ct.unpack('B'*l en(buf),buf))

byteOnes[n] is number is number of ones in byte n.

This solution looks nice, but how does it work? I'm guessing
struct.unpack will provide me with 8 bit bytes

unpack with 'B' format gives you int value equivalent to unsigned char
(1 byte).

(will this work on any system?)

Any system with 8-bit bytes, which would mean any system made after
1965. I'm not aware of any Python implementation for UNIVAC, so I
wouldn't worry ;-)

How does count1 work exactly?

1,2,4,8,16,32,6 4,128 in binary are
1,10,100,1000,1 0000,100000,100 0000,10000000
x&1 == 1 if x has first bit set to 1
x&2 == 2, so (x&2>0) == True if x has second bit set to 1
.... and so on.
In the context of int, True is interpreted as 1, False as 0.

[gregpinero]

On Mar 1, 12:46 pm, "Bart Ogryczak" <B.Ogryc...@gma il.comwrote:

This solution looks nice, but how does it work? I'm guessing
struct.unpack will provide me with 8 bit bytes

unpack with 'B' format gives you int value equivalent to unsigned char
(1 byte).

(will this work on any system?)

Any system with 8-bit bytes, which would mean any system made after
1965. I'm not aware of any Python implementation for UNIVAC, so I
wouldn't worry ;-)

How does count1 work exactly?

1,2,4,8,16,32,6 4,128 in binary are
1,10,100,1000,1 0000,100000,100 0000,10000000
x&1 == 1 if x has first bit set to 1
x&2 == 2, so (x&2>0) == True if x has second bit set to 1
... and so on.
In the context of int, True is interpreted as 1, False as 0.

Thanks Bart. That's perfect. The other suggestion was to precompute
count1 for all possible bytes, I guess that's 0-256, right?

Thanks again everyone for the help.

-Greg