Write bits in file

[Monica Leko]

Hi

I have a specific format and I need binary representation. Does
Python have some built-in function which will, for instance, represent
number 15 in exactly 10 bits?

[Ken Starks]

You want your file considered as a sequence of bits rather
than a sequence of 8-bit bytes, do you? is the 10-bit
bit-pattern to be stored at an arbitrary bit-position in
the file, or is the whole file regularly subdivided
at 10-bit intervals?
Monica Leko wrote:

Hi

I have a specific format and I need binary representation. Does
Python have some built-in function which will, for instance, represent
number 15 in exactly 10 bits?

[Monica Leko]

On May 18, 2:20*pm, Ken Starks <[e-mail]:

You want your file considered as a sequence of bits rather
than a sequence of 8-bit bytes, do you?

Yes.

is the 10-bit
bit-pattern to be stored at an arbitrary bit-position in
the file

Yes. I need arbitrary, 8bits, than 10 bits for something else, than
sequence of bytes, than 10 bits again, etc.

[Gabriel Genellina]

En Sun, 18 May 2008 10:36:28 -0300, Monica Leko <mo*********@gmail.comescribió:

On May 18, 2:20*pm, Ken Starks <[e-mail]:

>You want your file considered as a sequence of bits rather
than a sequence of 8-bit bytes, do you?

Yes.

>is the 10-bit
bit-pattern to be stored at an arbitrary bit-position in
the file

Yes. I need arbitrary, 8bits, than 10 bits for something else, than
sequence of bytes, than 10 bits again, etc.

If you really need arbitrary bit sequences that aren't synchonized into bytes, I think there is a BitVector o BitArray package somewhere.
But if you mostly have bytes and sparsely a different size, I think the struct module and some gymnastics involving bitwise operations would be enough.

--
Gabriel Genellina

[Ken Starks]

I admit that I was mostly just interested in getting your
question clarified, rather than having any great experise.

But a bit of Googling took me to the 'Bit vector' module,
[I googled: 'python ("bit array" OR "bit vector")']
which might be what you are after. I have no experience
with it, myself:

http://cobweb.ecn.purdue.edu/~kak/di...tor-1.4.1.html

Monica Leko wrote:

On May 18, 2:20 pm, Ken Starks <[e-mail]:

>You want your file considered as a sequence of bits rather
than a sequence of 8-bit bytes, do you?

Yes.

>is the 10-bit
bit-pattern to be stored at an arbitrary bit-position in
the file

Yes. I need arbitrary, 8bits, than 10 bits for something else, than
sequence of bytes, than 10 bits again, etc.

[John Nagle]

Monica Leko wrote:

Hi

I have a specific format and I need binary representation. Does
Python have some built-in function which will, for instance, represent
number 15 in exactly 10 bits?

The "struct" module will let you format Python data as a binary
object of specified format. But it doesn't support arbitrary bit-width
fields; integers can be 1, 2, 4, or 8 bytes.

If you have a space problem, you might use the "struct" module to
convert to a reasonably concise binary representation, then use the
"gzip" module to compress the file down further.

John Nagle

[Tim Roberts]

Monica Leko <mo*********@gmail.comwrote:

>
I have a specific format and I need binary representation. Does
Python have some built-in function which will, for instance, represent
number 15 in exactly 10 bits?

For the record, I'd like to point out that even C cannot do this. You need
to use shifting and masking to produce a stream of 8-bit bytes, or to
extract your values from a stream of 8-bit bytes.
--
Tim Roberts, ti**@probo.com
Providenza & Boekelheide, Inc.

[pataphor]

On Sun, 18 May 2008 06:36:28 -0700 (PDT)
Monica Leko <mo*********@gmail.comwrote:

Yes. I need arbitrary, 8bits, than 10 bits for something else, than
sequence of bytes, than 10 bits again, etc.

Here's something to get you started. No guarantees, but I managed to
write four 10 bit numbers to a file producing 5 bytes, saying hello. I
was just looking for an excuse to use send.

P.

def gen(f):
L = []

def flush(L):
L.reverse()
s = ''.join(map(str,L))
j = int(s,2)
f.write(chr(j))

while 1:
x = yield
if x in [0,1]:
L.append(x)
else:
break
if len(L) == 8:
flush(L)
L = []
if L:
while len(L) < 8:
L.append(0)
flush(L)
yield

def tenbits(i):
for j in range(9,-1,-1):
yield i >j & 1

def charbits(s):
for c in s:
i = ord(c)
for j in range(8):
yield i >j &1

def byten(L):
while L:
yield L[:10]
L = L[10:]

def test():
f = file('out.dat','w')
g = gen(f)
g.send(None)
for x in [90,611,397,758]:
for bit in tenbits(x):
g.send(bit)
g.send('stop')
f.close()

def test1():
bits = list(charbits('hello'))
L = []
for x in byten(bits):
L.append(int(''.join(map(str,x)),2))
print L

if __name__=='__main__':
test()

[Nick Craig-Wood]

Monica Leko <mo*********@gmail.comwrote:

On May 18, 2:20?pm, Ken Starks <[e-mail]:

You want your file considered as a sequence of bits rather
than a sequence of 8-bit bytes, do you?

Yes.

is the 10-bit bit-pattern to be stored at an arbitrary
bit-position in the file

Yes. I need arbitrary, 8bits, than 10 bits for something else, than
sequence of bytes, than 10 bits again, etc.

You could try

http://construct.wikispaces.com/

which could well do exactly what you want.

--
Nick Craig-Wood <ni**@craig-wood.com-- http://www.craig-wood.com/nick

[castironpi]

On May 20, 10:30*am, Nick Craig-Wood <[e-mail]:

Monica Leko <[e-mail]:

*On May 18, 2:20?pm, Ken Starks <[e-mail]:

You want your file considered as a sequence of bits rather
than a sequence of 8-bit bytes, do you?

*Yes.

is the 10-bit bit-pattern to be stored at an arbitrary
bit-position in the file

*Yes. *I need arbitrary, 8bits, than 10 bits for something else, than
*sequence of bytes, than 10 bits again, etc.

You could try

*http://construct.wikispaces.com/

which could well do exactly what you want.

--
Nick Craig-Wood <[e-mail]--http://www.craig-wood.com/nick

Do you have space to waste? If so, how much? Worst case, you could
be looking at a factor of 32 or higher, if each bit took an entire
system word.

[sjdevnull]

On May 20, 12:14 am, Tim Roberts <[e-mail]:

Monica Leko <[e-mail]:

I have a specific format and I need binary representation. Does
Python have some built-in function which will, for instance, represent
number 15 in exactly 10 bits?

For the record, I'd like to point out that even C cannot do this. You need
to use shifting and masking to produce a stream of 8-bit bytes, or to
extract your values from a stream of 8-bit bytes.

Technically specifying 8-bits isn't quite accurate, as C allows for 9-
bit bytes and other variations depending on the architecture. But
that may be overly pedantic unless you have a PDP-10 laying around
that you're writing C code on or something like that.

[Tim Roberts]

"sj*******@yahoo.com" <sj*******@yahoo.comwrote:

>On May 20, 12:14 am, Tim Roberts <[e-mail]:

>Monica Leko <[e-mail]:

>I have a specific format and I need binary representation. Does
Python have some built-in function which will, for instance, represent
number 15 in exactly 10 bits?

For the record, I'd like to point out that even C cannot do this. You need
to use shifting and masking to produce a stream of 8-bit bytes, or to
extract your values from a stream of 8-bit bytes.

Technically specifying 8-bits isn't quite accurate, as C allows for 9-
bit bytes and other variations depending on the architecture. But
that may be overly pedantic unless you have a PDP-10 laying around
that you're writing C code on or something like that.

As long as we are being pedantic, and I don't mind that, I would point out
that I didn't actually say that C worked in 8-bit bytes. I was very
careful to say merely that, assuming you wanted a stream of 8-bit bytes,
you need to use shifting and masking to produce it.
--
Tim Roberts, ti**@probo.com
Providenza & Boekelheide, Inc.

[Andrew Lee]

Tim Roberts wrote:

Monica Leko <mo*********@gmail.comwrote:

>I have a specific format and I need binary representation. Does
Python have some built-in function which will, for instance, represent
number 15 in exactly 10 bits?

For the record, I'd like to point out that even C cannot do this. You need
to use shifting and masking to produce a stream of 8-bit bytes, or to
extract your values from a stream of 8-bit bytes.

Hmmmm, bitfields are exactly non-aligned bits in a structure and are
part of ANSI C. Although C gives no guarantee of the ordering of fields
within machine words ... so bitfieds are of limited use in portable
programs unless they are 0-initialized.

IIRC, Huffman code uses arbitrary length bit strings and is the basis of
many compression algorithms.