Write 10k integers to a file using struct

[Thomas]

What's the quickest way to write and read 10.000 integer values ( or
more ) to and from a file? Using struct somehow? The example in the
docs shows how to handle to or three arguments, but is the fastest way
to write and read just a simple :

while data:
f.write(struct.pack(format, integer-value))

or is there another, quicker and nicer way to do it????

Thomas

[John Hunter]

>>>>> "Thomas" == Thomas <20**@weholt.org> writes:

Thomas> What's the quickest way to write and read 10.000 integer
Thomas> values ( or more ) to and from a file? Using struct
Thomas> somehow? The example in the docs shows how to handle to or
Thomas> three arguments, but is the fastest way to write and read
Thomas> just a simple :

Thomas> while data: f.write(struct.pack(format, integer-value))

Thomas> or is there another, quicker and nicer way to do it????

When handling arrays of numbers of that size, you may want to think
about using numeric/numarray. In the example below, I use randint to
simply to create a list of random integers for demo purposes.

1 >>> import random

2 >>> nums = [random.randint(0,100000) for i in range(10000)]

3 >>> from numarray import array, fromfile

4 >>> a = array(nums)

5 >>> a.typecode
----> a.typecode()
Out[5]: 'l'

6 >>> a.tofile?
Type: instancemethod
Base Class: <type 'instancemethod'>
String Form: <bound method NumArray.tofile of array([ 5193, 43059, 91249, ..., 13735, 38070, 75129])>
Namespace: Interactive
File: /usr/local/lib/python2.3/site-packages/numarray/generic.py
Definition: a.tofile(self, file)
Docstring:
Write the array as a binary image to a file.

If file is a string, it attempts to open a file with that name,
otherwise it assumes file is a file object. At the moment if
special positioning is needed in the file one must do that with
the file object beforehand. More options may be added to this
method to allow positioning or appends.

Note that for numerical data, the system byte order in which
the data is represented is *not* recorded in the file. This
renders the file non-portable because extra information is
required to interpret it on different machines than the one it
was created on.
7 >>> a.tofile('mynums.dat')

8 >>> fromfile?
Type: function
Base Class: <type 'function'>
String Form: <function fromfile at 0xb4b2c614>
Namespace: Interactive
File: /usr/local/lib/python2.3/site-packages/numarray/numarraycore.py
Definition: fromfile(file, type, shape=None)
Docstring:
Create an array from binary file data

If file is a string then that file is opened, else it is assumed
to be a file object. No options at the moment, all file positioning
must be done prior to this function call with a file object
9 >>> b = fromfile('mynums.dat', 'l')

10 >>> b.shape
Out[10]: (10000,)

11 >>> a is b
Out[11]: False

12 >>> a==b
Out[12]: array([1, 1, 1, ..., 1, 1, 1], type=Bool)

[Josiah Carlson]

Thomas <20**@weholt.org> wrote:

What's the quickest way to write and read 10.000 integer values ( or
more ) to and from a file? Using struct somehow? The example in the
docs shows how to handle to or three arguments, but is the fastest way
to write and read just a simple :

while data:
f.write(struct.pack(format, integer-value))

or is there another, quicker and nicer way to do it????

Look at the array module.

- Josiah