473,321 Members | 1,916 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,321 software developers and data experts.

Saving Numeric arrays to file?

I am having problems saving arrays to file. e.g.
t1 array([30]) t2 array([ 0, 0, 5, 5, 5, 10]) t3 array([20, 5]) t4 array([[ 0, 5, 10, 5, 0, 10],
[ 0, 0, 0, 5, 0, 0]]) f = file('c:\Data\IS_rounded.txt', 'w')
f.write(t1.tostring() + '\n')
f.write(t2.tostring() + '\n')
f.write(t3.tostring() + '\n')
f.write(t4.tostring() + '\n')
f.close()

f = file('c:\Data\IS_rounded.txt', 'r')
Numeric.fromstring(f.readline().strip()) array([30]) Numeric.fromstring(f.readline().strip()) array([0, 0, 5, 5, 5]) # missing
final element Numeric.fromstring(f.readline().strip())
Traceback (most recent call last):
File "<pyshell#181>", line 1, in -toplevel-
Numeric.fromstring(f.readline().strip())
ValueError: string size must be a multiple of element size


Anyone any idea where I'm going wrong? 2.3 on Win2k. Cheers.

Duncan
Jul 18 '05 #1
6 2715
Of course, the problem is reasonably obvious (doh!). But I still need a
workaround, so I can (ideally) save an array as a single line of text.

Duncan
Jul 18 '05 #2
Duncan Smith asks about writing Numeric arrays to disk:
[ 0, 0, 0, 5, 0, 0]])
f = file('c:\Data\IS_rounded.txt', 'w')


Perhaps with 'wb' ?

Emile van Sebille
em***@fenx.com
Jul 18 '05 #3

"Emile van Sebille" <em***@fenx.com> wrote in message
news:bl************@ID-11957.news.uni-berlin.de...
Duncan Smith asks about writing Numeric arrays to disk:
[ 0, 0, 0, 5, 0, 0]])
>> f = file('c:\Data\IS_rounded.txt', 'w')


Perhaps with 'wb' ?

Emile van Sebille
em***@fenx.com


It doesn't work for me. Looks like I'll maybe have to avoid \n altogether,
use spaces to separate the strings and read characters individually.
Basically I have a few hundred thousand sets of 4 tables, each associated
with a unique key. The idea was to store them as a text file so that I
could quickly fire them into a dictionary (memory permitting?) when I need
to use them. Maybe I should be using MySQL for persistence? What I *would*
like is a reasonably generic (and not terribly inefficient) solution for
storing strings that contain escape characters in text files. Cheers.

Duncan
Jul 18 '05 #4
Duncan Smith wrote:
...
What I
*would* like is a reasonably generic (and not terribly inefficient)
solution for
storing strings that contain escape characters in text files. Cheers.


I'm not quite sure what you mean by "contain escape characters" in this
context. But maybe the quoted-printable coding is what you're looking
for? See module quopri in the standard Python library.
Alex

Jul 18 '05 #5

"Alex Martelli" <al***@aleax.it> wrote in message
news:GL**********************@news2.tin.it...
Duncan Smith wrote:
...
What I
*would* like is a reasonably generic (and not terribly inefficient)
solution for
storing strings that contain escape characters in text files. Cheers.


I'm not quite sure what you mean by "contain escape characters" in this
context. But maybe the quoted-printable coding is what you're looking
for? See module quopri in the standard Python library.
Alex


Thanks Alex,
I was writing to file strings such as
'\n\x00\x00\x00\x00\x00\x00\x00' and then trying to retrieve them via
readline(). It turns out that querying a MySQL table is easily quick enough
for my purposes (the bottlenecks are elsewhere). But I'll certainly check
out quopri (seeing as I wasn't aware of it). Thanks again.

Duncan
Jul 18 '05 #6
Duncan Smith wrote:
I am having problems saving arrays to file. e.g. [snipped]


Maybe you'd better use pickle or cPickle modules?

anton.

Jul 18 '05 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Jacek Generowicz | last post by:
The Numeric docs state that "Subclassing Numeric arrays is not possible due to a limitation of Python." What is this limitation? My first guess is that it is the unsbclassability of built-in...
0
by: Travis Oliphant | last post by:
Numarray is making great progress and is quite usable for many purposes. An idea that was championed by some is that the Numeric code base would stay static and be replaced entirely by Numarray. ...
9
by: drife | last post by:
Hello, Could someone please provide instructions for install Numeric with ATLAS and LAPACK? I've actually done this correctly, I think. But I don't see any difference in the speed. I'm...
5
by: George Sakkis | last post by:
What's the fastest and most elegant equivalent of zip() in Numeric/Numarray between two equally sized 1D arrays ? That is, how to combine two (N,)-shaped arrays to one (N,2) (or (2,N)) shaped ? I...
5
by: bandw | last post by:
I am having a problem using Numeric-24.0b2 in conjunction with the NetCDF module from ScientificPython (version 2.4.9). This problem does not surface using Numeric-23.8. The problem arises in...
3
by: daniel.neilson | last post by:
I have two one-dimensional Numeric arrays, and I need to know the indices in the second array of elements from the first. so if i had: a=array() b=array() i want a function match that does...
1
by: Bill Maher | last post by:
I have an array that I use to write to the file on the C:\ drive. I want to print the file. How can I do it. Here is my code for the write to the file: I'm using "writeline". To allow for...
7
by: Sheldon | last post by:
Hi, I have the following loop that I think can be written to run faster in Numeric. I am currently using Numeric. range_va = main.xsize= 600 main.ysize= 600 #msgva is an (600x600) Numeric...
6
by: Sheldon | last post by:
Hi, I am trying to build a large array using concatenate function in python. So as I loop over the number of arrays, of which there are 12 (4 down and 3 across), I create 3 long arrays by...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.