binary input and memory address passing

[Eric Carlson]

Hello,

I can open, read, and convert data to a numeric (double) array from a
binary file using

nc = #something given
nr = #something given
f_o=open('junk.bin','rb')
x=reshape(array('d',f_o.read()),(nr,nc))

Is there a way in python that gives better performance? These commands
take three to 4 times longer than loading the data into an array in (for
example) octave.

Also, is there a way to pass the address of a block of memory, and then
access the data located there (pass by reference and let the python
program know what data type to use)?

My situation: I have two programs that can not communicate directly, but
I can get memory addresses for any data in my programs. I currently save
to a file, then load the data from file into my other program. Clearly
it would be much faster if I could access the block of memory directly
rather than making a copy.

Any suggestions greatly appreciated.

Cheers,

Eric Carlson

[Scott David Daniels]

Eric Carlson wrote:

Hello,

I can open, read, and convert data to a numeric (double) array from a
binary file using

nc = #something given
nr = #something given
f_o=open('junk.bin','rb')
x=reshape(array('d',f_o.read()),(nr,nc))

Is there a way in python that gives better performance? These commands
take three to 4 times longer than loading the data into an array in (for
example) octave.

Also, is there a way to pass the address of a block of memory, and then
access the data located there (pass by reference and let the python
program know what data type to use)?

My situation: I have two programs that can not communicate directly, but
I can get memory addresses for any data in my programs. I currently save
to a file, then load the data from file into my other program. Clearly
it would be much faster if I could access the block of memory directly
rather than making a copy.

Any suggestions greatly appreciated.

You might look over my (admittedly old)
http://members.dsl-only.net/~daniels/Block.html
It only has a Python 2.2 and 2.3 installer, but the sources are there
for you to compile (in which case you might toss a package back to me,
and I'll put it up). If you look it over and have questions, let me
know.

--Scott David Daniels
sc***********@acm.org

[Eric Carlson]

Thanks Dennis and Scott for both responses. Since Dennis has slam dunked
my notion of sharing memory addresses (my ignorance in computing is
pretty much unbounded), I guess I will need to continue on with sharing
through files.

Opening up and loading the binary info into a string variable is very
fast. The process bogs when changing the representation to float. So it
appears that Scott's Block module might do what I need.

My original:

nc = #something given
nr = #something given
f_o=open('junk.bin','rb')
x=reshape(array('d',f_o.read()),(nr,nc))

gives me any array with nr rows and nc columns. Using Block I think I
would use:

f_o=open('junk.bin','rb')
x = View('d', Block(f_o.read()))

Okay, so now how can I use this? Is this like an array type so that

x = reshape(x,(nr,nc))

makes sense?

Regards,
Eric