Hello-
I have a function to generate a multi-dimensional array, which then
gets summed over one axis. The problem is that the dimensions
are large, and I run out of memory when I create the entire array,
so I'm trying to do the sum *within* the function.
Example-- variables x,y,z,t; dimensions numX, numY, numZ, numT;
functions f1(x,y,z,t), f2(y,z,t); want to calculate f1*f2 and
sum over t to get out[x,y,z].
With loops, I could do it like--
out = zeros((numX,numY,numZ))
for x in range(numX):
for y in range(numY):
for z in range(numZ):
for t in range(numT):
tempval = f1(x,y,z,t) * f2(y,z,t)
out[x,y,z] = out[x,y,z] + tempval
With numarray, if I had enough memory, I could just do--
temp1 = fromfunction(f1,(numX,numY,numZ,numT))
temp2 = resize(fromfunction(f2,(numY,numZ,numT)),(numX,num Y,numZ,numT))
out = sum(temp1 * temp2, axis = 3)
Instead, I'm trying to do something like--
def f3(x,y,z):
for t in range(numT):
tempval = f1(x,y,z,t) * f2(y,z,t)
outval = sum(tempval,axis = 3)
return outval
out = fromfunction(f3,(numX,numY,numZ))
I've been trying various slicing and indexing, but I can't seem to
get that *extra* dimension within the 3-D function. I've scoured the
documentation and list archives, but haven't found exactly what I need.
Any suggestions? Am I stuck with generating the entire 4-D array?
Thanks in advance,
Jim Cser 5 1688
On Sat, Aug 14, 2004 at 02:07:30PM -0700, Jim Cser wrote: Instead, I'm trying to do something like-- def f3(x,y,z): for t in range(numT): tempval = f1(x,y,z,t) * f2(y,z,t) outval = sum(tempval,axis = 3) return outval
Here, "tempval" takes on a series of values during the for loop, then
"sum" is used on value from the final iteration of the loop.
Perhaps you want something like
def f3(x, y, z):
temps = []
for t in range(numT):
temps.append(f1(...) * f(...))
return sum(temps, axis=3)
Jeff
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
iD8DBQFBH3LzJd01MZaTXX0RAgTnAJwMvn8IgLCswgQXuRnonH Z1DHiObACfZSFv
adV/zmpFDyZiy5S+Nuf2Syo=
=znkn
-----END PGP SIGNATURE-----
Jeff Epler wrote: On Sat, Aug 14, 2004 at 02:07:30PM -0700, Jim Cser wrote:
Instead, I'm trying to do something like-- def f3(x,y,z): for t in range(numT): tempval = f1(x,y,z,t) * f2(y,z,t)
outval = sum(tempval,axis = 3) return outval
Here, "tempval" takes on a series of values during the for loop, then "sum" is used on value from the final iteration of the loop.
Perhaps you want something like def f3(x, y, z): temps = [] for t in range(numT): temps.append(f1(...) * f(...)) return sum(temps, axis=3)
Jeff
Thanks, that works, although someone gave me one that is faster:
def f3(x,y,z):
tempval = 0.*x
for t in range(numT):
tempval += f1(x,y,z,t) * f2(y,z,t)
return tempval
In either case, unfortunately, looping over t is extremely slow.
Ideally, there would be a way to use fromfunction() with slices as
arguments.
-Jim
On Sat, 14 Aug 2004, Jim Cser wrote: I have a function to generate a multi-dimensional array, which then gets summed over one axis. The problem is that the dimensions are large, and I run out of memory when I create the entire array, so I'm trying to do the sum *within* the function.
Example-- variables x,y,z,t; dimensions numX, numY, numZ, numT; functions f1(x,y,z,t), f2(y,z,t); want to calculate f1*f2 and sum over t to get out[x,y,z].
Would something like the following work?
def f3(x, y, z, t=arange(numT)):
return sum(f1(x,y,z,t)*f2(y,z,t))
This assumes that f1() and f2() can operate on array objects. If not, the
following (much slower) method should work:
def f3(x, y, z):
return sum(fromfunction(lambda t: f1(x,y,z,t)*f2(y,z,t),(numT,)))
To make the output matrix, you could use either this:
out = fromfunction(f3, (numX, numY, numZ))
which will save memory, but will be slow, or (assuming f1 and f2 support
arrays):
out = f3(arange(numX), arange(numY), arange(numZ))
which will eat lots of memory (more that you say you have, if you use the
first f3()), but will be very fast.
Hope this helps.
Christopher T King wrote: On Sat, 14 Aug 2004, Jim Cser wrote:
I have a function to generate a multi-dimensional array, which then gets summed over one axis. The problem is that the dimensions are large, and I run out of memory when I create the entire array, so I'm trying to do the sum *within* the function.
Example-- variables x,y,z,t; dimensions numX, numY, numZ, numT; functions f1(x,y,z,t), f2(y,z,t); want to calculate f1*f2 and sum over t to get out[x,y,z].
Cobbling together a number of suggestions, what finally worked was--
def f3(x, y, z, t_range=arange(numT)):
tempval = 0.* x
for t in t_range:
tempval += f1(x,y,z,t) + f2(y,z,t)
return tempval
out = fromfunction(f3,(numX,numY,numZ,1))
I couldn't quite get sum() to work inside the function, but this
is definitely good enough for now. Thanks to all for your help.
-Jim Cser I have a function to generate a multi-dimensional array, which then gets summed over one axis. The problem is that the dimensions are large, and I run out of memory when I create the entire array, so I'm trying to do the sum *within* the function.
Example-- variables x,y,z,t; dimensions numX, numY, numZ, numT; functions f1(x,y,z,t), f2(y,z,t); want to calculate f1*f2 and sum over t to get out[x,y,z].
Cobbling together a number of suggestions, what finally worked was--
def f3(x, y, z, t_range=arange(numT)):
tempval = 0.* x
for t in t_range:
tempval += f1(x,y,z,t) + f2(y,z,t)
return tempval
out = fromfunction(f3,(numX,numY,numZ,1))
I couldn't quite get sum() to work inside the function, but this
is definitely good enough for now. Thanks to all for your help.
-Jim Cser
[wrong identity on last post, sorry] This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Colin J. Williams |
last post by:
numarray is a package which is under development and intended to replace
Numeric, an efficient and operational package.
One of the classes in numarray is NumArray. As currently implemented,...
|
by: Alexander Schwaigkofler |
last post by:
Hi!
I have the following problem with numarray. I read the install.txt manual,
but it doesn't already work.
OS: Microsoft Windows 2000
python: Python 2.2.3 (#42, May 30 2003, 18:12:08) on...
|
by: Dan Williams |
last post by:
Hi people
I'm getting a little annoyed with the way the print function always adds a
space character between print statements unless there has been a new line.
The manual mentions that "In some...
|
by: Marc Schellens |
last post by:
Following the NumPy documentation, I took over some C code,
but run into an error.
Does anybody have a suggestion?
Thanks,
marc
gdlpython.cpp:225: `PyArray_Type' undeclared (first use this...
|
by: Matt Feinstein |
last post by:
Is there an optimal way to apply a function to the elements of a two-d
array?
What I'd like to do is define some function:
def plone(x):
return x+1
and then apply it elementwise to a 2-D...
|
by: Matt Feinstein |
last post by:
I spent all day yesterday trying to figure out how to do file IO in
the numarray module-- I -did- (I think) figure it all out, eventually,
but it's left me in a rather sour mood.
1. The basic...
|
by: meng |
last post by:
Hi, there,
I got different results by running the same lines of code on windows and
debian. Here is the code:
a = kroneckerproduct(ones((4195,1)), identity(12))
print a.mean()
This works...
|
by: Raphaël MARC |
last post by:
Hello,
Can anyone tell me how to open an image
and transform it into a list so that
the functions of the multi dimensionnal module of
numarray (numarray.nd image) can process it ?
Do I have...
|
by: andrewfelch |
last post by:
Below is the code to/from Boolean arrays and Unsigned integers. On my
Pentium 4, functions such as "bitwise_and" are 32 times faster when run
on 32-bit integers instead of the...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
| |