P: n/a

Hello
I have a function to generate a multidimensional array, which then
gets summed over one axis. The problem is that the dimensions
are large, and I run out of memory when I create the entire array,
so I'm trying to do the sum *within* the function.
Example variables x,y,z,t; dimensions numX, numY, numZ, numT;
functions f1(x,y,z,t), f2(y,z,t); want to calculate f1*f2 and
sum over t to get out[x,y,z].
With loops, I could do it like
out = zeros((numX,numY,numZ))
for x in range(numX):
for y in range(numY):
for z in range(numZ):
for t in range(numT):
tempval = f1(x,y,z,t) * f2(y,z,t)
out[x,y,z] = out[x,y,z] + tempval
With numarray, if I had enough memory, I could just do
temp1 = fromfunction(f1,(numX,numY,numZ,numT))
temp2 = resize(fromfunction(f2,(numY,numZ,numT)),(numX,num Y,numZ,numT))
out = sum(temp1 * temp2, axis = 3)
Instead, I'm trying to do something like
def f3(x,y,z):
for t in range(numT):
tempval = f1(x,y,z,t) * f2(y,z,t)
outval = sum(tempval,axis = 3)
return outval
out = fromfunction(f3,(numX,numY,numZ))
I've been trying various slicing and indexing, but I can't seem to
get that *extra* dimension within the 3D function. I've scoured the
documentation and list archives, but haven't found exactly what I need.
Any suggestions? Am I stuck with generating the entire 4D array?
Thanks in advance,
Jim Cser  
Share this Question
P: n/a

On Sat, Aug 14, 2004 at 02:07:30PM 0700, Jim Cser wrote: Instead, I'm trying to do something like def f3(x,y,z): for t in range(numT): tempval = f1(x,y,z,t) * f2(y,z,t) outval = sum(tempval,axis = 3) return outval
Here, "tempval" takes on a series of values during the for loop, then
"sum" is used on value from the final iteration of the loop.
Perhaps you want something like
def f3(x, y, z):
temps = []
for t in range(numT):
temps.append(f1(...) * f(...))
return sum(temps, axis=3)
Jeff
BEGIN PGP SIGNATURE
Version: GnuPG v1.2.4 (GNU/Linux)
iD8DBQFBH3LzJd01MZaTXX0RAgTnAJwMvn8IgLCswgQXuRnonH Z1DHiObACfZSFv
adV/zmpFDyZiy5S+Nuf2Syo=
=znkn
END PGP SIGNATURE  
P: n/a

Jeff Epler wrote: On Sat, Aug 14, 2004 at 02:07:30PM 0700, Jim Cser wrote:
Instead, I'm trying to do something like def f3(x,y,z): for t in range(numT): tempval = f1(x,y,z,t) * f2(y,z,t)
outval = sum(tempval,axis = 3) return outval
Here, "tempval" takes on a series of values during the for loop, then "sum" is used on value from the final iteration of the loop.
Perhaps you want something like def f3(x, y, z): temps = [] for t in range(numT): temps.append(f1(...) * f(...)) return sum(temps, axis=3)
Jeff
Thanks, that works, although someone gave me one that is faster:
def f3(x,y,z):
tempval = 0.*x
for t in range(numT):
tempval += f1(x,y,z,t) * f2(y,z,t)
return tempval
In either case, unfortunately, looping over t is extremely slow.
Ideally, there would be a way to use fromfunction() with slices as
arguments.
Jim  
P: n/a

On Sat, 14 Aug 2004, Jim Cser wrote: I have a function to generate a multidimensional array, which then gets summed over one axis. The problem is that the dimensions are large, and I run out of memory when I create the entire array, so I'm trying to do the sum *within* the function.
Example variables x,y,z,t; dimensions numX, numY, numZ, numT; functions f1(x,y,z,t), f2(y,z,t); want to calculate f1*f2 and sum over t to get out[x,y,z].
Would something like the following work?
def f3(x, y, z, t=arange(numT)):
return sum(f1(x,y,z,t)*f2(y,z,t))
This assumes that f1() and f2() can operate on array objects. If not, the
following (much slower) method should work:
def f3(x, y, z):
return sum(fromfunction(lambda t: f1(x,y,z,t)*f2(y,z,t),(numT,)))
To make the output matrix, you could use either this:
out = fromfunction(f3, (numX, numY, numZ))
which will save memory, but will be slow, or (assuming f1 and f2 support
arrays):
out = f3(arange(numX), arange(numY), arange(numZ))
which will eat lots of memory (more that you say you have, if you use the
first f3()), but will be very fast.
Hope this helps.  
P: n/a

Christopher T King wrote: On Sat, 14 Aug 2004, Jim Cser wrote:
I have a function to generate a multidimensional array, which then gets summed over one axis. The problem is that the dimensions are large, and I run out of memory when I create the entire array, so I'm trying to do the sum *within* the function.
Example variables x,y,z,t; dimensions numX, numY, numZ, numT; functions f1(x,y,z,t), f2(y,z,t); want to calculate f1*f2 and sum over t to get out[x,y,z].
Cobbling together a number of suggestions, what finally worked was
def f3(x, y, z, t_range=arange(numT)):
tempval = 0.* x
for t in t_range:
tempval += f1(x,y,z,t) + f2(y,z,t)
return tempval
out = fromfunction(f3,(numX,numY,numZ,1))
I couldn't quite get sum() to work inside the function, but this
is definitely good enough for now. Thanks to all for your help.
Jim Cser  
P: n/a
 I have a function to generate a multidimensional array, which then gets summed over one axis. The problem is that the dimensions are large, and I run out of memory when I create the entire array, so I'm trying to do the sum *within* the function.
Example variables x,y,z,t; dimensions numX, numY, numZ, numT; functions f1(x,y,z,t), f2(y,z,t); want to calculate f1*f2 and sum over t to get out[x,y,z].
Cobbling together a number of suggestions, what finally worked was
def f3(x, y, z, t_range=arange(numT)):
tempval = 0.* x
for t in t_range:
tempval += f1(x,y,z,t) + f2(y,z,t)
return tempval
out = fromfunction(f3,(numX,numY,numZ,1))
I couldn't quite get sum() to work inside the function, but this
is definitely good enough for now. Thanks to all for your help.
Jim Cser
[wrong identity on last post, sorry]   This discussion thread is closed Replies have been disabled for this discussion.   Question stats  viewed: 1462
 replies: 5
 date asked: Jul 18 '05
