434,741 Members | 2,009 Online + Ask a Question
Need help? Post your question and get tips & solutions from a community of 434,741 IT Pros & Developers. It's quick & easy.

Numarray: Using sum() within functions

 P: n/a Hello- I have a function to generate a multi-dimensional array, which then gets summed over one axis. The problem is that the dimensions are large, and I run out of memory when I create the entire array, so I'm trying to do the sum *within* the function. Example-- variables x,y,z,t; dimensions numX, numY, numZ, numT; functions f1(x,y,z,t), f2(y,z,t); want to calculate f1*f2 and sum over t to get out[x,y,z]. With loops, I could do it like-- out = zeros((numX,numY,numZ)) for x in range(numX): for y in range(numY): for z in range(numZ): for t in range(numT): tempval = f1(x,y,z,t) * f2(y,z,t) out[x,y,z] = out[x,y,z] + tempval With numarray, if I had enough memory, I could just do-- temp1 = fromfunction(f1,(numX,numY,numZ,numT)) temp2 = resize(fromfunction(f2,(numY,numZ,numT)),(numX,num Y,numZ,numT)) out = sum(temp1 * temp2, axis = 3) Instead, I'm trying to do something like-- def f3(x,y,z): for t in range(numT): tempval = f1(x,y,z,t) * f2(y,z,t) outval = sum(tempval,axis = 3) return outval out = fromfunction(f3,(numX,numY,numZ)) I've been trying various slicing and indexing, but I can't seem to get that *extra* dimension within the 3-D function. I've scoured the documentation and list archives, but haven't found exactly what I need. Any suggestions? Am I stuck with generating the entire 4-D array? Thanks in advance, Jim Cser Jul 18 '05 #1
5 Replies

 P: n/a On Sat, Aug 14, 2004 at 02:07:30PM -0700, Jim Cser wrote: Instead, I'm trying to do something like-- def f3(x,y,z): for t in range(numT): tempval = f1(x,y,z,t) * f2(y,z,t) outval = sum(tempval,axis = 3) return outval Here, "tempval" takes on a series of values during the for loop, then "sum" is used on value from the final iteration of the loop. Perhaps you want something like def f3(x, y, z): temps = [] for t in range(numT): temps.append(f1(...) * f(...)) return sum(temps, axis=3) Jeff -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFBH3LzJd01MZaTXX0RAgTnAJwMvn8IgLCswgQXuRnonH Z1DHiObACfZSFv adV/zmpFDyZiy5S+Nuf2Syo= =znkn -----END PGP SIGNATURE----- Jul 18 '05 #2

 P: n/a Jeff Epler wrote: On Sat, Aug 14, 2004 at 02:07:30PM -0700, Jim Cser wrote:Instead, I'm trying to do something like--def f3(x,y,z): for t in range(numT): tempval = f1(x,y,z,t) * f2(y,z,t) outval = sum(tempval,axis = 3) return outval Here, "tempval" takes on a series of values during the for loop, then "sum" is used on value from the final iteration of the loop. Perhaps you want something like def f3(x, y, z): temps = [] for t in range(numT): temps.append(f1(...) * f(...)) return sum(temps, axis=3) Jeff Thanks, that works, although someone gave me one that is faster: def f3(x,y,z): tempval = 0.*x for t in range(numT): tempval += f1(x,y,z,t) * f2(y,z,t) return tempval In either case, unfortunately, looping over t is extremely slow. Ideally, there would be a way to use fromfunction() with slices as arguments. -Jim Jul 18 '05 #3

 P: n/a On Sat, 14 Aug 2004, Jim Cser wrote: I have a function to generate a multi-dimensional array, which then gets summed over one axis. The problem is that the dimensions are large, and I run out of memory when I create the entire array, so I'm trying to do the sum *within* the function. Example-- variables x,y,z,t; dimensions numX, numY, numZ, numT; functions f1(x,y,z,t), f2(y,z,t); want to calculate f1*f2 and sum over t to get out[x,y,z]. Would something like the following work? def f3(x, y, z, t=arange(numT)): return sum(f1(x,y,z,t)*f2(y,z,t)) This assumes that f1() and f2() can operate on array objects. If not, the following (much slower) method should work: def f3(x, y, z): return sum(fromfunction(lambda t: f1(x,y,z,t)*f2(y,z,t),(numT,))) To make the output matrix, you could use either this: out = fromfunction(f3, (numX, numY, numZ)) which will save memory, but will be slow, or (assuming f1 and f2 support arrays): out = f3(arange(numX), arange(numY), arange(numZ)) which will eat lots of memory (more that you say you have, if you use the first f3()), but will be very fast. Hope this helps. Jul 18 '05 #4

 P: n/a Christopher T King wrote: On Sat, 14 Aug 2004, Jim Cser wrote:I have a function to generate a multi-dimensional array, which thengets summed over one axis. The problem is that the dimensionsare large, and I run out of memory when I create the entire array,so I'm trying to do the sum *within* the function.Example-- variables x,y,z,t; dimensions numX, numY, numZ, numT;functions f1(x,y,z,t), f2(y,z,t); want to calculate f1*f2 andsum over t to get out[x,y,z]. Cobbling together a number of suggestions, what finally worked was-- def f3(x, y, z, t_range=arange(numT)): tempval = 0.* x for t in t_range: tempval += f1(x,y,z,t) + f2(y,z,t) return tempval out = fromfunction(f3,(numX,numY,numZ,1)) I couldn't quite get sum() to work inside the function, but this is definitely good enough for now. Thanks to all for your help. -Jim Cser Jul 18 '05 #5

 P: n/a I have a function to generate a multi-dimensional array, which thengets summed over one axis. The problem is that the dimensionsare large, and I run out of memory when I create the entire array,so I'm trying to do the sum *within* the function.Example-- variables x,y,z,t; dimensions numX, numY, numZ, numT;functions f1(x,y,z,t), f2(y,z,t); want to calculate f1*f2 andsum over t to get out[x,y,z]. Cobbling together a number of suggestions, what finally worked was-- def f3(x, y, z, t_range=arange(numT)): tempval = 0.* x for t in t_range: tempval += f1(x,y,z,t) + f2(y,z,t) return tempval out = fromfunction(f3,(numX,numY,numZ,1)) I couldn't quite get sum() to work inside the function, but this is definitely good enough for now. Thanks to all for your help. -Jim Cser [wrong identity on last post, sorry] Jul 18 '05 #6

This discussion thread is closed

Replies have been disabled for this discussion. 