Is there an optimal way to apply a function to the elements of a two-d
array?
What I'd like to do is define some function:
def plone(x):
return x+1
and then apply it elementwise to a 2-D numarray. I intend to treat the
function as a variable, so ufuncs are probably not appropriate-- I
realize that what I'm looking for won't be terrifically efficient,
but I'd like to avoid doing it in the -worst- possible way.
Some things I've looked at include things like
def applyfun(m,f):
elist = [f(e) for e in m]
return reshape(elist,m.shape)
however, I can see that this looks neat but probably generates several
copies of the array, which is not so neat.
Is there a better way?
Matt Feinstein
--
There is no virtue in believing something that can be proved to be true. 6 1467
Matt Feinstein wrote: Is there an optimal way to apply a function to the elements of a two-d array?
What I'd like to do is define some function:
def plone(x): return x+1
and then apply it elementwise to a 2-D numarray. I intend to treat the function as a variable, so ufuncs are probably not appropriate-- I realize that what I'm looking for won't be terrifically efficient, but I'd like to avoid doing it in the -worst- possible way.
Some things I've looked at include things like
def applyfun(m,f): elist = [f(e) for e in m] return reshape(elist,m.shape)
however, I can see that this looks neat but probably generates several copies of the array, which is not so neat.
Is there a better way?
I must be missing something, because the simplest possible thing seems
to work for me:
py> import numarray as na
py> def plus1(arr):
.... return arr + 1
....
py> def apply_func(arr, f):
.... return f(arr)
....
py> a = na.arange(20, shape=(4, 5))
py> a
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]])
py> apply_func(a, plus1)
array([[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20]])
Is this not what you wanted?
STeVe
On Mon, 16 May 2005 11:07:06 -0600, Steven Bethard
<st************@gmail.com> wrote: I must be missing something, because the simplest possible thing seems to work for me:
py> import numarray as na py> def plus1(arr): ... return arr + 1 ... py> def apply_func(arr, f): ... return f(arr) ... py> a = na.arange(20, shape=(4, 5)) py> a array([[ 0, 1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, 13, 14], [15, 16, 17, 18, 19]]) py> apply_func(a, plus1) array([[ 1, 2, 3, 4, 5], [ 6, 7, 8, 9, 10], [11, 12, 13, 14, 15], [16, 17, 18, 19, 20]])
Is this not what you wanted?
The problem is that I chose an example function that's too simple.
Non-trivial functions aren't so polymorphic, unfortunately.
Sorry for the confusion.
Matt Feinstein
--
There is no virtue in believing something that can be proved to be true.
Matt Feinstein wrote: On Mon, 16 May 2005 11:07:06 -0600, Steven Bethard <st************@gmail.com> wrote: I must be missing something, because the simplest possible thing seems to work for me:
py> import numarray as na py> def plus1(arr): ... return arr + 1 ... py> def apply_func(arr, f): ... return f(arr) ... py> a = na.arange(20, shape=(4, 5)) py> a array([[ 0, 1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, 13, 14], [15, 16, 17, 18, 19]]) py> apply_func(a, plus1) array([[ 1, 2, 3, 4, 5], [ 6, 7, 8, 9, 10], [11, 12, 13, 14, 15], [16, 17, 18, 19, 20]])
Is this not what you wanted?
The problem is that I chose an example function that's too simple. Non-trivial functions aren't so polymorphic, unfortunately.
Can you give an example of what you really want to do? Probably there
are numarray functions that you can use. In general, you'll do better
applying a sequence of numarray functions than operating element-wise on
an array and converting it from a list back to an array...
STeVe
On Mon, 16 May 2005 12:03:24 -0600, Steven Bethard
<st************@gmail.com> wrote: Can you give an example of what you really want to do? Probably there are numarray functions that you can use. In general, you'll do better applying a sequence of numarray functions than operating element-wise on an array and converting it from a list back to an array...
Well, for example, suppose I want to modify the elements of the matrix
in some fashion. However, I'm not entirely sure how I want to do it.
As a strawman, I generate a function with a Boolean test in it that
multiplies by one factor if the matrix element is in an interval and
by a different factor if it is outside the interval
def genfunc(xmin, xmax, f_in, f_out):
def booltest(x):
in_interval = x > xmin and x < xmax
if in_interval:
return x*f_in
else:
return x*f_out
return booltest
Generating the function in this way gives me great flexibility in
deciding exactly what function I apply to the matrix. It's why I want
to use Python for this analysis. The part of the function I vary and
play around with is localized in one place in the 'genfunc' function--
I can change that and everything else stays the same. However, I
realize that the gain in flexibility means a loss in efficiency. I'm
limited to not-so-efficient ways of. For this work, it's OK-- I just
want to know the best not-so-efficient way of doing the calculation.
Matt Feinstein
--
There is no virtue in believing something that can be proved to be true.
Matt Feinstein wrote: Well, for example, suppose I want to modify the elements of the matrix in some fashion. However, I'm not entirely sure how I want to do it. As a strawman, I generate a function with a Boolean test in it that multiplies by one factor if the matrix element is in an interval and by a different factor if it is outside the interval
def genfunc(xmin, xmax, f_in, f_out): def booltest(x): in_interval = x > xmin and x < xmax if in_interval: return x*f_in else: return x*f_out return booltest
If efficiency was a concern, I'd probably write this instead as:
py> import numarray as na
py> def genfunc(xmin, xmax, f_in, f_out):
.... def booltest(arr):
.... is_in = na.logical_and(arr > xmin, arr < xmax)
.... is_out = na.logical_not(is_in)
.... return arr*is_in*f_in + arr*is_out*f_out
.... return booltest
....
py> b = genfunc(5, 11, 2, -2)
py> arr = na.arange(20, shape=(4,5))
py> b(arr)
array([[ 0, -2, -4, -6, -8],
[-10, 12, 14, 16, 18],
[ 20, -22, -24, -26, -28],
[-30, -32, -34, -36, -38]])
Sure, it's a little more complex that your version, and you have to
understand that you're manipulating arrays, not elements of arrays, but
if you want the efficiency of numarray or Numeric, something like this
is probably the way to go.
Generating the function in this way gives me great flexibility in deciding exactly what function I apply to the matrix. It's why I want to use Python for this analysis. The part of the function I vary and play around with is localized in one place in the 'genfunc' function-- I can change that and everything else stays the same. However, I realize that the gain in flexibility means a loss in efficiency. I'm limited to not-so-efficient ways of. For this work, it's OK-- I just want to know the best not-so-efficient way of doing the calculation.
If you're not worried about efficiency, your initial suggestion:
def applyfun(m,f):
elist = [f(e) for e in m]
return reshape(elist,m.shape)
seems pretty reasonable. OTOH, if you're using numarray or Numeric in
the first place, you're obviously somewhat concerned about efficiency.
STeVe
Matt Feinstein wrote: On Mon, 16 May 2005 12:03:24 -0600, Steven Bethard <st************@gmail.com> wrote:
Can you give an example of what you really want to do? Probably there are numarray functions that you can use. In general, you'll do better applying a sequence of numarray functions than operating element-wise on an array and converting it from a list back to an array...
Well, for example, suppose I want to modify the elements of the matrix in some fashion. However, I'm not entirely sure how I want to do it. As a strawman, I generate a function with a Boolean test in it that multiplies by one factor if the matrix element is in an interval and by a different factor if it is outside the interval
As Steven Bethard suggests, using the ufuncs provided to express the
function in a vectorial way is always the best option *if* it's possible.
Otherwise, you may want to look at Scipy's vectorize() function. I don't
think the numarray port is quite working yet, so you may have to use
Numeric. You will still have the same function call overhead since you
will be calling the Python function for each element, but the loops will
be in C.
Type: classobj
String Form: scipy_base.function_base.vectorize
Namespace: Interactive
File:
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/scipy_base/function_base.py
Docstring:
vectorize(somefunction) Generalized Function class.
Description:
Define a vectorized function which takes nested sequence objects or
numerix arrays as inputs and returns a numerix array as output,
evaluating the function over successive tuples of the input arrays like
the python map function except it uses the broadcasting rules of numerix
Python.
Input:
somefunction -- a Python function or method
Example:
def myfunc(a,b):
if a > b:
return a-b
else:
return a+b
vfunc = vectorize(myfunc) vfunc([1,2,3,4],2)
array([3,4,1,2])
--
Robert Kern rk***@ucsd.edu
"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: RJS |
last post by:
Hi all,
I can't get a py2exe compiled app to run with numarray (numarray-0.5.win32-
py2.2).
Also wxPythonWIN32-2.3.3.1-Py22 and ActivePython-2.2.1-222.
In the sample below, commenting out...
|
by: Alexander Schwaigkofler |
last post by:
Hi!
I have the following problem with numarray. I read the install.txt manual,
but it doesn't already work.
OS: Microsoft Windows 2000
python: Python 2.2.3 (#42, May 30 2003, 18:12:08) on...
|
by: andrewfelch |
last post by:
Below is the code to/from Boolean arrays and Unsigned integers. On my
Pentium 4, functions such as "bitwise_and" are 32 times faster when run
on 32-bit integers instead of the...
|
by: robert |
last post by:
just a note - some speed comparisons :
0.60627370238398726
0.42836673376223189
0.36965815487747022
0.016557970357098384
0.15692469294117473
0.01951756438393204
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new...
| |