By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
455,532 Members | 1,374 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 455,532 IT Pros & Developers. It's quick & easy.

Easier way to save result of a function?

P: n/a
Sorry for the clunky subject line - I have a feeling that not knowing
the proper terms for this is part of my problem.

I'm trying to write a class that analyses some data. I only want it to
do as much work as necessary, so it saves method results to a
dictionary, like so:
class MyClass:
def __init__(self):
self.data = {}

def doSomething(self):
return self.data.setdefault("someresult", self._doSomething())

def _doSomething(self):
# heavy processing here
result = 1
return result

def doMore(self):
return self.data.setdefault("moreresult", self._doMore())

def _doMore(self):
# more heavy processing here
return self.doSomething() + 1
This also seems kind of clunky. Is there a better way to acheive this?
Or is this the usual idiom for this kind of thing? I've looked at
decorators as they seem like they could be used for this, but I've no
idea how exactly, or even if they're the appropriate tool.

Thanks.

--
James Mitchelhill
ja***@disorderfeed.net
http://disorderfeed.net
Jul 4 '06 #1
Share this Question
Share on Google+
5 Replies


P: n/a
James Mitchelhill wrote:
Sorry for the clunky subject line - I have a feeling that not knowing
the proper terms for this is part of my problem.

I'm trying to write a class that analyses some data. I only want it to
do as much work as necessary, so it saves method results to a
dictionary, like so:
The term for this is 'memoization'. You can find several recipes for
this in the online Python Cookbook; for example, see
http://aspn.activestate.com/ASPN/Coo.../Recipe/466320

Jul 4 '06 #2

P: n/a
James Mitchelhill <ja***@disorderfeed.netwrote:
Sorry for the clunky subject line - I have a feeling that not knowing
the proper terms for this is part of my problem.

I'm trying to write a class that analyses some data. I only want it to
do as much work as necessary, so it saves method results to a
dictionary, like so:
class MyClass:
def __init__(self):
self.data = {}

def doSomething(self):
return self.data.setdefault("someresult", self._doSomething())

def _doSomething(self):
# heavy processing here
result = 1
return result

def doMore(self):
return self.data.setdefault("moreresult", self._doMore())

def _doMore(self):
# more heavy processing here
return self.doSomething() + 1
This also seems kind of clunky. Is there a better way to acheive this?
Or is this the usual idiom for this kind of thing? I've looked at
decorators as they seem like they could be used for this, but I've no
idea how exactly, or even if they're the appropriate tool.
No, this code is calling the heavy-processing functions
*unconditionally* -- Python's semantics is to compute every argument of
a function or method before starting to execute the code of that
function or method (almost all languages behave that way, save a very
few "pure" functional languages!), and that applies to the setdefault
methods of dictionaries too.

What you want to do is called "memoization" (sometimes "caching", but
that's a very generic term used in a bazillion different situations and
you may get overwhelmed by search results if you try searching for
it!-), and you can find plenty of recipes for it in the Python Cookbook
and elsewhere on the net.

For your needs, I would suggest first defining your class "normally"
(writing the methods without memoization) and then looping on all
methods of interest decorating them with automatic memoization; I just
suggested a similar kind of looping for decoration purposes in my very
latest post on another thread;-). You won't use the _syntax_ of
decorators, but, hey, who cares?-)

Assuming (for simplicity but with no real loss of conceptual generality)
that all of your heavy-processing methods have names starting with 'do',
and that each of them takes only 'self' as its argument, you could for
example do something like:

def memoizeMethod(cls, n, m):
def decorated(self):
if n in self._memo: return self._memo[n]
result = self._memo[n] = m(self)
return result
decorated.__name__ = n
setattr(cls, n, decorated)

def memoizeClass(cls):
cls._memo = {}
for n,m in inspect.getmembers(cls, inspect.ismethoddescriptors):
if not n.startswith('do'): continue
memoizeMethod(cls, n, m)

Now just write your MyClass with the doThis doThat methods you want, and
right after the class statement add

memoizeClass(MyClass)
Voila, that's it (untested details, but the general idea is sound;-).
Alex
Jul 4 '06 #3

P: n/a
On Tue, 4 Jul 2006 16:53:27 -0700, Alex Martelli wrote:
James Mitchelhill <ja***@disorderfeed.netwrote:
<snip>
>I'm trying to write a class that analyses some data. I only want it to
do as much work as necessary, so it saves method results to a
dictionary
<snip>
For your needs, I would suggest first defining your class "normally"
(writing the methods without memoization) and then looping on all
methods of interest decorating them with automatic memoization; I just
suggested a similar kind of looping for decoration purposes in my very
latest post on another thread;-). You won't use the _syntax_ of
decorators, but, hey, who cares?-)

Assuming (for simplicity but with no real loss of conceptual generality)
that all of your heavy-processing methods have names starting with 'do',
and that each of them takes only 'self' as its argument, you could for
example do something like:
Thanks, that's awesome! Definitely not something I'd have ever been able
to work out myself - I think I need to learn more about nested functions
and introspection.
def memoizeMethod(cls, n, m):
def decorated(self):
if n in self._memo: return self._memo[n]
result = self._memo[n] = m(self)
return result
decorated.__name__ = n
setattr(cls, n, decorated)

def memoizeClass(cls):
cls._memo = {}
for n,m in inspect.getmembers(cls, inspect.ismethod):
if not n.startswith('do'): continue
memoizeMethod(cls, n, m)
(I've changed inspect.ismethoddescriptors to inspect.ismethod and
unindented the last line.)
Now just write your MyClass with the doThis doThat methods you want, and
right after the class statement add

memoizeClass(MyClass)
Voila, that's it (untested details, but the general idea is sound;-).
Thanks again.

--
James Mitchelhill
ja***@disorderfeed.net
http://disorderfeed.net
Jul 5 '06 #4

P: n/a
Ant
Thanks, that's awesome! Definitely not something I'd have ever been able
to work out myself - I think I need to learn more about nested functions
and introspection.
I've recently found nested functions incredibly useful in many places
in my code, particularly as a way of producing functions that are
pre-set with some initialization data. I've written a bit about it
here: http://antroy.blogspot.com/ (the entry about Partial Functions)
def memoizeMethod(cls, n, m):
def decorated(self):
if n in self._memo: return self._memo[n]
result = self._memo[n] = m(self)
return result
decorated.__name__ = n
setattr(cls, n, decorated)
Couldn't this be more simply written as:

def memoizeMethod(cls, n, m):
def decorated(self):
if not n in self._memo:
self._memo[n] = m(self)
return self._memo[n]
decorated.__name__ = n
setattr(cls, n, decorated)

I've not seen the use of chained = statements before. Presumably it
sets all variables to the value of the last one?

Jul 5 '06 #5

P: n/a
Ant <an****@gmail.comwrote:
Thanks, that's awesome! Definitely not something I'd have ever been able
to work out myself - I think I need to learn more about nested functions
and introspection.

I've recently found nested functions incredibly useful in many places
in my code, particularly as a way of producing functions that are
pre-set with some initialization data. I've written a bit about it
here: http://antroy.blogspot.com/ (the entry about Partial Functions)
def memoizeMethod(cls, n, m):
def decorated(self):
if n in self._memo: return self._memo[n]
result = self._memo[n] = m(self)
return result
decorated.__name__ = n
setattr(cls, n, decorated)

Couldn't this be more simply written as:

def memoizeMethod(cls, n, m):
def decorated(self):
if not n in self._memo:
self._memo[n] = m(self)
return self._memo[n]
decorated.__name__ = n
setattr(cls, n, decorated)
Yes, at a tiny performance cost (probably hard to measure), though I
would spell the guard "if n not in self._memo:" :-).

I've not seen the use of chained = statements before. Presumably it
sets all variables to the value of the last one?
It sets all ``variables'' ("left-hand sides", LHS) to the right-hand
side (RHS) value; here, it lets you avoid an extra indexing in order to
obtain the value to return (a tiny cost, to be sure).
Alex
Jul 5 '06 #6

This discussion thread is closed

Replies have been disabled for this discussion.