473,219 Members | 2,017 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,219 software developers and data experts.

How to memoize/cache property access?

I seem to be writing the following boilerplate/pattern quite
frequently to avoid hitting the database until absolutely necessary,
and to only do it at most once:

class Foo(object):
@property
def expensive(self):
if not hasattr(self, '_expensiv'):
self._expensive = <insert expensive db call here>
return self._expensive

it's a bit verbose, and it violates the DRY (Don't Repeat Yourself)
principle -- and it has on at least one occasion resulted in really
slow code that produces the correct results (did you spot the typo
above?).

It would have been nice to be able to write

class Foo(object):
@property
def expensive(self):
self.expensive = <insert expensive db call here>
return self.expensive

but apparently I "can't set [that] attribute" :-(

I'm contemplating using a decorator to hide the first pattern:

def memprop(fn):
def _fn(self): # properties only take self
memname = '_' + fn.__name__
if not hasattr(self, memname):
setattr(self, memname, fn(self))
return getattr(self, memname)
return property(fget=_fn, doc=fn.__doc__)

which means I can very simply write

class Foo(object):
@memprop
def expensive(self):
return <insert expensive db call here>

I'm a bit hesitant to start planting home-grown memprop-s all over the
code-base though, so I'm wondering... does this seem like a reasonable
thing to do? Am I re-inventing the wheel? Is there a better way to
approach this problem?

-- bjorn
Dec 20 '07 #1
4 3012
thebjorn <Bj************************@gmail.comwrote:
It would have been nice to be able to write

class Foo(object):
@property
def expensive(self):
self.expensive = <insert expensive db call here>
return self.expensive

but apparently I "can't set [that] attribute" :-(
You can set and access it directly in __dict__. How about something along
these lines:

def cachedproperty(f):
name = f.__name__
def getter(self):
try:
return self.__dict__[name]
except KeyError:
res = self.__dict__[name] = f(self)
return res
return property(getter)

class Foo(object):
@cachedproperty
def expensive(self):
print "expensive called"
return 42
>>f = Foo()
f.expensive
expensive called
42
>>f.expensive
42

It is still calling the getter every time though, so not as fast as a plain
attribute lookup.
Dec 20 '07 #2
On Dec 20, 5:02 pm, thebjorn <BjornSteinarFjeldPetter...@gmail.com>
wrote:
I seem to be writing the following boilerplate/pattern quite
frequently to avoid hitting the database until absolutely necessary ...
I use the following module:

$ cat cache.py
class cached(property):
'Convert a method into a cached attribute'
def __init__(self, method):
private = '_' + method.__name__
def fget(s):
try:
return getattr(s, private)
except AttributeError:
value = method(s)
setattr(s, private, value)
return value
def fdel(s):
del s.__dict__[private]
super(cached, self).__init__(fget, fdel=fdel)
@staticmethod
def reset(self):
cls = self.__class__
for name in dir(cls):
attr = getattr(cls, name)
if isinstance(attr, cached):
delattr(self, name)

if __name__ == '__main__': # a simple test
import itertools
counter = itertools.count()
class Test(object):
@cached
def foo(self):
return counter.next()
reset = cached.reset

p = Test()
print p.foo
print p.foo
p.reset()
print p.foo
print p.foo
p.reset()
print p.foo

Michele Simionato
Dec 20 '07 #3
On Dec 20, 5:43 pm, Michele Simionato <michele.simion...@gmail.com>
wrote:
On Dec 20, 5:02 pm, thebjorn <BjornSteinarFjeldPetter...@gmail.com>
wrote:
I seem to be writing the following boilerplate/pattern quite
frequently to avoid hitting the database until absolutely necessary ...

I use the following module:
[...]

I love it! much better name too ;-)

I changed your testcase to include a second cached property (the
naming is in honor of my late professor in Optimization of Functional
Languages class: "...any implementation that calls bomb_moscow is per
definition wrong, even if the program produces the correct result..."
-- it was a while ago ;-)

if __name__ == '__main__': # a simple test
import itertools
counter = itertools.count()
class Test(object):
@cached
def foo(self):
return counter.next()

@cached
def bomb_moscow(self):
print 'fire missiles'
return counter.next()

reset = cached.reset

it didn't start WWIII, but I had to protect attribute deletion to get
it to run:

def fdel(s):
if private in s.__dict__:
del s.__dict__[private]

I'm a bit ambivalent about the reset functionality. While it's a
wonderful demonstration of a staticmethod, the very few times I've
felt the need to "freshen-up" the object, I've always felt it was best
to create it again from scratch. Do you have many uses of it in your
code?

-- bjorn

Dec 20 '07 #4
On Dec 20, 6:40 pm, thebjorn
I'm a bit ambivalent about the reset functionality. While it's a
wonderful demonstration of a staticmethod, the very few times I've
felt the need to "freshen-up" the object, I've always felt it was best
to create it again from scratch. Do you have many uses of it in your
code?
My use case is for web applications where the configuration
parameters are stored in a database. If you change them,
you can re-read them by resetting the configuration object
(you may have a reset button in the administrator Web user
interface) without restarting the application.

Michele Simionato

Dec 20 '07 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Chris Reedy | last post by:
I would like to memoize (I think I've got that right) a collection of functions for a project I'm working. <Aside for those not familiar with memoizing functions:> Given: def foo(a,b,c):...
3
by: Michael Hohn | last post by:
Hi, under python 2.2, the pickle/unpickle sequence incorrectly restores a larger data structure I have. Under Python 2.3, these structures now give an explicit exception from...
13
by: km | last post by:
Hi all, was going thru the new features introduced into python2.4 version. i was stuck with 'decorators' - can someone explain me the need of such a thing called decorators ? tia KM
3
by: George Ter-Saakov | last post by:
What is HttpContext.Cache for ? The usual HttpRequest lasts miliseconds. Why would someone to cache anything? We do have HttpContext.Items to keep any extra info during the request. Can...
2
by: Keith Chadwick | last post by:
Need some help on access the httpApplication cache. I was informed to use the application cache over the old application object and also to use a classes versus modules. do I need to Imprt...
1
by: Christopher | last post by:
In one of our ASP.NET Pages, we are starting a new background thread that we do not need to go and get any status on or use after the page finishes. The thread merely does some background stuff on...
11
by: thattommyhallll | last post by:
im plugging away at the problems at http://www.mathschallenge.net/index.php?section=project im trying to use them as a motivator to get into advanced topics in python. one thing that Structure...
4
by: Harry Haller | last post by:
What's wrong with this: Error 3 'System.Web.Caching.Cache' is a 'type', which is not valid in the given context public List<AssetSummaryGetAssetSummary() { return...
1
by: Andrus | last post by:
I need to create Dlinq entity property value caches for every entity and every property so that all entity caches can cleared if entity of this type is changed. Entity class method uses this as:...
1
isladogs
by: isladogs | last post by:
The next online meeting of the Access Europe User Group will be on Wednesday 6 Dec 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, Mike...
0
by: veera ravala | last post by:
ServiceNow is a powerful cloud-based platform that offers a wide range of services to help organizations manage their workflows, operations, and IT services more efficiently. At its core, ServiceNow...
0
by: VivesProcSPL | last post by:
Obviously, one of the original purposes of SQL is to make data query processing easy. The language uses many English-like terms and syntax in an effort to make it easy to learn, particularly for...
3
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 3 Jan 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). For other local times, please check World Time Buddy In...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
2
by: jimatqsi | last post by:
The boss wants the word "CONFIDENTIAL" overlaying certain reports. He wants it large, slanted across the page, on every page, very light gray, outlined letters, not block letters. I thought Word Art...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.