By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
445,870 Members | 1,189 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 445,870 IT Pros & Developers. It's quick & easy.

Wrapping classes

P: n/a
Is it possible to implement some sort of "lazy" creation of objects only
when the object is used, but behaving in the same way as the object?

For instance:

class Foo:
def __init__(self, val):
"""This is really slow."""
self.num = val

# this doesn't call Foo.__init__ yet
a = lazyclass(Foo, 6)

# Foo is only initalised here
print a.num

What I really want to do is make an object which looks like a numarray,
but only computes its contents the first time it is used.

Thanks

Jeremy
Sep 22 '05 #1
Share this Question
Share on Google+
13 Replies


P: n/a
Jeremy Sanders wrote:
Is it possible to implement some sort of "lazy" creation of objects only
when the object is used, but behaving in the same way as the object?

For instance:

class Foo:
def __init__(self, val):
"""This is really slow."""
self.num = val

# this doesn't call Foo.__init__ yet
a = lazyclass(Foo, 6)

# Foo is only initalised here
print a.num

What I really want to do is make an object which looks like a numarray,
but only computes its contents the first time it is used.


Almost anything is possible in Python, though whether the underlying
design idea is sound is a completely different question. (Translation:
try the following pseudo-code, but I have my suspicions about whether
what you're doing is a good idea. :-) )

class lazyclass(object):
'''should probably be called lazyobject though...'''
def __init__(self, class_, *args, **kwargs):
self.class_ = class_
self.args = args
self.kwargs = kwargs
self.obj = None

def _getnum(self):
if self.obj is None:
self.obj = self.class_(*args, **kwargs)
return self.obj.num
num = property(_getnum)

Now that "should" do precisely what you've asked for above, though it is
obviously very limited in supporting only a single attribute name even
though the __init__ method is somewhat generalized. I didn't try
testing the code so there could be typos.

-Peter
Sep 22 '05 #2

P: n/a
Jeremy Sanders wrote:
Is it possible to implement some sort of "lazy" creation of objects only
when the object is used, but behaving in the same way as the object?

A generic approach would override __getattribute__ to let it perform the
__init__ method on not initialized objects.This is a case for using
metaclasses as even __init__ method must be overridden ad hoc to
register the arguments for the lazy initialization.
Probably you want to fine-tune the triggering (specifing which attribute
should make it happen ),as every look up would trigger.....

class NotInitializedObjects(type):
def __init__(cls,*_):
realInit=cls.__init__
def __newInit__(self,*pos,**key):
def _init():
realInit(self,*pos,**key)
self._init=_init
cls.__init__=__newInit__
def __getattribute__(self,attr):
def getter(attr):
return object.__getattribute__(self,attr)
if '_init' in getter('__dict__'):
getter('_init')()
del self._init
return getter(attr)
cls.__getattribute__=__getattribute__
if __name__=='__main__':
class Class:
__metaclass__=NotInitializedObjects
def __init__(self,*pos,**key):
self.initialized=True
print 'initializing with',pos,key
a=Class('arg',key='key') # a fake initialization

try:
object.__getattribute__(a,'initialized')
except AttributeError: # should raise
print 'not initialized'
else:
raise
try:
a.initialized #every look up would do ,even a print
except AttributeError:
raise
else:
print 'initialized'
Have fun Paolino

___________________________________
Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB
http://mail.yahoo.it
Sep 23 '05 #3

P: n/a
Jeremy Sanders wrote:
Is it possible to implement some sort of "lazy" creation of objects only
when the object is used, but behaving in the same way as the object?
Smells like a Proxy pattern...
For instance:

class Foo:
def __init__(self, val):
"""This is really slow."""
self.num = val

# this doesn't call Foo.__init__ yet
a = lazyclass(Foo, 6)

# Foo is only initalised here
print a.num

What I really want to do is make an object which looks like a numarray,
but only computes its contents the first time it is used.


Here's a Q&D, stupid simple, possibly flawed solution:
class LazyProxy(object):
def __init__(self, klass, *args, **kwargs):
self.__klass = klass
self.__args = args
self.__kwargs = kwargs
self.__subject = None

def __lazy_init(self):
if self.__subject is None:
self.__subject = self.__klass(*self.__args,**self.__kwargs)

def __getattr__(self, name):
self.__lazy_init()
return getattr(self.__subject, name)

def __setattr__(self, name, value):
# TODO : there's a better way to do this,
# but I don't remember it ruight now and
# don't have time to search...
if name in ['_LazyProxy__klass',
'_LazyProxy__args',
'_LazyProxy__kwargs',
'_LazyProxy__subject']:
self.__dict__[name] = value
else:
self.__lazy_init()
setattr(self.__subject, name, value)
if __name__ == '__main__':
class Greeter(object):
def __init__(self, name):
self.name = name
def greet(self, who):
return "hello %s, my name is %s" % (who, self.name)

lazy1 = LazyProxy(Greeter, 'toto')
print lazy1.greet('titi')

lazy2 = LazyProxy(Greeter, 'lolo')
lazy2.name = "lili"
print lazy2.greet(lazy1.name)

Every comment, fix etc welcome.

Now there are probably better ways to do this playing with decorators or
meta-classes.

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"
Sep 23 '05 #4

P: n/a
Paolino wrote:
class NotInitializedObjects(type):
def __init__(cls,*_):
realInit=cls.__init__
def __newInit__(self,*pos,**key):
def _init():
realInit(self,*pos,**key)
self._init=_init
cls.__init__=__newInit__
def __getattribute__(self,attr):
def getter(attr):
return object.__getattribute__(self,attr)
if '_init' in getter('__dict__'):
getter('_init')()
del self._init
return getter(attr)
cls.__getattribute__=__getattribute__


A lighter solution can be overriding __getattr__.
This will produce more object-like behaving instances even when not
initialized, aka you can call methods and access class attributes
without triggering the init (not very useful)

class NotInitializedObjects(type):
def __init__(cls,*_):
realInit=cls.__init__
def __newInit__(self,*pos,**key):
def _init():
realInit(self,*pos,**key)
self._init=_init
cls.__init__=__newInit__
def __getattr__(self,attr):
if hasattr(self,'_init'):
self._init()
del self._init
if hasattr(self,attr):
return getattr(self,attr)
raise AttributeError
cls.__getattr__=__getattr__

### Test with previous testing code

A cleaner solution is decoupling the intensive calculation attributes
from __init__ and use descriptors for them.But this is impossible if
/the/ instance value is the intensive one to be calculated.

Ciao Paolino

___________________________________
Aggiungi la toolbar di Yahoo! Search sul tuo Browser, e'gratis!
http://it.toolbar.yahoo.com
Sep 23 '05 #5

P: n/a
Peter Hansen wrote:
Almost anything is possible in Python, though whether the underlying
design idea is sound is a completely different question. (Translation:
try the following pseudo-code, but I have my suspicions about whether
what you're doing is a good idea. :-) )


What I'd like to do precisely is to be able to evaluate an expression like
"a+2*b" (using eval) where a and b are objects which behave like numarray
arrays, but whose values aren't computed until their used.

I need to compute the values when used because the arrays could depend on
each other, and the easiest way to get the evaluation order correct is to
only evaluate them when they're used.

An alternative way is to do some string processing to replace a with
computearray("a") in the expression or something horrible like that.

Thanks

Jeremy

--
Jeremy Sanders
http://www.jeremysanders.net/
Sep 23 '05 #6

P: n/a
Jeremy Sanders wrote:
Peter Hansen wrote:

Almost anything is possible in Python, though whether the underlying
design idea is sound is a completely different question. (Translation:
try the following pseudo-code, but I have my suspicions about whether
what you're doing is a good idea. :-) )

What I'd like to do precisely is to be able to evaluate an expression like
"a+2*b" (using eval) where a and b are objects which behave like numarray
arrays, but whose values aren't computed until their used.


Maybe you can do that by passing eval your own globals dictionary -
which in its __getitem__ method will then compute the value lazy.

The heck, we're in python. Lets try:

class Foo(object):

def __init__(self):
self.a = 10
self.b = 20
#return dict.__new__(self)

def __getitem__(self, key):
print "computing %s" % key
return getattr(self, key)

l = Foo()
print l.a

print eval("10 * a + b", globals(), l)
It works - in python 2.4!! I tried subclassing dict, but my
__getitem__-method wasn't called - most probably because it's a C-type,
but I don't know for sure. Maybe someone can elaborate on that?

Regards,

Diez
Sep 23 '05 #7

P: n/a
Diez B. Roggisch wrote:
It works - in python 2.4!! I tried subclassing dict, but my
__getitem__-method wasn't called - most probably because it's a C-type,
but I don't know for sure. Maybe someone can elaborate on that?


Yes - I tried that (see thread below). Unfortunately it needs Python 2.4,
and I can't rely on my users having that.

Traceback (most recent call last):
File "test.py", line 15, in ?
print eval("10 * a + b", globals(), l)
TypeError: eval() argument 3 must be dict, not Foo

If you subclass dict it doesn't call the __getitem__ method.

Jeremy

--
Jeremy Sanders
http://www.jeremysanders.net/
Sep 23 '05 #8

P: n/a
Jeremy Sanders wrote:
Diez B. Roggisch wrote:

It works - in python 2.4!! I tried subclassing dict, but my
__getitem__-method wasn't called - most probably because it's a C-type,
but I don't know for sure. Maybe someone can elaborate on that?

Yes - I tried that (see thread below). Unfortunately it needs Python 2.4,
and I can't rely on my users having that.

Traceback (most recent call last):
File "test.py", line 15, in ?
print eval("10 * a + b", globals(), l)
TypeError: eval() argument 3 must be dict, not Foo

If you subclass dict it doesn't call the __getitem__ method.


Could it work with a UserDict subclass ?
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"
Sep 23 '05 #9

P: n/a
bruno modulix wrote:
Could it work with a UserDict subclass ?


Unfortunately not:

Traceback (most recent call last):
File "test.py", line 17, in ?
print eval("10 * a + b", globals(), l)
TypeError: eval() argument 3 must be dict, not instance

Thanks

Jeremy

--
Jeremy Sanders
http://www.jeremysanders.net/
Sep 23 '05 #10

P: n/a
Jeremy Sanders wrote:
Peter Hansen wrote:

Almost anything is possible in Python, though whether the underlying
design idea is sound is a completely different question. (Translation:
try the following pseudo-code, but I have my suspicions about whether
what you're doing is a good idea. :-) )

What I'd like to do precisely is to be able to evaluate an expression like
"a+2*b" (using eval) where a and b are objects which behave like numarray
arrays, but whose values aren't computed until their used.

Could you not have functions a and b each of which returns a NumArray
instance?

Your expression would then be something like a(..)+2*b(..).

Colin W.
I need to compute the values when used because the arrays could depend on
each other, and the easiest way to get the evaluation order correct is to
only evaluate them when they're used.

An alternative way is to do some string processing to replace a with
computearray("a") in the expression or something horrible like that.

Thanks

Jeremy

Sep 23 '05 #11

P: n/a
Colin J. Williams wrote:
Could you not have functions a and b each of which returns a NumArray
instance?

Your expression would then be something like a(..)+2*b(..).


The user enters the expression (yes - I'm aware of the possible security
issues), as it is a scientific application. I don't think they'd like to
put () after each variable name.

I could always munge the expression after the user enters it, of course.

Jeremy

--
Jeremy Sanders
http://www.jeremysanders.net/
Sep 23 '05 #12

P: n/a

I agree this is a case for using metaclasses. What about an
implementation like this ? Seems like checking if init was already
called will slow down all attribute access significantly, but, I don't
like this approach of changing the __init__ method.
class LazyInit(type):
def __new__(self, name, bases, dict):

def __getattribute__(self, attr):
attrs = object.__getattribute__(self, "__dict__")
init = attrs["_init"]
if not init:
args = attrs["_args"]
kwds = attrs["_kwds"]
__init__ = object.__getattribute__(self, "__init__")
__init__(*args, **kwds)
attrs["_init"] = True

return object.__getattribute__(self, attr)

dict['__getattribute__'] = __getattribute__
return type.__new__(self, name, bases, dict)

def __call__(cls, *args, **kwds):
o = object.__new__(cls, *args, **kwds)
o._args = args
o._kwds = kwds
o._init = False

return o
And some simple testing:
class Foo: .... __metaclass__ = LazyInit
.... def __init__(self, x, y):
.... print "init was called", x, y
.... self.x = x
.... self.y = y
.... o = Foo(1, None)
o <__main__.Foo object at 0x402cc96c> o.x init was called 1 None
1 o.y


Regards,

Pedro

On Fri, 23 Sep 2005 10:28:42 +0200
Paolino <pa*************@tiscali.it> wrote:
Jeremy Sanders wrote:
Is it possible to implement some sort of "lazy" creation of objects
only when the object is used, but behaving in the same way as the
object?

A generic approach would override __getattribute__ to let it perform
the
__init__ method on not initialized objects.This is a case for using
metaclasses as even __init__ method must be overridden ad hoc to
register the arguments for the lazy initialization.
Probably you want to fine-tune the triggering (specifing which
attribute should make it happen ),as every look up would trigger.....

class NotInitializedObjects(type):
def __init__(cls,*_):
realInit=cls.__init__
def __newInit__(self,*pos,**key):
def _init():
realInit(self,*pos,**key)
self._init=_init
cls.__init__=__newInit__
def __getattribute__(self,attr):
def getter(attr):
return object.__getattribute__(self,attr)
if '_init' in getter('__dict__'):
getter('_init')()
del self._init
return getter(attr)
cls.__getattribute__=__getattribute__
if __name__=='__main__':
class Class:
__metaclass__=NotInitializedObjects
def __init__(self,*pos,**key):
self.initialized=True
print 'initializing with',pos,key
a=Class('arg',key='key') # a fake initialization

try:
object.__getattribute__(a,'initialized')
except AttributeError: # should raise
print 'not initialized'
else:
raise
try:
a.initialized #every look up would do ,even a print
except AttributeError:
raise
else:
print 'initialized'
Have fun Paolino

___________________________________
Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB
http://mail.yahoo.it
--
http://mail.python.org/mailman/listinfo/python-list

--
Pedro Werneck
Sep 23 '05 #13

P: n/a
Jeremy Sanders wrote:
Colin J. Williams wrote:

Could you not have functions a and b each of which returns a NumArray
instance?

Your expression would then be something like a(..)+2*b(..).

The user enters the expression (yes - I'm aware of the possible security
issues), as it is a scientific application. I don't think they'd like to
put () after each variable name.

I could always munge the expression after the user enters it, of course.

Jeremy

Alternatively, you could build your own expression calculator, and initialize
the objects if necessary as they are evaluated. If you are happy with Python
syntax for your expressiones then the stdlib compiler package is helpful. The
example below is not tested beyond what you see. It's a bit verbose, but most
of the code is boilerplate.
a = 3
b = 4
calc('a * b') using a
using b
12 calc('a * b ** (b - a) * "a"') using a
using b
using b
using a
'aaaaaaaaaaaa' calc("0 and a or b") using b
4 calc("1 and a or b") using a
3 calc("1 and a or c") using a
3 calc("0 and a or c") Undefined symbol: c

HTH, Michael

-----------------

import compiler
class CalcError(Exception):
def __init__(self,error,descr = None,node = None):
self.error = error
self.descr = descr
self.node = node

def __repr__(self):
return "%s: %s" % (self.error, self.descr)
__str__ = __repr__
class LazyCalc(object):

def __init__(self, namespace):
self._cache = {} # dispatch table
self.context = namespace

def visit(self, node,**kw):
cls = node.__class__
meth = self._cache.setdefault(cls,
getattr(self,'visit'+cls.__name__,self.default))
return meth(node, **kw)

def visitExpression(self, node, **kw):
return self.visit(node.node)
# Binary Ops
def visitAdd(self,node,**kw):
return self.visit(node.left) + self.visit(node.right)
def visitDiv(self,node,**kw):
return self.visit(node.left) / self.visit(node.right)
def visitFloorDiv(self,node,**kw):
return self.visit(node.left) // self.visit(node.right)
def visitLeftShift(self,node,**kw):
return self.visit(node.left) << self.visit(node.right)
def visitMod(self,node,**kw):
return self.visit(node.left) % self.visit(node.right)
def visitMul(self,node,**kw):
return self.visit(node.left) * self.visit(node.right)
def visitPower(self,node,**kw):
return self.visit(node.left) ** self.visit(node.right)
def visitRightShift(self,node,**kw):
return self.visit(node.left) >> self.visit(node.right)
def visitSub(self,node,**kw):
return self.visit(node.left) - self.visit(node.right)

# Unary ops
def visitNot(self,node,*kw):
return not self.visit(node.expr)
def visitUnarySub(self,node,*kw):
return -self.visit(node.expr)
def visitInvert(self,node,*kw):
return ~self.visit(node.expr)
def visitUnaryAdd(self,node,*kw):
return +self.visit(node.expr)

# Flow Control
def visitAnd(self,node,**kw):
for arg in node.nodes:
val = self.visit(arg)
if not val:
return val
return val
def visitOr(self,node,**kw):
for arg in node.nodes:
val = self.visit(arg)
if val:
return val
return val

# Logical Ops
def visitBitand(self,node,**kw):
return reduce(lambda a,b: a & b,[self.visit(arg) for arg in node.nodes])
def visitBitor(self,node,**kw):
return reduce(lambda a,b: a | b,[self.visit(arg) for arg in node.nodes])
def visitBitxor(self,node,**kw):
return reduce(lambda a,b: a ^ b,[self.visit(arg) for arg in node.nodes])
def visitCompare(self,node,**kw):
comparisons = {
"<": operator.lt, # strictly less than
"<=": operator.le,# less than or equal
">": operator.gt, # strictly greater than
">=": operator.ge, # greater than or equal
"==": operator.eq, # equal
"!=": operator.ne, # not equal
"<>": operator.ne, # not equal
"is": operator.is_, # object identity
"is not": operator.is_not # negated object identity
}
obj = self.visit(node.expr)
for op, compnode in node.ops:
compobj = self.visit(compnode)
if not comparisons[op](obj, compobj):
return False
obj = compobj
return True
# Values
def visitCallFunc(self,node,**kw):
raise CalcError("Functions not supported", node.node)

def visitName(self, node, **kw):
"""LazyEvaluation"""
name = node.name
try:
val = eval(name, self.context)
except NameError:
raise CalcError("Undefined symbol",name)
except:
raise
print "using %s" % name # init if necessary here
return val

def visitConst(self, node, **kw):
return node.value

# Other
def default(self, node, **kw):
"""Anything not expressly allowed is forbidden"""
raise CalcError("Not Allowed",
node.__class__.__name__,node)
def calc(source, context = None):
walker = LazyCalc(context or globals())
try:
ast = compiler.parse(source,"eval")
except SyntaxError, err:
raise
try:
return walker.visit(ast)
except CalcError, err:
return err
Sep 24 '05 #14

This discussion thread is closed

Replies have been disabled for this discussion.