override a property

[Robin Becker]

Is there a way to override a data property in the instance? Do I need to create
another class with the property changed?
--
Robin Becker

[SPE - Stani's Python Editor]

No, you can just do it on the fly. You can even create properties
(attributes) on the fly.

class Dummy:
property = True

d = Dummy()
d.property = False
d.new = True

Stani
--
SPE - Stani's Python Editor http://pythonide.stani.be

[Bruno Desthuilliers]

Robin Becker a écrit :

Is there a way to override a data property in the instance? Do I need to
create another class with the property changed?

Do you mean attributes or properties ?

[Bengt Richter]

On Mon, 17 Oct 2005 18:52:19 +0100, Robin Becker <ro***@reportlab.com> wrote:

Is there a way to override a data property in the instance? Do I need to create
another class with the property changed?

How do you need to "override" it? Care to create a toy example with a
"wish I could <override action> here" comment line? ;-)

Regards,
Bengt Richter

[Bengt Richter]

On 17 Oct 2005 11:13:32 -0700, "SPE - Stani's Python Editor" <sp**********@gmail.com> wrote:

No, you can just do it on the fly. You can even create properties
(attributes) on the fly.

class Dummy:
property = True

d = Dummy()
d.property = False
d.new = True

a simple attribute is not a property in the sense Robin meant it,
and a "data property" is even more specific. See

http://docs.python.org/ref/descriptor-invocation.html

also

help(property)

Help on class property in module __builtin__:

class property(object)
| property(fget=None, fset=None, fdel=None, doc=None) -> property attribute
|
| fget is a function to be used for getting an attribute value, and likewise
| fset is a function for setting, and fdel a function for del'ing, an
| attribute. Typical use is to define a managed attribute x:
| class C(object):
| def getx(self): return self.__x
| def setx(self, value): self.__x = value
| def delx(self): del self.__x
| x = property(getx, setx, delx, "I'm the 'x' property.")
|
Regards,
Bengt Richter

[Robin Becker]

Bruno Desthuilliers wrote:

Robin Becker a écrit :

Is there a way to override a data property in the instance? Do I need
to create another class with the property changed?

Do you mean attributes or properties ?

I mean property here. My aim was to create an ObserverProperty class
that would allow adding and subtracting of set/get observers. My current
implementation works fine for properties on the class, but when I need
to specialize an instance I find it's quite hard.

--
Robin Becker

[bruno modulix]

Robin Becker wrote:

Bruno Desthuilliers wrote:

Robin Becker a écrit :

Is there a way to override a data property in the instance? Do I need
to create another class with the property changed?
Do you mean attributes or properties ?

I mean property here.

Ok, wasn't sure... And sorry, but I've now answer.
My aim was to create an ObserverProperty class
that would allow adding and subtracting of set/get observers.
Could you elaborate ? Or at least give an exemple ?
My current
implementation works fine for properties on the class, but when I need
to specialize an instance I find it's quite hard.

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"

[Alex Martelli]

Robin Becker <ro***@SPAMREMOVEjessikat.fsnet.co.uk> wrote:

Bruno Desthuilliers wrote:

Robin Becker a écrit :

Is there a way to override a data property in the instance? Do I need
to create another class with the property changed?

Do you mean attributes or properties ?

I mean property here. My aim was to create an ObserverProperty class
that would allow adding and subtracting of set/get observers. My current
implementation works fine for properties on the class, but when I need
to specialize an instance I find it's quite hard.

A property is an 'overriding descriptor', AKA 'data descriptor', meaning
it "captures" assignments ('setattr' kinds of operations), as well as
accesses ('getattr' kinds), when used in a newstyle class. If for some
reason you need an _instance_ to bypass the override, you'll need to set
that instance's class to one which has no overriding descriptor for that
attribute name. A better design might be to use, instead of the builtin
type 'property', a different custom descriptor type that is specifically
designed for your purpose -- e.g., one with a method that instances can
call to add or remove themselves from the set of "instances overriding
this ``property''" and a weak-key dictionary (from the weakref module)
mapping such instances to get/set (or get/set/del, if you need to
specialize "attribute deletion" too) tuples of callables.
Alex

[Robin Becker]

bruno modulix wrote:
......

Could you elaborate ? Or at least give an exemple ? ......
in answer to Bengt & Bruno here is what I'm sort of playing with. Alex suggests
class change as an answer, but that looks really clunky to me. I'm not sure what
Alex means by
A better design might be to use, instead of the builtin
type 'property', a different custom descriptor type that is specifically
designed for your purpose -- e.g., one with a method that instances can
call to add or remove themselves from the set of "instances overriding
this ``property''" and a weak-key dictionary (from the weakref module)
mapping such instances to get/set (or get/set/del, if you need to
specialize "attribute deletion" too) tuples of callables.

I see it's clear how to modify the behaviour of the descriptor instance, but is
he saying I need to mess with the descriptor magic methods so they know what
applies to each instance?
## my silly example
class ObserverProperty(property):
def __init__(self,name,observers=None,validator=None):
self._name = name
self._observers = observers or []
self._validator = validator or (lambda x: x)
self._pName = '_' + name
property.__init__(self,
fset=lambda inst, value: self.__notify_fset(inst,value),
)

def __notify_fset(self,inst,value):
value = self._validator(value)
for obs in self._observers:
obs(inst,self._pName,value)
inst.__dict__[self._pName] = value

def add(self,obs):
self._observers.append(obs)

def obs0(inst,pName,value):
print 'obs0', inst, pName, value

def obs1(inst,pName,value):
print 'obs1', inst, pName, value

class A(object):
x = ObserverProperty('x')

a=A()
A.x.add(obs0)

a.x = 3

b = A()
b.x = 4

#I wish I could get b to use obs1 instead of obs0
#without doing the following
class B(A):
x = ObserverProperty('x',observers=[obs1])

b.__class__ = B

b.x = 7
--
Robin Becker

[Kay Schluehr]

Robin Becker wrote:

Is there a way to override a data property in the instance? Do I need to create
another class with the property changed?
--
Robin Becker

It is possible to decorate a method in a way that it seems like
property() respects overridden methods. The decorator cares
polymorphism and accesses the right method.

def overridable(f):
def __wrap_func(self,*args,**kwd):
func = getattr(self.__class__,f.func_name)
if func.func_name == "__wrap_func":
return f(self,*args,**kwd)
else:
return func(self,*args,**kwd)
return __wrap_func
class A(object):
def __init__(self, x):
self._x = x

@overridable
def get_x(self):
return self._x

x = property(get_x)

class B(A):

def get_x(self):
return self._x**2

class C(B):pass

a = A(7)
a.x 7 b = B(7)
b.x 49 c = C(7)
c.x

49

[Robin Becker]

Kay Schluehr wrote:

Robin Becker wrote:

Is there a way to override a data property in the instance? Do I need to create
another class with the property changed?
--
Robin Becker

It is possible to decorate a method in a way that it seems like
property() respects overridden methods. The decorator cares
polymorphism and accesses the right method.

def overridable(f):
def __wrap_func(self,*args,**kwd):
func = getattr(self.__class__,f.func_name)
if func.func_name == "__wrap_func":
return f(self,*args,**kwd)
else:
return func(self,*args,**kwd)
return __wrap_func
class A(object):
def __init__(self, x):
self._x = x

@overridable
def get_x(self):
return self._x

x = property(get_x)

class B(A):

def get_x(self):
return self._x**2

class C(B):pass

a = A(7)
a.x
7
b = B(7)
b.x
49
c = C(7)
c.x

49

I thought that methods were always overridable. In this case the lookup on the
class changes the behaviour of the one and only property.

--
Robin Becker

[Steven Bethard]

Robin Becker wrote:

## my silly example
class ObserverProperty(property):
def __init__(self,name,observers=None,validator=None):
self._name = name
self._observers = observers or []
self._validator = validator or (lambda x: x)
self._pName = '_' + name
property.__init__(self,
fset=lambda inst, value: self.__notify_fset(inst,value),
)

def __notify_fset(self,inst,value):
value = self._validator(value)
for obs in self._observers:
obs(inst,self._pName,value)
inst.__dict__[self._pName] = value

def add(self,obs):
self._observers.append(obs)

def obs0(inst,pName,value):
print 'obs0', inst, pName, value

def obs1(inst,pName,value):
print 'obs1', inst, pName, value

class A(object):
x = ObserverProperty('x')

a=A()
A.x.add(obs0)

a.x = 3

b = A()
b.x = 4

#I wish I could get b to use obs1 instead of obs0
#without doing the following
class B(A):
x = ObserverProperty('x',observers=[obs1])

b.__class__ = B

b.x = 7

Can you add the object to be observed as another parameter to the add
method?

py> class ObservableProperty(property):
.... def __init__(self, *args, **kwargs):
.... super(ObservableProperty, self).__init__(*args, **kwargs)
.... self._observers = {}
.... def __set__(self, obj, value):
.... super(ObservableProperty, self).__set__(obj, value)
.... for observer in self._observers.get(obj, []):
.... observer(obj)
.... def add(self, obj, observer):
.... self._observers.setdefault(obj, []).append(observer)
....
py> class A(object):
.... def _getx(self):
.... return self._x
.... def _setx(self, value):
.... self._x = value
.... x = ObservableProperty(_getx, _setx)
....
py> def obs1(obj):
.... print 'obs1:', obj.x
....
py> def obs2(obj):
.... print 'obs2:', obj.x
....
py> a = A()
py> a.x = 3
py> A.x.add(a, obs1)
py> a.x = 4
obs1: 4
py> A.x.add(a, obs2)
py> a.x = 5
obs1: 5
obs2: 5
py> b = A()
py> b.x = 6
py> A.x.add(b, obs2)
py> b.x = 7
obs2: 7

Probably "self._observers" should use some sort of weakref dict instead
of a regular dict, but hopefully the idea is clear.

STeVe

[Robin Becker]

Steven Bethard wrote:

Robin Becker wrote:
........
Can you add the object to be observed as another parameter to the add
method?

py> class ObservableProperty(property):
... def __init__(self, *args, **kwargs): ....... py> A.x.add(b, obs2)
py> b.x = 7
obs2: 7

Probably "self._observers" should use some sort of weakref dict instead
of a regular dict, but hopefully the idea is clear.

STeVe

yes I think this is what Alex is proposing. It probably means abandoning the
class based observers entirely otherwise there would have to be a decision on
whether the instance observers take priority and some argument convention on
whether the class or the instance was being added to.
--
Robin Becker

[Kay Schluehr]

Robin Becker wrote:

I thought that methods were always overridable.
In this case the lookup on the
class changes the behaviour of the one and only property.

How can something be made overridable that is actually overridable? I
didn't know how to better express the broken polymorphism of Pythons
properties than by stating it as a pleonasm about the used get and set
methods. This way a property don't ever have to be redefined in
subclasses if get_x, set_x etc. are changed.

Kay

[Robin Becker]

Kay Schluehr wrote:

Robin Becker wrote:

I thought that methods were always overridable.
In this case the lookup on the
class changes the behaviour of the one and only property.

How can something be made overridable that is actually overridable? I
didn't know how to better express the broken polymorphism of Pythons
properties than by stating it as a pleonasm about the used get and set
methods. This way a property don't ever have to be redefined in
subclasses if get_x, set_x etc. are changed.

Kay

well I guess that's the ambiguity of human language. Clearly when I
assign to a normal attribute I am changing its value; assigning to a
property or descriptor does something that is not so obvious. Changing
the behaviour of such an attribute could be done by inheritance as
suggested. The new class has overridden the property. When I want to do
that on an instance I have first to create a mutable version of the
descriptor where the mutability is on the instance not the class. I call
the action of changing the base descriptor behaviour 'overriding', but
perhaps that's not the correct word. What do you suggest?
--
Robin Becker

[Bengt Richter]

On Tue, 18 Oct 2005 08:00:51 +0000, Robin Becker <ro***@SPAMREMOVEjessikat.fsnet.co.uk> wrote:

Bruno Desthuilliers wrote:

Robin Becker a écrit :

Is there a way to override a data property in the instance? Do I need
to create another class with the property changed?

Do you mean attributes or properties ?

I mean property here. My aim was to create an ObserverProperty class
that would allow adding and subtracting of set/get observers. My current
implementation works fine for properties on the class, but when I need
to specialize an instance I find it's quite hard.

Sorry, but my news feed went belly-up for a few days and I had to go to
google groups to see what had transpired.

ISTM you are already a fair way along the lines of Alex's suggestion of a custom
descriptor class, having chosen to create it by subclassing property.

If it is ok to add attributes to instances, you could put overriding
observer functions there and just check for them in your __notify_fset
loop, e.g., by checking for a matching name with an '__override_' prefixed
to the name of the obs function you want to override (see example below).

If it's not ok to add instance attributes, you could store the information
elsewhere, using weak-ref stuff as Alex suggests.

E.g. (I'm pasting in as a quote what I copied from google):

bruno modulix wrote:
.....

Could you elaborate ? Or at least give an exemple ?.....
in answer to Bengt & Bruno here is what I'm sort of playing with. Alex suggests
class change as an answer, but that looks really clunky to me. I'm not sure what
Alex means by

A better design might be to use, instead of the builtin
type 'property', a different custom descriptor type that is specifically
designed for your purpose -- e.g., one with a method that instances can
call to add or remove themselves from the set of "instances overriding
this ``property''" and a weak-key dictionary (from the weakref module)
mapping such instances to get/set (or get/set/del, if you need to
specialize "attribute deletion" too) tuples of callables. I see it's clear how to modify the behaviour of the descriptor instance, but is
he saying I need to mess with the descriptor magic methods so they know what
applies to each instance?
I think maybe only insofar as __notify_fset is magic ;-)
## my silly example
class ObserverProperty(property):
def __init__(self,name,observers=None,validator=None):
self._name = name
self._observers = observers or []
self._validator = validator or (lambda x: x)
self._pName = '_' + name
property.__init__(self,
fset=lambda inst, value: self.__notify_fset(inst,value),
) def __notify_fset(self,inst,value):
value = self._validator(value)
for obs in self._observers: obs = inst.__dict__.get('__override_'+obs.func_name, obs)
obs(inst,self._pName,value)
inst.__dict__[self._pName] = value def add(self,obs):
self._observers.append(obs) def obs0(inst,pName,value):
print 'obs0', inst, pName, value def obs1(inst,pName,value):
print 'obs1', inst, pName, value class A(object):
x = ObserverProperty('x') a=A()
A.x.add(obs0) a.x = 3 b = A()
b.x = 4 #I wish I could get b to use obs1 instead of obs0
#without doing the following I think your class assignment would eliminate all x-observing functions,
but did you mean strictly just to "use obs1 instead of obs0" -- which
would mean to leave others operable?

## >class B(A):
## > x = ObserverProperty('x',observers=[obs1])
##
## >b.__class__ = B

b.__override_obs0 = obs1

# Of course you could wrap that last line functionality in some helper thing ;-)
b.x = 7

With the above mods put in becker.py, I get:

import becker obs0 <becker.A object at 0x02EF3C2C> _x 3
obs0 <becker.A object at 0x02EF3C4C> _x 4
obs1 <becker.A object at 0x02EF3C4C> _x 7

But adding another observer doesn't eliminate the other(s):
def obs3(inst,pName,value): ... print 'obs3', inst, pName, value
... becker.A.x.add(obs3)
becker.b.x = 777 obs1 <becker.A object at 0x02EF3C4C> _x 777
obs3 <becker.A object at 0x02EF3C4C> _x 777 becker.a.x = 777

obs0 <becker.A object at 0x02EF3C2C> _x 777
obs3 <becker.A object at 0x02EF3C2C> _x 777

HTH

Regards,
Bengt Richter

[Alex Martelli]

Robin Becker <ro***@reportlab.com> wrote:
...

in answer to Bengt & Bruno here is what I'm sort of playing with. Alex
suggests class change as an answer, but that looks really clunky to me.
I'm not sure what
Changing class is indeed 'clunky', though it might have been necessary
depending on how one interpreted your original specs.
Alex means by

A better design might be to use, instead of the builtin
type 'property', a different custom descriptor type that is specifically
designed for your purpose -- e.g., one with a method that instances can
call to add or remove themselves from the set of "instances overriding
this ``property''" and a weak-key dictionary (from the weakref module)
mapping such instances to get/set (or get/set/del, if you need to
specialize "attribute deletion" too) tuples of callables.
I see it's clear how to modify the behaviour of the descriptor instance,
but is he saying I need to mess with the descriptor magic methods so they
know what applies to each instance?

If (e.g.) __set__ needs to behave differently when applied to certain
instances rather than others, then it had better be "messed with"
(overridden) compared to property.__set__ since the latter has no such
proviso. Of course, your architecture as sketched below (taking
advantage of the fact that property.__set__ always calls a certain
callable, and you get to control that callable) is OK too.

## my silly example
class ObserverProperty(property):
def __init__(self,name,observers=None,validator=None):
self._name = name
self._observers = observers or []
self._validator = validator or (lambda x: x)
self._pName = '_' + name
property.__init__(self,
fset=lambda inst, value: self.__notify_fset(inst,value),
)
Why not just fset=self.__notify_fset ? I fail to see the added value of
this lambda. Anyway...:
def __notify_fset(self,inst,value):
value = self._validator(value)
for obs in self._observers:
obs(inst,self._pName,value)
inst.__dict__[self._pName] = value

def add(self,obs):
self._observers.append(obs)
....this class only offers sets of observers *per-descriptor instance*,
not ones connected to a specific 'inst' being observed. My point is,
you could add the latter pretty easily.

def obs0(inst,pName,value):
print 'obs0', inst, pName, value

def obs1(inst,pName,value):
print 'obs1', inst, pName, value

class A(object):
x = ObserverProperty('x')

a=A()
A.x.add(obs0)

a.x = 3

b = A()
b.x = 4

#I wish I could get b to use obs1 instead of obs0
#without doing the following
class B(A):
x = ObserverProperty('x',observers=[obs1])

b.__class__ = B

b.x = 7

You can, if you have a way to call, say, b.x.add_per_inst(b, obs1).
Such as, adding within ObserverProperty:

self._observers_per_inst = {}

in the init, and changing the notification method to do:

def __notify_fset(self,inst,value):
value = self._validator(value)
observers = self._observers_per_inst.get(inst)
if not observers: observers = self._observers
for obs in observers:
obs(inst,self._pName,value)
inst.__dict__[self._pName] = value

and a new method add_per_inst:

def add_per_inst(self, inst, obs):
self._observers_per_inst.setdefault(inst,[]).append(obs)

Of course, you most likely want to use weak rather than normal
references here (probably to both instances and observers), but that's a
separate issue.
Alex

[Robin Becker]

Alex Martelli wrote:
.......

If (e.g.) __set__ needs to behave differently when applied to certain
instances rather than others, then it had better be "messed with"
(overridden) compared to property.__set__ since the latter has no such
proviso. Of course, your architecture as sketched below (taking
advantage of the fact that property.__set__ always calls a certain
callable, and you get to control that callable) is OK too.
...... I think I at last got this
.......
Why not just fset=self.__notify_fset ? I fail to see the added value of
this lambda. Anyway...:
duh just being Homerish

def __notify_fset(self,inst,value):
value = self._validator(value) ........ def add(self,obs):
self._observers.append(obs)

...this class only offers sets of observers *per-descriptor instance*,
not ones connected to a specific 'inst' being observed. My point is,
you could add the latter pretty easily.

.......
......... You can, if you have a way to call, say, b.x.add_per_inst(b, obs1).
Such as, adding within ObserverProperty:

self._observers_per_inst = {}

in the init, and changing the notification method to do:

def __notify_fset(self,inst,value):
value = self._validator(value)
observers = self._observers_per_inst.get(inst)
if not observers: observers = self._observers
for obs in observers:
obs(inst,self._pName,value)
inst.__dict__[self._pName] = value

and a new method add_per_inst:

def add_per_inst(self, inst, obs):
self._observers_per_inst.setdefault(inst,[]).append(obs)

Of course, you most likely want to use weak rather than normal
references here (probably to both instances and observers), but that's a
separate issue.

.......
yes I begin at last to see the full complexity of this. There are really
three possible attachments, the descriptor, instance class and the
instance. Since the descriptor is attached to the class or a base class
one could argue about whether observers should be inherited etc etc, but
perhaps that's a step too far.

Thanks to Alex and Bengt and others for clarifying a bunch of issues.
--
Robin Becker

[Kay Schluehr]

Robin Becker wrote:

Kay Schluehr wrote:

Robin Becker wrote:

I thought that methods were always overridable.
In this case the lookup on the
class changes the behaviour of the one and only property.

How can something be made overridable that is actually overridable? I
didn't know how to better express the broken polymorphism of Pythons
properties than by stating it as a pleonasm about the used get and set
methods. This way a property don't ever have to be redefined in
subclasses if get_x, set_x etc. are changed.

Kay

well I guess that's the ambiguity of human language. Clearly when I
assign to a normal attribute I am changing its value; assigning to a
property or descriptor does something that is not so obvious. Changing
the behaviour of such an attribute could be done by inheritance as
suggested. The new class has overridden the property. When I want to do
that on an instance I have first to create a mutable version of the
descriptor where the mutability is on the instance not the class. I call
the action of changing the base descriptor behaviour 'overriding', but
perhaps that's not the correct word. What do you suggest?
--
Robin Becker

I would suggest to take a step back and start with Raymond Hettingers
descriptor definition:

"In general, a descriptor is an object attribute with "binding
behavior", one whose attribute access has been overridden by methods in
the descriptor protocol. Those methods are __get__, __set__, and
__delete__. If any of those methods are defined for an object, it is
said to be a descriptor."

http://users.rcn.com/python/download/Descriptor.htm

The definition is a little confusing since we have to identify the
descriptor object that implements one of the descriptor methods
__get__, __set__ and __del__ with an object attribute that is assigned
by the descriptor ( as object not attribute ). Otherwise we can assert
that a descriptor becomes effective only if it is used as an object
attribute. The meaning / essence of a descriptor is to assign
attributes by a descriptor to alter binding behaviour but it's
essentially an object that can be handled quite differently. To make
the destinction clear I will talk about "descripted" attributes.

Now we can talk unambigously about "overriding" the descriptor by means
of overriding the descriptor methods in subclasses. In case of
"properties" we pass certain functions into property() that will be
wrapped into descriptor methods. Thereby property() is itself a
descriptor. If we use methods of the class where the descripted
attribute is defined, overriding has no effect on the descriptor. The
inheritance hierarchies of descriptors and classes that define
descripted attributes do not correspond.

One way of establishing a pseudo-correspondence I've already presented.
But maybe one can do it better without decorators? Remember that
property() is a descriptor factory and there is no intrinsic need to
pass functions into a factory function. Why not passing strings that
are names of methods?

For brevity only __get__ should be defined here:

class described_by(object):
def __init__(self, fget=""):
assert isinstance(fget, str)
self.fget = fget

def __get__(self, obj, objtype=None):
if obj is None:
return self
if self.fget is None:
raise AttributeError, "unreadable attribute"
return getattr(obj.__class__, self.fget)(obj)
class A(object):
def __init__(self, x):
self._x = x

def get_x(self):
return self._x

x = described_by("get_x")

class B(A):
def get_x(self):
return self._x**2

a = A(7)
a.x 7 b = B(7)
b.x

49

Regards,
Kay