Class Variable Access and Assignment

[Graham]

This has to do with class variables and instances variables.

Given the following:

<code>

class _class:
var = 0
#rest of the class

instance_b = _class()

_class.var=5

print instance_b.var # -> 5
print _class.var # -> 5

</code>

Initially this seems to make sense, note the difference between to last
two lines, one is refering to the class variable 'var' via the class
while the other refers to it via an instance.

However if one attempts the following:

<code>

instance_b.var = 1000 # -> _class.var = 5
_class.var = 9999 # -> _class.var = 9999

</code>

An obvious error occurs. When attempting to assign the class variable
via the instance it instead creates a new entry in that instance's
__dict__ and gives it the value. While this is allowed because of
pythons ability to dynamically add attributes to a instance however it
seems incorrect to have different behavior for different operations.

There are two possible fixes, either by prohibiting instance variables
with the same name as class variables, which would allow any reference
to an instance of the class assign/read the value of the variable. Or
to only allow class variables to be accessed via the class name itself.

Many thanks to elpargo and coke. elpargo assisted in fleshing out the
best way to present this.

perhaps this was intended, i was just wondering if anyone else had
noticed it, and if so what form would you consider to be 'proper'
either referring to class variables via the class itself or via
instances of that class. Any response would be greatly appreciated.
Graham

[Steven D'Aprano]

On Sat, 05 Nov 2005 16:27:00 -0800, Paul Rubin wrote:

Steven D'Aprano <st***@REMOVETH IScyber.com.au> writes:

But do you want x += y to work for immutable objects as well? Then
__iadd__ cannot be a statement, because x can't be modified in place.
It never occurred to me that immutable objects could implement __iadd__.
If they can, I'm puzzled as to why.

???

The classic += idiom comes from C, where you typically use it on ints and
pointers.

In C, ints aren't objects, they are just bytes, so you can modify them
in place. I'm surprised that it never occurred to you that people might
want to do something like x = 1; x += 1 in Python, especially as the
lack of such a feature (as I recall) was one of the biggest complaints
from C programmers crossing over to Python.

Personally, I'm not fussed about +=. Now that it is in the language, I'll
use it, but I never missed it when it wasn't in the language.

While I am enjoying the hoops people are jumping through to modify the
language so that b.a += 2 assigns b.a in the same scope as it was
accessed, I'm still rather perplexed as to why you would want that
behaviour.

Weren't you the one saying += acting differently for mutables and
immutables was a wart?

Nope, not me.
If it's such a wart, why are do you find it so
important to be able to rely on the more bizarre consequences of the
wartiness? Warts should be (if not fixed) avoided, not relied on.

The consequences of instance.attrib ute += 1 may be unexpected for those
who haven't thought it through, or read the documentation, but they aren't
bizarre. Whether that makes it a feature or a wart depends on whether you
think non-method attributes should be inherited or not. I think they
should be.

I can respect the position of somebody who says that only methods
should be inherited -- somebody, I think it was you, suggested that there
is at least one existing OO language that doesn't allow inheritance for
attributes, but never responded to my asking what language it was.
Personally, I would not like an OO language that didn't inherit
attributes, but at least that is consistent. (At least, if you don't
consider methods to be a particular sort of attribute.)

But I can't understand the position of folks who want inheritance but
don't want the behaviour that Python currently exhibits.
instance.attrib ute sometimes reading from the class attribute is a feature
of inheritance; instance.attrib ute always writing to the instance is a
feature of OOP; instance.attrib ute sometimes writing to the instance and
sometimes writing to the class would be, in my opinion, not just a wart
but a full-blown misfeature.

I ask and I ask and I ask for some use of this proposed behaviour, and
nobody is either willing or able to tell me where how or why it would be
useful. What should I conclude from this?
--
Steven.

[Paul Rubin]

Steven D'Aprano <st***@REMOVETH IScyber.com.au> writes:

It never occurred to me that immutable objects could implement __iadd__.
If they can, I'm puzzled as to why.

I'm surprised that it never occurred to you that people might
want to do something like x = 1; x += 1 in Python,
But I wouldn't expect that to mean that ints implement __iadd__. I'd expect
the x+=1 to just use __add__. I haven't checked the spec though.
I can respect the position of somebody who says that only methods
should be inherited -- somebody, I think it was you, suggested that there
is at least one existing OO language that doesn't allow inheritance for
attributes, but never responded to my asking what language it was.
I was thinking of Flavors. You use a special function (send) to do method
calls. But people generally felt that was kludgy and CLOS eliminated it.
I'm not sure what happens in Smalltalk.
instance.attrib ute sometimes reading from the class attribute is a feature
of inheritance; instance.attrib ute always writing to the instance is a
feature of OOP; instance.attrib ute sometimes writing to the instance and
sometimes writing to the class would be, in my opinion, not just a wart
but a full-blown misfeature.

But that is what you're advocating: x.y+=1 writes to the instance or
the class depending on whether x.y is mutable or not. Say you have an
immutable class with a mutable subclass or vice versa. You'd like to
be able to replace a class instance with a subclass instance and not
have the behavior change (Liskov substitution principle), etc.

[Steve Holden]

Steven D'Aprano wrote:
[...]

But I can't understand the position of folks who want inheritance but
don't want the behaviour that Python currently exhibits.
instance.attrib ute sometimes reading from the class attribute is a feature
of inheritance; instance.attrib ute always writing to the instance is a
feature of OOP; instance.attrib ute sometimes writing to the instance and
sometimes writing to the class would be, in my opinion, not just a wart
but a full-blown misfeature.

I ask and I ask and I ask for some use of this proposed behaviour, and
nobody is either willing or able to tell me where how or why it would be
useful. What should I conclude from this?

You should conclude that some readers of this group are happier
designing languages with theoretical purity completely disconnected from
users' needs. But of course we pragmatists know that practicality beats
purity :-)

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC www.holdenweb.com
PyCon TX 2006 www.python.org/pycon/

[Steven D'Aprano]

On Sat, 05 Nov 2005 18:14:03 -0800, Paul Rubin wrote:

instance.attrib ute sometimes reading from the class attribute is a feature
of inheritance; instance.attrib ute always writing to the instance is a
feature of OOP; instance.attrib ute sometimes writing to the instance and
sometimes writing to the class would be, in my opinion, not just a wart
but a full-blown misfeature.
But that is what you're advocating: x.y+=1 writes to the instance or
the class depending on whether x.y is mutable or not.

Scenario 1:

Pre-conditions: class.a exists; instance.a exists.
Post-conditions: class.a unchanged; instance.a modified.

I give that a big thumbs up, expected and proper behaviour.

Scenario 2:

Pre-conditions: class.a exists and is immutable; instance.a does not
exist.
Post-conditions: class.a unchanged; instance.a exists.

Again, expected and proper behaviour.

(Note: this is the scenario that Antoon's proposed behaviour would change
to class.a modified; instance.a does not exist.)

Scenario 3:

Pre-conditions: class.a exists and is mutable; instance.a exists.
Post-conditions: class.a unchanged; instance.a is modified.

Again, expected and proper behaviour.

Scenario 4:

Pre-conditions: class.a exists and is mutable; instance.a does
not exist.
Post-conditions: class.a modified; instance.a does not exist.

Well, that is a wart. It is the same wart, and for the same reasons, as
the behaviour of:

def function(value=[]):
value.append(No ne)

I can live with that. It is a familiar wart, and keeps inheritance of
attributes working the right way. And who knows? If your attributes are
mutable, AND you want Antoon's behaviour, then you get it for free just by
using b.a += 1 instead of b.a = b.a + 1.

Say you have an
immutable class with a mutable subclass or vice versa. You'd like to
be able to replace a class instance with a subclass instance and not
have the behavior change (Liskov substitution principle), etc.

That's easy. You just have to make sure that the subclass implements
__iadd__ the same way that the immutable parent class does.

You can't expect a class that performs += in place to act the
same as a class that doesn't perform += in place. Excessive data
abstraction, remember?

L = list("Liskov substitution principle")
L.sort() # sorts in place
print L # prints the sorted list

class immutable_list( list):
# __init__ not shown, but does the right thing
def sort(self):
tmp = list(self)
tmp.sort()
return immutable_list( tmp)

L = immutable_list( "Liskov substitution principle")
L.sort() # throws the sorted list away
print L # prints the unsorted list

The only way the Liskov substitution principle works is if everything
works the same way, which means that all subclasses, all *possible*
subclasses, must have no more functionality than the subclass that does
the absolute least. Since the least is nothing, well, you work it out.

--
Steven.

[Bengt Richter]

On Sun, 06 Nov 2005 15:17:18 +1100, Steven D'Aprano <st***@REMOVETH IScyber.com.au> wrote:

On Sat, 05 Nov 2005 18:14:03 -0800, Paul Rubin wrote:

instance.attrib ute sometimes reading from the class attribute is a feature
of inheritance; instance.attrib ute always writing to the instance is a
feature of OOP; instance.attrib ute sometimes writing to the instance and
sometimes writing to the class would be, in my opinion, not just a wart
but a full-blown misfeature.
But that is what you're advocating: x.y+=1 writes to the instance or
the class depending on whether x.y is mutable or not.

Scenario 1:

Pre-conditions: class.a exists; instance.a exists.
Post-conditions: class.a unchanged; instance.a modified.

I give that a big thumbs up, expected and proper behaviour.

Scenario 2:

Pre-conditions: class.a exists and is immutable; instance.a does not
exist.
Post-conditions: class.a unchanged; instance.a exists.

Again, expected and proper behaviour.

(Note: this is the scenario that Antoon's proposed behaviour would change
to class.a modified; instance.a does not exist.)

Scenario 3:

Pre-conditions: class.a exists and is mutable; instance.a exists.
Post-conditions: class.a unchanged; instance.a is modified.

Again, expected and proper behaviour.

Scenario 4:

Pre-conditions: class.a exists and is mutable; instance.a does
not exist.
Post-conditions: class.a modified; instance.a does not exist.

Are you saying the above is what happens or what should happen or not happen?
It's not what happens. Post-conditions are that class.a is modified AND
instance.a gets a _separate_ reference to the same result. Note:

Python 2.4b1 (#56, Nov 3 2004, 01:47:27)
[GCC 3.2.3 (mingw special 20030504-1)] on win32
Type "help", "copyright" , "credits" or "license" for more information.

class A(object): ... a = []
... b=A()
id(A.__dict__['a']) 49230700 b.a += [123]
id(A.__dict__['a']) 49230700 id(b.__dict__['a']) 49230700 (b.__dict__['a']) [123] (A.__dict__['a']) [123]

Let's eliminate the inheritable class variable A.a:
del A.a
b.a [123] id(b.__dict__['a']) 49230700 vars(b) {'a': [123]}

Make sure we did eliminate A.a vars(A) <dictproxy object at 0x02E817AC> vars(A).keys()

['__dict__', '__module__', '__weakref__', '__doc__']

Is that the "wart" you were thinking of, or are you actually happier? ;-)
Well, that is a wart. It is the same wart, and for the same reasons, as
the behaviour of:

def function(value=[]):
value.append(No ne) IMO that's not a wart at all, that's a direct design decision, and it's
different from the dual referencing that happens in Scenario 4.

I can live with that. It is a familiar wart, and keeps inheritance of
attributes working the right way. And who knows? If your attributes are
mutable, AND you want Antoon's behaviour, then you get it for free just by
using b.a += 1 instead of b.a = b.a + 1.

Not quite, because there is no way to avoid the binding of the __iadd__
return value to b.a by effective setattr (unless you make type(b).a
a descriptor that intercepts the attempt -- see another post for example).

Regards,
Bengt Richter

[Steven D'Aprano]

On Sun, 06 Nov 2005 08:40:00 +0000, Bengt Richter wrote:

Pre-conditions: class.a exists and is mutable; instance.a does
not exist.
Post-conditions: class.a modified; instance.a does not exist.
Are you saying the above is what happens or what should happen or not happen?

Er, it's what I thought was happening without actually checking it...
It's not what happens. Post-conditions are that class.a is modified AND
instance.a gets a _separate_ reference to the same result. Note:
[snip demonstration]
Is that the "wart" you were thinking of, or are you actually happier?
;-)

In other words, the post-condition for all four scenarios includes that
the instance attribute now exists. I'm actually happier. My brain was full
of people talking about __iadd__ modifying mutables in place and I wasn't
thinking straight.

Well, that is a wart. It is the same wart, and for the same reasons, as
the behaviour of:

def function(value=[]):
value.append(No ne)

IMO that's not a wart at all, that's a direct design decision, and it's
different from the dual referencing that happens in Scenario 4.

Okay, perhaps wart is not quite the right word... but it is certainly
unexpected if you haven't come across it before, or thought *deeply* about
what is going on. A gotcha perhaps.
--
Steven.

[Christopher Subich]

Bengt Richter wrote:

On Fri, 04 Nov 2005 10:28:52 -0500, Christopher Subich <cs************ ****@spam.subic h.block.com> wrote:

is very much within the language specification. Indeed, the language
specificati on dictates that an instance variable b.a is created if one
didn't exist before; this is true no matter if type(b.a) == int, or if
b.a is some esoteric mutable object that just happens to define
__iadd__(self ,type(other) == int).

But if it is an esoteric descriptor (or even a simple property, which is
a descriptor), the behaviour will depend on the descriptor, and an instance
variable can be created or not, as desired, along with any side effect you like.

Right, and that's also language-specification. Voodoo, yes, but
language specification nonetheless. :)

[Bengt Richter]

On Sun, 06 Nov 2005 12:23:02 -0500, Christopher Subich <sp************ *********@subic h.nospam.com> wrote:

Bengt Richter wrote:

On Fri, 04 Nov 2005 10:28:52 -0500, Christopher Subich <cs************ ****@spam.subic h.block.com> wrote:

is very much within the language specification. Indeed, the language
specificatio n dictates that an instance variable b.a is created if one
didn't exist before; this is true no matter if type(b.a) == int, or if
b.a is some esoteric mutable object that just happens to define
__iadd__(sel f,type(other) == int).

But if it is an esoteric descriptor (or even a simple property, which is
a descriptor), the behaviour will depend on the descriptor, and an instance
variable can be created or not, as desired, along with any side effect you like.

Right, and that's also language-specification. Voodoo, yes, but
language specification nonetheless. :)

I guess http://docs.python.org/ref/augassign.html is the spec.
I notice its example at the end uses an old-style class, so maybe
it's understandable that when it talks about getattr/setattr, it doesn't
mention the possible role of descriptors, nor narrow the meaning of
"evaluate once" for a.x to exclude type(a).x in the setattr phase of execution.

I.e., if x is a descriptor, "evaluate" apparently means only

type(a).x.__get __(a, type(a))

since that is semantically getting the value behind x, and so both of the ".x"s in

type(a).x.__set __(a, type(a).x.__get __(a, type(a)).__add_ _(1)) # (or __iadd__ if defined, I think ;-)

don't count as "evaluation " of the "target" x, even though it means that a.x got evaluated twice
(via getattr and setattr, to get the same descriptor object (which was used two different ways)).

I think the normal, non-descriptor case still results in (optimized) probes for type(a).x.__get __
and type(a).x.__set __ before using a.__dict__['x'].

ISTM also that it's not clear that defining __iadd__ does _not_ prevent the setattr phase from going ahead.
I.e., a successful __iadd__ in-place mutation does not happen "instead" of the setattr.

Regards,
Bengt Richter

[Antoon Pardon]

Op 2005-11-04, Christopher Subich schreef <cs************ ****@spam.subic h.block.com>:

Antoon Pardon wrote:

Except when your default is a list

class foo:
x = [] # default

a = foo()
a.x += [3]

b = foo()
b.x

This results in [3]. So in this case using a class variable x to
provide a default empty list doesn't work out in combination
with augmented operators.
This has nothing to do with namespacing at all,

Yes it has.
it's the Python
idiosyncracy about operations on mutable types. In this case, +=
mutates an object, while + returns a new one -- as by definition, for
mutables.

It is the combination of the two.

If python had chosen for an approach like function namespaces, the
problem wouldn't have occured either. What would have happened then
is that the compilor would have noticed the a.x on the right hand
side and based on that fact would then have deciced that all a.x
references should be instance reference (at least in that function
block). The a.x += ... would then result in an AttributeError being raised.

You may prefer the current behaviour over this, but that is not the
point. The point is that resolution of name spaces does play its
role in this problem.
It also has little to do with mutable vs immutable types.
Someone could implement an immutable type, but take advantage
of some implemtation details to change the value inplace
in the __iadd__ method. Such an immutable type would show
the same problems.

--
Antoon Pardon

[Antoon Pardon]

Op 2005-11-04, Steven D'Aprano schreef <st***@REMOVETH IScyber.com.au> :

On Fri, 04 Nov 2005 09:03:56 +0000, Antoon Pardon wrote:

Op 2005-11-03, Steven D'Aprano schreef <st***@REMOVETH IScyber.com.au> :

On Thu, 03 Nov 2005 13:01:40 +0000, Antoon Pardon wrote:

> Seems perfectly sane to me.
>
> What would you expect to get if you wrote b.a = b.a + 2?

I would expect a result consistent with the fact that both times
b.a would refer to the same object.

class RedList(list):
colour = "red"

L = RedList(())

What behaviour would you expect from len(L), given that L doesn't have a
__len__ attribute?
Since AFAICT there is no single reference to the __len__ attribute that
will be resolved to two different namespace I don't see the relevance.

Compare:

b.a += 2

Before the assignment, instance b does not have an attribute "a", so class
attribute "a" is accessed. You seem to be objecting to this inheritance.

I object to the inheritance in a scope where b.a also refers to the
instance.

If there is no problem that a reference can refer to different objects
in the same scope, then the following should work too.

a = 0
def f():
a += 2

One can reason just the same that before the assignment f doesn't have
a local variable yet, so the global should be accessed. People who
don't agree don't want functions to have access to outer scope
variables.
Do you object to import searching multiple directories?

Why do you object to attribute resolution searching multiple namespaces?
I don't.

I don't see the relevance of these pieces of code. In none of them is
there an occurence of an attribute lookup of the same attribute that
resolves to different namespaces.

Look a little more closely. In all three pieces of code, you have a
conflict between the class attribute 'ls' and an instance attribute 'ls'.

No you look a little more clearly.
In the first scenario, that conflict is resolved by insisting that
instances explicitly define an attribute, in other words, by making
instance attribute ONLY search the instance namespace and not the class
namespace.

No it isn't. You seem unable to make a difference between a resolution
in general, and a resolution in a scope where an assignment has been
made.

--
Antoon Pardon