Class Variable Access and Assignment

[Graham]

This has to do with class variables and instances variables.

Given the following:

<code>

class _class:
var = 0
#rest of the class

instance_b = _class()

_class.var=5

print instance_b.var # -> 5
print _class.var # -> 5

</code>

Initially this seems to make sense, note the difference between to last
two lines, one is refering to the class variable 'var' via the class
while the other refers to it via an instance.

However if one attempts the following:

<code>

instance_b.var = 1000 # -> _class.var = 5
_class.var = 9999 # -> _class.var = 9999

</code>

An obvious error occurs. When attempting to assign the class variable
via the instance it instead creates a new entry in that instance's
__dict__ and gives it the value. While this is allowed because of
pythons ability to dynamically add attributes to a instance however it
seems incorrect to have different behavior for different operations.

There are two possible fixes, either by prohibiting instance variables
with the same name as class variables, which would allow any reference
to an instance of the class assign/read the value of the variable. Or
to only allow class variables to be accessed via the class name itself.

Many thanks to elpargo and coke. elpargo assisted in fleshing out the
best way to present this.

perhaps this was intended, i was just wondering if anyone else had
noticed it, and if so what form would you consider to be 'proper'
either referring to class variables via the class itself or via
instances of that class. Any response would be greatly appreciated.
Graham

[Paul Rubin]

Steven D'Aprano <st***@REMOVETH IScyber.com.au> writes:

Then you don't approve of inheritance? That's fine, it is your choice, but
as far as I know, all OO languages include inheritance.

Some OO languages only implement inheritance for method calls. Class
variables don't get inherited.

[Mike Meyer]

Antoon Pardon <ap*****@forel. vub.ac.be> writes:

What would you expect to get if you wrote b.a = b.a + 2? I would expect a result consistent with the fact that both times
b.a would refer to the same object.

Except they *don't*. This happens in any language that resolves
references at run time. Changing that would be changing a fundamental
- and *important* - feature of Python. Arbitrary restrictions to
prevent a single case of this from doing something people who aren't
used to suvh behavior are kludges, and would constitute a wart on the
language, pure and simple.

If you think this is bad, you should consider Jensen's device. It uses
call-by-name, which Python doesn't have. This defers evauation of an
argument until the actual use of the argument. You can fake it in
python by passing strings that you eval and/or lambdas that you call -
I'm not going to work out the details.

Here's the function:

real procedure SIGMA(x, i, n);
value n;
real x; integer i, n;
begin
real s;
s := 0;
for i := 1 step 1 until n do
s := s + x;
SIGMA := s;
end

The object referred to by "x" in the body of the loop is different
*every pass through the loop*, at least in the expected usage. That
usage is SIGMA(a(i), i, 10). Call-by-name evaluates a(i) when x is
mentioned. Since i changes with each pass through the loop, a(i) is a
different element in the array. So SIGMA(a(i), i, 10) is sum(a[:10])
(give or take). But SIGMA(a(i) * b(i), i, 10) is sum([a(i) * b(i) for
i in xrange(10)]) (more or less).

This isn't insane, it's a feature. It's a *very powerful*
feature. Yes, it causes behavior that's unexpected and appears to be
wrong to people who aren't used to resolving names at run time. They
need to get used to it. Then they can start taking advantage of it.
I think it even less sane, if the same occurce of b.a refers to two
different objects, like in b.a += 2

That's a wart in +=, nothing less. The fix to that is to remove +=
from the language, but it's a bit late for that.

<mike
--
Mike Meyer <mw*@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Graham]

Thanks to all of you for your reply's i had no idea it would get this
sort of response,
i've read through most of the posts and it seems that its a active
topic.

My question remains however, i suppose i'm not familiar with how this
functions in
other languages, but what is the common way of referring to a class
variable.

is <class>.<var> the norm?
or <instance>.<var > the norm.

I just seems to me that <instance>.<var > shouldn't defer to the class
variable if
an instance variable of the same name does not exists, it should, at
least how i
understand it raise an exception.

Is my thinking way off here?

Graham

[Mike Meyer]

"Graham" <gr***********@ gmail.com> writes:

I just seems to me that <instance>.<var > shouldn't defer to the class
variable if
an instance variable of the same name does not exists, it should, at
least how i
understand it raise an exception.

Is my thinking way off here?

Yes. This behavior is how you get inheritted access to class
variables. Consider:

class Counter(object) :
"A mutable counter."
# implementation elided

class A(object):
instance_count = Counter()
def __init__(self):
self.instance_c ount.increment( )

class B(A):
instance_count = Counter()

This is sufficient to count instances of A and B. If
self.instance_c ount raised an exception, you'd have to do something
like A.instance_coun t in A.__init__, which would mean the __init__ B
inherited from A would do the wrong thing. Currently, you can either
reference class variables by explicit class, or via inheritance, and
both behaviors are desirable. If you're going to disallow
self.class_vari able, you need to come up with a mechanism to replace
the latter behavior.

<mike
--
Mike Meyer <mw*@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Steven D'Aprano]

On Thu, 03 Nov 2005 10:01:04 -0800, Paul Rubin wrote:

Steven D'Aprano <st***@REMOVETH IScyber.com.au> writes:

Then you don't approve of inheritance? That's fine, it is your choice, but
as far as I know, all OO languages include inheritance.

Some OO languages only implement inheritance for method calls. Class
variables don't get inherited.

Fascinating. Which languages, and what is the reasoning behind that?

I bet they are languages that force the class designer to spend half their
time writing setters and getters. Am I right?
--
Steven.

[Graham]

Many thanks your explaination cleared up many of the questions I had.
I know think i can understand the purpose, regardless of my opinion, i
do however think that one should be able to assign the value in the
same way it is accessed.
Given your previous example:

class Counter(object) :
"A mutable counter."
# implementation elided class A(object):
instance_count = Counter()
def __init__(self):
self.instance_c ount.increment( )

if you changed class A(object):
instance_count = Counter()
def __init__(self):
self.instance_c ount.increment( )
to
class A(object):
instance_count = 0
def __init__(self):
self.instance_c ount = self.instance_c ount + 1

It would not work as planned. I understand all the reasons why this
occurs, but i dont understand why its implemented this way. Because it
acts in a different way than you expect. It seems to me that
self.instance_c ount should not create a new entry in the __dict__ if a
class variable of that name is already present anywhere in that objects
hierarchy.

Does that make sense?

Again thank you for explaination.

graham

[Bengt Richter]

On 3 Nov 2005 12:20:35 GMT, Antoon Pardon <ap*****@forel. vub.ac.be> wrote:

Op 2005-11-03, Stefan Arentz schreef <st***********@ gmail.com>:

Antoon Pardon <ap*****@forel. vub.ac.be> writes:

Op 2005-11-03, Steven D'Aprano schreef <st***@REMOVETH IScyber.com.au> :

>> There are two possible fixes, either by prohibiting instance variables
>> with the same name as class variables, which would allow any reference
>> to an instance of the class assign/read the value of the variable. Or
>> to only allow class variables to be accessed via the class name itself.
>
> There is also a third fix: understand Python's OO model, especially
> inheritance, so that normal behaviour no longer surprises you.

No matter wat the OO model is, I don't think the following code
exhibits sane behaviour:

class A:
a = 1

b = A()
b.a += 2
print b.a
print A.a

Which results in

3
1

I find it confusing at first, but I do understand what happens :-)

I understand what happens too, that doesn't make it sane behaviour.

But really, what should be done different here?

I don't care what should be different. But a line with only one
referent to an object in it, shouldn't be referring to two different
objects.

In the line: b.a += 2, the b.a should be refering to the class variable
or the object variable but not both. So either it could raise an
attribute error or add two to the class variable.

Sure one could object to those sematics too, but IMO they are preferable
to what we have now.

A somewhat similar name space problem, where you could argue
that "a" prior to += should be seen as defined in the outer scope,
but lookahead determines that a is local to inner, period, so that
is the reference that is used (and fails).

def outer(): ... a = 1
... def inner():
... a += 2
... print a
... print 'outer a', a
... inner()
... print 'outer a', a
... outer()

outer a 1
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 7, in outer
File "<stdin>", line 4, in inner
UnboundLocalErr or: local variable 'a' referenced before assignment

Regards,
Bengt Richter

[Steven D'Aprano]

Graham wrote:

Many thanks your explaination cleared up many of the questions I had.
I know think i can understand the purpose, regardless of my opinion, i
do however think that one should be able to assign the value in the
same way it is accessed.
You mean, like this?

# access the value of an instance attribute:
x = instance.name

# assign the value of an instance attribute:
x = instance.name

No, of course that's not what you *meant* -- but it is
what you said. So what do you actually mean? I'll
assume you mean something like "the scoping rules for
access and assignment should be the same".

The problem is, they are. When you access an attribute,
Python looks in the instance scope, and if that lookup
fails, it looks in the class scope. When you assign an
attribute, at least ignoring the complication of slots,
Python uses the same scoping rules: assign to the
instance scope, WHICH ALWAYS SUCCEEDS, and if it fails,
assign to the class scope.

In fact, Python may not even bother to include code to
push the assignment to the class, since the assignment
at the instance level is guaranteed to either succeed,
or fail in such a way that you have no choice but to
raise an exception. But conceptually, assignment and
access are using the same scoping rules.

(Again, I stress that new-style classes with slots may
do things differently.)

Now that I've demonstrated that what you want is not
either "assignment and access should be the same", nor
"the scoping rules should be the same", can you
describe precisely what you do want?
See below for further details.

Given your previous example:

class Counter(object) :
"A mutable counter."
# implementation elided

class A(object):
instance_count = Counter()
def __init__(self):
self.instance_c ount.increment( )

if you changed

class A(object):
instance_count = Counter()
def __init__(self):
self.instance_c ount.increment( )

to

class A(object):
instance_count = 0
def __init__(self):
self.instance_c ount = self.instance_c ount + 1

It would not work as planned.

Why not?

For starters, what is the plan? Do you want all
instances of class A to share state? Are all instances
of Counter supposed to share state?

Depending on whether you want the answers of those to
be Yes or No, you would pick one technique or the
other. Sometimes you want to increment mutables in
place, and sometimes you don't.

But I would suggest very strongly that in general, you
usually don't want instances to share state.

I understand all the reasons why this
occurs, but i dont understand why its implemented this way. Because it
acts in a different way than you expect. It seems to me that
self.instance_c ount should not create a new entry in the __dict__ if a
class variable of that name is already present anywhere in that objects
hierarchy.

But that would stop inheritance from working the
expected way.

In standard OO programming, you expect instances to
inherit behaviour from their class (and superclasses)
unless over-ridden. This lets you do something like this:

class Paper:
size = A4

Now all instances of Paper are created with a default
size of A4 -- they inherit that size from the class.

If you are localising your application for the US
market, you simply change the class attribute:

Paper.size = USLetter

and all the instances that inherit from the class will
now reflect the new default.

Now suppose you have a specific instance that needs a
different paper size:

instance = Paper()
instance.size = Foolscap

What do you expect should happen? Should all Paper
instances suddenly be foolscap size, or just the one?
If you say "just the one", then you want the current
behaviour. If you say "all of them", then you want
shared state -- but do you really want all class
instances, all the time, to have shared state?
You can get ride of that behaviour by getting rid of
inheritance, or at least inheritance of non-method
attributes. Then you have to write code like this:

class PrintableThing:
"""Prints a Thing object with prefix and suffix.
Customize the prefix and suffix by setting the
appropriate instance attributes.
"""

prefix = "START "
suffix = " STOP"

def __str__(self):
try:
# access the instance attributes,
# if they exist
prefix = self.prefix
suffix = self.suffix
except AttributeError:
# fall back to class attributes
prefix = self.__class__. prefix
suffix = self.__class__. suffix
# have you spotted the subtle bug in this code?
return prefix + self.thing + suffix

instead of:

class PrintableThing:
def __str__(self):
return self.prefix + self.thing + self.suffix
Even worse would be the suggestion that Python allowed
accessing instance.attrib ute to refer to either a class
or instance attribute, decided at runtime as it does
now, but *remembered* which it was so that assignment
went back to the same object.

That would mean that class attributes would mask
instance attributes -- or vice versa, depending on
which was created first. I assume that in general,
class attributes would be created before instances.

If you had a class with a default attribute, like
Paper.size above, you couldn't over-write it at the
instance level because instance.size would always be
masked by class.size. You would need to write code like
this:

class Paper:
default_size = A4

def __init__(self, size=None):
self.size = size

def print(self):
if self.size is None:
papersize = self.__class__. default_size
else:
papersize = self.size
do_something(pa persize)
The standard inheritance model used by Python and all
OO languages I know of is, in my opinion, the optimal
model. It gives you the most convenient behaviour for
the majority of cases, and in those few cases where you
want non-standard behaviour (e.g. shared state) it is
easy to do with some variant of self.__class__. attribute.
--
Steven.

[Mike Meyer]

"Graham" <gr***********@ gmail.com> writes:

Many thanks your explaination cleared up many of the questions I had.
I know think i can understand the purpose, regardless of my opinion, i
do however think that one should be able to assign the value in the
same way it is accessed.
That's not true in lots of cases, and would be a serious restriction
on the language. The rules for where a binding takes place are the way
the are for good reason. Restricting lookups to those rules would make
a number of things more difficult, and would make some features
(cough-closures-cough) nearly useless.
Given your previous example:

class Counter(object) :
"A mutable counter."
# implementation elided

class A(object):
instance_count = Counter()
def __init__(self):
self.instance_c ount.increment( )

if you changed

class A(object):
instance_count = Counter()
def __init__(self):
self.instance_c ount.increment( )

to

class A(object):
instance_count = 0
def __init__(self):
self.instance_c ount = self.instance_c ount + 1

It would not work as planned. I understand all the reasons why this
occurs, but i dont understand why its implemented this way. Because it
acts in a different way than you expect.

No, it acts in a different way than *you* expect. It does exactly what
I expect, which is why I didn't write it that way.
It seems to me that
self.instance_c ount should not create a new entry in the __dict__ if a
class variable of that name is already present anywhere in that objects
hierarchy.

Does that make sense?
Yes, but such behavior would make Python worse, not better. Right
now, binding instance.name *always* works(*) on the value of the name
attribute of instance. Creating a special case for when one of the
classes instance belongs to happens to have an attribute "name" would
be less consistent than the current behavior. Yes, the current
behavior is surprising to people who aren't used to dynamic
languages. But there are lots of such things - they're part of the
power of dynamic languages. People who want to program in dynamic
languages just have to get used to those things. That's a lesser price
to pay than having a language cluttered with special cases just to
avoid surprising people who aren't used to the language yet.
Again thank you for explaination.

You're welcome.

<mike

*) I almost said "binds", but that's not true. Attempting to bind an
attribute can invoke arbitrary code - but it's got the instance and
attribute name to work with. If you really wanted to, you could create
a class for which trying to set an attribute behaved as you wanted. Be
warned - it's not as easy as it looks.
--
Mike Meyer <mw*@mired.or g> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Steve Holden]

Antoon Pardon wrote:

Op 2005-11-03, Stefan Arentz schreef <st***********@ gmail.com>:

Antoon Pardon <ap*****@forel. vub.ac.be> writes:

...

>No matter wat the OO model is, I don't think the following code
>exhibits sane behaviour:
>
>class A:
> a = 1
>
>b = A()
>b.a += 2
>print b.a
>print A.a
>
>Which results in
>
>3
>1

I find it confusing at first, but I do understand what happens :-)

I understand what happens too, that doesn't make it sane behaviour.
But really, what should be done different here?

I don't care what should be different. But a line with only one
referent to an object in it, shouldn't be referring to two different
objects.
It doesn't.

Yes it does. If the b.a refers to the instance variable, then an
AttributeError should be raised, because the instance variable doesn't
exist yet, so you can't add two to it.

Excuse me. The statement

a += 2

causes a to refer to a different object after the assignment than it did
before. So does the statement

self.a += 2

So why are you so concerned that the pre-assignment reference comes from
a different scope to the post-assignment reference? The fact remains
that after both assignments the rebound name can no longer (ever) be
used to refer to its former referent without a further rebinding taking
place.
If the b.a refers to the class variable then two should be added to it.
Wring, wring, wring. (Sorry, got that wrong :-)
Neither happens instead we get some hybrid in which an instance varible
is created that gets the value of class variable incrented by two.
Yes. So does this mean you also have a problem with

def f(x):
x += 2

g = 3
print f(g)

When the function call executes, the name x is bound to an object in the
call's containing scope. Is it then your contention that the augmented
assignment in the function should add two to that object, changing the
value of g?

For extra marks please explain the difference between augmented
assignment to a function argument and augmented assignment to a class
variable referenced through self.

In the line: b.a += 2, the b.a should be refering to the class variable
or the object variable but not both. So either it could raise an
attribute error or add two to the class variable. I'm not sure where this moral imperative comes from, and your arguments
singularly fail to convince me.
It does exactly what you say. It adds 2 to the a *instance variable* of
the object instance in 'b'.

There is no instance variable at that point. How can it add 2, to
something that doesn't exist at the moment.

It doesn't, it simply proceeds along the lines of all Python assignments
and resolves the name as a reference to a specific object. It then
computes a new value from the referenced object and the augmented
assignment operator's right operand, and rebinds the name to the
newly-computed value.

Please stop talking about variables.

Although augmented assignment operators have the *option* of updating
objects in place, surely not even you can require that they do so when
they are bound to an immutable object such as an integer.

It doesn't touch the *class variable* A.a which is still 1.

Why "should" it? Why, why, why? And gain, just for good measure, why?
Augmented assignment to a function argument doesn't modify the passed
object when immutable, and you have no problem with that (I know as I
write that this is just asking for trouble, and it will turn out that
you also find that behavior deeply controversial ...)

But it accesses the class variable.

Repeat after me: "Python assignment binds values to names".

When I write

class something:
a = 1
def __init__(self, val=None):
if val:
self.a += val

then in the last statement the augmented assignment rebinds "self.a"
from the class "variable" to a newly-created instance "variable". I am
of course using the word "variable" here in a Pythonic sense, rather
than in the sense that, say, a C programmer would use. In Python I
prefer to talk about binding names because talking of variables leads
people to expect that a name is bound to an area of memory whose value
is modified by assignment, but this is in fact not so.

The initial access to self.a uses the defined name resolution order to
locate a value that was bound to the name "a" in class scope. So what?
This is a long-documented fact of Python life. It's *supposed* to be
that way, dammit.

I fail to understand why this is such a problem for you. But then it's
clear from long past experience that our perceptions of Python's
execution model differ quite radically, and that I seem to find it quite
satisfactory overall, whereas you are forever banging on about what
"should" be true of Python and what Python "should" do. Which, as is
probably obvious by now, I sometimes find just a teeny bit irritating.
Kindly pardon my tetchiness.

I suppose ultimately I'm just more pragmatic than you. Plus I started
using Icon, whose assignment semantics are very similar, back in the
1970's, so Python's way of doing things fits my brain quite nicely,
thank you.

regards
Steve

PS As a total non-sequitur added for light relief at the end of what
seems even to me to be a slightly tedious post, I discover I managed to
misspell "assignment " in four distinct ways during the composition of
the above.
--
Steve Holden [phone] [phone]
Holden Web LLC www.holdenweb.com
PyCon TX 2006 www.python.org/pycon/