Class Variable Access and Assignment

[Graham]

This has to do with class variables and instances variables.

Given the following:

<code>

class _class:
var = 0
#rest of the class

instance_b = _class()

_class.var=5

print instance_b.var # -> 5
print _class.var # -> 5

</code>

Initially this seems to make sense, note the difference between to last
two lines, one is refering to the class variable 'var' via the class
while the other refers to it via an instance.

However if one attempts the following:

<code>

instance_b.var = 1000 # -> _class.var = 5
_class.var = 9999 # -> _class.var = 9999

</code>

An obvious error occurs. When attempting to assign the class variable
via the instance it instead creates a new entry in that instance's
__dict__ and gives it the value. While this is allowed because of
pythons ability to dynamically add attributes to a instance however it
seems incorrect to have different behavior for different operations.

There are two possible fixes, either by prohibiting instance variables
with the same name as class variables, which would allow any reference
to an instance of the class assign/read the value of the variable. Or
to only allow class variables to be accessed via the class name itself.

Many thanks to elpargo and coke. elpargo assisted in fleshing out the
best way to present this.

perhaps this was intended, i was just wondering if anyone else had
noticed it, and if so what form would you consider to be 'proper'
either referring to class variables via the class itself or via
instances of that class. Any response would be greatly appreciated.
Graham

[Antoon Pardon]

Op 2005-11-04, Christopher Subich schreef <cs************ ****@spam.subic h.block.com>:

Antoon Pardon wrote:

Op 2005-11-03, Stefan Arentz schreef <st***********@ gmail.com>:

The model makes sense in my opinion. If you don't like it then there are
plenty of other languages to choose from that have decided to implement
things differently.

And again this argument. Like it or leave it, as if one can't in general
like the language, without being blind for a number of shortcomings.

It is this kind of recations that make me think a number of people is
blindly devoted to the language to the point that any criticism of
the language becomes intollerable.

No, it's just that a goodly number of people actually -like- the
relatively simple conceputal model of Python.

Why /shouldn't/

a.x = foo
correspond exactly to
setattr(a,'x ',foo) #?
Similarly, why shouldn't
foo = a.x
correspond exactly to
foo = getattr(a,'x') #?
With that in mind, the logical action for
a.x = f(a.x)
is
setattr(a,'x ',f(a,'x')) #,
and since
a.x += foo
is equal to
a.x = A.__iadd__(a.x, foo) # (at least for new-style classes
# that have __iadd__ defined. Otherwise, it falls back on
# __add__(self,ot her) to return a new object, making this
# evern more clear),
why shouldn't this translate into
setattr(a,'x ',A.__iadd__(ge tattr(a,'x'),fo o)) #?

Well maybe because as far as I understand the same kind of logic
can be applied to something like

lst[f()] += foo

In order to decide that this should be equivallent to

lst[f()] = lst[f()] + foo.

But that isn't the case.

So it seems applying augmented operators is not a matter of just
substituting straight translations to get the right result.

Looking at it this way, it's obvious that the setattr and getattr may do
different things, if the programmer understands that "instances (can)
look up object attributes, and (always) set instance attributes." In
fact, it is always the case (so far as I can quickly check) that += ends
up setting an instance attribute. Consider this code:

Looking at lists in a similar way, it would be obvious that the
__setitem__ and __getitem__ can do different things and so we
should expect lst[f()] += foo to behave exactly as lst[f()] = lst[f()] +
foo.

--
Antoon Pardon

[Antoon Pardon]

Op 2005-11-04, Christopher Subich schreef <cs************ ****@spam.subic h.block.com>:

Antoon Pardon wrote:

Op 2005-11-03, Stefan Arentz schreef <st***********@ gmail.com>:

The model makes sense in my opinion. If you don't like it then there are
plenty of other languages to choose from that have decided to implement
things differently.

class foo:
x = [5]
a = foo()
a += [6]
a.x [5,6] foo.x [5,6] foo.x = [7]
a.x

[5,6]

In truth, this all does make perfect sense -- if you consider class
variables mostly good for "setting defaults" on instances.

Except when your default is a list

class foo:
x = [] # default

a = foo()
a.x += [3]

b = foo()
b.x

This results in [3]. So in this case using a class variable x to
provide a default empty list doesn't work out in combination
with augmented operators.

This however would work:

class foo:
x = [] # default

a = foo()
a.x = a.x + [3]

b = foo()
b.x

This results in []

--
Antoon Pardon

[Steven D'Aprano]

On Fri, 04 Nov 2005 09:03:56 +0000, Antoon Pardon wrote:

Op 2005-11-03, Steven D'Aprano schreef <st***@REMOVETH IScyber.com.au> :

On Thu, 03 Nov 2005 13:01:40 +0000, Antoon Pardon wrote:

Seems perfectly sane to me.

What would you expect to get if you wrote b.a = b.a + 2?

I would expect a result consistent with the fact that both times
b.a would refer to the same object.

class RedList(list):
colour = "red"

L = RedList(())

What behaviour would you expect from len(L), given that L doesn't have a
__len__ attribute?

Since AFAICT there is no single reference to the __len__ attribute that
will be resolved to two different namespace I don't see the relevance.

Compare:

b.a += 2

Before the assignment, instance b does not have an attribute "a", so class
attribute "a" is accessed. You seem to be objecting to this inheritance.

len(L) => L.__len__()

Instance L also does not have an attribute "__len__", so class attribute
"__len__" is accessed. You don't appear to object to this inheritance.

Why object to one and not the other?

If you object to b.a resolving to b.__class__.a, why don't you object to
L.__len__ resolving to L.__class__.__l en__ also?

Perhaps you don't object to that half of the problem. Perhaps you object
to the assignment: you expect that assigning to b.a should assign to
b.__class__.a instead.

Should assigning to L[0] assign to L.__class__[0] also, so that all
lists share not only the same behaviour, but also the same data?
[snip]

b is a name, and any reference to b (in the same namespace) will refer
to the same object. At least until you rebind it to another object.

But some namespaces take great care not to allow a rebinding that would
result in the same name being resolved to a different namespace during
this namespace's lifetime.

And some take great care to allow such a rebinding, because that is the
right thing to do to make inheritance work correctly.

But b.a is not a name, it is an attribute lookup,

An other implementation detail. b.a is a name search of 'a' in the
namespace b.

Factually incorrect. b.a is the name search for 'a' in the namespaces
[note plural] of b, b.__class__, and any superclasses of b, *in that order*.

Do you object to import searching multiple directories?

Why do you object to attribute resolution searching multiple namespaces?
[snip]

I think it even less sane, if the same occurce of b.a refers to two
different objects, like in b.a += 2

Then it seems to me you have some serious design problems. Which would
you prefer to happen?

# Scenario 1
# imaginary pseudo-Python code with no inheritance: class Paragraph:
ls = '\n' # line separator

para = Paragraph()
para.ls

=> AttributeError - instance has no attribute 'ls'
# Scenario 2
# imaginary pseudo-Python code with special inheritance:
class Paragraph:
ls = '\n' # line separator

linux_para = Paragraph()
windows_para = Paragraph()
windows_para.ls = '\n\r' # magically assigns to the class attribute
linux_para.ls

=> prints '\n\r'

# Scenario 3
# Python code with standard inheritance:
class Paragraph:
ls = '\n' # line separator

linux_para = Paragraph()
windows_para = Paragraph()
windows_para.ls = '\n\r'
linux_para.ls

=> prints '\n'

I don't see the relevance of these pieces of code. In none of them is
there an occurence of an attribute lookup of the same attribute that
resolves to different namespaces.

Look a little more closely. In all three pieces of code, you have a
conflict between the class attribute 'ls' and an instance attribute 'ls'.

In the first scenario, that conflict is resolved by insisting that
instances explicitly define an attribute, in other words, by making
instance attribute ONLY search the instance namespace and not the class
namespace.

In the second scenario, that conflict is resolved by insisting that
instance.name assigns to instance.__clas s__.name, just as you asked for.

The third scenario is the way Python actually operates.

--
Steven.

[Christopher Subich]

Antoon Pardon wrote:

Since ints are immutable objects, you shouldn't expect the value of b.a
to be modified in place, and so there is an assignment to b.a, not A.a.

You are now talking implementation details. I don't care about whatever
explanation you give in terms of implementation details. I don't think
it is sane that in a language multiple occurence of something like b.a
in the same line can refer to different objects

This isn't an implementation detail; to leading order, anything that
impacts the values of objects attached to names is a specification issue.

An implementation detail is something like when garbage collection
actually happens; what happens to:

b.a += 2

is very much within the language specification. Indeed, the language
specification dictates that an instance variable b.a is created if one
didn't exist before; this is true no matter if type(b.a) == int, or if
b.a is some esoteric mutable object that just happens to define
__iadd__(self,t ype(other) == int).

[Christopher Subich]

Antoon Pardon wrote:

Well I wonder. Would the following code be considered a name binding
operation:

b.a = 5

Try it, it's not.

Python 2.2.3 (#1, Nov 12 2004, 13:02:04)
[GCC 3.2.3 20030502 (Red Hat Linux 3.2.3-42)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.

a Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'a' is not defined b = object()
b.a

Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'object' object has no attribute 'a'

Once it's attached to an object, it's an attribute, not a base name.
The distinction is subtle and possibly something that could (should?) be
unified for Py3k, but in cases like this the distinction is important.

[Magnus Lycka]

Graham wrote:

My question remains however, i suppose i'm not familiar with how this
functions in
other languages, but what is the common way of referring to a class
variable.

is <class>.<var> the norm?
or <instance>.<var > the norm.
It's not always that easy, due to inheritance. You might want
the <var> defined in the class where a method you define now
is implemented (A.<var> if we're in a method defined in class A),
or you might want <var> in the class of the instance object
(which could be a subclass of A). You can get that through
self.__class__. <var>, so I guess you could always manage without
Python searching in the class scope after searching the instance
scope, if it wasn't for the problem below...
I just seems to me that <instance>.<var > shouldn't defer to the class
variable if
an instance variable of the same name does not exists, it should, at
least how i
understand it raise an exception.

So, you want this:

class A: .... def f(self):
.... print 'Hello'
.... a = A()
a.f() Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: A instance has no attribute 'f'

A bit boring that we need to make method calls
with a.__class__.f(a ) in your Python...

Python is more consistent than you have thought...
You know, it could be that we want to assign the
method to another variable, as in:

o = A()
o_f = o.f # This might look as I'm getting a normal
# attribute, but f is a method
for i in range(gazillion ):
o_f() # This is slightly faster than o.f()

Or, we might want to do:
o.f=len
o.f('Hello')

Here, o.f is no longer a method in o's class hierarchy, but
it's still callable.

If you think about it, you'll understand that in such a dynamic
language as Python, there is no way that the interpreter can
know before lookup whether it will find a method or a simple
attribute. If it's going to look in different places depending
on what it will find when it has looked...we have a Catch 22.

Normal methods in Python are defined in the scope of the class,
and they are passed the instance object as their first argument
when they are called. The call (where the instance object is
implicitly called in case of a bound method) is something that
comes after the lookup, as you can see in the a_f() example.

Python is *very* dynamic. The behaviour of the class can change
after the instance has been created.
class A: .... def f(self):
.... print 'Hello'
.... a = A()
a.f() Hello del A.f
a.f()

Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: A instance has no attribute 'f'

How would this work if 'a.f' doesn't cause a lookup in A if it's
missing in a? Do you want a.f to first search the instance a, then
the class A, and if it finds f in A, issue an AttributeError if it
turns out that f isn't a method?

[Magnus Lycka]

Antoon Pardon wrote:

I have looked and didn't find it in the language reference.

This is what I have found:

An augmented assignment expression like x += 1 can be rewritten
as x = x + 1 to achieve a similar, but not exactly equal effect.
It's just a little further down. I'll post the quote once more (but
this is the last time ;^):

"""For targets which are attribute references, the initial value is
retrieved with a getattr() and the result is assigned with a setattr().
Notice that the two methods do not necessarily refer to the same
variable. When getattr() refers to a class variable, setattr() still
writes to an instance variable. For example:

class A:
x = 3 # class variable
a = A()
a.x += 1 # writes a.x as 4 leaving A.x as 3"""

I'd say it's documented...
That doesn't change the fact that the current behaviour is
on occasions awkward or whatever you want to call it.
I fear that this has to do with the way reality works. Perhaps
it's due to Gödel's incompleteness theorems... :)

Sure, Python has evolved and grown for about 15 years, and
backward compatibility has always been an issue, but the
management and development of Python is dynamic and fairly
open-minded. If there had been an obvious way to change this
in a way that solved more problems than it caused, I suspect
that change would have happened already.
I also find that people underestimate the magic that
is going on in python. But just because you are familiar with
the magic, doesn't make it less magic. IMO python shows its
history a little.
If Guido would design a new language today, that was aiming at
the kind of tasks Python solves, I'm sure it wouldn't be identical
to the current Python. The languages I'm most experienced in besides
Python are C++ and SQL. Compared to those beasts, Python is a wonder
in clarity and consistency. I don't think a single vendor has managed
to fully implement the SQL standard, and it's known that the standard
contains bugs, inconsitencies and gaps, even though (or because) it's
been worked on for more than 20 years. The C++ spec is marginally
better. Of course, there isn't a formal specification for Python. I
don't know if the language reference is so complete that someone
could actually write another really compatible Python implementation
based on just the reference manual. Still the difference is drastic.
I suspect that only few people in the SQL standard committee fully
understand the spec (maybe C.J. Date and Hugh Darwen does) and all
the things happening under the hood in C++ is staggering, considering
that this language is really just a fancy assmbler that can't even
manage memory for the programmer!
I'm after nothing particular. The only thing I'm frustrated about
is the way in which some people seem willing to defend python
just because it is python. If the only reaction I would have
gotten would have been something like: Yeah that seems a bit
awkward but fixing this would break more than it would cure,
I would have left it as it is.
That's probably what most people think, but we're not entirely
rational. We're human. An emotional posting will probably attract
equally emotional responses.
I have rarely indicated I wanted things to be fixed. Sure I would
like it if some things were different, but I recognize that there
are more important things that needs to be resolved.

Does that mean I shouldn't mention things that IMO could have been
better or that I should only mention them in the softest of
language that certainly can't be interpreted as emotional language.

Personally, I want comp.lang.pytho n to work as a way for me to learn
new things about Python, to get help if I'm stuck with something, and
as a way for me to inform others about Python stuff. It's also a part
of the Python community--an arena where I communicate with other
Pythonistas. This is something important for me, both professionally
and personally. I try to think an extra time before I post messages.
It this message meaningful? Does it add anything of value? Am I just
repeating what is already said? Is this message likely to have some
kind of positive impact or will it just be ignored? Might I hurt
someone? Am I building useful relationships?

When it works as it should, people you've never met before will buy
you beer or lunch when you happen to be in their neighbourhood. At
least they might come up to you and chat if you go to a Python
conference. They might also offer you jobs or contracts etc. This
is really nice and valuable. Something to handle with care.

[Christopher Subich]

Antoon Pardon wrote:

Except when your default is a list

class foo:
x = [] # default

a = foo()
a.x += [3]

b = foo()
b.x

This results in [3]. So in this case using a class variable x to
provide a default empty list doesn't work out in combination
with augmented operators.

This has nothing to do with namespacing at all, it's the Python
idiosyncracy about operations on mutable types. In this case, +=
mutates an object, while + returns a new one -- as by definition, for
mutables.

[Christopher Subich]

Antoon Pardon wrote:

Well maybe because as far as I understand the same kind of logic
can be applied to something like

lst[f()] += foo

In order to decide that this should be equivallent to

lst[f()] = lst[f()] + foo.

But that isn't the case.

Because, surprisingly enough, Python tends to evaluate expressions only
once each time they're invoked.

In this case, [] is being used to get an item and set an item --
therefore, it /has/ to be invoked twice -- once for __getitem__, and
once for __setitem__.

Likewises, lst appears once, and it is used once -- the name gets looked
up once (which leads to a += 1 problems if a is in an outer scope).

f() also appears once -- so to evaluate it more trhan one time is odd,
at best.

If you know very much about modern lisps, it's similar to the difference
between a defun and a defmacro.

[Steve Holden]

Antoon Pardon wrote:

Op 2005-11-04, Steve Holden schreef <st***@holdenwe b.com>:

Antoon Pardon wrote:

Op 2005-11-04, Steve Holden schreef <st***@holdenwe b.com>:
[...]

I suppose ultimately I'm just more pragmatic than you.
It has nothing to do with being more pragmatic. Being pragmatic
is about how you handle things with real life projects. It has
little to do with the degree in which you agree with the design
of the tool you have to work with. I would say I am more pragmatic
than most defenders of python, because when it comes done to
do my work, I just use python as best as I can, while a lot
of people here seem to think that every little criticism I have
is enough to go and look for a different language.

No, being pragmatic is to do with accepting what is rather than wasting
time wishing it were otherwise,

Just accepting what is, is not pragmatic. Not much progress would have
been made if we just accepted what is.

Pragmatically, I accept that whatever I say you are likely to respond
with a nit-picking argument. Pragmatically I accept this situation for
what it is. This does not stop me imagining a world where our dialogues
are about meaningful issues rather than whether a particular facet of
Python's design is "unsane". Pragmatically I accept that such a world is
likely to exist only in my imagination.

particularl y when the "insane" behavior
was actually a deliberate design choice. Which is why it doesn't work
the same as non-local references in nested scopes.

That b.a = b.a + 2

works as a result of a design choice, that I can accept.

But IMO b.a += 2, working as it does, is more the result of
earlier design and implementation decisions than it was
a deliberate design decision.

You are wrong.

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC www.holdenweb.com
PyCon TX 2006 www.python.org/pycon/