473,416 Members | 1,559 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,416 software developers and data experts.

Class attributes, instances and metaclass __getattribute__


Hi all
I noticed something strange here while explaining decorators to someone.
Not any real use code, but I think it's worth mentioning.

When I access a class attribute, on a class with a custom metaclass with
a __getattribute__ method, the method is used when acessing some
attribute directly with the class object, but not when you do it from
the instance.

<code type='prompt'>
>>class M(type):
.... def __getattribute__(cls, attr):
.... print cls, attr
.... return type.__getattribute__(cls, attr)
....
>>class C(object):
.... __metaclass__ = M
....
>>C.x = 'foo'
C.x
<class '__main__.C'x
'foo'
>>o = C()
o.x
'foo'
>>>
</code>
Someone at freenode #python channel involved with python-dev sprint
suggested it might be a bug, worth mentioning... to me it seems like a
decision to avoid some problems with method and descriptors creation,
since someone using metaclasses and custom __getattribute__ at the same
time is asking for trouble, but... I googled for it and tried to find
something on the list but, nothing. From the source it seems like a
generic wrapper is used. What's the real case here ?
Regards,

--
Pedro Werneck
Aug 7 '06 #1
4 3274
Pedro Werneck wrote:
When I access a class attribute, on a class with a custom metaclass with
a __getattribute__ method, the method is used when acessing some
attribute directly with the class object, but not when you do it from
the instance.

<code type='prompt'>
>class M(type):
... def __getattribute__(cls, attr):
... print cls, attr
... return type.__getattribute__(cls, attr)
...
>class C(object):
... __metaclass__ = M
...
>C.x = 'foo'
C.x
<class '__main__.C'x
'foo'
>o = C()
o.x
'foo'
>>
</code>
Someone at freenode #python channel involved with python-dev sprint
suggested it might be a bug, worth mentioning... to me it seems like a
decision to avoid some problems with method and descriptors creation,
since someone using metaclasses and custom __getattribute__ at the same
time is asking for trouble, but... I googled for it and tried to find
something on the list but, nothing. From the source it seems like a
generic wrapper is used. What's the real case here ?
Regards,

--
Pedro Werneck
To me, it seems consistent. As said in
http://www-128.ibm.com/developerwork...ary/l-pymeta2/

"""The availability of metaclass attributes is not transitive; in other
words, the attributes of a metaclass are available to its instances,
but not to the instances of the instances. Just this is the main
difference between metaclasses and superclasses."""

Since this happens for real attributes, it looks natural that the same
should
happen for 'virtual' attributes implemented via '__getattr__' or
'__getattribute__'.

Michele Simionato

Aug 8 '06 #2

Hi

On 8 Aug 2006 00:10:39 -0700
"Michele Simionato" <mi***************@gmail.comwrote:
To me, it seems consistent. As said in
http://www-128.ibm.com/developerwork...ary/l-pymeta2/

"""The availability of metaclass attributes is not transitive; in
other words, the attributes of a metaclass are available to its
instances, but not to the instances of the instances. Just this is the
main difference between metaclasses and superclasses."""

Well... I'm not talking about metaclass attributes... that's perfectly
consistent, agreed.

I'm saying that when the class implements a custom __getattribute__,
when you try to access the instance attributes from itself, it uses it.
But if the class is a metaclass, instances of its instances have acess
to the attribute anyway, but don't use the custom __getattribute__ you
implemented.

Like the example I mentioned on the previous mail, or (I think in this
case it's more obvious):
>>class N(type):
.... def __getattribute__(cls, attr):
.... print 'using N.__getattribute for "%s"'%(attr)
.... return type.__getattribute__(cls, attr)
....
>>class M(type):
.... __metaclass__ = N
....
>>class C(object):
.... __metaclass__ = M
....
>>M.x = 'foo'
M.x
using N.__getattribute for "x"
'foo'
>>C.x
'foo'
So, in both cases I have access to the class attribute; but in one case
it's using the bound M.__getattribute__ implemented at the base
metaclass, but not in the other. It was supposed to use it after the
bound 'C.__getattribute__' raises AttributeError as expected, but it
doesn't...

It's inconsistent, but seems a reasonable decision to me since someone
can easily mess with descriptors doing with this... but, someone else
involved with Python dev I talked about thinks it may be a bug.

And, I'm curious anyway... is it possible to customize attribute access
in this case in any other way ? What really happens here ?
>From the typeobject.c source code, seems like when the metaclass
implements __getattribute__, it uses type.__getattribute__ in this case,
but I'm not sure.

Since this happens for real attributes, it looks natural that the same
should happen for 'virtual' attributes implemented via '__getattr__'
or '__getattribute__'.
Well... as I think it's clear now, the case you mentioned is not exactly
what I'm talking about, but it's good you mentioned about 'virtual'
attributes, because in this case we have another problem, it's
inconsistent too and there's no reason for it...
>>class M(type):
.... def __getattr__(cls, attr):
.... if attr == 'x':
.... return 'foo'
....
>>class C(object):
.... __metaclass__ = M
....
>>C.y = 'bar'
C.x
'foo'
>>C.y
'bar'
>>o = C()
>>o.x
....
AttributeError: 'C' object has no attribute 'x'
>>o.y
'bar'
>>>
So... both 'x' and 'y' are class attributes, but 'x' is a virtual
attribute implemented with M.__getattr__. From the instance I have
access to 'y' but not to 'x'.

Regards,

--
Pedro Werneck
Aug 8 '06 #3
Pedro Werneck wrote:
Hi
[snip]
Well... I'm not talking about metaclass attributes... that's perfectly
consistent, agreed.

I'm saying that when the class implements a custom __getattribute__,
when you try to access the instance attributes from itself, it uses it.
But if the class is a metaclass, instances of its instances have acess
to the attribute anyway, but don't use the custom __getattribute__ you
implemented.
Attribute lookup for instances of a class never calls metaclass'
__getattribute__() method. This method is called only when you
access attributes directly on the class.

[snip]
And, I'm curious anyway... is it possible to customize attribute access
in this case in any other way ? What really happens here ?
There are two distinct methods involved in your example; attribute
lookup for classes is controled by metaclass' __getattribute__()
method, while instance attribute lookup is controled by class'
__getattribute__() method.

They are basicaly the same, but they never use ``type(obj).attr`` to
access the class' attributes. The code for these methods would look
something like this in Python:

class Object(object):
"""
Emulates object's and type's behaviour in attribute lookup.
"""

def __getattribute__(self, name):
cls = type(self)

# you normally access this as self.__dict__
try:
dict_descriptor = cls.__dict__['__dict__']
except KeyError:
# uses __slots__ without dict
mydict = {}
else:
mydict = dict_descriptor.__get__(self, cls)

# Can't use cls.name because we would get descriptors
# (methods and similar) that are provided by class'
# metaclass and are not meant to be accessible from
# instances.
classdicts = [c.__dict__ for c in cls.__mro__]

# We have to look in class attributes first, since it can
# be a data descriptor, in which case we have to ignore
# the value in the instance's dict.
for d in classdicts:
if name in d:
classattr = d[name]
break
else:
# None of the classes provides this attribute; perform
# the normal lookup in instance's dict.
try:
return mydict[name]
except KeyError:
# Finally if everything else failed, look for the
# __getattr__ hook.
for d in classdicts:
if '__getattr__' in d:
return d['__getattr__'](self, name)
msg = "%r object has no attribute %r"
raise AttributeError(msg % (cls.__name__, name))

# Check if class' attribute is a descriptor.
if hasattr(classattr, '__get__'):
# If it is a non-data descriptor, then the value in
# instance's dict takes precedence
if not hasattr(classattr, '__set__') and name in mydict:
return mydict[name]
return classattr.__get__(self, cls)

# Finally, look into instance's dict.
return mydict.get(name, classattr)

As you can see, it completely avoids calling metaclass'
__getattribute__()
method. If it wouldn't do that, then the metaclass' attributes would
'leak' to instances of its classes. For example, __name__, __mro__
and mro() are some of the descriptors provided by type to every class,
but they are not accesible through instances of these classes,
and shouldn't be, otherwise they could mask some errors in user's code.

Ziga

Aug 8 '06 #4
On 8 Aug 2006 07:24:54 -0700
"Ziga Seilnacht" <zi************@gmail.comwrote:
[snip]
Well... I'm not talking about metaclass attributes... that's
perfectly consistent, agreed.

I'm saying that when the class implements a custom __getattribute__,
when you try to access the instance attributes from itself, it uses
it. But if the class is a metaclass, instances of its instances have
acess to the attribute anyway, but don't use the custom
__getattribute__ you implemented.

Attribute lookup for instances of a class never calls metaclass'
__getattribute__() method. This method is called only when you
access attributes directly on the class.
Well... thanks for the answer.

As I said on the first mail, I noticed this when I was explaining
descriptors and methods to someone else... I implemented a pure-python
Method class to show him exactly how it works.

But, since their __get__ call is available at the class too, my first
thought was that it was implemented at the metaclass __getattribute__,
and when an instance tries to get a class attribute it would fail on its
own __getattribute__, use the bound method at its class and make the
call.

After implementing a test metaclass I noticed it doesn't work this way,
and even if it's a bit inconsistent (especially in the case of 'virtual'
attributes), it seemed to me the reasonable thing to do, exactly for
what you mentioned on your code... someone could easily break a lot of
stuff doing it the wrong way, instead if not using __dict__.

I mailed the list because someone else thought it might be a bug and I
was in doubt... now it's clear it was the right thing to do.
Regards,

--
Pedro Werneck
Aug 8 '06 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
by: Ruud de Jong | last post by:
I have the situation where I need to construct the name of a static method, and then retrieve the corresponding function from a class object. I thought I could just use __getattribute__ for this...
50
by: Dan Perl | last post by:
There is something with initializing mutable class attributes that I am struggling with. I'll use an example to explain: class Father: attr1=None # this is OK attr2= # this is wrong...
7
by: Carlos Ribeiro | last post by:
I'm looking for ways to load new class definitions at runtime (I'm not talking about object instances here, so a persistent object database isn't what I am looking for ). I'm aware of a few...
3
by: Sylvain Ferriol | last post by:
hello when i define __getattribute__ in a class, it is for the class instances but if i want to have a __getattribute__ for class attributes how can i do that ? sylvain
5
by: Laszlo Zsolt Nagy | last post by:
Hughes, Chad O wrote: > Is there any way to create a class method? I can create a class > variable like this: > Hmm, seeing this post, I have decided to implement a 'classproperty'...
9
by: sashang | last post by:
Hi I'd like to use metaclasses to dynamically generate a class based on a parameter to the objects init function. For example: class MetaThing(type): def __init__(cls, name, bases, dict,...
14
by: Harlin Seritt | last post by:
Hi, How does one get the name of a class from within the class code? I tried something like this as a guess: self.__name__ Obviously it didn't work. Anyone know how to do that?
5
by: JH | last post by:
Hi I found that a type/class are both a subclass and a instance of base type "object". It conflicts to my understanding that: 1.) a type/class object is created from class statement 2.) a...
5
by: Neal Becker | last post by:
After spending the morning debugging where I had misspelled the name of an attribute (thus adding a new attr instead of updating an existing one), I would like a way to decorate a class so that...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.