assigning a custom mapping type to __dict__

[Steven Bethard]

I tried to Google for past discussion on this topic, but without much
luck. If this has been discussed before, I'd be grateful for a pointer.

Does anyone know why you can't assign a custom mapping type to an
object's __dict__?

py> class M(object):
.... def __getitem__(self, key):
.... return 42
.... def __setitem__(self, key, value):
.... pass
....
py> class C(object):
.... pass
....
py> c = C()
py> c.__dict__ = M()
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
TypeError: __dict__ must be set to a dictionary

I looked at the source in typeobject.c (where this error originates),
but I'm not fluent enough in CPython yet to be able to tell why a true
dict type is preferred here over just a mapping type...

STeVe

[Daniel Cer]

Why not just inherit from dict? That seems to work.

class M(dict): .... def __getitem__(self,key):
.... return 42
.... def __setitem__(self,key,value):
.... pass
.... class C(object): .... pass
.... c = C()
c.__dict__ = M()
c.__dict__['x']

42

-Dan

Steven Bethard wrote:
I tried to Google for past discussion on this topic, but without much
luck. If this has been discussed before, I'd be grateful for a pointer.

Does anyone know why you can't assign a custom mapping type to an
object's __dict__?

py> class M(object):
... def __getitem__(self, key):
... return 42
... def __setitem__(self, key, value):
... pass
...
py> class C(object):
... pass
...
py> c = C()
py> c.__dict__ = M()
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
TypeError: __dict__ must be set to a dictionary

I looked at the source in typeobject.c (where this error originates),
but I'm not fluent enough in CPython yet to be able to tell why a true
dict type is preferred here over just a mapping type...

STeVe

[Nick Coghlan]

Daniel Cer wrote:

Why not just inherit from dict? That seems to work.

Because that isn't the question - Steven knows how to make it work, what he's
curious about is why things are the way they are :)

Anyway, a quick look suggests that it is due to typeobject.c using the concrete
PyDict_* API calls [1] to manipulate tp_dict, rather than the abstract
PyMapping_* calls [2]. The reason behind using the concrete API is, presumably,
a question of speed :)

Cheers,
Nick.

[1] http://www.python.org/dev/doc/devel/...ctObjects.html
[2] http://www.python.org/dev/doc/devel/api/mapping.html
--
Nick Coghlan | nc******@email.com | Brisbane, Australia
---------------------------------------------------------------
http://boredomandlaziness.skystorm.net

[Duncan Booth]

Daniel Cer wrote:

Why not just inherit from dict? That seems to work.

class M(dict): ... def __getitem__(self,key):
... return 42
... def __setitem__(self,key,value):
... pass
... class C(object): ... pass
... c = C()
c.__dict__ = M()
c.__dict__['x'] 42

Didn't test this very much, did you?

c.x
Traceback (most recent call last):
File "<pyshell#23>", line 1, in -toplevel-
c.x
AttributeError: 'C' object has no attribute 'x'

Or even:
c = C()
c.__dict__ = M({'x': 1})
c.x 1 c.__dict__['x'] 42

[Daniel Cer]

> > Why not just inherit from dict? That seems to work.

Because that isn't the question - Steven knows how to make it work, what he's
curious about is why things are the way they are :)
Sorry, didn't mean to be a pest :)

I guess I assumed Steve already knew that he could inherit from dict.
That being said, I was wondering why pragmatically this wouldn't be the
right thing to do (in order to do what he seemed to want to do).

<me> braces self for the true but not always too informative response of
'in principle, it's best to use the most abstract interface possible'</me>

-Dan


Anyway, a quick look suggests that it is due to typeobject.c using the concrete
PyDict_* API calls [1] to manipulate tp_dict, rather than the abstract
PyMapping_* calls [2]. The reason behind using the concrete API is, presumably,
a question of speed :)

Cheers,
Nick.

[1] http://www.python.org/dev/doc/devel/...ctObjects.html
[2] http://www.python.org/dev/doc/devel/api/mapping.html
--
Nick Coghlan | nc******@email.com | Brisbane, Australia
---------------------------------------------------------------
http://boredomandlaziness.skystorm.net
--
http://mail.python.org/mailman/listinfo/python-list

[Steven Bethard]

Daniel Cer wrote:

Why not just inherit from dict? That seems to work.

Because that isn't the question - Steven knows how to make it work, what he's
curious about is why things are the way they are :)

Sorry, didn't mean to be a pest :)

I guess I assumed Steve already knew that he could inherit from dict.
That being said, I was wondering why pragmatically this wouldn't be the
right thing to do (in order to do what he seemed to want to do).

The problem with inheriting from dict is that you then need to override
*all* the methods in the dict object, because they all go straight to
Python's dict'c C code functions. So just because you redefine
__getitem__ doesn't mean you don't still have to redefine __contains__,
get, update, etc. UserDict.DictMixin can help with this some, but the
ideal situation would be to only have to define the methods you actually
support. Inheriting from dict likely means you have to redefine a bunch
of functions to raise Exceptions saying that they're unsupported.

STeVe

[Nick Coghlan]

Steven Bethard wrote:

The problem with inheriting from dict is that you then need to override
*all* the methods in the dict object, because they all go straight to
Python's dict'c C code functions. So just because you redefine
__getitem__ doesn't mean you don't still have to redefine __contains__,
get, update, etc. UserDict.DictMixin can help with this some, but the
ideal situation would be to only have to define the methods you actually
support. Inheriting from dict likely means you have to redefine a bunch
of functions to raise Exceptions saying that they're unsupported.

You're just lucky the affected class is already overriding __getattribute__, so
the __dict__ is generally getting accessed from Python code :)

If it weren't for that, object.c's direct calls to the PyDict_* API would be
making things even more fun for you than they already are (as Duncan pointed out).

Cheers,
Nick.

--
Nick Coghlan | nc******@email.com | Brisbane, Australia
---------------------------------------------------------------
http://boredomandlaziness.skystorm.net

[Nick Coghlan]

Steven Bethard wrote:

support. Inheriting from dict likely means you have to redefine a bunch
of functions to raise Exceptions saying that they're unsupported.

Hmm. . .

We've got the NotImplemented singleton already to let special methods say "I
thought I might be able to handle this, but I can't".

Maybe "__op__ = NotImplemented" should clear the associated slot. It would also
make it easier to inherit from list and handle slices in __getitem__ by writing
"__getslice__ = NotImplemented" instead of overriding __getslice__ to delegate
to __getitem__.

Cheers,
Nick.

--
Nick Coghlan | nc******@email.com | Brisbane, Australia
---------------------------------------------------------------
http://boredomandlaziness.skystorm.net

[Steven Bethard]

Nick Coghlan wrote:

Steven Bethard wrote:

The problem with inheriting from dict is that you then need to
override *all* the methods in the dict object, because they all go
straight to Python's dict'c C code functions. So just because you
redefine __getitem__ doesn't mean you don't still have to redefine
__contains__, get, update, etc. UserDict.DictMixin can help with this
some, but the ideal situation would be to only have to define the
methods you actually support. Inheriting from dict likely means you
have to redefine a bunch of functions to raise Exceptions saying that
they're unsupported.

You're just lucky the affected class is already overriding
__getattribute__, so the __dict__ is generally getting accessed from
Python code :)

If it weren't for that, object.c's direct calls to the PyDict_* API
would be making things even more fun for you than they already are (as
Duncan pointed out).

Yup, I noticed that. Lucky us. =)

STeVe