Inconsistency in dictionary behaviour: dict(dict) not calling __setitem__

[Almad]

Hello,

I discovered this behaviour in dictionary which I find confusing. In
SneakyLang, I've tried to extend dictionary so it visits another class
after something is added:

class RegisterMap(dict):
def __setitem__(self, k, v):
dict.__setitem__(self, k,v)
self[k].visit_register_map(self)
However, when constructing dictionary with dictionary in constructor
like d = RegisterMap({'k':'v'}), __setitem__ is not called, so
workaround is needed:

class RegisterMap(dict):
def __init__(self, *args, **kwargs):
dict.__init__(self, *args, **kwargs)
for k in self:
self.__after_add(k)

def __after_add(self, k):
self[k].visit_register_map(self)

def __setitem__(self, k, v):
dict.__setitem__(self, k,v)
self.__after_add(k)
What is the reason for this behavior? Am I doing something wrong and
better approach is needed? Or should this be considered as minor bug in
Python? (tried this only in 2.4 so far)

Thank You,

Almad

[Fredrik Lundh]

Almad wrote:

However, when constructing dictionary with dictionary in constructor
like d = RegisterMap({'k':'v'}), __setitem__ is not called

why should it do that? dict() is a concrete implementation, not a
template class for the creation of dict-like objects.

Or should this be considered as minor bug in Python?

no.

</F>

[Mitja Trampus]

Fredrik Lundh wrote:

Almad wrote:

>However, when constructing dictionary with dictionary in constructor
like d = RegisterMap({'k':'v'}), __setitem__ is not called

why should it do that? dict() is a concrete implementation, not a
template class for the creation of dict-like objects.

I think what was unexpected for the OP is that dict.__init__
does not use __setitem__ to create its internal structures.
This is independent of dict being a concrete implementation
or not:

>>class H:

.... def __init__(self): self.hello()
.... def hello(self): print 'oi oi'

>>class K(H):

.... def hello(self): print 'hello'

>>K()

hello
<__main__.K instance at 0x00AC9878>

At least, I know it surprised me when I first met this
behavior. Or is my reasoning incorrect?

Further to the OP: This is due to optimizations -
initialization is much faster if it's done by modifying
dict's internal state directly and thus without calling a
(possibly custom-made) method for each item to be inserted.
What I find an even nastier surprise is that dict.update
behaves this way as well:

>>class D(dict):

.... def __setitem__(self,k,v):
.... print 'setting',k,'to',v
.... dict.__setitem__(self,k,v)

>>d=D({1:'a', 2:int})

# No output here - OK, half-expected

>>d.update({3:'s',6:23})

# No output here, either...

The docstring is, at best, misguiding on this particular point:

>>print d.update.__doc__

D.update(E, **F) -None. Update D from E and F: for k in
E: D[k] = E[k]
(if E has keys else: for (k, v) in E: D[k] = v) then: for k
in F: D[k] = F[k]
It's not a big issue, but an almost sure one-time surprise
for everyone, I suppose...

[Fredrik Lundh]

Mitja Trampus wrote:

I think what was unexpected for the OP is that dict.__init__
does not use __setitem__ to create its internal structures.

you can implement most methods on core objects in terms of other
methods. picking one by random, and then complaining that other
methods don't use the one you picked, strikes me as a rather
naive view of how object-oriented design works in practice.

</F>

[Andrea Griffini]

Mitja Trampus wrote:

....

At least, I know it surprised me when I first met this behavior. Or is
my reasoning incorrect?

Why len() doesn't call iteritems() ? :-)

Kidding apart for example it would be ok for __setitem__
to call either an internal "insert_new_item" or
"update_existing_item" depending on if the key is
already present in the dictionary.
In this case I suppose you agree it would make a lot
of sense to go directly for "insert_new_item" in the
constructor from a dict instead of calling the public
__setitem__...

The key point is that you're not authorized to assume
constructing a dictionary from a dictionary will use
__setitem__ unless this is explicitly stated in the
interface.

....

What I find an even nastier surprise is that dict.update behaves this
way as well:

....

The docstring is, at best, misguiding on this particular point:

>>print d.update.__doc__

D.update(E, **F) -None. Update D from E and F: for k in E: D[k] = E[k]
(if E has keys else: for (k, v) in E: D[k] = v) then: for k in F: D[k] =
F[k]

I cannot understand this doc string at all.

The explanation in the manual however just talks about
"updating", with no reference to assignments. The manual
of 2.3 instead was using a code example and I'd say this
would qualify as a binding to actually implement calls
to __setitem__. This kind of error (i.e. over-specifying
by providing actual code that implies specific side-effects)
was also present in the C++ standard, and in at least
one case an implementation would have to be very
inefficient to comply on the issue (this fortunately is
not what happened, the standard was "fixed" instead).

If there is a bug in this case is IMO a docstring bug.

Andrea

[Terry Reedy]

"Almad" <bu**@almad.netwrote in message
news:11*********************@j72g2000cwa.googlegro ups.com...

Hello,

I discovered this behaviour in dictionary which I find confusing

....

However, when constructing dictionary with dictionary in constructor
like d = RegisterMap({'k':'v'}), __setitem__ is not called,

d.__setitem__(k,v) is the internal translation of d[k] = v (when such is
explicitly written in this code). d.__init__(otherdict) need not go thru
that interface if a more direct means is available.

>so workaround is needed:

Your 'workaround' seems to be the proper solution.

Am I doing something wrong

Denigrating a solution because it does not meet your expectation.

Your are not the first to do this ;-).

tjr

[Raymond Hettinger]

[Almad]

I discovered this behaviour in dictionary which I find confusing. In
SneakyLang, I've tried to extend dictionary so it visits another class
after something is added:

class RegisterMap(dict):
def __setitem__(self, k, v):
dict.__setitem__(self, k,v)
self[k].visit_register_map(self)
However, when constructing dictionary with dictionary in constructor
like d = RegisterMap({'k':'v'}), __setitem__ is not called,

Try subclassing from UserDict.DictMixin.
Raymond

[Almad]

Thanks to everybody for replies, I'm now satisfied ^_^

Almad

[Terry Reedy]

Final note: one of the developers ran into a similar issue with dict and
has opened a discussion on pydev about how the C implementation might be
changed to have derived classes act more consistently without imposing a
time penalty on the normal use of dict. There might possibly be a change
by 2.6 but I am not following the details.

tjr