By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,077 Members | 1,309 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,077 IT Pros & Developers. It's quick & easy.

"specialdict" module

P: n/a
Hello,

in follow-up to the recent "dictionary accumulator" thread, I wrote a
little module with several subclassed dicts.

Comments (e.g. makes it sense to use super), corrections, etc.? Is this
PEP material?

Docstrings, Documentation and test cases are to be provided later.

mfg
Georg

----------------------------------------------------------------------

class defaultdict(dict):
# _defaulttype: 0=no default, 1=defaultvalue, 2=defaultfactory
__slots__ = ['_defaulttype', '_default']

def __init__(self, *args, **kwargs):
self._defaulttype = 0

super(defaultdict, self).__init__(self, *args, **kwargs)

def setdefaultvalue(self, value):
self._defaulttype = 1
self._default = value

def setdefaultfactory(self, factory, *args, **kwargs):
if not callable(factory):
raise TypeError, 'default factory must be a callable'
self._defaulttype = 2
self._default = (factory, args, kwargs)

def cleardefault(self):
self._defaulttype = 0

def __getitem__(self, key):
try:
return super(defaultdict, self).__getitem__(key)
except KeyError:
if self._defaulttype == 0:
raise
elif self._defaulttype == 1:
return self.setdefault(key, self._default)
else:
return self.setdefault(key,
self._default[0](*self._default[1], **self._default[2]))

class keytransformdict(dict):
__slots__ = ['_transformer']

def __init__(self, *args, **kwargs):
self._transformer = lambda x: x

super(keytransformdict, self).__init__(self, *args, **kwargs)

def settransformer(self, transformer):
if not callable(transformer):
raise TypeError, 'transformer must be a callable'
self._transformer = transformer

def __setitem__(self, key, value):
super(keytransformdict, self).__setitem__(self,
self._transformer(key), value)

def __getitem__(self, key):
return super(keytransformdict, self).__getitem__(self,
self._transformer(key))

def __delitem__(self, key):
super(keytransformdict, self).__delitem__(self,
self._transformer(key))

class sorteddict(dict):
def __iter__(self):
for key in sorted(super(sorteddict, self).__iter__(self)):
yield key

def keys(self):
return list(self.iterkeys())

def items(self):
return list(self.iteritems())

def values(self):
return list(self.itervalues())

def iterkeys(self):
return iter(self)

def iteritems(self):
return ((key, self[key]) for key in self)

def itervalues(self):
return (self[key] for key in self)

if __name__ == '__main__':
x = sorteddict(a=1, b=3, c=2)

print x.keys()
print x.values()
print x.items()
Jul 18 '05 #1
Share this Question
Share on Google+
12 Replies


P: n/a
The software you used to post this message wrapped some of the lines of
code. For example:
def __delitem__(self, key):
super(keytransformdict, self).__delitem__(self,
self._transformer(key))


In defaultdict, I wonder whether everything should be viewed as a
factory:
def setdefaultvalue(self, value):
def factory(): return value
self.setdefaultfactory(factory)

and the "no-default" mode would either cease to exist, or
def cleardefault(self):
def factory():
raise KeyError, "key does not exist and no default defined"
self.setdefaultfactory(factory)
(too bad that the key isn't available in the factory, this degrades the
quality of the error messge)

if so, __getitem__ becomes simpler:
__slots__ = ['_default']
def __getitem__(self, key):
try:
return super(defaultdict, self).__getitem__(key)
except KeyError:
return self.setdefault(key, apply(*self._default))

I don't ever have an itch for sorted dictionaries, as far as I can
remember, and I don't immediately understand the use of
keytransformdict. Can you give an example of it?

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)

iD8DBQFCUCb2Jd01MZaTXX0RAujvAJ4mi7cu3l85k0eeTUsd42 UDtGVAzACgp1um
6Iq2DhOo529q83oRXqquZ5I=
=Ytjv
-----END PGP SIGNATURE-----

Jul 18 '05 #2

P: n/a
Jeff Epler wrote:
The software you used to post this message wrapped some of the lines of
code. For example:
def __delitem__(self, key):
super(keytransformdict, self).__delitem__(self,
self._transformer(key))

Somehow I feared that this would happen.
In defaultdict, I wonder whether everything should be viewed as a
factory:
def setdefaultvalue(self, value):
def factory(): return value
self.setdefaultfactory(factory)
That's a reasonable approach. __init__ must be changed too, but this
shouldn't hurt too badly.
and the "no-default" mode would either cease to exist, or
def cleardefault(self):
def factory():
raise KeyError, "key does not exist and no default defined"
self.setdefaultfactory(factory)
(too bad that the key isn't available in the factory, this degrades the
quality of the error messge)
That can be worked around with a solution in __getitem__, see below.
if so, __getitem__ becomes simpler:
__slots__ = ['_default']
def __getitem__(self, key):
try:
return super(defaultdict, self).__getitem__(key)
except KeyError:
return self.setdefault(key, apply(*self._default))
You are peculating the kwargs. Also, apply() is on the verge of being
deprecated, so better not use it.

def __getitem__(self, key):
try:
return super(defaultdict, self).__getitem__(key)
except KeyError, err:
try:
return self.setdefault(key,
self._default[0](*self._default[1],
**self._default[2]))
except KeyError:
raise err

Although I'm not sure whether KeyError would be the right one to raise
(perhaps a custom error?).
I don't ever have an itch for sorted dictionaries, as far as I can
remember, and I don't immediately understand the use of
keytransformdict. Can you give an example of it?


See the thread "Case-insensitive dict, non-destructive, fast, anyone?",
starting at 04/01/05 12:38.

mfg
Georg
Jul 18 '05 #3

P: n/a
Georg Brandl wrote:
Hello,

in follow-up to the recent "dictionary accumulator" thread, I wrote a
little module with several subclassed dicts.

Comments (e.g. makes it sense to use super), corrections, etc.? Is this
PEP material?

Docstrings, Documentation and test cases are to be provided later.

mfg
Georg


Georg:

A few reactions:

1. Given that these are specializations, why not have:

class defaultvaluedict(dict):
...

class defaultfactorydict(dict):
...

rather than having to jump through hoops to make one implementation satisfy both
cases

2. I would really prefer to have the default value specified in the constructor

I realize that this is tricky due to the kw arguments of dict.__init__, but I
would favor either breaking compatibility with that interface, or adopting some
workaround to make something like d= defaultvaluedict(__default__ = 0) possible.

One worksaround would be to store the default in the dict, not as an attribute
of the dict. By default the default value would be associated with the key
"__default__", but that keyname could be changed for the (I guess very few)
cases where that key conflicted with non-default content of the dict. Then
dict.__init__ would simply take __default__ = value as a keyword argument, as it
does today, and __getitem__ for a missing key would return
dict.__getitem__(self, "__default__")

Alternatively, you could provide factory functions to construct the defaultdict.
Someone (Michele?) recently posted an implementation of this

3. Can you work in the tally and listappend methods that started this whole
thread off?

4. On super, no I don't think it's necessary or particularly desirable. These
specializations have a close association with dict. dict.method(self,...) feels
more appropriate in this case.

Michael
Jul 18 '05 #4

P: n/a
Michael Spencer wrote:
1. Given that these are specializations, why not have:

class defaultvaluedict(dict):
...

class defaultfactorydict(dict):
...

rather than having to jump through hoops to make one implementation satisfy both
cases
I think I like Jeff's approach more (defaultvalues are just special
cases of default factories); there aren't many "hoops" required.
Apart from that, the names just get longer ;)
2. I would really prefer to have the default value specified in the constructor

I realize that this is tricky due to the kw arguments of dict.__init__, but I
would favor either breaking compatibility with that interface, or adopting some
workaround to make something like d= defaultvaluedict(__default__ = 0) possible.
Too much specialcased for my liking.
One worksaround would be to store the default in the dict, not as an attribute
of the dict. By default the default value would be associated with the key
"__default__", but that keyname could be changed for the (I guess very few)
cases where that key conflicted with non-default content of the dict. Then
dict.__init__ would simply take __default__ = value as a keyword argument, as it
does today, and __getitem__ for a missing key would return
dict.__getitem__(self, "__default__")
I thought about this too (providing a singleton instance named Default,
just like None is, and using it as a key), but you would have to
special-case the (iter)keys,values,items methods to exclude the default
- definitely too much work, and too much magic.
Alternatively, you could provide factory functions to construct the defaultdict.
Someone (Michele?) recently posted an implementation of this
Yes, I think this could be reasonable.
3. Can you work in the tally and listappend methods that started this whole
thread off?
They aren't necessary any longer.

Use defaultdict.setdefaultvalue(0) instead of the tally approach and
defaultdict.setdefaultfactory(list) instead of listappend.
4. On super, no I don't think it's necessary or particularly desirable. These
specializations have a close association with dict. dict.method(self,...) feels
more appropriate in this case.


Any other opinions on this?

Thanks for the comments,

mfg
Georg
Jul 18 '05 #5

P: n/a
Georg Brandl wrote:

I think I like Jeff's approach more (defaultvalues are just special
cases of default factories); there aren't many "hoops" required.
Apart from that, the names just get longer ;)
Yes Jeff's approach does simplify the implementation and more-or-less eliminates
my complexity objection

But why do you write:

def __getitem__(self, key):
try:
return super(defaultdict, self).__getitem__(key)
except KeyError, err:
try:
return self.setdefault(key,
self._default[0](*self._default[1],
**self._default[2]))
except KeyError:
raise err

rather than:

def __getitem__(self, key):
return self.setdefault(key,
self._default[0](*self._default[1],
**self._default[2]))

(which could catch AttributeError in the case of _default not set)
I'm sure there's a reason, but I can't see it.

2. I would really prefer to have the default value specified in the constructor ....

Too much specialcased for my liking.
It does set up some gotchas I concede ;-)
....
Alternatively, you could provide factory functions to construct the defaultdict.
Someone (Michele?) recently posted an implementation of this

Yes, I think this could be reasonable.

....though this would more naturally complement a fixed-default dictionary IMO
Your design permits - even encourages (by providing convenient setters) the
default to change over the lifetime of the dictionary. I'm not sure whether
that's good or bad, but it's a feature worth discussing.
3. Can you work in the tally and listappend methods that started this whole
thread off?

They aren't necessary any longer.

Use defaultdict.setdefaultvalue(0) instead of the tally approach and
defaultdict.setdefaultfactory(list) instead of listappend.


Oops, I see what you mean... then use += or append as required. I still prefer
the clarity of tally for its specific use-case, but it does suffer from lack of
generality.

Michael

Jul 18 '05 #6

P: n/a
Michael Spencer wrote:
Georg Brandl wrote:

I think I like Jeff's approach more (defaultvalues are just special
cases of default factories); there aren't many "hoops" required.
Apart from that, the names just get longer ;)


Yes Jeff's approach does simplify the implementation and more-or-less eliminates
my complexity objection

But why do you write:

def __getitem__(self, key):
try:
return super(defaultdict, self).__getitem__(key)
except KeyError, err:
try:
return self.setdefault(key,
self._default[0](*self._default[1],
**self._default[2]))
except KeyError:
raise err

rather than:

def __getitem__(self, key):
return self.setdefault(key,
self._default[0](*self._default[1],
**self._default[2]))

(which could catch AttributeError in the case of _default not set)
I'm sure there's a reason, but I can't see it.


In your version, the default factory is called every time a value is
retrieved, which might be a performance problem.
Alternatively, you could provide factory functions to construct the defaultdict.
Someone (Michele?) recently posted an implementation of this

Yes, I think this could be reasonable.

...though this would more naturally complement a fixed-default dictionary IMO
Your design permits - even encourages (by providing convenient setters) the
default to change over the lifetime of the dictionary. I'm not sure whether
that's good or bad, but it's a feature worth discussing.


It's certainly more in the spirit of Python -- we're consenting adults,
and so we are allowed to change the default.

mfg
Georg
Jul 18 '05 #7

P: n/a
Michael Spencer:
Alternatively, you could provide factory functions to construct the defaultdict. Someone (Michele?) recently posted an implementation of this


Yes, here is the link for the ones who missed that thread:

http://groups-beta.google.com/group/...67d30b796423c7

Jul 18 '05 #8

P: n/a
About not using super: you might have problems in multiple inheritance.
Suppose I want to use both your defaultdict and a thirdpartdict. A
subclass

class mydict(defaultdict, thirdpartdict):
pass

would not work if thirdpartdict requires a non-trivial __init__ , since
without super in defaultdict.__init__ you would just call dict.__init__
and not thirdpartdict.

Michele Simionato

Jul 18 '05 #9

P: n/a
Michele Simionato wrote:
About not using super: you might have problems in multiple inheritance.
Suppose I want to use both your defaultdict and a thirdpartdict. A
subclass

class mydict(defaultdict, thirdpartdict):
pass

would not work if thirdpartdict requires a non-trivial __init__ , since
without super in defaultdict.__init__ you would just call dict.__init__
and not thirdpartdict.


Right. I thought about a combined defaultdict/keytransformdict,
which seems to be easy to create with the current implementation:

class defaultkeytransformdict(defaultdict, keytransformdict):
pass

At least I hope so. This is another argument against the initializing
of defaultfactory or keytransformer in __init__.

mfg
Georg

Here comes the current module (keytransformdict should be working now
for all dict methods):

# specialdict - subclasses of dict for common tasks
#

class NoDefaultGiven(Exception):
pass

class defaultdict(dict):
__slots__ = ['_default']

def __init__(self, *args, **kwargs):
self._defaulttype = 0

super(defaultdict, self).__init__(*args, **kwargs)

def setdefaultvalue(self, value):
def defaultfactory():
return value
self._default = (defaultfactory, (), {})

def setdefaultfactory(self, factory, *args, **kwargs):
if not callable(factory):
raise TypeError, 'default factory must be a callable'
self._default = (factory, args, kwargs)

def cleardefault(self):
def defaultfactory():
raise NoDefaultGiven
self._default = (defaultfactory, (), {})

def __getitem__(self, key):
try:
return super(defaultdict, self).__getitem__(key)
except KeyError, err:
try:
return self.setdefault(key, self._default[0](*self._default[1], **self._default[2]))
except NoDefaultGiven:
raise err

class keytransformdict(dict):
__slots__ = ['_transformer']

def __init__(self, *args, **kwargs):
self._transformer = lambda x: x

super(keytransformdict, self).__init__(*args, **kwargs)

def settransformer(self, transformer):
if not callable(transformer):
raise TypeError, 'transformer must be a callable'
self._transformer = transformer

def __setitem__(self, key, value):
print "setitem"
super(keytransformdict, self).__setitem__(self._transformer(key), value)

def __getitem__(self, key):
print "getitem"
return super(keytransformdict, self).__getitem__(self._transformer(key))

def __delitem__(self, key):
super(keytransformdict, self).__delitem__(self._transformer(key))

def has_key(self, key):
return super(keytransformdict, self).has_key(self._transformer(key))

def __contains__(self, key):
return self.has_key(key)

def get(self, key, default):
return super(keytransformdict, self).get(self._transformer(key), default)

def setdefault(self, key, default):
return super(keytransformdict, self).setdefault(self._transformer(key), default)

def pop(self, key, default):
return super(keytransformdict, self).pop(self._transfomer(key), default)

def update(self, other=None, **kwargs):
if other is not None:
if hasattr(other, "keys"):
super(keytransformdict, self).update((self._transformer(k), other[k]) for k in other.keys())
else:
super(keytransformdict, self).update((self._transformer(k), v) for (k, v) in other)
if kwargs:
super(keytransformdict, self).update((self._transformer(k), v) for (k, v) in kwargs.iteritems())

class sorteddict(dict):
def __iter__(self):
for key in sorted(super(sorteddict, self).__iter__()):
yield key

def keys(self):
return list(self.iterkeys())

def items(self):
return list(self.iteritems())

def values(self):
return list(self.itervalues())

def iterkeys(self):
return iter(self)

def iteritems(self):
return ((key, self[key]) for key in self)

def itervalues(self):
return (self[key] for key in self)
Jul 18 '05 #10

P: n/a
Georg Brandl wrote:
Michele Simionato wrote:
About not using super: you might have problems in multiple inheritance.
Suppose I want to use both your defaultdict and a thirdpartdict. A
subclass

class mydict(defaultdict, thirdpartdict):
pass

would not work if thirdpartdict requires a non-trivial __init__ , since
without super in defaultdict.__init__ you would just call dict.__init__
and not thirdpartdict.


Right. I thought about a combined defaultdict/keytransformdict,
which seems to be easy to create with the current implementation:

class defaultkeytransformdict(defaultdict, keytransformdict):
pass

At least I hope so. This is another argument against the initializing
of defaultfactory or keytransformer in __init__.

mfg
Georg

Here comes the current module (keytransformdict should be working now
for all dict methods):

# specialdict - subclasses of dict for common tasks
#

class NoDefaultGiven(Exception):
pass

class defaultdict(dict):
__slots__ = ['_default']

def __init__(self, *args, **kwargs):
self._defaulttype = 0

^^^^^^^^^^^^^^^^^^^^^

This must read "self.cleardefault()", of course.

mfg
Georg
Jul 18 '05 #11

P: n/a
Georg Brandl wrote:
Hello,

in follow-up to the recent "dictionary accumulator" thread, I wrote a
little module with several subclassed dicts.

Comments (e.g. makes it sense to use super), corrections, etc.? Is this
PEP material?

Docstrings, Documentation and test cases are to be provided later.


So, no further interest in this? Or should I write a PEP before?

mfg
Georg
Jul 18 '05 #12

P: n/a
Georg Brandl wrote:
Georg Brandl wrote:
Hello,

in follow-up to the recent "dictionary accumulator" thread, I wrote a
little module with several subclassed dicts.

Comments (e.g. makes it sense to use super), corrections, etc.? Is this
PEP material?

Docstrings, Documentation and test cases are to be provided later.


So, no further interest in this? Or should I write a PEP before?


I'd personally like to see a PEP. It'd give people a chance to come up
with some use cases that really justify adding such specialized dicts to
the standard library. The only use cases I can come up with would be
better served by functions that take a sequence and produce a dict (e.g.
by counting items or by grouping items into lists). Once the dict is
built, I usually have no more need for the default-specialized methods.

I'm sure there are some good use cases out there for such dicts, but
before I get behind this, I wouldn't mind seeing a few of them. =)

STeVe
Jul 18 '05 #13

This discussion thread is closed

Replies have been disabled for this discussion.