Pythonic way for missing dict keys

[Alex Popescu]

Hi all!

I am pretty sure this has been asked a couple of times, but I don't seem
to find it on the archives (Google seems to have a couple of problems
lately).

I am wondering what is the most pythonic way of dealing with missing
keys and default values.

According to my readings one can take the following approaches:

1/ check before (this has a specific name and acronym that I haven't
learnt yet by heart)

if not my_dict.has_key (key):
my_obj = myobject()
my_dict[key] = my_obj
else:
my_obj = my_dict[key]

2/ try and react on error (this has also a specific name, but...)

try:
my_obj = my_dict[key]
except AttributeError:
my_obj = myobject()
my_dict[key] = my_obj

3/ dict.get usage:

my_obj = my_dict.get(key , myobject())

I am wondering which one is the most recommended way? get usage seems
the clearest, but the only problem I see is that I think myobject() is
evaluated at call time, and so if the initialization is expensive you
will probably see surprises.

thanks in advance,
../alex
--
..w( the_mindstorm )p.

[Bruno Desthuilliers]

Alex Popescu a écrit :

Jakub Stolarski <ja************ *@gmail.comwrot e in
news:11******** **************@ k79g2000hse.goo glegroups.com:

>>Version 1 and 2 do different thing than version 3. The latter doesn't
add value to dict.

As it was mentioned before, use:
1 - if you expect that there's no key in dict
2 - if you expect that there is key in dict

I may be missing something

You are.

but I think the 3 approaches are completely
equivalent in terms of functionality.

d = dict()
answer = d.get('answer', 42)
answer in d
=False

[Alex Martelli]

Carsten Haese <ca*****@uniqsy s.comwrote:

On Sat, 21 Jul 2007 09:22:32 +0530, Rustom Mody wrote

Can someone who knows about python internals throw some light on why

>>x in dic

is cheaper than

>>dic.has_key(x )

??

I won't claim to know Python internals, but compiling and disassembling the
expressions in question reveals the reason:

>from compiler import compile
from dis import dis
dis(compile("d ic.has_key(x)", "","eval"))

1 0 LOAD_NAME 0 (dic)
3 LOAD_ATTR 1 (has_key)
6 LOAD_NAME 2 (x)
9 CALL_FUNCTION 1
12 RETURN_VALUE

>dis(compile( "x in dic","","eval") )

1 0 LOAD_NAME 0 (x)
3 LOAD_NAME 1 (dic)
6 COMPARE_OP 6 (in)
9 RETURN_VALUE

"dic.has_key(x) " goes through an attribute lookup to find the function that
looks for the key. "x in dic" finds the function more directly.

Yup, it's mostly that, as microbenchmarki ng can confirm:

brain:~ alex$ python -mtimeit -s'd={}; f=d.has_key' 'f(23)'
10000000 loops, best of 3: 0.146 usec per loop
brain:~ alex$ python -mtimeit -s'd={}; f=d.has_key' '23 in d'
10000000 loops, best of 3: 0.142 usec per loop
brain:~ alex$ python -mtimeit -s'd={}; f=d.has_key' 'f(23)'
10000000 loops, best of 3: 0.146 usec per loop
brain:~ alex$ python -mtimeit -s'd={}; f=d.has_key' '23 in d'
10000000 loops, best of 3: 0.142 usec per loop
brain:~ alex$ python -mtimeit -s'd={}; f=d.has_key' 'd.has_key(23)'
1000000 loops, best of 3: 0.278 usec per loop
brain:~ alex$ python -mtimeit -s'd={}; f=d.has_key' 'd.has_key(23)'
1000000 loops, best of 3: 0.275 usec per loop

the in operator still appears to have a tiny repeatable advantage (about
4 nanoseconds on my laptop) wrt even the hoisted method, but the
non-hoisted method, due to repeated lookup, is almost twice as slow
(over 100 nanoseconds penalty, on my laptop).
Alex

[Alex Popescu]

Bruno Desthuilliers <bd************ *****@free.quel quepart.frwrote in
news:46******** *************** @news.free.fr:

Alex Popescu a écrit :

>Jakub Stolarski <ja************ *@gmail.comwrot e in

>
[snip...]
d = dict()
answer = d.get('answer', 42)
answer in d
=False

Thanks. I think to make the 3rd approach completely equivalent I should
have been using d.setdefault(ke y, myojbect()).

../alex
--
..w( the_mindstorm )p.

[Duncan Booth]

"Rustom Mody" <ru*********@gm ail.comwrote:

Can someone who knows about python internals throw some light on why

>>>x in dic

is cheaper than

>>>dic.has_key( x)

??

Some special methods are optimised by having a reserved slot in the data
structure used to implement a class. The 'in' operator uses one of these
slots so it can bypass all the overheads of looking up an attribute such as
'has_key'.

[Zentrader]

On Jul 21, 7:48 am, Duncan Booth <duncan.bo...@i nvalid.invalidw rote:

"Rustom Mody" <rustompm...@gm ail.comwrote:

Can someone who knows about python internals throw some light on why

>>x in dic

is cheaper than

>>dic.has_key(x )

??

>From the 2.6 PEP #361 (looks like dict.has_key is deprecated)

Python 3.0 compatability: ['compatibility'-->someone should use a
spell-checker for 'official' releases]
- warnings were added for the following builtins which no
longer exist in 3.0:
apply, callable, coerce, dict.has_key, execfile, reduce,
reload

[genro]

On Jul 19, 6:29 am, Bruno Desthuilliers
<bdesth.quelque [e-mail] quepart.frwrote :

>
Myobject will be instanciated each time, yes.

and so if the initialization is expensive you
will probably see surprises.

No "surprise" here, but it can indeed be suboptimal if instanciating
myobject is costly.

What about this way ?

my_obj = my_dict.get(key ) or my_dict.setdefa ult(key,myobjec t())

Ciao
G.

[Marc 'BlackJack' Rintsch]

On Sat, 21 Jul 2007 16:20:37 -0700, genro wrote:

On Jul 19, 6:29 am, Bruno Desthuilliers
<bdesth.quelque [e-mail] quepart.frwrote :

>No "surprise" here, but it can indeed be suboptimal if instanciating
myobject is costly.

What about this way ?

my_obj = my_dict.get(key ) or my_dict.setdefa ult(key,myobjec t())

Reduces the unnecessary instantiation of `myobject` to "false" objects.
May be not good enough.

Ciao,
Marc 'BlackJack' Rintsch

[Duncan Booth]

Steven D'Aprano <st***@REMOVE.T HIS.cybersource .com.auwrote:

>
Instead of doing:
if callable(functi on): function()

you should do:

try:
function()
except TypeError:
pass
That should work for most uses of callable(), but isn't quite the
same. (What if function() has side-effects, or is expensive, and you
want to determine if it is callable, but not actually call it _now_?)

The replacement for callable(x) is simply hasattr(x, '__call__').
Once upon a time it may have been that functions didn't have a __call__
attribute (I haven't checked back on really old Pythons top see if this
was the case), but these

>Also, what is the replacement of reduce? I think I remember seeing
somewhere that lists comprehension would be (but also remember the
advise that reduce will be quicker).

No, a list comprehension isn't equivalent to reduce(). There is no
replacement for reduce().

<snip>
There are of course several replacements for specialised uses of reduce:
sum, any, all. There is no general purpose replacement, but you can
write one in a few lines if you really need it.

>
It's a shame really. Oh well, maybe it will sneak back in via a
functional module, or itertools, or something. What a waste, what a
waste.

I'm sure it will reappear in some other module, but that's the correct
place for a little used function, not in builtins.

[Steven D'Aprano]

On Mon, 30 Jul 2007 07:37:05 +0000, Duncan Booth wrote:

Steven D'Aprano <st***@REMOVE.T HIS.cybersource .com.auwrote:

>>
Instead of doing:
if callable(functi on): function()

you should do:

try:
function()
except TypeError:
pass
That should work for most uses of callable(), but isn't quite the
same. (What if function() has side-effects, or is expensive, and you
want to determine if it is callable, but not actually call it _now_?)

The replacement for callable(x) is simply hasattr(x, '__call__').
Once upon a time it may have been that functions didn't have a __call__
attribute (I haven't checked back on really old Pythons top see if this
was the case), but these

Your sentence seems to have run out early :)

Thanks, I didn't know that -- I remember that Back In The Day checking if
something was callable was tricky, because it could be a function, a
method, or a class with __call__. It makes much more sense to make all
callables go through __call__.

>>Also, what is the replacement of reduce? I think I remember seeing
somewhere that lists comprehension would be (but also remember the
advise that reduce will be quicker).

No, a list comprehension isn't equivalent to reduce(). There is no
replacement for reduce().

<snip>
There are of course several replacements for specialised uses of reduce:
sum, any, all. There is no general purpose replacement, but you can
write one in a few lines if you really need it.

True. But writing the same basic algorithm over and over again is about as
far from best practice as you can get without being a PHP programmer.
*wink* Currying is easy to do too, but Python has grown a function
functools.parti al() to do it properly. That's as it should be.

(Yes, I know that's a misuse of the term currying, but it's a common one,
and easier than saying "Making a partial function".)

>It's a shame really. Oh well, maybe it will sneak back in via a
functional module, or itertools, or something. What a waste, what a
waste.

I'm sure it will reappear in some other module, but that's the correct
place for a little used function, not in builtins.

Oh, I would have no problem at all with reduce being banished to the
functtools module. But it's a shame to have to go through the whole PEP
process, and risk Guido saying no.
--
Steven.

[John J. Lee]

Steve Holden <st***@holdenwe b.comwrites:
[...]

Yup. Anyway there's a trivial translation for uses of apply.

apply(f, *args, **kw) = f(*args, **kw)

[...]

Steve means:

apply(f, args, kw) = f(*args, **kw)
John