dict.has_key(x) versus 'x in dict'

[Paul Melis]

Hello,

I've always been using the has_key() method to test if a dictionary
contains a certain key. Recently I tried the same using 'in', e.g.

d = { ... }
if k in d:
...

and found that it seems to perform a lot better when lots of key-tests
are to be performed. I also noticed that has_key() is scheduled to be
removed from future (C)Python versions.

Does the 'in' way of testing have an optimized implementation compared
to has_key()? Is that the reason has_key() is being phased out?

Thanks,
Paul

[Fredrik Lundh]

Paul Melis wrote:

I've always been using the has_key() method to test if a dictionary
contains a certain key. Recently I tried the same using 'in', e.g.

d = { ... }
if k in d:
...

and found that it seems to perform a lot better when lots of key-tests
are to be performed. I also noticed that has_key() is scheduled to be
removed from future (C)Python versions.

Does the 'in' way of testing have an optimized implementation compared
to has_key()?

no, but full method calls are a lot slower than the under-the-hood C-level
call used by the "in" operator.

this is why e.g.

string[:len(prefix)] == prefix

is often a lot faster than

string.startswith(prefix)

and why

if character in string:
string = string.replace(character, replacement)

is faster than

string = string.replace(character, replacement)

if the character isn't there most of the time, despite the fact that the "replace"
method doesn't actually do something if the character isn't found.

</F>

[Bjoern Schliessmann]

Paul Melis wrote:

I've always been using the has_key() method to test if a
dictionary contains a certain key. Recently I tried the same using
'in', e.g.

d = { ... }
if k in d:
...

Wouldn't be "if k in d.keys()" be the exact replacement?

Regards,
Björn

--
BOFH excuse #17:

fat electrons in the lines

[Fredrik Lundh]

Bjoern Schliessmann wrote:

Wouldn't be "if k in d.keys()" be the exact replacement?

no. that would convert an O(1) operation to an O(n) operation, which
would be rather silly.

</F>

[Peter Otten]

Bjoern Schliessmann wrote:

Paul Melis wrote:

>I've always been using the has_key() method to test if a
dictionary contains a certain key. Recently I tried the same using
'in', e.g.

d = { ... }
if k in d:
...

Wouldn't be "if k in d.keys()" be the exact replacement?

No, 'k in d' is equivalent to 'd.has_key(k)', only with less (constant)
overhead for the function call. 'k in d.keys()' on the other hand creates a
list of keys which is then searched linearly -- about the worst thing you
can do about both speed and memory footprint. Some numbers:

$ python2.5 -m timeit -s"N = 1; d = dict.fromkeys(range(N))" "N in d.keys()"
1000000 loops, best of 3: 0.693 usec per loop
$ python2.5 -m timeit -s"N = 1000; d = dict.fromkeys(range(N))" "N in
d.keys()"
10000 loops, best of 3: 77.2 usec per loop # ouch!

$ python2.5 -m timeit -s"N = 1; d = dict.fromkeys(range(N))" "N in d"
10000000 loops, best of 3: 0.192 usec per loop
~ $ python2.5 -m timeit -s"N = 1000; d = dict.fromkeys(range(N))" "N in d"
10000000 loops, best of 3: 0.192 usec per loop

$ python2.5 -m timeit -s"N = 1; d = dict.fromkeys(range(N))" "d.has_key(N)"
1000000 loops, best of 3: 0.376 usec per loop
$ python2.5 -m timeit -s"N = 1000; d = dict.fromkeys(range(N))"
"d.has_key(N)"
1000000 loops, best of 3: 0.385 usec per loop

Peter

[Roberto Bonvallet]

Fredrik Lundh wrote:
[...]

this is why e.g.

string[:len(prefix)] == prefix

is often a lot faster than

string.startswith(prefix)

This is interesting. In which cases does the former form perform better?
[I won't stop using str.startswith anyway :) ]

Regards.
--
Roberto Bonvallet

[Andy Dingley]

Paul Melis wrote:

I've always been using the has_key() method to test if a dictionary
contains a certain key. Recently I tried the same using 'in', e.g.

I've found using the set type to be the quickest way to do many of
these tasks. That leads me to another problem: how to cast / convert
sets as something else afterwards
What's the best (i.e. Pythonic) way to do this?
I need to generate a set (lots of intersections involved), but then I
need to display it sorted

lstBugsChanged = [ bugId for bugId in setBugsChanged ]
lstBugsChanged.sort()

[Fredrik Lundh]

Andy Dingley wrote:

I need to generate a set (lots of intersections involved), but then I
need to display it sorted

lstBugsChanged = [ bugId for bugId in setBugsChanged ]
lstBugsChanged.sort()

lstBugsChanged = sorted(setBugsChanged)

</F>

[Roberto Bonvallet]

Andy Dingley wrote:

I need to generate a set (lots of intersections involved), but then I
need to display it sorted

lstBugsChanged = [ bugId for bugId in setBugsChanged ]
lstBugsChanged.sort()

In Python 2.4:

sorted(setBugsChanged)

--
Roberto Bonvallet

[Roberto Bonvallet]

I wrote:

In Python 2.4:

lastline.replace(">", ">=")

--
Roberto Bonvallet

[Bjoern Schliessmann]

Peter Otten wrote:

No, 'k in d' is equivalent to 'd.has_key(k)', only with less
(constant) overhead for the function call.

Ah, thx. Thought the "x in d" syntax might search in d.values() too.

Regards,
Björn

--
BOFH excuse #12:

dry joints on cable plug

[Paul Melis]

Bjoern Schliessmann wrote:

Peter Otten wrote:

>No, 'k in d' is equivalent to 'd.has_key(k)', only with less
(constant) overhead for the function call.

Ah, thx. Thought the "x in d" syntax might search in d.values() too.

I don't think it does

Python 2.4.3 (#1, Nov 19 2006, 13:16:36)
[GCC 3.4.5 (Gentoo 3.4.5-r1, ssp-3.4.5-1.0, pie-8.7.9)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>d={1:'a',2:'b'}
1 in d

True

>>'a' in d

False
Paul

[Fredrik Lundh]

Roberto Bonvallet wrote:

>this is why e.g.

string[:len(prefix)] == prefix

is often a lot faster than

string.startswith(prefix)

This is interesting. In which cases does the former form perform better?

no time to doublecheck right now, but iirc, last time we benchmarked
this, slicing was faster when len(prefix) < 300 characters or so.

</F>

[Andy Dingley]

Roberto Bonvallet wrote:

lstBugsChanged = [ bugId for bugId in setBugsChanged ]

In Python 2.4:

Hmmm. Thanks. Another reason to twist the admin's arm and get them to
upgrade the last 2.3.4 boxen

sorted(setBugsChanged)

Out of interest, whats the Pythonic way to simply cast (sic) the set to
a list, assuming I don't need it sorted? The list comprehension?

[Roberto Bonvallet]

Andy Dingley wrote:

Out of interest, whats the Pythonic way to simply cast (sic) the set to
a list, assuming I don't need it sorted? The list comprehension?

mySet = set(myList)
myList = list(mySet)

--
Roberto Bonvallet

[Peter Otten]

Andy Dingley wrote:

> sorted(setBugsChanged)

Out of interest, whats the Pythonic way to simply cast (sic) the set to
a list, assuming I don't need it sorted? The list comprehension?

list(setBugsChanged)

Peter

[Paul McGuire]

"Peter Otten" <__*******@web.dewrote in message
news:el*************@news.t-online.com...

Andy Dingley wrote:

>> sorted(setBugsChanged)

>Out of interest, whats the Pythonic way to simply cast (sic) the set to
a list, assuming I don't need it sorted? The list comprehension?

list(setBugsChanged)

Peter

Note that this is not really a "cast" in the C sense of the word.
list(setBugsChanged) constructs and returns a new list containing the
elements of setBugsChanges, and leaves setBugsChanged unchanged.

-- Paul

[Bjoern Schliessmann]

Paul Melis wrote:

I don't think it does

Thanks for trying, I was too lazy ;)

Regards,
Björn

--
BOFH excuse #52:

Smell from unhygienic janitorial staff wrecked the tape heads

[Duncan Booth]

Paul Melis <pa**@floorball-flamingos.nlwrote:

>Ah, thx. Thought the "x in d" syntax might search in d.values() too.

I don't think it does

Python 2.4.3 (#1, Nov 19 2006, 13:16:36)
[GCC 3.4.5 (Gentoo 3.4.5-r1, ssp-3.4.5-1.0, pie-8.7.9)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>>d={1:'a',2:'b'}
1 in d

True

>>>'a' in d

False

It is easy enough to to check if you remember that 'in' maps to the
__contains__ method:

>>help({}.__contains__)

Help on built-in function __contains__:

__contains__(...)
D.__contains__(k) -True if D has a key k, else False

[skip]

PeterBjoern Schliessmann wrote:

>Wouldn't be "if k in d.keys()" be the exact replacement?

PeterNo, 'k in d' is equivalent to 'd.has_key(k)', only with less
Peter(constant) overhead for the function call. 'k in d.keys()' on the
Peterother hand creates a list of keys which is then searched linearly
Peter-- about the worst thing you can do about both speed and memory
Peterfootprint.

I will admit that way back when (maybe 8 yrs ago) I actually did this in a
piece of frequently executed code that's been stable for a looong time. I
have no idea why I might have written it that way. Brain fart I suppose. I
only noticed my mistake a couple months ago during a trip into the
containing function for some other stuff. Man, was I embarrassed...

Skip

[Hendrik van Rooyen]

<sk**@pobox.comwrote:

PeterBjoern Schliessmann wrote:

>Wouldn't be "if k in d.keys()" be the exact replacement?

PeterNo, 'k in d' is equivalent to 'd.has_key(k)', only with less
Peter(constant) overhead for the function call. 'k in d.keys()' on the
Peterother hand creates a list of keys which is then searched linearly
Peter-- about the worst thing you can do about both speed and memory
Peterfootprint.

I will admit that way back when (maybe 8 yrs ago) I actually did this in a
piece of frequently executed code that's been stable for a looong time. I
have no idea why I might have written it that way. Brain fart I suppose. I
only noticed my mistake a couple months ago during a trip into the
containing function for some other stuff. Man, was I embarrassed...

why? - the main point is actually that the code worked, and was stable -
that should make you proud, not embarrassed. I think that there is far too much
emphasis on doing things the quickest way - as long as it works, and is fast
enough,
its not broken, so don't fix it...

- Hendrik

[Stephen Eilert]

Hendrik van Rooyen escreveu:

<sk**@pobox.comwrote:

PeterBjoern Schliessmann wrote:

>Wouldn't be "if k in d.keys()" be the exact replacement?

PeterNo, 'k in d' is equivalent to 'd.has_key(k)', only with less
Peter(constant) overhead for the function call. 'k in d.keys()' on the
Peterother hand creates a list of keys which is then searched linearly
Peter-- about the worst thing you can do about both speed and memory
Peterfootprint.

I've always used has_key(), thinking it was the only way to do it.
Given that Python says that "There Should Be Only One Way to Do It", I
didn't bother searching for alternatives.

Is there a list somewhere listing those not-so-obvious-idioms? I've
seen some in this thread (like the replacement for .startswith).

I do think that, if it is faster, Python should translate
"x.has_key(y)" to "y in x".
Stephen

[Georg Brandl]

Stephen Eilert wrote:

Hendrik van Rooyen escreveu:

><sk**@pobox.comwrote:

PeterBjoern Schliessmann wrote:
>Wouldn't be "if k in d.keys()" be the exact replacement?

PeterNo, 'k in d' is equivalent to 'd.has_key(k)', only with less
Peter(constant) overhead for the function call. 'k in d.keys()' on the
Peterother hand creates a list of keys which is then searched linearly
Peter-- about the worst thing you can do about both speed and memory
Peterfootprint.

I've always used has_key(), thinking it was the only way to do it.
Given that Python says that "There Should Be Only One Way to Do It", I
didn't bother searching for alternatives.

Is there a list somewhere listing those not-so-obvious-idioms? I've
seen some in this thread (like the replacement for .startswith).

I do think that, if it is faster, Python should translate
"x.has_key(y)" to "y in x".

How and when should it do that?

Georg

[Steven D'Aprano]

On Thu, 07 Dec 2006 04:08:18 -0800, Stephen Eilert wrote:

Given that Python says that "There Should Be Only One Way to Do It",

Python says no such thing.

Perhaps you should run "import this" at the prompt, and read _carefully_
and take note of the difference between the Zen of Python and what you
wrote, because the difference is very, very significant.

--
Steven.

[Duncan Booth]

"Stephen Eilert" <sp******@gmail.comwrote:

I've always used has_key(), thinking it was the only way to do it.
Given that Python says that "There Should Be Only One Way to Do It", I
didn't bother searching for alternatives.

The full quote is actually:
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.

It might be more accurate to add a third line:
And the one true way may change with new versions of Python

In this case, dict objects used to not support the 'in' operator, but at
some point it was added. I believe it wasn't there originally because Guido
wasn't sure whether people would expect it should match keys or keys and
values.

>
Is there a list somewhere listing those not-so-obvious-idioms? I've
seen some in this thread (like the replacement for .startswith).

The release notes for each new version. Unfortunately the rest of the
documentation sometimes lags behind the release notes.

I do think that, if it is faster, Python should translate
"x.has_key(y)" to "y in x".

Python doesn't know whether x has a __contains__ method until it tries to
call it. In particular there may be user-defined dictionary-like objects in
your program which support has_key but not 'in'. e.g. in at least some
versions of Zope HTTPRequest objects support has_key but not __contains__.

[Daniel Dittmar]

Duncan Booth wrote:

In this case, dict objects used to not support the 'in' operator, but at
some point it was added. I believe it wasn't there originally because Guido
wasn't sure whether people would expect it should match keys or keys and
values.

And he was right:

import sys
'sys' in sys.modules =True
sys in sys.modules =False

Doesn't this look wrong?

Daniel

[Stephen Eilert]

Duncan Booth escreveu:

>

Is there a list somewhere listing those not-so-obvious-idioms? I've
seen some in this thread (like the replacement for .startswith).

The release notes for each new version. Unfortunately the rest of the
documentation sometimes lags behind the release notes.

Right. If and when time allows, I'll try to compile a list.

>

I do think that, if it is faster, Python should translate
"x.has_key(y)" to "y in x".

Python doesn't know whether x has a __contains__ method until it tries to
call it. In particular there may be user-defined dictionary-like objects in
your program which support has_key but not 'in'. e.g. in at least some
versions of Zope HTTPRequest objects support has_key but not __contains__.

You're right. Brain fart.
Stephen

[Fredrik Lundh]

Stephen Eilert wrote:

I do think that, if it is faster, Python should translate
"x.has_key(y)" to "y in x".

http://svn.python.org/view/sandbox/t...py?view=markup

</F>

[skip]

>I will admit that way back when (maybe 8 yrs ago) I actually did this
in a piece of frequently executed code that's been stable for a
looong time. I have no idea why I might have written it that way.
Brain fart I suppose. I only noticed my mistake a couple months ago
during a trip into the containing function for some other stuff.
Man, was I embarrassed...

Hendrikwhy? - the main point is actually that the code worked, and was
Hendrikstable - that should make you proud, not embarrassed. that
Hendrikthere is far too much emphasis on doing things the quickest way
Hendrik- as long as it works, and is fast enough, its not broken, so
Hendrikdon't fix it...

That's the rub. It wasn't fast enough. I only realized that had been a
problem once I fixed it though.

Skip

[Danny Colligan]

Is there a list somewhere listing those not-so-obvious-idioms?

I don't know about lists of not-so-obvious idioms, but here's some
gotchas (there may be some overlap with what you're asking about):

http://zephyrfalcon.org/labs/python_pitfalls.html
http://www.ferg.org/projects/python_gotchas.html
http://www.onlamp.com/pub/a/python/2...rn_python.html

Danny

[John Machin]

Daniel Dittmar wrote:

Duncan Booth wrote:

In this case, dict objects used to not support the 'in' operator, but at
some point it was added. I believe it wasn't there originally because Guido
wasn't sure whether people would expect it should match keys or keys and
values.

And he was right:

import sys
'sys' in sys.modules =True
sys in sys.modules =False

Doesn't this look wrong?

No it doesn't look wrong to anyone who has read the docs on
sys.modules. And it's irrelevant. The comparison should be between
'sys' in sys.modules # match keys
and
('sys', sys) in sys.modules # match keys and values
not
sys in sys.modules # match values

In any case, dict values can be any old object and may not even support
an equality test -- matching keys only seems the only reasonable choice
to me.

Cheers,
John

[George Sakkis]

Danny Colligan wrote:

Is there a list somewhere listing those not-so-obvious-idioms?

I don't know about lists of not-so-obvious idioms, but here's some
gotchas (there may be some overlap with what you're asking about):

http://zephyrfalcon.org/labs/python_pitfalls.html
http://www.ferg.org/projects/python_gotchas.html
http://www.onlamp.com/pub/a/python/2...rn_python.html

Danny

I'm surprized that none of these pages mentions the incompatible type
comparison gotcha:

>>5 < "4"

True

I'm sure this has bitten many folks, particularly (ex) Perl'ers.

George

[Steven D'Aprano]

On Thu, 07 Dec 2006 13:52:33 -0800, George Sakkis wrote:

I'm surprized that none of these pages mentions the incompatible type
comparison gotcha:

>>>5 < "4"

True

I'm sure this has bitten many folks, particularly (ex) Perl'ers.

Why is this a gotcha?

I can't speak for others, but except for sorting, I've never been tempted
to compare ints to strings like that, but thinking about the code I've
written, I can't think of any bad consequences that would have happened if
I had.

I'm not saying that there are never bad consequences for comparing
incompatible types, but I'm questioning that it is a gotcha, let alone a
common gotcha. What do others think? Ever been bitten by 5 < "4"?

--
Steven.

[Ben Finney]

"Stephen Eilert" <sp******@gmail.comwrites:

Is there a list somewhere listing those not-so-obvious-idioms?

They're only not-so-obvious to those who learn one version of Python
and then ignore release notes on future versions. The "k in some_dict"
was a wonderful thing to read in the release notes, and I was glad
enough that I never forgot it :-)

Hmm, that sounds rather snarky. I guess I'm only saying that one's
duty to oneself as a programmer is to keep up with the improved idioms
as they appear in new releases of one's language of choice.

--
\ "Money is always to be found when men are to be sent to the |
`\ frontiers to be destroyed: when the object is to preserve them, |
_o__) it is no longer so." -- Voltaire, _Dictionnaire Philosophique_ |
Ben Finney

[Gabriel Genellina]

At Thursday 7/12/2006 19:45, Steven D'Aprano wrote:

I'm surprized that none of these pages mentions the incompatible type
comparison gotcha:

>>5 < "4"

True

I'm sure this has bitten many folks, particularly (ex) Perl'ers.

Why is this a gotcha?

I can't speak for others, but except for sorting, I've never been tempted
to compare ints to strings like that, but thinking about the code I've
written, I can't think of any bad consequences that would have happened if
I had.

I'm not saying that there are never bad consequences for comparing
incompatible types, but I'm questioning that it is a gotcha, let alone a
common gotcha. What do others think? Ever been bitten by 5 < "4"?

Yes. By example, on a web application. If you forget to convert the
form or query values to their correct type, you could mix things
inadvertidely like that.
Of course, not exactly as written, but

def test(self, value):
if value>self.value: self.value=value

(when self.value is an int, but value comes as a string)

As always, once you know that, you make sure it wont happen again...
--
Gabriel Genellina
Softlab SRL

__________________________________________________
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis!
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar

[Hendrik van Rooyen]

<sk**@pobox.comwrote:

Hendrik- as long as it works, and is fast enough, its not broken, so
Hendrikdon't fix it...

That's the rub. It wasn't fast enough. I only realized that had been a
problem once I fixed it though.

LOL - this is kind of weird - it was working, nobody complained, you fiddled
with it to make it faster, (just because you could, not because you had to, or
were asked to), it became faster, and then, suddenly, retrospectively,
it became a problem ????

Would it have been no problem if it so happened that you were unable to make it
go faster?

I don't really follow that logic - but then I have never believed that I could
change yesterday...

;-) - Hendrik

[George Sakkis]

Steven D'Aprano wrote:

On Thu, 07 Dec 2006 13:52:33 -0800, George Sakkis wrote:

I'm surprized that none of these pages mentions the incompatible type
comparison gotcha:

>>5 < "4"

True

I'm sure this has bitten many folks, particularly (ex) Perl'ers.

Why is this a gotcha?

I can't speak for others, but except for sorting,

Well, sorting alone is a big issue, don't you think ?

I've never been tempted to compare ints to strings like that, but thinking about the code I've
written, I can't think of any bad consequences that would have happened if
I had.

Of course nobody (except perhaps from newbies coming from perl) would
intentionally compare ints with strings. The gotcha is when you compare
two values whose type you *think* you know, and since duck typing is
typically preferred over explicit type checking, it's one of the very
few cases that the "errors should never pass silently" rule is
violated. Thankfully this will raise TypeError in 3.0.

George

[Roel Schroeven]

Hendrik van Rooyen schreef:

<sk**@pobox.comwrote:

> Hendrik- as long as it works, and is fast enough, its not broken, so
Hendrikdon't fix it...

That's the rub. It wasn't fast enough. I only realized that had been a
problem once I fixed it though.

LOL - this is kind of weird - it was working, nobody complained, you fiddled
with it to make it faster, (just because you could, not because you had to, or
were asked to), it became faster, and then, suddenly, retrospectively,
it became a problem ????

Would it have been no problem if it so happened that you were unable to make it
go faster?

I don't really follow that logic - but then I have never believed that I could
change yesterday...

Have you never experienced the following:

A customer reports a bug. Upon investaging you find the source of the
problem, but from studying the code you don't understand anymore how it
has ever been able to function correctly. From that moment, it indeed
stops working even on computers where it always have worked correctly.

You fix the bug and all is well again.

Very strange, but it has happened to me on a few occasions. There's
probably a perfectly logical explanation for what happened, but I never
found it.

--
If I have been able to see further, it was only because I stood
on the shoulders of giants. -- Isaac Newton

Roel Schroeven

[Daniel Dittmar]

John Machin wrote:

No it doesn't look wrong to anyone who has read the docs on
sys.modules.

My point was really that there is no obvious implementation for 'in' on
dictionaries, so it should have been left out. And that GvR thought so
for quite some time as well (until he got mixed up with a crowd of ex
C++ programmers).

Daniel

[Roel Schroeven]

Roel Schroeven schreef:

Have you never experienced the following:

A customer reports a bug. Upon investaging you find the source of the
problem, but from studying the code you don't understand anymore how it
has ever been able to function correctly. From that moment, it indeed
stops working even on computers where it always have worked correctly.

You fix the bug and all is well again.

Very strange, but it has happened to me on a few occasions. There's
probably a perfectly logical explanation for what happened, but I never
found it.

BTW this is known as a Schroedinbug.

--
If I have been able to see further, it was only because I stood
on the shoulders of giants. -- Isaac Newton

Roel Schroeven

[Marc 'BlackJack' Rintsch]

In <el**********@news.sap-ag.de>, Daniel Dittmar wrote:

John Machin wrote:

>No it doesn't look wrong to anyone who has read the docs on
sys.modules.

My point was really that there is no obvious implementation for 'in' on
dictionaries, so it should have been left out. And that GvR thought so
for quite some time as well (until he got mixed up with a crowd of ex
C++ programmers).

Maybe there's no obvious implementation for ``in`` and dictionaries but
there is the semantics of ``in`` and iterables. Dictionaries are iterable
so it's clearly defined what ``'answer' in {'answer': 42}`` should return.

In [84]: class A:
....: def __iter__(self):
....: return iter((1,2,3))
....:

In [85]: a = A()

In [86]: 2 in a
Out[86]: True

In [87]: 5 in a
Out[87]: False

If ``in`` shouldn't work with dictionaries, either `__contains__()` must
be implemented to throw an exception or dictionaries shouldn't be iterable.

Ciao,
Marc 'BlackJack' Rintsch

[Daniel Dittmar]

Marc 'BlackJack' Rintsch wrote:

If ``in`` shouldn't work with dictionaries, either `__contains__()` must
be implemented to throw an exception or dictionaries shouldn't be iterable.

I agree completely (in the sense that dictionaries shouldn't be iterable
directly). Probably even more strongly, at least every time I see some
code where someone iterates over the keys, only to use the key to look
up the value (instead if using iteritms).

For strings, the 1:1 relationship between 'in' and iteration has already
been broken. But then, iteration over strings happens in my code only
when a pass a string where I should have passed a list of strings.

Daniel

[Fredrik Lundh]

Daniel Dittmar wrote:

I agree completely (in the sense that dictionaries shouldn't be iterable
directly). Probably even more strongly, at least every time I see some
code where someone iterates over the keys, only to use the key to look
up the value (instead if using iteritms).

so? that's the original Python way of doing things, and is both very
readable and surprisingly efficient:

$ timeit -s "d = vars()" "for k in d: v = d[k]"
1000000 loops, best of 3: 0.472 usec per loop

$ timeit -s "d = vars()" "for k, v in d.iteritems(): pass"
1000000 loops, best of 3: 0.663 usec per loop

</F>

[Stephen Eilert]

Ben Finney escreveu:

"Stephen Eilert" <sp******@gmail.comwrites:

Is there a list somewhere listing those not-so-obvious-idioms?

They're only not-so-obvious to those who learn one version of Python
and then ignore release notes on future versions. The "k in some_dict"
was a wonderful thing to read in the release notes, and I was glad
enough that I never forgot it :-)

Hmm, that sounds rather snarky. I guess I'm only saying that one's
duty to oneself as a programmer is to keep up with the improved idioms
as they appear in new releases of one's language of choice.

That's fine.

I do need to read those release notes retroactively, though.
Stephen

[Hendrik van Rooyen]

"Roel Schroeven" <rs****************@fastmail.fmwrote:

Hendrik van Rooyen schreef:

<sk**@pobox.comwrote:

Hendrik- as long as it works, and is fast enough, its not broken, so
Hendrikdon't fix it...

That's the rub. It wasn't fast enough. I only realized that had been a
problem once I fixed it though.

LOL - this is kind of weird - it was working, nobody complained, you fiddled
with it to make it faster, (just because you could, not because you had to,

or

were asked to), it became faster, and then, suddenly, retrospectively,
it became a problem ????

Would it have been no problem if it so happened that you were unable to make

it

go faster?

I don't really follow that logic - but then I have never believed that I

could

change yesterday...

Have you never experienced the following:

A customer reports a bug. Upon investaging you find the source of the
problem, but from studying the code you don't understand anymore how it
has ever been able to function correctly. From that moment, it indeed
stops working even on computers where it always have worked correctly.

You fix the bug and all is well again.

Very strange, but it has happened to me on a few occasions. There's
probably a perfectly logical explanation for what happened, but I never
found it.

This is simply a manifestation of the faith that can move mountains - while
everybody believed that it was working, it did, and stopped working only because
of the heretical doubt of some infidel...

:-) - Hendrik

[Steven D'Aprano]

On Fri, 08 Dec 2006 17:42:56 +0100, Daniel Dittmar wrote:

Marc 'BlackJack' Rintsch wrote:

>If ``in`` shouldn't work with dictionaries, either `__contains__()` must
be implemented to throw an exception or dictionaries shouldn't be iterable.

I agree completely (in the sense that dictionaries shouldn't be iterable
directly).

Why on earth not???

Probably even more strongly, at least every time I see some
code where someone iterates over the keys, only to use the key to look
up the value (instead if using iteritms).

Is there really that much difference between the two?

>>D = dict(zip(xrange(10000), xrange(10000)))
def d1(D):

.... for key in D:
.... value = D[key]
....

>>def d2(D):

.... for key, value in D.iteritems():
.... pass
....

>>timeit.Timer("d1(D)", "from __main__ import d1, d2, D").timeit(1000)

5.9806718826293945

>>timeit.Timer("d2(D)", "from __main__ import d1, d2, D").timeit(1000)

4.7772250175476074

Using iteritems is slightly faster, and probably more Pythonic, but the
difference isn't so great to argue that "for key in dict" is always the
wrong thing to do.

One could argue until the cows come home whether "for x in dict" should
iterate over keys, values or key/value pairs, but it really doesn't
matter. All three possibilities are available, all three values are
useful, and having to copy the keys/values into a list in order to iterate
over them is just wasteful.
--
Steven.

[skip]

>>>>"Hendrik" == Hendrik van Rooyen <ma**@microcorp.co.zawrites:

Hendrik<sk**@pobox.comwrote:
Hendrik- as long as it works, and is fast enough, its not broken, so
Hendrikdon't fix it...

>That's the rub. It wasn't fast enough. I only realized that had
been a problem once I fixed it though.

HendrikLOL - this is kind of weird - it was working, nobody
Hendrikcomplained, you fiddled with it to make it faster, (just
Hendrikbecause you could, not because you had to, or were asked to),
Hendrikit became faster, and then, suddenly, retrospectively, it
Hendrikbecame a problem ????

No, I think you misunderstand. I was poking around in that function for
other reasons, saw the "k in d.keys()" and realized that the wrong way to
write it. Rewrote it and noticed the performance increase. What's so weird
about that?

Skip

[Hendrik van Rooyen]

<sk**@pobox.comwrote:

>>>"Hendrik" == Hendrik van Rooyen <ma**@microcorp.co.zawrites:

Hendrik<sk**@pobox.comwrote:
Hendrik- as long as it works, and is fast enough, its not broken, so
Hendrikdon't fix it...

>That's the rub. It wasn't fast enough. I only realized that had
>been a problem once I fixed it though.

HendrikLOL - this is kind of weird - it was working, nobody
Hendrikcomplained, you fiddled with it to make it faster, (just
Hendrikbecause you could, not because you had to, or were asked to),
Hendrikit became faster, and then, suddenly, retrospectively, it
Hendrikbecame a problem ????

No, I think you misunderstand. I was poking around in that function for
other reasons, saw the "k in d.keys()" and realized that the wrong way to
write it. Rewrote it and noticed the performance increase. What's so weird
about that?

Nothing wrong with that - it is meet and good to make improvements as you spot
them - what seemed weird to me was not that you made the change, but that you
described it as a problem, retrospectively - why I say this, I expect, is that
in my
mind, such a definition would list as a "problem" all working code that could,
by rummaging around in it, be improved in some way.

I suspect that such a definition would condemn almost *all* production code to
problem status...

I just don't see it that way - my attitude, I suppose, is a wimpy one, as I
think that working code in production use should be left alone, as every
time you touch it, there is a finite non zero probability that you will do
something stupid to break it, and the breakage may be subtle, and
undiscovered for a while, and then it will really be a problem...

So the weirdness for me was that you called a piece of perfectly good working
code a problem, just because it was faster after you fixed it, even though you
had no prior knowledge of the offending line that was doing the right thing,
albeit slowly. - I would only have called such a piece of code a problem if I
had been inundated with prior requests to fix the slow-running dog...

I dunno if all this makes any sense - and we are getting way off topic. :-)

- Hendrik

[Kent Johnson]

Fredrik Lundh wrote:

Stephen Eilert wrote:

>I do think that, if it is faster, Python should translate
"x.has_key(y)" to "y in x".

http://svn.python.org/view/sandbox/t...py?view=markup

Seems to have moved to here:
http://svn.python.org/view/sandbox/t...py?view=markup

>
</F>