469,645 Members | 1,174 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,645 developers. It's quick & easy.

Strange Behavior with Old-Style classes and implicit __contains__

A co-worker of mine came across some interesting behavior in the
Python interpreter today and I'm hoping someone more knowledgeable in
Python internals can explain this to me.

First, we create an instance of an Old-Style class without defining a
__contains__ but instead define a __getitem__ method in which we raise
KeyError. Next we repeatedly use the 'in' operator to test to see
whether something, a string, an int, etc is an attribute of this new
instance.

Here's the strange part: The first test will return False as if the
__getitem__ was never called. The next test will raise a KeyError as
we'd expect. The test after that will again return False. This goes on
ad infinitum.

In Code:
Python 2.5 (r25:51908, Jan 21 2007, 03:10:25)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more
information.
>>class Foo:
... def __getitem__(self, key):
... raise KeyError
...
>>foo = Foo()
"asdf" in foo
False
>>"asdf" in foo
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __getitem__
KeyError
>>"asdf" in foo
False
>>"asdf" in foo
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __getitem__
KeyError
According to the language reference, if __contains__ isn't defined for
Old-Style classes and __getitem__ is defined, __getitem__ will be
called. So, how then can False ever be returned?

And to make matters worse, I've set up a situation where Python will
flat-out return incorrect results:

Python 2.5 (r25:51908, Jan 21 2007, 03:10:25)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>class Foo:
.... def __getitem__(self, key):
.... raise KeyError
....
>>foo = Foo()
"asdf" in foo
False
>>1 in set([1,2,3]) <---- So the prior KeyError from another class is interacting and producing bad output
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __getitem__
KeyError

According to our cursory testing, this funny business doesn't happen
with New-Style Classes or when using PyPy.

If anyone can provide some insight into this, it would be greatly
appreciated.

Thanks,
Rick Harris

Apr 11 '07 #1
5 1341
On Wed, 11 Apr 2007 16:37:35 -0700, rconradharris wrote:
A co-worker of mine came across some interesting behavior in the
Python interpreter today and I'm hoping someone more knowledgeable in
Python internals can explain this to me.

First, we create an instance of an Old-Style class without defining a
__contains__ but instead define a __getitem__ method in which we raise
KeyError. Next we repeatedly use the 'in' operator to test to see
whether something, a string, an int, etc is an attribute of this new
instance.

Here's the strange part: The first test will return False as if the
__getitem__ was never called. The next test will raise a KeyError as
we'd expect. The test after that will again return False. This goes on
ad infinitum.
I can confirm that. It looks like __getitem__ is only being called every
second time.

class Parrot:
def __getitem__(self, n):
print "Checking index %s..." % n
raise KeyError

def tester(n):
parrot = Parrot()
results = []
for i in range(n):
try:
results.append(i in parrot)
except KeyError:
results.append("KeyError")
return results

Here are the results under Python 2.5:
>>tester(10)
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
[False, 'KeyError', False, 'KeyError', False,
'KeyError', False, 'KeyError', False, 'KeyError']
And here are the results under Python 2.4.3:
>>tester(10)
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
['KeyError', 'KeyError', 'KeyError', 'KeyError', 'KeyError',
'KeyError', 'KeyError', 'KeyError', 'KeyError', 'KeyError']
Looks like a bug to me.

--
Steven.

Apr 12 '07 #2
En Wed, 11 Apr 2007 20:37:35 -0300, <rc***********@gmail.comescribió:
First, we create an instance of an Old-Style class without defining a
__contains__ but instead define a __getitem__ method in which we raise
KeyError. Next we repeatedly use the 'in' operator to test to see
whether something, a string, an int, etc is an attribute of this new
instance.

Here's the strange part: The first test will return False as if the
__getitem__ was never called. The next test will raise a KeyError as
we'd expect. The test after that will again return False. This goes on
ad infinitum.

In Code:
Python 2.5 (r25:51908, Jan 21 2007, 03:10:25)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] on linux2
Type "help", "copyright", "credits" or "license" for more
information.
>>class Foo:
... def __getitem__(self, key):
... raise KeyError
...
>>foo = Foo()
>>"asdf" in foo
False
>>"asdf" in foo
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __getitem__
KeyError
First I want to say that __getitem__ should raise IndexError, not
KeyError, to indicate "not found" - just to make clear the observed
behavior. (Using IndexError, you always get False, as expected).
Python 2.4 and 2.3 never return False, always showing the KeyError
exception, also as expected.
>>>foo = Foo()
"asdf" in foo
False
>>>1 in set([1,2,3]) <---- So the prior KeyError from another class
is interacting and producing bad output
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __getitem__
KeyError
I have a displayhook installed, and it also were interfering with the
results; it appears that the next __getitem__ call after the KeyError was
raised (wherever it is called) "sees" the previous exception.

You should file a bug at http://sourceforge.net/bugs/?group_id=5470

--
Gabriel Genellina

Apr 12 '07 #3
On Thu, 12 Apr 2007 00:32:32 -0300, Gabriel Genellina wrote:
First I want to say that __getitem__ should raise IndexError, not
KeyError, to indicate "not found"
How do you know the Original Poster's class was meant to be a sequence
rather than a mapping?

--
Steven.

Apr 12 '07 #4
Steven D'Aprano wrote:
On Wed, 11 Apr 2007 16:37:35 -0700, rconradharris wrote:
...
Here are the results under Python 2.5:
>>>tester(10)
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
Checking index 0...
[False, 'KeyError', False, 'KeyError', False,
'KeyError', False, 'KeyError', False, 'KeyError']
And here are the results under Python 2.4.3:
>>>tester(10)
[works]
>
Looks like a bug to me.
No problem with 2.5.1c1 here.

--Scott David Daniels
sc***********@acm.org
Apr 12 '07 #5
En Thu, 12 Apr 2007 01:23:08 -0300, Steven D'Aprano
<st***@REMOVE.THIS.cybersource.com.auescribió:
On Thu, 12 Apr 2007 00:32:32 -0300, Gabriel Genellina wrote:
>First I want to say that __getitem__ should raise IndexError, not
KeyError, to indicate "not found"

How do you know the Original Poster's class was meant to be a sequence
rather than a mapping?
Ouch, no, sorry, disregard that comment.

--
Gabriel Genellina

Apr 12 '07 #6

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

reply views Thread by Richard Hollenbeck | last post: by
1 post views Thread by Default | last post: by
5 posts views Thread by John | last post: by
4 posts views Thread by solex | last post: by
reply views Thread by birdinhand | last post: by
reply views Thread by ivb | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.