473,573 Members | 2,259 Online

# Rationale behind the deprecation of __getslice__?

Hi all,

I was wondering if someone can help me understand why __getslice__ has been
deprecated, yet it remains necessary to implement it for simple slices (i:j),
while __getitem__ gets called for extended slices (i:j:k).

The problem with this approach, besides a bit of code duplication, is that
classes which implement slicing must now do runtime type-checking inside
__getitem__. Here's a trivial example:
############### ############### ############### ############### ####
import types

class Vector(list):
def __init__(self,a rg,wrap=0):
'''
If arg is an integer, make a new zero vector of length n.
Otherwise, arg must be a python sequence or another vector,
and a new (deep) copy is generated.
'''
if isinstance(arg, int):
list.__init__(s elf,[0.0]*arg)
else:
list.__init__(s elf,arg)

def __getitem__(sel f,key):
"""called for single-element OR slice access"""
if type(key) is types.SliceType :
return Vector(list.__g etitem__(self,k ey))
else:
return list.__getitem_ _(self,key)

def __getslice__(se lf,i,j):
"""Deprecat ed since 2.0, but still called for non-extended slices.

Since extended slices are handled by __getitem__, I'm just deferring
to that one so all actual implementation is done there. Why this is
not the default (since they deprecated __getslice__) is beyond me."""

return self.__getitem_ _(slice(i,j))

print 'a is a vector'
a = Vector(5)
print a
print type(a)
print

print 'b is a slice of a'
b = a[1:3]
print b
print type(b)
print

print 'c is an element of a'
c = a[1]
print c
print type(c)

############### ############### ############### ############### ####

What bugs me is that typecheck for slicing which seems to be necessary inside
of the __getitem__ method. I have the feeling that the code would be much
cleaner if I could simply use __getslice__ for slices and __getitem__ for
items, and that bundling the two in this hybrid mode is reall ugly and
unpythonic. Am I just missing something?

Thanks for any help,

f

Jul 18 '05 #1
18 2726
Fernando Perez wrote:
I was wondering if someone can help me understand why __getslice__ has been
deprecated, yet it remains necessary to implement it for simple slices (i:j),
while __getitem__ gets called for extended slices (i:j:k).

I don't think this is true -- everything goes to __getitem__:
class C(object): .... def __getitem__(sel f, *args, **kwds):
.... print args, kwds
.... c = C()
c[1] (1,) {} c[1:2] (slice(1, 2, None),) {} c[1:2:-1] (slice(1, 2, -1),) {}

All you have to do is check the type of the single argument to __getitem__:
class C(object): .... def __getitem__(sel f, x):
.... if isinstance(x, slice):
.... return x.indices(10)
.... else:
.... return x
.... c = C()
c[1] 1 c[1:2] (1, 2, 1) c[1:2:-1]

(1, 2, -1)

Steve
Jul 18 '05 #2
Fernando Perez wrote:
classes which implement slicing must now do runtime type-checking inside
__getitem__.

Just in case you thought that they wouldn't have to do runtime
type-checking otherwise:
class C(object): .... def __getitem__(sel f, x):
.... print type(x), x
.... c = C()
c[1] <type 'int'> 1 c[1:2] <type 'slice'> slice(1, 2, None) c[1:2:-1] <type 'slice'> slice(1, 2, -1) c[1,2] <type 'tuple'> (1, 2) c[1,2:3] <type 'tuple'> (1, slice(2, 3, None)) c['1'] <type 'str'> 1 c[[]]

<type 'list'> []

You can put just about anything in a __getitem__ call. Do you really
want a method for each of the variants above?

Steve
Jul 18 '05 #3
Steven Bethard wrote:
Fernando Perez wrote:
I was wondering if someone can help me understand why __getslice__ has been
deprecated, yet it remains necessary to implement it for simple slices
(i:j), while __getitem__ gets called for extended slices (i:j:k).

I don't think this is true -- everything goes to __getitem__:
>>> class C(object): ... def __getitem__(sel f, *args, **kwds):
... print args, kwds
... >>> c = C()
>>> c[1] (1,) {} >>> c[1:2] (slice(1, 2, None),) {} >>> c[1:2:-1]

(slice(1, 2, -1),) {}

Not if you subclass builtin types like list:

In [6]: class mylist(list):
...: def __getitem__(sel f,*args,**kwds) :
...: print 'mylist getitem'
...: print args,kwds
...:

In [7]: a=mylist()

In [8]: a[1]
mylist getitem
(1,) {}

In [9]: a[1:2]
Out[9]: []

In [10]: a[1:2:3]
mylist getitem
(slice(1, 2, 3),) {}

I did this testing, which is what forced me to implement __getslice__
separately in my little example, to satisfy calls with simple i:j slices.

Best,

f

Jul 18 '05 #4
Steven Bethard wrote:
Fernando Perez wrote:
classes which implement slicing must now do runtime type-checking inside
__getitem__.
Just in case you thought that they wouldn't have to do runtime
type-checking otherwise:
>>> class C(object):

... def __getitem__(sel f, x):
... print type(x), x
...

[...]
You can put just about anything in a __getitem__ call. Do you really
want a method for each of the variants above?

I guess that conceptually it just felt natural to me to keep separate methods
for dealing with a slice (get many elements out) and other types of indexing,
which I tend to think of as 'scalar' indexing.

Regards,

f

Jul 18 '05 #5
Fernando Perez wrote:
I was wondering if someone can help me understand why __getslice__ has been deprecated, yet it remains necessary to implement it for simple slices (i:j), while __getitem__ gets called for extended slices (i:j:k).

The problem with this approach, besides a bit of code duplication, is that classes which implement slicing must now do runtime type-checking inside __getitem__.

I'm pretty sure it's to support multidimensiona l array slicing.
Consider an array reference such as a[1,2:5,4,6:9,10:]. Now what do
you do? You have mixed slices and indices. The
__getslice__/__getitem__ paradigm isn't versatile enough to handle this
situation. In that light, I'd say checking for slices is the lesser
evil.

As for why list objects still use getslice--they probably shouldn't.
I'd file a bug report.
--
CARL BANKS

Jul 18 '05 #6
Fernando Perez wrote:
Steven Bethard wrote:
Fernando Perez wrote:
I was wondering if someone can help me understand why __getslice__ has been
deprecated , yet it remains necessary to implement it for simple slices
(i:j), while __getitem__ gets called for extended slices (i:j:k).
I don't think this is true -- everything goes to __getitem__:
[snip]
Not if you subclass builtin types like list:

Ahh, I didn't catch that your problem was with list. Yeah, so if a
__getslice__ exists, this is used first. Unfortunately, by inheriting
from list, you inherit __getslice__ from list. Another example without
builtin types:
class C(object): .... def __getitem__(sel f, x):
.... print "C:getitem"
.... def __getslice__(se lf, *args):
.... print "C:getslice "
.... class D(C): .... def __getitem__(sel f, x):
.... print "D:getitem"
.... d = D()
d[1] D:getitem d[1:2] C:getslice d[1:2:-1]

D:getitem

While the D class doesn't define __getslice__, it's parent class does,
so it has the same behavior you're running into. I don't see how to fix
this other than overriding __getslice__ to call __getitem__ like you have.

Unfortunately, I don't think __getslice__ can be removed from list (and
str and tuple) because of backwards compatibility constraints...

Steve
Jul 18 '05 #7
Steven Bethard wrote:
Unfortunately, I don't think __getslice__ can be removed from list (and str and tuple) because of backwards compatibility constraints...

Wouldn't it work to have __getslice__ call __getitem__? And, since
that would be too much of a performance hit, have it check whether its
type is list (or str or tuple), and only call __getitem__ if it is not
(i.e., only for subclasses). I don't think that would be too bad.

Subclasses would still be free to override __getslice__, but wouldn't
have to.
--
CARL BANKS

Jul 18 '05 #8
Fernando Perez wrote:
I guess that conceptually it just felt natural to me to keep separate methods
for dealing with a slice (get many elements out) and other types of indexing,
which I tend to think of as 'scalar' indexing.

Yeah, I can see that a bit.

Ignoring dicts for the moment (and concerning ourselves only with
"sequences" ), you're probably right in thinking that that slice objects
are the second most common thing to get in __getitem__ (second to ints
of course). But there is heavy use of other objects in various other
modules, most notably tuples in numarray:
import numarray as na
a = na.arange(24, shape=(2, 3, 4))
a array([[[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]],

[[12, 13, 14, 15],
[16, 17, 18, 19],
[20, 21, 22, 23]]]) a[0,1,2] 6 a[0,1] array([4, 5, 6, 7]) a[...,3] array([[ 3, 7, 11],
[15, 19, 23]]) a[1,:,0]

array([12, 16, 20])

Presumably the numarray code has to do quite a bit of type checking to
perform all these slicings right (and I didn't even show you what
happens when you use another array as an "index"). I'm not necessarily
saying that all this type checking is a good thing, but because people
will always find new things that they want to index by, adding
__getxxx__ methods for each of the index types is probably not the right

Steve
Jul 18 '05 #9
Carl Banks wrote:
As for why list objects still use getslice--they probably shouldn't.
I'd file a bug report.

I'm not convinced this is actually a bug; it works just like the docs
promise:

------------------------------------------------------------
http://docs.python.org/ref/sequence-methods.html
__getslice__( self, i, j)
....
Called to implement evaluation of self[i:j].
....
If no __getslice__() is found, a slice object is created instead, and
....
For slice operations involving extended slice notation, or in absence of
the slice methods, __getitem__(), __setitem__() or __delitem__() is
called with a slice object as argument.
------------------------------------------------------------

So the docs imply that if __getslice__ exists, it will be used before
trying __getitem__. Since list defines __getslice__, list.__getslice __
will be used before __getitem__ in any class that inherits from list.

This is certainly a wart though. I'd love to see list.__getslice __
removed, leaving only list.__getitem_ _, but I suspect that this kind of
backwards-incompatible change would be frowned on...

Guess we could file a feature request?

Steve
Jul 18 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.