negative stride list slices

[David Abrahams]

Can anyone explain the logic behind the behavior of list slicing with
negative strides? For example:

print range(10)[:-3:-1]

[9,8]

I found this result very surprising, and would just like to see the
rules written down somewhere.

Thanks,
--
Dave Abrahams
Boost Consulting
http://www.boost-consulting.com

[Shalabh Chaturvedi]

Shalabh Chaturvedi wrote:

I'd like to add:

+---+---+---+---+---+
| H | e | l | p | A |
+---+---+---+---+---+
0 1 2 3 4 5
-5 -4 -3 -2 -1
| |
start end (defaults for +ve step)
end start (defaults for -ve step)

(Is this correct?)

Apparently not.

"HelpA"[-1:-2:-1] 'A'

I would expect 'p'. Also,
"HelpA"[:-1:-1]

''

And I definitely expected 'A'. Now I find it confusing too.

Does anyone else find it intuitive to expect that S[b:a:-1] be exactly
reverse of S[a:b:1]?

This is only true for S[::1] and S[::-1]. In other cases, for -ve step,
the -ve indices seem to indicate a shifted position. I'm using Python 2.3.3.

--
Shalabh

[Peter Hansen]

(Please don't top-post. It really buggers up the quoting
and makes discussion hard.)

Julio Oña wrote:

Slice has three arguments, [begin:end:step]

when doing s[:-3:-1] you are asking the las to elements of the list in
reversed order.

Uh, yeah. Okay. So let's say that part was obvious. Now
please explain *which* elements are being listed in reverse
order, referring to the index value -3 and the elided
index value. Presumably one of those refers to the beginning
or end of the list, while the other one refers to something
that is referenced as "-3". Can you point to a diagram or
description in the documentation or tutorial which actually
explains this? Or can you (or anyone) explain it in your
own words? Or is everyone who thinks this makes sense just
pretending to actually understand it?

-Peter

[Julio Oña]

On Wed, 01 Sep 2004 18:05:22 -0400, Peter Hansen <pe***@engcorp. com> wrote:

Uh, yeah. Okay. So let's say that part was obvious. Now
please explain *which* elements are being listed in reverse
order, referring to the index value -3 and the elided
index value. Presumably one of those refers to the beginning
or end of the list, while the other one refers to something
that is referenced as "-3". Can you point to a diagram or
description in the documentation or tutorial which actually
explains this? Or can you (or anyone) explain it in your
own words? Or is everyone who thinks this makes sense just
pretending to actually understand it?

Sorry Peter to be to lassy.

Well in the case [:-3:-1] I think this is what is happening
- revers the list
- this left with [0:2] (remember [-3:] returns 2 characters only

in the case [-1:-2:-1]
- revers the string 'Apleh'
- this left with [:1] (this is tricky but -1 is equivalent to 0 as
-2 is to 1, remember you are counting backwards from the end)

The problem is the mening of -1 when reversing the list. The slice is
closed in the lower side and open in the upper one. When reversing the
list the lower end is open and no close as with positive step.

Any comments?
--
Julio

[Tim Hochberg]

Peter Hansen wrote:

(Please don't top-post. It really buggers up the quoting
and makes discussion hard.)

Julio Oña wrote:

Slice has three arguments, [begin:end:step]

when doing s[:-3:-1] you are asking the las to elements of the list in
reversed order.

Uh, yeah. Okay. So let's say that part was obvious. Now
please explain *which* elements are being listed in reverse
order, referring to the index value -3 and the elided
index value. Presumably one of those refers to the beginning
or end of the list, while the other one refers to something
that is referenced as "-3". Can you point to a diagram or
description in the documentation or tutorial which actually
explains this? Or can you (or anyone) explain it in your
own words? Or is everyone who thinks this makes sense just
pretending to actually understand it?

I've been using extended slicing in Numeric and now numarray for years
and I even wrote some classes that implement extended slicing, but I
still often find it confusing to use negative indices in practice.
However, despite that confusion, I'll give a shot at explaining it.

There are (at least) two issues: what to do with missing indices and
given all the indices, what does it all mean. I'll start with the first
issue, since it's simpler. Of the three indices (start, stop and step),
only start and stop are potentially confusing, since step simply
defaults to 1. Start defaults to the first element (0) if step is
greater than zero, otherwise it defaults to the last element (len(s)-1).
Stop, on the other hand, defaults to one past the edge of the sequence.
If step is positive, it's one past the far edge (len(s)), otherwise it's
one past the near edge. One would be tempted to write this as -1, but
that won't works since that means the end of the sequence, so we'll call
it (-len(s)-1). In summary the missing indices are supplied as:

start = 0 if (step > 0) else len(s)-1
stop = len(s) is (step > 0) else -len(s)-1
step = 1

On to issue number two. I've always understood extended slicing in
relation to the range. In Numeric this relation is best expressed using
Numeric.take, however for general python consumption I'll cast it in
terms of list comprehensions. Since we've already disposed of how to
supply the missing parts of the slice, this will only deal with the case
where all parts are supplied. In addition, if start or stop is less than
zero, we add len(s) so that all values are nonnegative unless stop was
-len(s)-1, in which case it ends up as -1. With that wordy preamble,
here's how I've always understood extended slicing:

s[start:stop:step] ~ [s[i] for i in range(start,sto p,step)]

Let me show a few examples:

s = 'python' s[:-3:-1] 'no' s[len(s)-1:-3:-1] # Fill in the default values 'no' [s[i] for i in range(len(s)-1, len(s)-3,-1)] # equiv to s[:-3:-1] ['n', 'o']
s[4::] 'on' s[4:len(s):1] # Fill in the default values 'on' [s[i] for i in range(4,len(s), 1)] # equiv to s[4::] ['o', 'n']
s[4::-1] 'ohtyp' s[4:-len(s)-1:-1] # Fill in the default values 'ohtyp' [s[i] for i in range(4,-1,-1)] # equiv to s[4::-1]

['o', 'h', 't', 'y', 'p']
Is this confusing? Probably. However, I suspect it's a natural
consequence of Python's zero based indexing. Zero based indexing is
great in a lot of ways, but it has some unfortunate corner cases for
negative indices (another topic) and gets downright odd for negative
strides. So it goes.

I-hope-I-didn't screw-that-up-too-badly-ly yours,

-tim

[Peter Hansen]

Julio Oña wrote:

On Wed, 01 Sep 2004 18:05:22 -0400, Peter Hansen <pe***@engcorp. com> wrote:

please explain *which* elements are being listed in reverse
order, referring to the index value -3 and the elided
index value.

Well in the case [:-3:-1] I think this is what is happening
- revers the list
- this left with [0:2] (remember [-3:] returns 2 characters only

in the case [-1:-2:-1]
- revers the string 'Apleh'
- this left with [:1] (this is tricky but -1 is equivalent to 0 as
-2 is to 1, remember you are counting backwards from the end)

The problem is the mening of -1 when reversing the list. The slice is
closed in the lower side and open in the upper one. When reversing the
list the lower end is open and no close as with positive step.

Any comments?

Okay, so let me see how this looks with the previous diagram.

+---+---+---+---+---+
| H | e | l | p | A |
+---+---+---+---+---+
0 1 2 3 4 5
-5 -4 -3 -2 -1

But that only works for the forward or positive step case.
With a negative/reverse step, you first have to reverse
the string:

+---+---+---+---+---+
| A | p | l | e | H |
+---+---+---+---+---+
-1 -2 -3 -4 -5 -6

And now you're in effect going from the start (i.e. the
missing value before the colon) to the -3 point.

Thus 'Ap'.

I don't find it at all intuitive, and certainly the existing
diagram needs to be updated if this is really how it works,
but at least you have an explanation that corresponds to the
values, so thank you.

It does seem very odd that the step of -1 in effect changes
the normal meaning or position of the slice indices, by
first reversing the string. I'm guessing that this is
necessary because any other interpretation leads to even
worse situations than the confusion this inspires, but
perhaps that's not true.

I would have found either 'Apl' or 'eH' more intuitive...

-Peter

[Christopher Koppler]

On Wed, 01 Sep 2004 14:57:57 -0700, Shalabh Chaturvedi wrote:

Shalabh Chaturvedi wrote:

I'd like to add:

+---+---+---+---+---+
| H | e | l | p | A |
+---+---+---+---+---+
0 1 2 3 4 5
-5 -4 -3 -2 -1
| |
start end (defaults for +ve step)
end start (defaults for -ve step)

(Is this correct?)
Apparently not.

That should be

+---+---+---+---+---+
| H | e | l | p | A |
+---+---+---+---+---+
0 1 2 3 4 5
-6 -5 -4 -3 -2 -1
| |
start end (defaults for +ve step)
end start (defaults for -ve step)

>>> "HelpA"[-1:-2:-1] 'A'

I would expect 'p'. Also,
>>> "HelpA"[:-1:-1]

''

And I definitely expected 'A'. Now I find it confusing too.

Does anyone else find it intuitive to expect that S[b:a:-1] be exactly
reverse of S[a:b:1]?

As others have explained, that's not easily possible because of Python's
zero-based indexing, which in general is great, but can lead to confusion
because using negative strides indices begin with -1.

This is only true for S[::1] and S[::-1]. In other cases, for -ve step,
the -ve indices seem to indicate a shifted position. I'm using Python 2.3.3.

The defaults here map to S[0:len(S):1] and S[-1:-len(S)-1:-1].

--
Christopher

[Christopher T King]

On Wed, 1 Sep 2004, Shalabh Chaturvedi wrote:

Does anyone else find it intuitive to expect that S[b:a:-1] be exactly
reverse of S[a:b:1]?

No. S[a:b:1] includes the start index (a), but excludes the stop index
(b). Similarly, S[b:a:-1] includes the start index (b), but excludes the
stop index (a). Only if the stop indices were included (or the start
indices were excluded), thus making the indexing symmetrical, would I
expect the results to be symmetric.

[Michael Hudson]

da**@boost-consulting.com (David Abrahams) writes:

Can anyone explain the logic behind the behavior of list slicing with
negative strides?
You can read PySlice_GetIndi cesEx in Objects/sliceobject.c if you're
really keen...
For example:

>>> print range(10)[:-3:-1]

[9,8]

I found this result very surprising,

What would you have expected instead?

range(10-1, 10-3, -1) == [9,8]

was my guiding principle when I wrote the code.
and would just like to see the rules written down somewhere.

It seems to be hardly any easier to write the rules than to write the
algorithm, in practice...

In practice, I only ever use negative strides as "[::-1]", the result
of which hopefully surprises noone.

Cheers,
mwh

--
<glyph> yo ho ho and a bottle of internet
-- from Twisted.Quotes

[Shalabh Chaturvedi]

Christopher T King wrote:

On Wed, 1 Sep 2004, Shalabh Chaturvedi wrote:

Does anyone else find it intuitive to expect that S[b:a:-1] be exactly
reverse of S[a:b:1]?

No. S[a:b:1] includes the start index (a), but excludes the stop index
(b). Similarly, S[b:a:-1] includes the start index (b), but excludes the
stop index (a). Only if the stop indices were included (or the start
indices were excluded), thus making the indexing symmetrical, would I
expect the results to be symmetric.

That's a good explanation. My initial confusion arose because I
remembered how slices work by the suggestion (and diagram) under:

http://docs.python.org/tut/node5.htm...00000000000000

"The best way to remember how slices work is to think of the indices as
pointing /between/ characters". When negative step is in the picture,
the indices are no longer in the same inbetween places and need to be
shifted to the right. Associating indices with characters themselves
dissipates the confusion.

Thanks,
Shalabh

[Alex Martelli]

Shalabh Chaturvedi <sh*****@cafepy .com> wrote:
...

Does anyone else find it intuitive to expect that S[b:a:-1] be exactly
reverse of S[a:b:1]?

Not me, because I _know_ that S[x:y:anything] INCLUDES S[x] and EXCLUDES
S[y]. So, swappiing x and y can't POSSIBILY "exactly reverse' things,
*ever*, NO MATTER HOW 'anything' changes.

That's a good reason why 2.4 introduces the new built-in 'reversed': if
you want to loop on "exactly the reverse of" S[a:b], you loop on
reversed(S[a:b]) and live happily ever after.
Alex