negative indices for sequence types

[dan]

I was recently surprised, and quite shocked in fact, to find that
Python treats negative indices into sequence types as if they were
mod(length-of-sequence), at least up to -len(seq).

This fact is *deeply* buried in the docs, and is not at all intuitive.
One of the big advantages of a high-level language such as Python is
the ability to provide run-time bounds checking on array-type
constructs. To achieve this I will now have to subclass my objects
and add it myself, which seems silly and will add significant
overhead. If you want this behavior, how hard is it to say a = b[x %
len(b)] ??

Can anyone explain why this anomaly exists, and why it should continue
to exist?

[Martin v. Löwis]

da*******@yahoo.com (dan) writes:

This fact is *deeply* buried in the docs, and is not at all intuitive.
I find it highly intuitive and very convenient.
If you want this behavior, how hard is it to say a = b[x %
len(b)] ??

*This* I would call un-intuitive. It is also much slower.

To get the last element, you currently write b[-1]. If that was not
available, you would have to write b[len(b)-1], which is still
significantly slower. Also, you might not have a variable name, so try
rewriting foo()[-1].

Regards,
Martin

[Peter Otten]

dan wrote:

I was recently surprised, and quite shocked in fact, to find that
Python treats negative indices into sequence types as if they were
mod(length-of-sequence), at least up to -len(seq).

This fact is *deeply* buried in the docs, and is not at all intuitive.
One of the big advantages of a high-level language such as Python is
the ability to provide run-time bounds checking on array-type
constructs. To achieve this I will now have to subclass my objects
and add it myself, which seems silly and will add significant
overhead. If you want this behavior, how hard is it to say a = b[x %
len(b)] ??

Can anyone explain why this anomaly exists, and why it should continue
to exist?

After you have recovered from the shock, you probably will admit that
(1) the most common "out of bounds" case is caught:

l = list("abc")
l[3] Traceback (most recent call last):
File "<stdin>", line 1, in ?
IndexError: list index out of range

and
(2) that accessing elements from the end of the list is something you will
soon appreciate: l[-1] 'c' l[-2:] ['b', 'c']

I think that more code enjoys the beauty of accessing the end of a list than
suffers from uncaught <0 index errors. See the possibilities rather than
the danger :-)

Peter

[Bengt Richter]

On 7 Sep 2003 11:26:28 -0700, da*******@yahoo.com (dan) wrote:

I was recently surprised, and quite shocked in fact, to find that
Python treats negative indices into sequence types as if they were
mod(length-of-sequence), at least up to -len(seq).

This fact is *deeply* buried in the docs, and is not at all intuitive.
One of the big advantages of a high-level language such as Python is
the ability to provide run-time bounds checking on array-type
constructs. To achieve this I will now have to subclass my objects
and add it myself, which seems silly and will add significant
overhead. If you want this behavior, how hard is it to say a = b[x %
len(b)] ??
That isn't really the exact behavior. E.g.,

range(5) [0, 1, 2, 3, 4] range(5)[-4] 1 range(5)[-5] 0 range(5)[-6] Traceback (most recent call last):
File "<stdin>", line 1, in ?
IndexError: list index out of range
range(5)[4] 4 range(5)[5]

Traceback (most recent call last):
File "<stdin>", line 1, in ?
IndexError: list index out of range
Can anyone explain why this anomaly exists, and why it should continue
to exist?

It has apparently proven more useful to have it so than not, though I sympathize
with your frustration in for your use.

Perhaps a .no_negative_indexing attribute or something could be added to the C implementation,
so that you could specify your desired checking without a performance hit.

Meanwhile, maybe an assert i>=0 in the index-supplier side of the contract might work too?

Regards,
Bengt Richter

[Terry Reedy]

"dan" <da*******@yahoo.com> wrote in message
news:fb**************************@posting.google.c om...

I was recently surprised, and quite shocked in fact, to find that
Python treats negative indices into sequence types as if they were
mod(length-of-sequence), at least up to -len(seq).
No, it adds len(seq). Changing + to % would be slower and more
obscure.
This fact is *deeply* buried in the docs,
No more so than everything else in chapter subsections. From the Ref
Man table of contents I went directly to the most obvious place 5.3.2
Subscriptions, and found
'''
If the primary is a sequence, the expression (list) must evaluate to a
plain integer. If this value is negative, the length of the sequence
is added to it (so that, e.g., x[-1] selects the last item of x.) The
resulting value must be a nonnegative integer less than the number of
items in the sequence, and the subscription selects the item whose
index is that value (counting from zero).
'''
Translated to Python, letting idex be result of index expression:

if not isinstance(idex, (int,long)): raise TypeError()
if idex < 0: idex += seqlen
if idex < 0 or idex >= seqlen: raise IndexError()
<get seq[idex]>
and is not at all intuitive.
Phrases like 'third from the end' are idiomatic English ;-)
One of the big advantages of a high-level language such as Python is the ability to provide run-time bounds checking on array-type
constructs. To achieve this I will now have to subclass my objects
and add it myself, which seems silly and will add significant
overhead. If you want this behavior, how hard is it to say a = b[x % len(b)] ??
Again, your innovation of using '% obscures rather than clarify.
Can anyone explain why this anomaly exists, and why it should continue to exist?

Being able to abbreviate seq(len(seq)-1] as seq[-1] is quite handy and
faster executing, , especially if seq is calculated from an
expression. Same for -2, etc. (And, of course, a change now would
break a noticeable fraction of existing programs.)

Terry J. Reedy

[Erik Max Francis]

dan wrote:

I was recently surprised, and quite shocked in fact, to find that
Python treats negative indices into sequence types as if they were
mod(length-of-sequence), at least up to -len(seq).
That is not the behavior of negative indices. Negative indices mean
index from the end of the sequence. So -1 means the _last_ element in
the list, -2 means the second to last element in the list, and so on.
-n (for n = len(seq) is the first element in the list.
This fact is *deeply* buried in the docs, and is not at all intuitive.
It's mentioned prominently (and early) in all the tutorials and books on
Python I've read, and it's a very common and convenient convention, so
I'm not sure how far you could have gotten through learning Python and
never been exposed to it.
One of the big advantages of a high-level language such as Python is
the ability to provide run-time bounds checking on array-type
constructs. To achieve this I will now have to subclass my objects
and add it myself, which seems silly and will add significant
overhead. If you want this behavior, how hard is it to say a = b[x %
len(b)] ??

That's simply not true. Negative indices have similar bounds
requirements. If you have a sequence of length n, then indices 0
through (n - 1) map to the elements of the sequence in order from left
to right, and indices -1 through -n map to the elements in order from
right to left. Indices greater than n or less than -n generate
IndexErrors. Bounds checking is always done, whether on positive or
negative indices.

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
__ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
/ \ Then you give me that Judas Kiss / Could you hurt me more than this
\__/ Lamya

[bigdog]

da*******@yahoo.com (dan) wrote in message news:<fb**************************@posting.google. com>...

As is often the case, I think this comes down to documentation. While
the behavior is mentioned early in the tutorial, I found it difficult
to find it in the reference -- but whatever, we can chalk this up to
RTFM on my part.

My explanation of the behavior is correct however. list[a] always
equals list[a % len(list)]. A negative number mod N = its absolute
value subtracted from N:

a % n == n - abs(a) # where -n <= a <= 0

However if I want to count from the end of the list, I would of course
write
list[len(list)-a]. I wasn't really considering that the purpose of
this feature was to count from the end of a list, which I admit could
come in handy.

Thanks for the responses.

Fernando Perez <fp*******@yahoo.com> wrote in message news:<bj**********@peabody.colorado.edu>...

dan wrote:

I was recently surprised, and quite shocked in fact, to find that
Python treats negative indices into sequence types as if they were
mod(length-of-sequence), at least up to -len(seq).

This fact is *deeply* buried in the docs, and is not at all intuitive.

Very deeply indeed: section 3.1.4 of the beginner's tutorial:

http://www.python.org/doc/current/tu...00000000000000

Of all places, this is the section on lists:

>> a = ['spam', 'eggs', 100, 1234]

[... snip ...]

>> a[-2] 100>> a[1:-1]

['eggs', 100]

Can anyone explain why this anomaly exists, and why it should continue
to exist?

Because this 'anomaly' is incredibly useful in many contexts, as many others
have already pointed out. Rest assured that it will continue to exist,
probably for as long as the language is around. Better get to like it :)

Cheers,

f.

Heck, I like it simply because I can read lines from files and easily
chop off the newline.

myStr = f.readline()[0:-1]

That alone is worth it's wait in gold to me, never mind all the other
things it makes easy.

[Lukasz Pankowski]

ms****@comshare.com (bigdog) writes:

myStr = f.readline()[0:-1]

this may eat you last character in the file (if last line does not end
with new line which happens, but this will not ::

myStr = f.readline().rstrip('\n')

but is 6 character longer :)

--

=*= Lukasz Pankowski =*=

[Jacek Generowicz]

da*******@yahoo.com (dan) hypothesizes:

My explanation of the behavior is correct however. list[a] always
equals list[a % len(list)]. A negative number mod N = its absolute
value subtracted from N:

Proof by counterexample:

Python 2.2.2 (#1, Feb 8 2003, 12:11:31)
[GCC 3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.

s = '0123'
s[-20 % len(s)] '0' s[-20]

Traceback (most recent call last):
File "<stdin>", line 1, in ?
IndexError: string index out of range
Your explanation of the behaviour is incorrect.

QED.