By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
425,587 Members | 1,891 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 425,587 IT Pros & Developers. It's quick & easy.

Possible improvement to slice opperations.

P: n/a

After considering several alternatives and trying out a few ideas with a
modified list object Bengt Richter posted, (Thank You), I think I've
found a way to make slice operation (especially far end indexing)
symmetrical and more consistent.

So to find out if this is indeed a possibility, it would be nice to get
a few opinions at this point. So blast away... or hopefully tell me
what you like about it instead. ;-)

(Any suggestions or contributions to make it better would be appreciated.)

Cheers,
Ron Adam

"""
IMPROVED SLICING
================

Slicing is one of the best features of Python in my opinion, but
when you try to use negative index's and or negative step increments
it can be tricky and lead to unexpected results.

This topic has come up fairly often on comp.lang.python, and often
times, the responses include:

* Beginners should avoid negative extended slices.

* Slices with negative step increments are for advanced
python programmers.

* It's not broke if you look at it in a different way.

* You should do it a different way.

All of these and various responses similar to them are unsatisfactory in
my opinion and it's a good indicator it's not as good as it could be.
Indexing and slice operations are vary common in Python and I don't
think we should be discouraging people from learning and using them.
COMPATIBILITY
-------------
Because the following suggested changes will break current code,
it probably can not be implemented prior to Python 3000.

+ Direct indexing with both positive and negative values
returns the same items as they do now.

+ Extended slices with all positive and or empty default
values remain unchanged.

- Extended slices with negative values return values that
have less items than currently.

- Slices with negative step values return entirely different
results.
REVERSE ORDER STEPPING
----------------------
When negative steps are used, a slice operation
does the following. (or the equivalent)

1. reverse the list
2. cut the reversed sequence using start and stop
3. iterate forward using the absolute value of step.

* This order results in an inverse selection and I believe should be
considered a bug.

Changing the order in the following way results in a much
more predictable pattern which is both easier to understand and use.

1. cut sequence using start and stop.
2 reverse the order of the results.
3. iterate forward using the absolute value of step.
CURRENT INDEXING
----------------

Given a range [a,b,c]:

Positive indexing

| a | b | c |
+---+---+---+
0 1 2 3

Current negative indexing.

| a | b | c |
+---+---+---+
-3 -2 -1 -0
When a single index is used the item to the
right of the index for both positive and
negative index's is returned.

With slices, the items between start, and
stop index's are returned.

Accessing a range at the end of a list numerically
becomes inconvenient when negative index's are used
as the '-0'th position can not be specified numerically
with negative values.
ONES BASED NEGATIVE INDEXING
----------------------------
Making negative index's Ones based, and selecting
individual item to the left of negative index's would enable
addressing the end of the list numerically.

Ones based negative index's.

| a | b | c |
+---+---+---+
-4 -3 -2 -1

Then:

a[-1] -> c # item left of index, same result as now.

a[-3:-2] -> b # item between index's

a[-1:-1] = [d] # insert item at the end.

USE OF '~' IN PLACE OF NEGATIVE INDEX'S
---------------------------------------

The '~' is the binary not symbol which when used
with integers returns the two's compliment. This
works nice with indexing from the end of a list
because convieniently ~0 == -1.

This creates a numerical symmetry between positive
indexing and '~' "End of list" indexing.

a[0] -> first item in the list.
a[~0] -> last item in the list.

a[0:~0] -> whole list.

a[1:~1] -> center, one position from both ends.

* Note: using '~' works as described here in place of single negative
index's in current versions of Python. It does not work as described
here for extended slices.

"""

# TEST LIST CLASS.
"""
A list class to Test '~' end of list indexing.

* This class modifies the slice before returning
a value. The final implementation may do this by
modifying slice objects directly or the underlying
C code of sequences.

"""

class nxlist(object):

def __init__(self, value):
self.value = value

def normslc(self, slc):
start,stop,step = slc.start, slc.stop, slc.step
if type(start) == int and start<0:
start = len(self.value)+start+1
if type(stop) == int and stop<0:
stop = len(self.value)+stop+1
return slice(start,stop,step)

def __getitem__(self, i):
tp = i.__class__
if tp == int:
if i>=0:
return self.value[i]
else:
return self.value[ len(self.value)+i ]
if tp == slice:
slc = self.normslc(i)
value = self.value[slc.start:slc.stop]
if type(i.step) == int and i.step<0:
value.reverse()
slc = slice(None,None,-i.step)
else:
slc = slice(None,None,i.step)
return value[slc]

#def __setitem__(self, i, v):
#Not emplimented yet.

def __repr__(self): return 'nxlist(%r)'%self.value
a = nxlist(range(10))
print a

testdata = [
('a[0]'),
('a[1]'),
('a[~0]'),
('a[~1]'),
('a[:]'),
('a[:~1]'),
('a[~1:]'),
('a[::]'),
('a[0:~0]'),
('a[1:~1]'),
('a[~1:1]'),
('a[::-2]'),
('a[:3]'),
('a[3:]'),
('a[~3:]'),
('a[:~3]'),
('a[:3:-1]'),
('a[3::-1]'),
('a[~3::-1]'),
('a[:~3:-1]'),
('a[:3:-2]'),
('a[3::-2]'),
('a[~3::-2]'),
('a[:~3:-2]'),
]

for n, s in enumerate(testdata):
print '%r. %s = %r' % (n,s,eval(s))
"""
nxlist([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
0. a[0] = 0
1. a[1] = 1
2. a[~0] = 9
3. a[~1] = 8
4. a[:] = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
5. a[:~1] = [0, 1, 2, 3, 4, 5, 6, 7, 8]
6. a[~1:] = [9]
7. a[::] = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
8. a[0:~0] = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
9. a[1:~1] = [1, 2, 3, 4, 5, 6, 7, 8]
10. a[~1:1] = []
11. a[::-2] = [9, 7, 5, 3, 1]
12. a[:3] = [0, 1, 2]
13. a[3:] = [3, 4, 5, 6, 7, 8, 9]
14. a[~3:] = [7, 8, 9]
15. a[:~3] = [0, 1, 2, 3, 4, 5, 6]
16. a[:3:-1] = [2, 1, 0]
17. a[3::-1] = [9, 8, 7, 6, 5, 4, 3]
18. a[~3::-1] = [9, 8, 7]
19. a[:~3:-1] = [6, 5, 4, 3, 2, 1, 0]
20. a[:3:-2] = [2, 0]
21. a[3::-2] = [9, 7, 5, 3]
22. a[~3::-2] = [9, 7]
23. a[:~3:-2] = [6, 4, 2, 0]
"""
r = range(10)
a = nxlist(r)

print r[~0],a[~0] # ok
print r[~3:],a[~3:] # one off
print r[~3::-1],a[~3::-1] # other side
print r[~3::-2],a[~3::-2] # other side
"""
Comparisons of negative indexing and '~'
indexing with same values.

current, proposed

9 9
[6, 7, 8, 9] [7, 8, 9]
[6, 5, 4, 3, 2, 1, 0] [9, 8, 7]
[6, 4, 2, 0] [9, 7]
"""

Sep 4 '05 #1
Share this Question
Share on Google+
40 Replies


P: n/a

"Ron Adam" <rr*@ronadam.com> wrote in message
news:FR******************@tornado.tampabay.rr.com. ..
Slicing is one of the best features of Python in my opinion, but
when you try to use negative index's and or negative step increments
it can be tricky and lead to unexpected results.

This topic has come up fairly often on comp.lang.python, and often times,
the responses include:
* Beginners should avoid negative extended slices.
* Slices with negative step increments are for advanced
python programmers.
* It's not broke if you look at it in a different way.
* You should do it a different way.


You omitted the slice-specific response I gave before and repeat here with
more detail by quoting from What's New in Python 2.3.
http://www.python.org/doc/2.3/whatsn...on-slices.html
"
15 Extended Slices
Ever since Python 1.4, the slicing syntax has supported an optional third
``step'' or ``stride'' argument. For example, these are all legal Python
syntax: L[1:10:2], L[:-1:1], L[::-1]. This was added to Python at the
request of the developers of Numerical Python, which uses the third
argument extensively. However, Python's built-in list, tuple, and string
sequence types have never supported this feature, raising a TypeError if
you tried it.
"
Again, extended slices were probably designed by and certainly designed for
Numerical Python and for 7 years were used at least mainly by Numerical
Python. They were not designed for other users like you. The responses
you summarized and distain pretty much all derive from this history.

So, I am pretty sure that changes to the core would have to be upwards
compatible with current usage. On the other hand, your nxlist subclass of
list seems to work pretty well now.The 2.2+ ability to do this sort of
thing is what type subclassing was made for.

Or one could just write an extended slice function similar to your .normslc
method. I might even use such a thing one day.

Terry J. Reedy
Sep 5 '05 #2

P: n/a
> After considering several alternatives and trying out a few ideas with a
modified list object Bengt Richter posted, (Thank You), I think I've
found a way to make slice operation (especially far end indexing)
symmetrical and more consistent.


I don't know that it makes it more consistent. I could be persuaded,
but it would have to be by real-life examples with calculated slice
indices and stride. I do this thing all the time, and find the current
rules simple and very consistent. Occasionally, I might wish that
things were a little different, but there is always a workaround. I
would have to see some real code examples, of sufficient scope to see
that there are fewer workarounds with this proposal than with the
current implementation.

FWIW, there is a reasonable workaround for the case where the indices
might be negative and you would like zero or greater to mean 'end of
list'. If "x" is the index variable, you can use the expression (x<0
and x or None) for the index value in the slice. If you find yourself
doing this often, you can write a little function for it -- def
EndIndex(x): return x<0 and x or None.

But in real code, I fear you might need a similar helper function for
similar issues with your change. I just don't know what those are
without more thought.

Regards,
Pat

Sep 5 '05 #3

P: n/a
Terry Reedy wrote:
"Ron Adam" <rr*@ronadam.com> wrote in message
news:FR******************@tornado.tampabay.rr.com. ..
Slicing is one of the best features of Python in my opinion, but
when you try to use negative index's and or negative step increments
it can be tricky and lead to unexpected results.

This topic has come up fairly often on comp.lang.python, and often times,
the responses include:
* Beginners should avoid negative extended slices.
* Slices with negative step increments are for advanced
python programmers.
* It's not broke if you look at it in a different way.
* You should do it a different way.

You omitted the slice-specific response I gave before and repeat here with
more detail by quoting from What's New in Python 2.3.
http://www.python.org/doc/2.3/whatsn...on-slices.html
"
15 Extended Slices
Ever since Python 1.4, the slicing syntax has supported an optional third
``step'' or ``stride'' argument. For example, these are all legal Python
syntax: L[1:10:2], L[:-1:1], L[::-1]. This was added to Python at the
request of the developers of Numerical Python, which uses the third
argument extensively. However, Python's built-in list, tuple, and string
sequence types have never supported this feature, raising a TypeError if
you tried it.
"


I didn't omit anything, I previously gave examples of using various
forms of slices with the intent of showing how some of those forms are
not symmetrical and difficult to understand. And then you explained
more precisely how those slices were indexed.

It would be useful to do a side by side comparison of those cases and
similar ones to this new method. Then maybe the benefits of making the
changes I'm suggesting will be clearer.

I looked at the documents in Numerical Python. From what I could tell,
it uses the form L[::-1] quite a bit, which will still work *exactly* as
it does now. There were no examples of partial negative step
operations, so I couldn't determine if/how they are used. I'll install
it and experiment with it and see if/how it works with Numerical Python.

After Python 3000 is released Numerical Python will need to be updated
even without these changes. But it is a good idea to consult with the
Numerical Python developers on this.

Again, extended slices were probably designed by and certainly designed for
Numerical Python and for 7 years were used at least mainly by Numerical
Python. They were not designed for other users like you. The responses
you summarized and distain pretty much all derive from this history.
Other users like me? And what kind of user am I?

Is it really a good idea to maintain that some things shouldn't be done
by some, but is ok for others?

I think you mean to say that extended slices with negative strides,
should only (or primarily) be used with Numerical Python. I however
disagree with that view. I think they have a broader range of uses.

So, I am pretty sure that changes to the core would have to be upwards
compatible with current usage. ...
This is why I said these changes probably couldn't be made before Python
3000. It is upward (backward?) compatible for the majority of use
cases, all direct indexing operations, and all positive extended slices.
In Python's library, only negative extended slices will need to be
adjusted. As far as I could tell there are no partial negative strides
cases that will need to be adjusted. I wish there was a few, because
then I could show how this would improve those cases.

Now even though this suggestion may be sound, that alone isn't reason
enough to include it in Python 3000. It would need clear use case
benefits and support by users as well as support from Guido.

At this point, I'm only trying to clarify the issues and develop working
examples.
... On the other hand, your nxlist subclass of list seems to work
pretty well now. The 2.2+ ability to do this sort of thing is what
type subclassing was made for.
Yes, it does work well, (so far) :-) Which is the motivation for me
posting it. Lets make it better.

Or one could just write an extended slice function similar to your .normslc
method. I might even use such a thing one day.
Unfortunately one can not sub-class slice objects directly. And the ':'
in a slice index isn't a true operator with it's own methods. So
changing existing slice behavior will require changing the core. But
for now I will write a function and a complete list sub-class and post
it here.

Cheers,
Ron
Terry J. Reedy

Sep 5 '05 #4

P: n/a
Patrick Maupin wrote:
After considering several alternatives and trying out a few ideas with a
modified list object Bengt Richter posted, (Thank You), I think I've
found a way to make slice operation (especially far end indexing)
symmetrical and more consistent.

I don't know that it makes it more consistent. I could be persuaded,
but it would have to be by real-life examples with calculated slice
indices and stride. I do this thing all the time, and find the current
rules simple and very consistent. Occasionally, I might wish that
things were a little different, but there is always a workaround. I
would have to see some real code examples, of sufficient scope to see
that there are fewer workarounds with this proposal than with the
current implementation.


I'll post a few examples once I'm sure the list object works correctly
and then you can play around with it and try it out as well.
FWIW, there is a reasonable workaround for the case where the indices
might be negative and you would like zero or greater to mean 'end of
list'. If "x" is the index variable, you can use the expression (x<0
and x or None) for the index value in the slice. If you find yourself
doing this often, you can write a little function for it -- def
EndIndex(x): return x<0 and x or None.
Thanks, this is a good example.

Yes it does remove the need for a work around in those cases.

You still need to check for cases where you may cross the -1,0 boundary
while incrementing or decrementing an index. But that is an expected
condition and easily handled with an 'if' or 'for' loop.

But in real code, I fear you might need a similar helper function for
similar issues with your change. I just don't know what those are
without more thought.

Regards,
Pat


Thanks for the feedback, it was helpful.

Cheers,
Ron


Sep 5 '05 #5

P: n/a
with the current syntax L[i:i+1] returns [L[i]], with nxlist it returns
L[i+1] if i<0.

L=range(10)
L[1:2]==[L[1]]==[1]
L[-2:-1]==[L[-2]]==[8]

L=nxlist(range(10))
L[1:2]==[L[1]]==[1]
L[-2:-1]==[L[-1]]==[9] # not [L[-2]]

IMHO in this case current list slicing is more consistent.

Sep 5 '05 #6

P: n/a
Ron Adam wrote:
Slicing is one of the best features of Python in my opinion, but
when you try to use negative index's and or negative step increments
it can be tricky and lead to unexpected results.
Hm... Just as with positive indexes, you just need to understand
the concept properly.
This topic has come up fairly often on comp.lang.python, and often
times, the responses include:

* Beginners should avoid negative extended slices.

* Slices with negative step increments are for advanced
python programmers.
I certainly wouldn't respond like that...
* It's not broke if you look at it in a different way.

* You should do it a different way.
Those are correct, but need to be complemented with helpful
explanations. It's often like this, that we think things seem
strange, because we don't see it right. We need a new perspective.
For instance, it took me a long time to understand OOP, but once
the concept was clear, things fell into place. Often, we fail to
see things due to preconceptions that mislead us. Like Yoda said:
"You need to unlearn what you have learnt."
- Extended slices with negative values return values that
have less items than currently.

- Slices with negative step values return entirely different
results.
Over my dead body! ;) Honestly, I can't imagine you'll get agreement
over such a change.
REVERSE ORDER STEPPING
----------------------
When negative steps are used, a slice operation
does the following. (or the equivalent)

1. reverse the list
2. cut the reversed sequence using start and stop
3. iterate forward using the absolute value of step.
I think you are looking at this from the wrong perspective.

Whatever sign c has:
For s[a:b:c], a is the index for the first item to include,
b is the item after the last to include (just like .end() in
C++ iterators for instance), and c describes the step size.

To get a non-empty result, you obviously must have a > b iff
c < 0.

a defaults to 0, b defaults to None (which represents the
item beyond the end of the sequence) and c defaults to 1.

This is basically all you need to understand before you use
a or b < 0. There are no special cases or exceptions.

The concept of negative indices are completely orthogonal
to the concept of slicing today. You can learn and
understand them independently, and will automatically
be able to understand how to use the concepts together,
iff you actually understood both concepts correctly.
1. cut sequence using start and stop.
2 reverse the order of the results.
3. iterate forward using the absolute value of step.
With this solution, you suddenly have two different cases
to consider. You're suggesting that with c < 0, a should be
the end, and b should be the start. Now, it's no longer
obvious whether a or b should be excluded from the result.
I'm pretty sure the number of bugs and questions regarding
negative slices would grow quite a lot.
CURRENT INDEXING
----------------

Given a range [a,b,c]:

Positive indexing

| a | b | c |
+---+---+---+
0 1 2 3

Current negative indexing.

| a | b | c |
+---+---+---+
-3 -2 -1 -0
Almost right, there's no such thing as -0 in Python.
It's more like this: Current negative indexing.

| a | b | c |
+---+---+---+
-3 -2 -1 None Of course, [a,b,c][None] won't work, since it would
access beyond the list. It actually returns a TypeError
rather than an IndexError, and you might argue whether this
is the right thing to do. No big deal in my opinion. For
slices, None works as intended, giving the default in all
three positions. Thid means that if you want a negative
index that might be "-0" for the slice end, you simply
write something like

seq[start:end or None]
Accessing a range at the end of a list numerically
becomes inconvenient when negative index's are used
as the '-0'th position can not be specified numerically
with negative values.
But None works as is...
ONES BASED NEGATIVE INDEXING
----------------------------
Making negative index's Ones based, and selecting
individual item to the left of negative index's would enable
addressing the end of the list numerically.

Ones based negative index's.

| a | b | c |
+---+---+---+
-4 -3 -2 -1
Right. 0-based positive indices, and 1-based negative indices, but
you don't actually get the indexed value, but the one before...
I fear that your cure is worse then the "disease".

Today, a[x:x+1 or None] is always the same as [a[n]] for cases
where the latter doesn't yield an exception. With your change,
it won't be. That's not an improvement in my mind.
a[-3:-2] -> b # item between index's But a[-3] -> a! Shrug!
The '~' is the binary not symbol which when used
with integers returns the two's compliment. This
works nice with indexing from the end of a list
because convieniently ~0 == -1.
But that's only helpful if you want to write literal sequence
indexes, and that's not how real programming works. Further,
it's only an advantage if we agree that your suggestion is
broken and we want some kind of half-baked fix, to make it
work as it should.
This creates a numerical symmetry between positive
indexing and '~' "End of list" indexing.

a[0] -> first item in the list.
a[~0] -> last item in the list.
You can use this silly trick today to, if you think
it looks prettier than a[-1], but for the newbie,
this either means that you introduce magic, or that
you must teach two concepts instead of one.
a[0:~0] -> whole list.
I don't see why that's better than a[:], or a[0:None].
After all, the visual aspect only appears when you type
literals, which isn't very interesting. For calculated
values on the slice borders, you still have -1 as end
value.
a[1:~1] -> center, one position from both ends.


This is just a convoluted way of writing a[1:-2], which
is exactly the same as you would write today.
Sep 5 '05 #7

P: n/a
Magnus Lycka wrote:
Ron Adam wrote:
ONES BASED NEGATIVE INDEXING
I think Ron's idea is taking off from my observation that if one's
complement, rather than negation, was used to specify measure-from-
right, we would have a simple consistent system (although I also
observed it is far too late to do that to Python now). Using such
a system should not define things as below:
| a | b | c |
+---+---+---+
-4 -3 -2 -1 but rather use a form like: | a | b | c |
+---+---+---+
~3 ~2 ~1 ~0 The '~' is the binary not symbol which when used
with integers returns the two's compliment. Actually, the ~ operator is the one's complement operator.
For calculated values on the slice borders, you still
have -1 as end value.

But if you are defining the from-right as ones complement,
you use one's complement on the calculated values and
all proceeds happily. Since this could happen in Python,
perhaps we should call it Pythoń.
a[1:~1] -> center, one position from both ends.


This is just a convoluted way of writing a[1:-2], which
is exactly the same as you would write today.


Actually, a[1 : -1] is how you get to drop the first and
last characters today. I suspect you knew this and were
just a bit in a hurry criticizing a lame-brained scheme.

-Scott David Daniels
Sc***********@Acm.Org
Sep 5 '05 #8

P: n/a
Scott David Daniels wrote:
Magnus Lycka wrote:
Ron Adam wrote:
ONES BASED NEGATIVE INDEXING

I think Ron's idea is taking off from my observation that if one's
complement, rather than negation, was used to specify measure-from-
right, we would have a simple consistent system (although I also
observed it is far too late to do that to Python now). Using such
a system should not define things as below:


I didn't start with your observation in mind, but it ended up there
because the only way to extend the indexes past the last item is to not
use 0 along with negative index's. And once that's done the '~' forms
"just work" without any additional changes. ;-)

Yes, it may be too late for changing Python's built in indexing. And it
may also be that the majority may rather use it as is, rather than
change it to fix this edge case.

| a | b | c |
+---+---+---+
-4 -3 -2 -1
but rather use a form like: >> | a | b | c |
>> +---+---+---+
>> ~3 ~2 ~1 ~0 The '~' is the binary not symbol which when used
with integers returns the two's compliment.
Actually, the ~ operator is the one's complement operator.


Yes, thanks, my mistake.

> For calculated values on the slice borders, you still
> have -1 as end value.

But if you are defining the from-right as ones complement,
you use one's complement on the calculated values and
all proceeds happily. Since this could happen in Python,
perhaps we should call it Pythoń.
a[1:~1] -> center, one position from both ends.

This is just a convoluted way of writing a[1:-2], which
is exactly the same as you would write today.

Actually, a[1 : -1] is how you get to drop the first and
last characters today. I suspect you knew this and were
just a bit in a hurry criticizing a lame-brained scheme.

-Scott David Daniels
Sc***********@Acm.Org

Sep 5 '05 #9

P: n/a
Scott David Daniels wrote:
Magnus Lycka wrote: [...]
The '~' is the binary not symbol which when used
with integers returns the two's compliment.
Actually, the ~ operator is the one's complement operator.

Actually the two are exactly the same thing. Could we argue about
substantive matters, please? ;-)
> For calculated values on the slice borders, you still
> have -1 as end value.

But if you are defining the from-right as ones complement,
you use one's complement on the calculated values and
all proceeds happily. Since this could happen in Python,
perhaps we should call it Pythoń.

:-) And how would that be pronounced? I understood that "ń" would only
appear between two vowels. a[1:~1] -> center, one position from both ends.
This is just a convoluted way of writing a[1:-2], which
is exactly the same as you would write today.

It does have the merit (if you think of it as a merit) of allowing
someone to write

a[n, ~n]

to remove n characters from each end of the string. Frankly I'd rather
deal with the Python that exists now than wrap my head around this
particular suggestion.
Actually, a[1 : -1] is how you get to drop the first and
last characters today. I suspect you knew this and were
just a bit in a hurry criticizing a lame-brained scheme.

Yes, I've been surprised how this thread has gone on and on.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

Sep 5 '05 #10

P: n/a
Steve Holden wrote:
Yes, I've been surprised how this thread has gone on and on.


it's of course a variation of

"You can lead an idiot to idioms, but you can't make him
think ;-)"

as long as you have people that insist that their original misunderstandings
are the only correct way to model the real world, and that all observed
inconsistencies in their models are caused by bugs in the real world, you'll
end up with threads like this.

</F>

Sep 5 '05 #11

P: n/a
Szabolcs Nagy wrote:
with the current syntax L[i:i+1] returns [L[i]], with nxlist it returns
L[i+1] if i<0.

L=range(10)
L[1:2]==[L[1]]==[1]
L[-2:-1]==[L[-2]]==[8]

L=nxlist(range(10))
L[1:2]==[L[1]]==[1]
L[-2:-1]==[L[-1]]==[9] # not [L[-2]]

IMHO in this case current list slicing is more consistent.

Your second case should be:
L[-3:-2]==[L[-2]]==[8]

The single index is also the right side index of the range selection
ending in the same position just as with positive number the single
index is the left side index of a range starting in the same position.

So for positive indexing...
L[n1:n2]==[L[n1]]==first value of selection from left

and negative indexing...
L[n1:n2]==[L[n2]]==first value of selection from right
Currently for negative values...

L[n1:n2]==[L[n2-1]]==first value of selection from right
The symmetry is easier to see when using the '~' values.

LL=list(range(10))
Lx=nxlist(range(10))

LL[ 1: 2]==[LL[ 1]]==[1]
Lx[~2:~1]==[Lx[~1]]==[8]
Cheers,
Ron

Sep 5 '05 #12

P: n/a
Magnus Lycka wrote:
Ron Adam wrote:
Slicing is one of the best features of Python in my opinion, but
when you try to use negative index's and or negative step increments
it can be tricky and lead to unexpected results.

Hm... Just as with positive indexes, you just need to understand
the concept properly.
This topic has come up fairly often on comp.lang.python, and often
times, the responses include:

* Beginners should avoid negative extended slices.

* Slices with negative step increments are for advanced
python programmers.

I certainly wouldn't respond like that...
* It's not broke if you look at it in a different way.

* You should do it a different way.

Those are correct, but need to be complemented with helpful
explanations. It's often like this, that we think things seem
strange, because we don't see it right. We need a new perspective.
For instance, it took me a long time to understand OOP, but once
the concept was clear, things fell into place. Often, we fail to
see things due to preconceptions that mislead us. Like Yoda said:
"You need to unlearn what you have learnt."


To me, seeing a lot responses of this sort is an indicator it needs to
be simplified/or that something isn't right about them. They shouldn't
be that difficult to use or explain.
- Extended slices with negative values return values that
have less items than currently.

- Slices with negative step values return entirely different
results.

Over my dead body! ;) Honestly, I can't imagine you'll get agreement
over such a change.


Yet, your description of how it works below is closer to what I'm
proposing than how they currently work.

REVERSE ORDER STEPPING
----------------------
When negative steps are used, a slice operation
does the following. (or the equivalent)

1. reverse the list
2. cut the reversed sequence using start and stop
3. iterate forward using the absolute value of step.

I think you are looking at this from the wrong perspective.

Whatever sign c has:
For s[a:b:c], a is the index for the first item to include,
b is the item after the last to include (just like .end() in
C++ iterators for instance), and c describes the step size.


Yes, and that is how it "should" work. But....

With current slicing and a negative step...

[ 1 2 3 4 5 6 7 8 9 ]
-9 -8 -7 -6 -5 -4 -3 -2 -1 -0

r[-3:] -> [7, 8, 9] # as expected
r[-3::-1] -> [7, 6, 5, 4, 3, 2, 1, 0] # surprise

The seven is include in both cases, so it's not a true inverse selection
either.

In most cases, negative step (or stride) values are used
to reverse the whole lists, so this issue doesn't come up.
To get a non-empty result, you obviously must have a > b iff
c < 0.

a defaults to 0, b defaults to None (which represents the
item beyond the end of the sequence) and c defaults to 1.

This is basically all you need to understand before you use
a or b < 0. There are no special cases or exceptions.
See above. ;-)

The concept of negative indices are completely orthogonal
to the concept of slicing today. You can learn and
understand them independently, and will automatically
be able to understand how to use the concepts together,
iff you actually understood both concepts correctly.


It's easy to think you understand something when you don't. I spend
quite a while figuring this out, And am sure about how it works. If
there are issues with this, then it will probably be in how I describe
it, what words or terminology is used, and weather or not it's the
proper approach.

There are actually two distinct proposals here, not just one.

1. Fixing negative strides so they return the slice indexed as you
say they should.

2. Using base one negative index's and picking item from the right
of negative index's instead of the right.
They don't both need to implemented, Item 1 could be fixed in 2.5.

Cheers,
Ron
Sep 5 '05 #13

P: n/a
Fredrik Lundh wrote:
Steve Holden wrote:

Yes, I've been surprised how this thread has gone on and on.

it's of course a variation of

"You can lead an idiot to idioms, but you can't make him
think ;-)"

as long as you have people that insist that their original misunderstandings
are the only correct way to model the real world, and that all observed
inconsistencies in their models are caused by bugs in the real world, you'll
end up with threads like this.

</F>


Ok, -1 from both Fredrik and Steven

No problem,

But I think you are mistaken that I don't understand how slicing works.
(IMHO of course) ;-)
A more valid argument against this would be that more users may prefer
the current zero based, select to the right, indexing than have to
remember to subtract one from negative index's or use '~'.

I'm completely aware of that, and don't argue that that viewpoint is valid.

It also has backwards compatibility issues as well. Which is probably
enough in itself to kill it as a PEP.
However, I would like the inverse selection of negative strides to be
fixed if possible. If you could explain the current reason why it does
not return the reverse order of the selected range. I would appreciated it.

Cheers,
Ron


Sep 5 '05 #14

P: n/a
Ron Adam wrote:
Magnus Lycka wrote:

Ron Adam wrote:

Slicing is one of the best features of Python in my opinion, but
when you try to use negative index's and or negative step increments
it can be tricky and lead to unexpected results.

Hm... Just as with positive indexes, you just need to understand
the concept properly.

This topic has come up fairly often on comp.lang.python, and often
times, the responses include:

* Beginners should avoid negative extended slices.

* Slices with negative step increments are for advanced
python programmers.

I certainly wouldn't respond like that...

* It's not broke if you look at it in a different way.

* You should do it a different way.

Those are correct, but need to be complemented with helpful
explanations. It's often like this, that we think things seem
strange, because we don't see it right. We need a new perspective.
For instance, it took me a long time to understand OOP, but once
the concept was clear, things fell into place. Often, we fail to
see things due to preconceptions that mislead us. Like Yoda said:
"You need to unlearn what you have learnt."

To me, seeing a lot responses of this sort is an indicator it needs to
be simplified/or that something isn't right about them. They shouldn't
be that difficult to use or explain.

It's a common misconception that all ideas should be explainable simply.
This is not necessarily the case, of course. When a subject is difficult
then all sorts of people bring their specific misconceptions to the
topic, and suggest that if only a few changes were made the whole thing
would be easier to understand.

Unfortunately, while that might make the topic in question easier to
understand for some it would make it difficult, and even
counter-intuitive, for others. As many have said before me, there's a
reason why slicing and indexing are the way they are. The interfaces
were designed by people with long years of programming and teaching
experience.

[...]
You said it quite well yourself:
It's easy to think you understand something when you don't. I spend
quite a while figuring this out, And am sure about how it works. If
there are issues with this, then it will probably be in how I describe
it, what words or terminology is used, and weather or not it's the
proper approach.

There are actually two distinct proposals here, not just one.

1. Fixing negative strides so they return the slice indexed as you
say they should.

2. Using base one negative index's and picking item from the right
of negative index's instead of the right.
They don't both need to implemented, Item 1 could be fixed in 2.5.

Given that Python has a 1's-complement operator already I don;t see why
you can't just leave Python alone and use it, since it seems to keep you
happy. If "fixing" item 1 in 2.5 would induce major code breakage,
there's less than a snowball's chance in hell it will happen.

"Professor Einstein, could you tell our readers how general relativity
works?"

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

Sep 5 '05 #15

P: n/a
Ron Adam wrote:
However, I would like the inverse selection of negative strides to be
fixed if possible. If you could explain the current reason why it does
not return the reverse order of the selected range.


why? you're not listening anyway.

</F>

Sep 5 '05 #16

P: n/a
Steve Holden <st***@holdenweb.com> writes:
Given that Python has a 1's-complement operator already I don;t see
why you can't just leave Python alone and use it,


What's the meaning of the 1's complement operator (for example, what
is ~1), when ints and longs are the same?
Sep 5 '05 #17

P: n/a
On Mon, 5 Sep 2005 18:09:51 +0200, "Fredrik Lundh" <fr*****@pythonware.com> wrote:
Steve Holden wrote:
Yes, I've been surprised how this thread has gone on and on.


it's of course a variation of

"You can lead an idiot to idioms, but you can't make him
think ;-)"

as long as you have people that insist that their original misunderstandings
are the only correct way to model the real world, and that all observed
inconsistencies in their models are caused by bugs in the real world, you'll
end up with threads like this.

OTOH, ISTM we must be careful not to label an alternate alpha-version
"way to model the real world" as a "misunderstanding" just because it is alpha,
and bugs are apparent ;-)

BTW, it just occurred to me that alternate slice member semantics could be RELATIVE,
EACH depending on sign. I.e.,

x[start:stop:step]

could mean start with x[start] as now as a starting point reference point element,
then let stop be a relative range length in either direction from the
starting point, and step absolute value indicating step size, and its sign
indicating insertion side for zero length slices. In effect, stop would
be a count for the slice range in either direction starting with the starting
element

s = 'abcde'
s[2:2] => 'cd'
s[2:-2] => 'cb'
s[-2:-3] => 'dcb'
s[-2:0] => ''
s[2:0] => ''
s[-2:-3:2] => 'db'
r = range(10)
r[5:0] = 'a'
r => [0, 1, 2, 3, 4, 5, 'a', 6, 7, 8, 9]
r[-2:0:-1] = 'b'
r => [0, 1, 2, 3, 4, 5, 'a', 6, 7, 'b', 8, 9]
r[-2:0] = ['c', 'd']
r => [0, 1, 2, 3, 4, 5, 'a', 6, 7, 'b', 8, c, d, 9]

note that slice assignment would work by iterating through the right hand
sequence, which could do interesting things:

r = range(6)
r[3:-2] = 'ab'
r => [0, 1, 'b', 'a', 4, 5]
but two reverse relative slices match order, so
r = range(10)
r[5:-3] = range(10)[-1:-3] # rh seq is 9, 8
r => [0, 1, 8, 9, 4, 5]

I think this is kind of interesting, and I probably wouldn't have thought of
it if I had not been for Ron's enthusiasm for his "misunderstanding" ;-)

In a way, "misunderstandings" are the mutations of open source evolution of ideas,
most of which die, but some of which mutate again and may occasionally survive.
So we need them ;-)

Regards,
Bengt Richter
Sep 5 '05 #18

P: n/a
Bengt Richter wrote:
as long as you have people that insist that their original misunderstandings
are the only correct way to model the real world, and that all observed
inconsistencies in their models are caused by bugs in the real world, you'll
end up with threads like this.

OTOH, ISTM we must be careful not to label an alternate alpha-version
"way to model the real world" as a "misunderstanding" just because it is alpha,
and bugs are apparent ;-)


in my post, "the real world" is the existing Python implementation. what
theoretical construct are you discussing?

</F>

Sep 5 '05 #19

P: n/a
Paul Rubin wrote:
Steve Holden <st***@holdenweb.com> writes:
Given that Python has a 1's-complement operator already I don;t see
why you can't just leave Python alone and use it,

What's the meaning of the 1's complement operator (for example, what
is ~1), when ints and longs are the same?


Python 2.2.1 (#1, Aug 25 2004, 16:56:05)
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
~1L -2L ~1 -2 import sys
sys.maxint*4 8589934588L ~(sys.maxint*4) -8589934589L
$ python
Python 2.4.1 (#1, May 27 2005, 18:02:40)
[GCC 3.3.3 (cygwin special)] on cygwin
Type "help", "copyright", "credits" or "license" for more information. ~1L -2L ~1 -2 sys.maxint*4 8589934588L ~(sys.maxint*4) -8589934589L


What's going to change when ints and longs are finally integrated?

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

Sep 5 '05 #20

P: n/a
Steve Holden wrote:
It's a common misconception that all ideas should be explainable simply.
This is not necessarily the case, of course. When a subject is difficult
then all sorts of people bring their specific misconceptions to the
topic, and suggest that if only a few changes were made the whole thing
would be easier to understand.
What misconception do you think I have?

Unfortunately, while that might make the topic in question easier to
understand for some it would make it difficult, and even
counter-intuitive, for others.
True, and that's why asking and getting opinions on a subject is a good
idea.

As many have said before me, there's a reason why slicing and
indexing are the way they are. The interfaces were designed by people
with long years of programming and teaching experience.
No one has yet explained the reasoning (vs the mechanics) of the
returned value of the following.

L = range(10)
L[3::-1]

So far every attempt to explain it has either quoted the documents which
don't address that particular case, or assumed I'm misunderstanding
something, or implied it isn't neccisary to do.

It's quite easy to get me to change my mind on something, just show me a
convincing explanation and I will. :)

[...]
You said it quite well yourself:
It's easy to think you understand something when you don't. I spend
quite a while figuring this out, And am sure about how it works. If
there are issues with this, then it will probably be in how I describe
it, what words or terminology is used, and weather or not it's the
proper approach.

There are actually two distinct proposals here, not just one.

1. Fixing negative strides so they return the slice indexed as
you say they should.

2. Using base one negative index's and picking item from the
right of negative index's instead of the right.
They don't both need to implemented, Item 1 could be fixed in 2.5.

Given that Python has a 1's-complement operator already I don;t see why
you can't just leave Python alone and use it, since it seems to keep you
happy. If "fixing" item 1 in 2.5 would induce major code breakage,
there's less than a snowball's chance in hell it will happen.


I doubt fixing item (1) would induce major code breakage. As near as I
can tell, it's rarely, (nearly never), used in forms other than L[::-1].

Item (2) however would require a lot of small changes in index's.
Mostly changing.. L[:-1] to L[:~1] or to L[:-2]. So no, I don't expect
one's based indexing to be added any time soon. It could be useful as a
function or object in it's own.

"Professor Einstein, could you tell our readers how general relativity
works?"
Actually I can, but it would be off topic for this news group.

Cheers,
Ron

regards
Steve

Sep 5 '05 #21

P: n/a
Bengt Richter wrote:
On Mon, 5 Sep 2005 18:09:51 +0200, "Fredrik Lundh" <fr*****@pythonware.com> wrote:
OTOH, ISTM we must be careful not to label an alternate alpha-version
"way to model the real world" as a "misunderstanding" just because it is alpha,
and bugs are apparent ;-)
Thanks! I couldn't have said this myself. :-)

BTW, it just occurred to me that alternate slice member semantics could be RELATIVE,
EACH depending on sign. I.e.,

x[start:stop:step]

could mean start with x[start] as now as a starting point reference point element,
then let stop be a relative range length in either direction from the
starting point, and step absolute value indicating step size, and its sign
indicating insertion side for zero length slices. In effect, stop would
be a count for the slice range in either direction starting with the starting
element

s = 'abcde'
s[2:2] => 'cd'
s[2:-2] => 'cb'
s[-2:-3] => 'dcb'
s[-2:0] => ''
s[2:0] => ''
s[-2:-3:2] => 'db'
r = range(10)
r[5:0] = 'a'
r => [0, 1, 2, 3, 4, 5, 'a', 6, 7, 8, 9]
r[-2:0:-1] = 'b'
r => [0, 1, 2, 3, 4, 5, 'a', 6, 7, 'b', 8, 9]
r[-2:0] = ['c', 'd']
r => [0, 1, 2, 3, 4, 5, 'a', 6, 7, 'b', 8, c, d, 9]
Interesting, so it would be...

slice( position, count, step )

Items are addressed directly so there's no 'gap's to account for.

note that slice assignment would work by iterating through the right hand
sequence, which could do interesting things:

r = range(6)
r[3:-2] = 'ab'
r => [0, 1, 'b', 'a', 4, 5]
but two reverse relative slices match order, so
r = range(10)
r[5:-3] = range(10)[-1:-3] # rh seq is 9, 8
r => [0, 1, 8, 9, 4, 5]

I think this is kind of interesting, and I probably wouldn't have thought of
it if I had not been for Ron's enthusiasm for his "misunderstanding" ;-)

In a way, "misunderstandings" are the mutations of open source evolution of ideas,
most of which die, but some of which mutate again and may occasionally survive.
So we need them ;-)

Here's another possible "misunderstanding".

(or alternative alpha-version) ;-)
Have you thought of using a slice object as a buffer pointer for
indirect operations?

r = range(100)
d = [10:20]
r.range_add(d,5) # Adds 5 to items in range d

d = d[5:] -> [15:20] # relative range modification.
# slice of a slice object

r.range_sub(d,3) # subtract 3 from items in range d

Or more flexible ...
r.range_modify(d, add(), 5)

Using your suggestion that would be...

r = range(100)
d = [10:10]
r.range_add(d,5)

d = d[5:] -> [15:5] # interesting symmetry.
r.range_sub(d,3)

Of course adding and subtracting slice objects could also be possible.

d = [10:20]
e = [15:25]
f = d + e -> [10:25]

or ...

d = [10:10]
e = [15:10]
f = d + e -> [10:15]
Cheers,
Ron

Regards,
Bengt Richter

Sep 5 '05 #22

P: n/a
On Mon, 5 Sep 2005 22:56:29 +0200, "Fredrik Lundh" <fr*****@pythonware.com> wrote:
Bengt Richter wrote:
as long as you have people that insist that their original misunderstandings
are the only correct way to model the real world, and that all observed
inconsistencies in their models are caused by bugs in the real world, you'll
end up with threads like this.

OTOH, ISTM we must be careful not to label an alternate alpha-version
"way to model the real world" as a "misunderstanding" just because it is alpha,
and bugs are apparent ;-)


in my post, "the real world" is the existing Python implementation. what
theoretical construct are you discussing?

I guess I was talking about a "real world" of abstractions (if that is not an oxymoron ;-)
re sequences and slice access methods, of which the existing python implementation
provides a concrete example of one "model" and Ron's efforts provide some attempts at
an alternative "model". Of course, as far as the concrete "real world" goes, python does what
it does, and a model of _that_ that doesn't fit is a real misunderstanding ;-)

BTW, how did relative slice semantics strike you? They could live along side normal ones
by prefixing the slice brackets with a '.', like r.[3:-2] for 2 (abs(-2)) elements
leftwards(-2<0) starting with r[3]. The logic for None defaults and non-1 steps and
guaranteeing legal ranges is messy, but otherwise UIAM

r.[start:count] == r[start:start+count:-(count<0) or 1]

allowing signed start and count values. IMO r[start] is easy for newbies
to understand as a starting element, whether from end or beginning, and
an absolute value of count is easy, with the sign saying which direction
to scan from the start element, irrespective of how the latter was specified.

Regards,
Bengt Richter
Sep 5 '05 #23

P: n/a
> Ron Adam wrote:
However, I would like the inverse selection of negative strides to be
fixed if possible. If you could explain the current reason why it does
not return the reverse order of the selected range.


To repeat, the current reason is compatibility with the original design for
NumPy. Perhaps there is some explanation for that in sliceobject.c (or
whatever the file is called), the CVS checkin messages, or the newsgroup
archives before the first checkin (in 1996, I believe).

Terry J. Reedy

Sep 6 '05 #24

P: n/a
Ron Adam wrote:
Steve Holden wrote:

It's a common misconception that all ideas should be explainable simply.
This is not necessarily the case, of course. When a subject is difficult
then all sorts of people bring their specific misconceptions to the
topic, and suggest that if only a few changes were made the whole thing
would be easier to understand.

What misconception do you think I have?

This was not an ad hominem attack but a commentary on many attempts to
"improve" the language.

Unfortunately, while that might make the topic in question easier to
understand for some it would make it difficult, and even
counter-intuitive, for others.

True, and that's why asking and getting opinions on a subject is a good
idea.
As many have said before me, there's a reason why slicing and
indexing are the way they are. The interfaces were designed by people
with long years of programming and teaching experience.

No one has yet explained the reasoning (vs the mechanics) of the
returned value of the following.

L = range(10)
L[3::-1]

So far every attempt to explain it has either quoted the documents which
don't address that particular case, or assumed I'm misunderstanding
something, or implied it isn't neccisary to do.

It's quite easy to get me to change my mind on something, just show me a
convincing explanation and I will. :)
L[3::-1] [3, 2, 1, 0] L[3::1] [3, 4, 5, 6, 7, 8, 9]

I don;t see the problem here. The start specifies which element is the
first in the slice, the stop is the default (end of the sequence) and
the stride is "successive elements to the left" when it's -1 and
"successive elements to the right" when it's 1.

Or perhaps you can tell me what I've missed?

[...]
You said it quite well yourself:
It's easy to think you understand something when you don't. I spend
quite a while figuring this out, And am sure about how it works. If
there are issues with this, then it will probably be in how I describe
it, what words or terminology is used, and weather or not it's the
proper approach.

There are actually two distinct proposals here, not just one.

1. Fixing negative strides so they return the slice indexed as
you say they should.

2. Using base one negative index's and picking item from the
right of negative index's instead of the right.
They don't both need to implemented, Item 1 could be fixed in 2.5.


Given that Python has a 1's-complement operator already I don;t see why
you can't just leave Python alone and use it, since it seems to keep you
happy. If "fixing" item 1 in 2.5 would induce major code breakage,
there's less than a snowball's chance in hell it will happen.

I doubt fixing item (1) would induce major code breakage. As near as I
can tell, it's rarely, (nearly never), used in forms other than L[::-1].

Well, I don't see the problem yet, so I don't actually think that needs
"fixing".
Item (2) however would require a lot of small changes in index's.
Mostly changing.. L[:-1] to L[:~1] or to L[:-2]. So no, I don't expect
one's based indexing to be added any time soon. It could be useful as a
function or object in it's own.
My point was that you can make those changes in your own code, leaving
others to accept the situation as it is.
"Professor Einstein, could you tell our readers how general relativity
works?"

Actually I can, but it would be off topic for this news group.

Right, I wasn't trying to suggest that you didn't know what you were
talking about - or even that you didn't understand general relativity
(on which my grasp could be said to be tenuous) - merely that some
things are inherently difficult, and no matter how you twist the
implementations about, the difficulties will remain.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

Sep 6 '05 #25

P: n/a
On 05 Sep 2005 12:58:00 -0700, Paul Rubin <http://ph****@NOSPAM.invalid> wrote:
Steve Holden <st***@holdenweb.com> writes:
Given that Python has a 1's-complement operator already I don;t see
why you can't just leave Python alone and use it,


What's the meaning of the 1's complement operator (for example, what
is ~1), when ints and longs are the same?


assert ~x == -1-x # or -1^x

The only problem is seeing the result printed, since people insist
that hex(a) will be '-'[:a<0]+hex(abs(a))

which brings up base-complement representation for signed numbers,
where the first digit is always 0 or base-1 to indicate positive and negative:
(BTW, I found a bug when I dug this up from my disk, so a previous post with this
might have that bug (bad leading digit check wasn't sign-sensitive))
def basecompl(x, B=10, digits='0123456789abcdefghijklmnopqrstuvwxyz'): ... if not (2 <= B <= len(digits)): raise ValueError('bad base = %r'%B)
... if not x: return digits[0]
... s = []
... while x and x != -1:
... x, r = divmod(x, B)
... s.append(digits[r])
... if not s or s[-1] != (digits[0], digits[B-1])[x<0]:
... s.append(digits[x<0 and B-1 or 0])
... return ''.join(s[::-1])
... def bcdecode(s, B=10, digits='0123456789abcdefghijklmnopqrstuvwxyz'): ... if s == digits[0]: return 0
... acc = s[0].lower() == digits[B-1] and -B**len(s) or 0
... for i, c in enumerate(s[::-1]):
... acc += digits.index(c)*B**i
... return acc
... bc = basecompl # easier to type ;-) bc(~3L, 2) '100' bc(-1-3L, 2) '100' bc(-1^3L, 2) '100' bc(~3L, 8) '74' bc(-1-3L, 8) '74' bc(-1^3L, 8) '74' bc(~3L) '96' bc(-1-3L) '96' bc(-1^3L) '96' bc(bcdecode(bc(~3L))) '96' bc(bcdecode(bc(~3L, 2),2),2) '100' bc(~3L, 16) 'fc' bc(-1-3L, 16) 'fc' bc(-1^3L, 16) 'fc' bc(3L)

'03'

Regards,
Bengt Richter
Sep 6 '05 #26

P: n/a
On Mon, 05 Sep 2005 17:10:05 -0400, Steve Holden <st***@holdenweb.com> wrote:
Paul Rubin wrote:
Steve Holden <st***@holdenweb.com> writes:
Given that Python has a 1's-complement operator already I don;t see
why you can't just leave Python alone and use it,

What's the meaning of the 1's complement operator (for example, what
is ~1), when ints and longs are the same?


Python 2.2.1 (#1, Aug 25 2004, 16:56:05)
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
~1L-2L ~1-2 import sys
sys.maxint*48589934588L ~(sys.maxint*4)-8589934589L
$ python
Python 2.4.1 (#1, May 27 2005, 18:02:40)
[GCC 3.3.3 (cygwin special)] on cygwin
Type "help", "copyright", "credits" or "license" for more information. >>> ~1L-2L >>> ~1-2 >>> sys.maxint*48589934588L >>> ~(sys.maxint*4)-8589934589L >>>
What's going to change when ints and longs are finally integrated?

I suspect that Paul may be worrying that bit operations on longs will be
bit operations on signs and positive magnitudes rather than signed numbers
of arbitrary width. This may be promoted by the fact the so far we have no
builting format for showing the bits of negative numbers the way hex used to do.

I scratched the itch this way:
from ut.basecompl import basecompl as bc
import sys
bc(sys.maxint*4, 16) '01fffffffc' bc(~sys.maxint*4, 16) 'fe00000000'
oops, precendence ... bc(~(sys.maxint*4), 16) 'fe00000003'

vs. the useless hex(~(sys.maxint*4)) '-0x1FFFFFFFDL'

(at least for looking at bit operation results ;-)

BTW, note base complementing, with base-1 or 0 as sign:
bc(~(sys.maxint*4), 8) '700000000003' bc( (sys.maxint*4), 8) '077777777774' bc(~(sys.maxint*4), 10) '91410065411' bc( (sys.maxint*4), 10) '08589934588' bc(~(sys.maxint*4), 2) '1000000000000000000000000000000011' bc( (sys.maxint*4), 2)

'0111111111111111111111111111111100'

Regards,
Bengt Richter
Sep 6 '05 #27

P: n/a
> No one has yet explained the reasoning (vs the mechanics) of the
returned value of the following.

L = range(10)
L[3::-1]

So far every attempt to explain it has either quoted the documents which
don't address that particular case, or assumed I'm misunderstanding
something, or implied it isn't neccisary to do.

It's quite easy to get me to change my mind on something, just show me a
convincing explanation and I will. :)


Once I saw this, I was surprised, because slices currently work exactly
like I would and do expect on this example, so I now have to admit that
I didn't read your original post fully and carefully. I have gone back
to look to figure out what you don't like and what you want, and I am
very surprised.

To me, your way reeks of black magic -- if there is a sign on the
stride, then strange and wondrous transformations happen. I think it
is conceptually (and probably programatically) much simpler the way it
is.

In any case, you asked for a rationale. I'll give you mine:
L = range(10)
L[3:len(L):-1] == [L[i] for i in range(3,len(L),-1)] True


If you manage somehow to hange that 'True' to a 'False', I'll
personally be extremely unhappy.

Regards,
Pat

Sep 6 '05 #28

P: n/a

I previously wrote (in response to a query from Ron Adam):
In any case, you asked for a rationale. I'll give you mine:
L = range(10)
L[3:len(L):-1] == [L[i] for i in range(3,len(L),-1)] True
After eating supper, I just realized that I could probably make my
point a bit clearer with a slightly longer example:
L = range(10)
for stride in [-3, -2, -1, 1, 2, 3]:

.... for start in range(len(L)):
.... for end in range(len(L)):
.... P = L[start:end:stride]
.... Q = [L[i] for i in range(start, end, stride)]
.... assert P == Q

This should never fail with an assertion error. You will note that it
shows that, for non-negative start and end values, slicing behavior is
_exactly_ like extended range behavior. I cannot imagine that the
behavior of range() could be made any more intuitive than it already
is. I personally feel that your proposed change makes slice() less
intuitive on its own, but even if I did not feel that way, your way
would have to be SIGNIFICANTLY better than the current way to make it
worthwhile to make slice() behavior differ from that of range().

In my initial skimming of your post, I originally thought you were
referring to negative start and end values. Negative start and end
values will sometimes cause issues, but the utility of their current
implementation far outweighs the few corner cases which (as I mentioned
in an earlier post) sometimes need some special case logic to deal
with.

Regards,
Pat

Sep 6 '05 #29

P: n/a
Terry Reedy wrote:
Ron Adam wrote:

However, I would like the inverse selection of negative strides to be
fixed if possible. If you could explain the current reason why it does
not return the reverse order of the selected range.

To repeat, the current reason is compatibility with the original design for
NumPy. Perhaps there is some explanation for that in sliceobject.c (or
whatever the file is called), the CVS checkin messages, or the newsgroup
archives before the first checkin (in 1996, I believe).

Terry J. Reedy


Thanks, I'll check the cvs, so far I havn't been able to find any
references to it.




Sep 6 '05 #30

P: n/a
Patrick Maupin wrote:
I previously wrote (in response to a query from Ron Adam):

In any case, you asked for a rationale. I'll give you mine:

>L = range(10)
>L[3:len(L):-1] == [L[i] for i in range(3,len(L),-1)]
True


After eating supper, I just realized that I could probably make my
point a bit clearer with a slightly longer example:

L = range(10)
for stride in [-3, -2, -1, 1, 2, 3]:


... for start in range(len(L)):
... for end in range(len(L)):
... P = L[start:end:stride]
... #Q = [L[i] for i in range(start, end, stride)]


Q = [L[i] for i in range(start, end)][::stride]
... assert P == Q

With the changed line above it will pass with the example nxlist type.

The result is different because the method is different. So in order
for this test to not give an assert, the same order needs to be used.

This should never fail with an assertion error. You will note that it
shows that, for non-negative start and end values, slicing behavior is
_exactly_ like extended range behavior.
Yes, and it passes for negative start and end values as well.

I cannot imagine that the
behavior of range() could be made any more intuitive than it already
is.
If range were also changed.. (I'm not suggesting it should be)
range(0,10,-1), would count down from 10 to zero. The sign of the step
would determine which side to iterate from.

I think they are both fairly equivalent as far as intuitiveness. But I
think I agree, changing both range and slice is probably out of the
question.

I personally feel that your proposed change makes slice() less
intuitive on its own, but even if I did not feel that way, your way
would have to be SIGNIFICANTLY better than the current way to make it
worthwhile to make slice() behavior differ from that of range().
Well I did post it as a "possible" improvement, meaning I wasn't sure.
And did ask for opinions. This was the type of reply I was looking for.
Thanks for replying, and for taking a serious look. :-)

In my initial skimming of your post, I originally thought you were
referring to negative start and end values. Negative start and end
values will sometimes cause issues, but the utility of their current
implementation far outweighs the few corner cases which (as I mentioned
in an earlier post) sometimes need some special case logic to deal
with.
I was referring to both, it seemed to me that my suggestion had enough
good things in it that it would be worth asking others if it was
feasible, and also if it were desirable. In any case, it's an
interesting topic and looks like it could have been a valid alternative
if it were done this way from the start. Probably isn't good enough to
change now.

Thanks again, this pretty much explains why slices opperate the way they
do. And it explains why the edge case's happen as well I think.

Cheers,
Ron
Regards,
Pat

Sep 6 '05 #31

P: n/a
Steve Holden wrote:
Ron Adam wrote:
Steve Holden wrote:

What misconception do you think I have?
This was not an ad hominem attack but a commentary on many attempts to
"improve" the language.


Ok, No problem. ;-)

No one has yet explained the reasoning (vs the mechanics) of the
returned value of the following.

L = range(10)
L[3::-1]

So far every attempt to explain it has either quoted the documents
which don't address that particular case, or assumed I'm
misunderstanding something, or implied it isn't neccisary to do.

It's quite easy to get me to change my mind on something, just show me
a convincing explanation and I will. :)
>>> L[3::-1]

[3, 2, 1, 0] >>> L[3::1] [3, 4, 5, 6, 7, 8, 9] >>>


I don;t see the problem here. The start specifies which element is the
first in the slice, the stop is the default (end of the sequence) and
the stride is "successive elements to the left" when it's -1 and
"successive elements to the right" when it's 1.


This is how it works. But determining the correct index's to use in
some cases can be tricky.

Or perhaps you can tell me what I've missed?
Ok, lets see... This shows the problem with using the gap indexing model.

L = range(10)

[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] # elements
0 1 2 3 4 5 6 7 8 9 10 # index's

L[3::1] -> [3, 4, 5, 6, 7, 8, 9] 3rd index to end... ok
L[3:6:1] -> [3, 4, 5] 3rd index to 6th index... ok

L[3::-1] -> [3, 2, 1, 0] 4th index to beginning... umm
L[6:3:-1] -> [6, 5, 4] 7th index to 4th index... ?

So negative strides use a different index position?

[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] # elements
-1 0 1 2 3 4 5 6 7 8 9 # index's

L[3::-1] -> [3, 2, 1, 0] 3rd index to beginning... ok
L[6:3:-1] -> [6, 5, 4] 6th index to 3rd index... ok

To reach the '0' we have to check the index...

if i <= -1:
r = L[i+3::-1]
else:
r = L[i+3:i:-1]
Using negative index's ...

[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] # elements
-10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 # index's

L[-3::1] -> [7, 8, 9] ok
L[-6:-3:1] -> [4, 5, 6] ok

L[-3::-1] -> [7, 6, 5, 4, 3, 2, 1, 0] -2nd to -10th ?
L[-3:-6:-1] -> [7, 6, 5] -2nd index to -5th.. ?
So we have to shift all the index's to the right again.

[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] # elements
-11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 # index's

L[-3::-1] -> [7, 6, 5, 4, 3, 2, 1, 0] -3rd to -11th
L[-3:-6:-1] -> [7, 6, 5] -3rd index to -6th.

I feel it would be nicer if the index's didn't shift for negative strides.

Maybe gap addressing isn't really the best way to explain slicing?

Most of this function has to do with filling in the defaults, but it
seems to work. I'm sure theres a better way to determine the defaults
than this rather awkward if-else tree.

def slc(L, start, stop=None, step=1):
if stop == None:
if start == None:
if step >= 0:
start = 0
stop = len(L)
else:
start = len(L)-1
stop = -1
elif start >=0:
if step>0:
stop = len(L)
else:
stop = -1
else:
if step>0:
stop = 0
else:
stop = -(len(L)+1)
if start == None:
if stop >= 0:
if step>0:
start = 0
else:
start = len(L)-1
else:
if step>0:
start = -1
else:
start = 0
new = []
for i in range(start,stop,step):
new.append(L[i])
return new

This is more precise, but not neccisarily any easier to understand
without mentally tracing the conditions. But it's clear now why slices
behave the way they do.
Having to reverse the order of the index's along with using -steps is a
bit bothersome. It might be easier if it were of the form...

L[left:right:step]

Then positive and negative index's could be easily be mixed, and the
slice is always the space between. Step determines the output order and
stride.

Thats easy enough to explain even for beginners. But as Patrick pointed
out it's not consistent with range.

This alternate "suggestion" gets the slice first then reverses it. It
has the advantage that the above alternate indexing adjustments go away
also.

And.

L[3:6:1] = L[3:6:-1] reverses a sub sequence.
This disadvantages are: it's different, it's not compatible with current
range. It probably breaks more backward compatibility than I suspect.

My point was that you can make those changes in your own code, leaving
others to accept the situation as it is.
It's only a suggestion and an interesting idea I thought I would share
and see if anyone would like to discuss. I did title this as a 'possible
improvement', and not as proposed improvement.

Right, I wasn't trying to suggest that you didn't know what you were
talking about - or even that you didn't understand general relativity
(on which my grasp could be said to be tenuous) - merely that some
things are inherently difficult, and no matter how you twist the
implementations about, the difficulties will remain.
But shouldn't we try to make things easier when possible?

Cheers,
Ron

regards
Steve

Sep 6 '05 #32

P: n/a

Ron Adam wrote:
This should never fail with an assertion error. You will note that it
shows that, for non-negative start and end values, slicing behavior is
_exactly_ like extended range behavior.
Yes, and it passes for negative start and end values as well.
Umm, no:

..>> for stride in [-3, -2, -1, 1, 2, 3]:
.... for start in range(-1,len(L)):
.... for end in range(-1,len(L)):
.... P = L[start:end:stride]
.... Q = [L[i] for i in range(start, end, stride)]
.... assert P==Q, [start, end, stride, P, Q]
....
Traceback (most recent call last):
File "<stdin>", line 6, in ?
AssertionError: [-1, 0, -3, [9, 6, 3], []]
Thanks again, this pretty much explains why slices opperate the
way they do. And it explains why the edge case's happen as well I think.


You're welcome.

Regards,
Pat

Sep 6 '05 #33

P: n/a
Ron Adam wrote:
Ok, lets see... This shows the problem with using the gap indexing model.

L = range(10)

[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] # elements
0 1 2 3 4 5 6 7 8 9 10 # index's

L[3::1] -> [3, 4, 5, 6, 7, 8, 9] 3rd index to end... ok
L[3:6:1] -> [3, 4, 5] 3rd index to 6th index... ok

L[3::-1] -> [3, 2, 1, 0] 4th index to beginning... umm
L[6:3:-1] -> [6, 5, 4] 7th index to 4th index... ?

So negative strides use a different index position?

[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] # elements
-1 0 1 2 3 4 5 6 7 8 9 # index's

L[3::-1] -> [3, 2, 1, 0] 3rd index to beginning... ok
L[6:3:-1] -> [6, 5, 4] 6th index to 3rd index... ok


Ok, I see what you mean. The "view slices as indices standing
between the items in the sequence" model actually breaks down
with negative strides.

Then you need to view it more in the mathematical way of
half-open (or half-closed if you prefer) intervals.

[a,b) = { x | a <= x < b }

See http://en.wikipedia.org/wiki/Half-closed_interval

I still think the current sematics are the least surprising
though. For instance, if we want to implement a ring class
looking something like the one below, the logical way to do
that would allow it to be used as if it was a list.

Actually, such a ring class (extented to handle extended slices
correctly) would (I think) solve the tricky cases with things
such as l[0:-0] or l[9:-1:-1].

The complete implementation is left as an exercise to the
reader, although there's probably something like it in the
Python cookbook already.

class Ring(list):
def __init__(self, size):
self.size = size
def __setitem__(self, i,v):
return list.__setitem__(self, i%self.size, v)
def __getitem__(self, i):
return list.__getitem__(self, i%self.size)
def __setslice__(self, i, j, v):
return list.__setslice__(self, i%self.size, j%self.size,v)
def __getslice__(self, i, j):
return list.__getslice__(self, i%self.size, j%self.size)
Sep 6 '05 #34

P: n/a
Patrick Maupin wrote:
Ron Adam wrote:

This should never fail with an assertion error. You will note that it
shows that, for non-negative start and end values, slicing behavior is
_exactly_ like extended range behavior.
Yes, and it passes for negative start and end values as well.

Umm, no:

.>> for stride in [-3, -2, -1, 1, 2, 3]:
... for start in range(-1,len(L)):
... for end in range(-1,len(L)):
... P = L[start:end:stride]
... Q = [L[i] for i in range(start, end, stride)]
... assert P==Q, [start, end, stride, P, Q]
...
Traceback (most recent call last):
File "<stdin>", line 6, in ?
AssertionError: [-1, 0, -3, [9, 6, 3], []]


Ah, Yes... I it was way too late last night and I mistakenly changed the
values of L... which was meaningless.

Range lines in the for statements, need to read..

range(-len(l),0)

But then it doesn't include all the values of L.

Thanks again, this pretty much explains why slices opperate the
way they do. And it explains why the edge case's happen as well I think.

You're welcome.

Regards,
Pat

Sep 6 '05 #35

P: n/a
Magnus Lycka wrote:
Ron Adam wrote:
Ok, lets see... This shows the problem with using the gap indexing
model.

L = range(10)

[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] # elements
0 1 2 3 4 5 6 7 8 9 10 # index's

L[3::1] -> [3, 4, 5, 6, 7, 8, 9] 3rd index to end... ok
L[3:6:1] -> [3, 4, 5] 3rd index to 6th index... ok

L[3::-1] -> [3, 2, 1, 0] 4th index to beginning... umm
L[6:3:-1] -> [6, 5, 4] 7th index to 4th index... ?

So negative strides use a different index position?

[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] # elements
-1 0 1 2 3 4 5 6 7 8 9 # index's

L[3::-1] -> [3, 2, 1, 0] 3rd index to beginning... ok
L[6:3:-1] -> [6, 5, 4] 6th index to 3rd index... ok

Ok, I see what you mean. The "view slices as indices standing
between the items in the sequence" model actually breaks down
with negative strides.


Yes, that and the edge case's is why I this topic keeps coming up. Then
there's the 'None' default values that depend on both the stride sign,
and the index signs. These are all features in some context, and can be
annoyances in others. As long as you stick with positive stride values,
it's not much of a problem though, so an alternate solution will have to
be really good.

Then you need to view it more in the mathematical way of
half-open (or half-closed if you prefer) intervals.

[a,b) = { x | a <= x < b }

See http://en.wikipedia.org/wiki/Half-closed_interval
Interesting... Maybe just a different syntax that designates the stop
as being inclusive would work?

[a,b] = { x | a <= x <= b }

L[a;<b] a to b-1, same as L[a:b]
L[a;b] a to b
L[a>;b] a+1 to b
L[a>;<b] a+1 to b-1

L == L[;<i] + L[i;] == L[;i] + L[i>;]

I still think the current sematics are the least surprising
though. For instance, if we want to implement a ring class
looking something like the one below, the logical way to do
that would allow it to be used as if it was a list.

Actually, such a ring class (extented to handle extended slices
correctly) would (I think) solve the tricky cases with things
such as l[0:-0] or l[9:-1:-1].

The complete implementation is left as an exercise to the
reader, although there's probably something like it in the
Python cookbook already.

class Ring(list):
def __init__(self, size):
self.size = size
def __setitem__(self, i,v):
return list.__setitem__(self, i%self.size, v)
def __getitem__(self, i):
return list.__getitem__(self, i%self.size)
def __setslice__(self, i, j, v):
return list.__setslice__(self, i%self.size, j%self.size,v)
def __getslice__(self, i, j):
return list.__getslice__(self, i%self.size, j%self.size)


This is nice. I might even find a use for it. ;-)

Cheers,
Ron


Sep 6 '05 #36

P: n/a
On Tue, 06 Sep 2005 10:31:33 GMT, Ron Adam <rr*@ronadam.com> wrote:
Steve Holden wrote: [...]
My point was that you can make those changes in your own code, leaving
others to accept the situation as it is.


It's only a suggestion and an interesting idea I thought I would share
and see if anyone would like to discuss. I did title this as a 'possible
improvement', and not as proposed improvement.

Right, I wasn't trying to suggest that you didn't know what you were
talking about - or even that you didn't understand general relativity
(on which my grasp could be said to be tenuous) - merely that some
things are inherently difficult, and no matter how you twist the
implementations about, the difficulties will remain.


But shouldn't we try to make things easier when possible?

Sure ;-)

It occurs to me that maybe this discussion of slicing has a parallel
in mathematical intervals, and we might usefully check if it makes sense there.

IOW, let's compare [a, b] vs [a, b) vs (a, b] vs (a,b)

ISTM the python range model corresponds to [a, b) in terms of integers. The step thing
modifies that, but leave that aside to concetrate on the main issue.

For math, I think the notation requires a<=b, but for programming, python has a convention
for specifying related intervals and a subsetting function, with similar notation adding step.

Leaving aside abs(step)!=1 which specifies subsetting, we could say that

[a:b:1]
is
[a, b)
and
[a:b,-1]
is
(a, b]

but the latter returned in reverse order.

If we factor out the issues of reversing and subsetting, we just have
the indication of which kind of interval: half-open to the right or left.

That we could do by
[a:b] => [a, b)
and
.[a:b] => (a, b]

Then the question is, do we need sugar for reversed(x.[a:b])
or list(reversed(x.[a:b])) for the right hand side of a statement,
and do we want to to use both kinds of intervals in slice assignment?
(maybe and yes ;-)

The reason for yes is that it solves the which-gap problem in assigning to [a:a]
if you define [a, a) as an empty interval to the left of a and (a, a] as an empty
interval to the right of a, sort of like +0 and -0 in half-open intervals ;-)
Replacing the empty interval does the insertion on the side you want ;-)

I am choosing the convention to stay compatible with python's current behaviour,
even though I'm not sure what the math world says about a<x<=a vs a<=x<a as
different-in-some-sense intervals. I guess the intervals as point sets are the same,
but the interval definitions are different...

Other than the a:a distinction, in terms of integers and UIAM
.[a:b]
is just sugar for
[a+1:b+1]
but the sugar is nice, and the slice assignment to either side is nicer.

I'll deal with the subsetting another time ...

Regards,
Bengt Richter
Sep 7 '05 #37

P: n/a
On Tue, 06 Sep 2005 18:34:13 +0200, Magnus Lycka <ly***@carmen.se> wrote:
[...]

Then you need to view it more in the mathematical way of
half-open (or half-closed if you prefer) intervals.

[a,b) = { x | a <= x < b }

Funny, I just posted with the same thought (and some additional considerations)
I hadn't read yours yet, or I would have mentioned it ;-)

Regards,
Bengt Richter
Sep 7 '05 #38

P: n/a
Bengt Richter wrote:
Then the question is, do we need sugar for reversed(x.[a:b])
or list(reversed(x.[a:b])) for the right hand side of a statement,
and do we want to to use both kinds of intervals in slice assignment?
(maybe and yes ;-)
Yes, I think this is the better way to do it, as this address's the
underlying causes instead of treating the symptoms.

The reason for yes is that it solves the which-gap problem in assigning to [a:a]
if you define [a, a) as an empty interval to the left of a and (a, a] as an empty
interval to the right of a, sort of like +0 and -0 in half-open intervals ;-)
Replacing the empty interval does the insertion on the side you want ;-)

I am choosing the convention to stay compatible with python's current behaviour,
even though I'm not sure what the math world says about a<x<=a vs a<=x<a as
different-in-some-sense intervals. I guess the intervals as point sets are the same,
but the interval definitions are different...
Not sure either. I think intervals is an important concept and enabling
python to work with them would be good if it could be done in a simple
and consistent way. This extends a lot further than just slice
operations because of the relationship between ...

for x in range() -> slice(range)
So defining an interval object that can be used as an iterator in a for
loop might be the beginning, and then finally slice with an alternate
syntax to an abbreviated form. So then you would have the relationship
of...

for x in interval() -> slice(interval)

Other than the a:a distinction, in terms of integers and UIAM
.[a:b]
is just sugar for
[a+1:b+1]
but the sugar is nice, and the slice assignment to either side is nicer.
Wouldn't that be [a:b+1] ?

As sugar the index's are translated at compile time, it may run into the
current edge case indexing problems. So it will most likely need to be
an actual interval object, with an alternative syntax to use it.
I'll deal with the subsetting another time ...
No hurry, this isn't a hack it out because "the boss wants it on his
desk Monday" situation. ;-)
Regards,
Bengt Richter


Cheers,
Ron

Sep 7 '05 #39

P: n/a
Ron Adam <rr*@ronadam.com> writes:
Magnus Lycka wrote:
Ron Adam wrote: [...]
REVERSE ORDER STEPPING
----------------------
When negative steps are used, a slice operation
does the following. (or the equivalent)

1. reverse the list
2. cut the reversed sequence using start and stop
3. iterate forward using the absolute value of step.

I think you are looking at this from the wrong perspective.
Whatever sign c has:
For s[a:b:c], a is the index for the first item to include,
b is the item after the last to include (just like .end() in
C++ iterators for instance), and c describes the step size.


Yes, and that is how it "should" work. But....

With current slicing and a negative step...

[ 1 2 3 4 5 6 7 8 9 ]
-9 -8 -7 -6 -5 -4 -3 -2 -1 -0

r[-3:] -> [7, 8, 9] # as expected
r[-3::-1] -> [7, 6, 5, 4, 3, 2, 1, 0] # surprise

The seven is include in both cases, so it's not a true inverse
selection either.


Did you read what Magnus said: "a is the index for the first item to
include"? How could r[-3::x] for any x not include the 7?

Cheers,
mwh

--
Windows 2000: Smaller cow. Just as much crap.
-- Jim's pedigree of operating systems, asr
Sep 9 '05 #40

P: n/a
Michael Hudson wrote:
Ron Adam <rr*@ronadam.com> writes:

With current slicing and a negative step...

[ 1 2 3 4 5 6 7 8 9 ]
-9 -8 -7 -6 -5 -4 -3 -2 -1 -0

r[-3:] -> [7, 8, 9] # as expected
r[-3::-1] -> [7, 6, 5, 4, 3, 2, 1, 0] # surprise

The seven is include in both cases, so it's not a true inverse
selection either.


Did you read what Magnus said: "a is the index for the first item to
include"? How could r[-3::x] for any x not include the 7?

Cheers,
mwh


Yes, and I agreed this is how it works. The point above was the 7 was
outside the "in-between" index's. Not that it shouldn't be included.
And yes, I wasn't looking at it quite correctly as well, but my point is
still valid in the context it was given.

I originally posted that negative index's were one's based, but then I
got multiple responses that I was looking at it incorrectly (using the
wrong model) and I should look at index's as being the places between
the items. (Or on the left side of the items) This is also mentioned in
the tutorial as follows.
from 2.4.1 Docs, Section 3.1.2, Strings:
----------------------------------------
The best way to remember how slices work is to think of the indices as
pointing between characters, with the left edge of the first character
numbered 0. Then the right edge of the last character of a string of n
characters has index n, for example:

+---+---+---+---+---+
| H | e | l | p | A |
+---+---+---+---+---+
0 1 2 3 4 5
-5 -4 -3 -2 -1

The first row of numbers gives the position of the indices 0...5 in the
string; the second row gives the corresponding negative indices. The
slice from i to j consists of all characters between the edges labeled i
and j, respectively.
-----------------------------------------
But you need a different pair of index's for this to work with negative
step values as you do with positive step values. (or switch to right
side indexing.)

So I tried to find a consistent way for in-between index's to work and
solution I found "and asked if it might be a possibility" wasn't
desirable. (And I was told again I was using the wrong model.) ;-)

So we get back to how it actually works, (base 1 for negative index's),
Or more precisely .. (len(L)+i) indexing.) which is a little more
complex visually, but I think is better to teach it correctly from the
start and avoid the confusions that the in-between index's lead to
later. And also avoid the confusion of having two different way of
looking at it.

Here's the correct way....
From 2.4.1 Docs, section 2.3.6, Sequence Types:
-----------------------------------------------
s[i] i'th item of s, origin 0 (3)
s[i:j] slice of s from i to j (3), (4)
s[i:j:k] slice of s from i to j with step k (3), (5)

(3)
If i or j is negative, the index is relative to the end of the string:
len(s) + i or len(s) + j is substituted. But note that -0 is still 0.

(4)
The slice of s from i to j is defined as the sequence of items with
index k such that i <= k < j. If i or j is greater than len(s), use
len(s). If i is omitted, use 0. If j is omitted, use len(s). If i is
greater than or equal to j, the slice is empty.

(5)
The slice of s from i to j with step k is defined as the sequence of
items with index x = i + n*k such that . In other words, the indices are
i, i+k, i+2*k, i+3*k and so on, stopping when j is reached (but never
including j). If i or j is greater than len(s), use len(s). If i or j
are omitted then they become ``end'' values (which end depends on the
sign of k). Note, k cannot be zero.
-------------------------------------
This could be a bit clearer.

Notice this doesn't say to use index's between items. Looking at the
index's as being either on the left or right creates the 'apparent'
indexing inconsistency I was referring to. It looks good at first, but
breaks down later, so the examples using in-between index's in the docs
and tutorials should probably be rewritten to show how it actually works
right from the start. Something like the following examples with a walk
through would do.

['a', 'b', 'c', 'd', 'e', 'f', 'g']
0 1 2 3 4 5 6 <--- i
-7 -6 -5 -4 -3 -2 -1 <--- len(L)+i

example: i = -7 -> len(L) + -7 -> 7 + -7 -> 0

L[3] = 3
L[:3] = ['a','b','c'] # L[0:3]
L[3:] = ['d','e','f','g'] # L[3:7]

L[-3] -> 'e'
L[:-3] -> ['a','b','c','d'] # L[-7:-3]
L[-3:] -> ['e','f','g'] # L[-3:len(L)] (use len(L), not 0)

L[-3::-1] -> ['e','d','c','b','a'] # L[-3:-8:-1]
L[:-3:-1] -> ['g','f'] # L[-1:-3:-1]

L[3::-1] -> ['d','c','b','a'] # L[3:-Len(L)-1:-1](-1 wont' work here)
L[:3:-1] -> ['g','f','e'] # L[0:3:-1]

L[3:4:1] i-> ['d']
L[3:2:-1] -> ['d']

L[-4:-3: 1] -> ['e']
L[-4:-5:-1] -> ['e']

(*) The problems stituations are the cases where start and stop are on
each side of the -1/0 boundary. This causes difficulty reaching end
points in some situations and also requires having to do extra bounds
checking if you move the stop index around.

So maybe we can improve things in some situations when that occurs?
Or at least try. So here's another (last by me) attempt. The following
will break current code, but it is a stable and predictable method.

It has both advantages and disadvantages over the current method. (as
usual)

+ Bound checking the stop index isn't needed to avoid situations
where it crosses -1/0 . (works in both directions)

+ The order is always preserved from the starting position. No ugly
switches if the stop in incrimented past -1/0.

+ Only need to check start position if you are moving it.

- can't index from both ends using both positive and negative
indexing at the same time.
With the below function, L[1:-1] does not remove an item off each end,
but instead returns [] as the stop is less than the start. So you have
to do it in two slices.

r = L[1:][:-1]

Or use all positive indexing.

r = L[1:len(L)-1]

Or all negative indexing.

r = r[-len(L):-1]

This is because in order to resolve the -1/0 indexing situations you
have to disallow indexing from both sides at the same time. I would
guess that a lot of people wouldn't want to give that up now that
they've gotten used to it. (? shrug)
Anyway, this might give someone else an idea or two. ;-)

Cheers,
Ron


Given: L[i:j:k] as start,stop,step.

And the following diagrams:

|--------i------0-------->
j<=i [i......] j>0
<------+0-----i---------|
j<-1 [......i] j>=i
-1

This function could probably be cleaned up a bit. I think there may be
several possible short cuts that would speed this up and reduce the
number of comparisons.

def Slice(L, start=None, stop=None, step=1):
if step == None:
step = 1
if start != None \
and stop != None \
and (type(start) != int \
or type(stop) != int \
or type(step) != int):
raise TypeError("slice indices must be integers")
if step == 0:
raise ValueError("slice step cannot be zero")
if step >= 1:
if start == None:
start = 0
if stop == None:
stop = len(L)
if start < -len(L):
start = -len(L)
if start > len(L) or stop <= start:
return []
if start < 0 and stop > 0:
start = start+len(L)
stop = stop+len(L)
if stop > len(L):
stop = len(L)
elif step <= -1:
if start == None:
start = -1
if stop == None:
stop = -len(L)-1
if start >= len(L):
start = len(L)-1
if stop >= start or start < -len(L):
return []
if start >= 0 and stop < 0 :
start = start - len(L)
stop = stop - len(L)
if stop < -len(L)-1:
stop = -len(L)-1
new = []
for i in range(start,stop,step):
new.append(L[i])
return new
L = range(10)
data = [
(0,-0,1),
(None, None, None),
(None, None, 1),
(3, 6, 1),
(3, 100, 1),
(100,6,-1),
(3, -1, -1),
(-3, 0, 1), # this works
(5, -1, -1), # this too
(None, -100, -1),
(-100, 100, 1),
(6, 3, -1),
(-3, None, 1),
(-6, 5, 1),
(-50, -6, 1),
(-3, -100, -1),
(-3, -6, -1),
(-3, -100, 0),
(-3, -6, .5),
(-3, 34.5, 1),
(2.0, -6, -1),
]
import sys
for start,stop,step in data:
print 'L[%r:%r:%r]='%(start,stop,step),
try:
print Slice(L,start,stop,step)
except:
print sys.exc_type, ":", sys.exc_value
print

---------- Output ------------

L[0:0:1]= []
L[None:None:None]= [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
L[None:None:1]= [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
L[3:6:1]= [3, 4, 5]
L[3:100:1]= [3, 4, 5, 6, 7, 8, 9]
L[100:6:-1]= [9, 8, 7]
L[3:-1:-1]= [3, 2, 1, 0]
L[-3:0:1]= [7, 8, 9]
L[5:-1:-1]= [5, 4, 3, 2, 1, 0]
L[None:-100:-1]= [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
L[-100:100:1]= [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
L[6:3:-1]= [6, 5, 4]
L[-3:None:1]= [7, 8, 9]
L[-6:5:1]= [4, 5, 6, 7, 8, 9]
L[-50:-6:1]= [0, 1, 2, 3]
L[-3:-100:-1]= [7, 6, 5, 4, 3, 2, 1, 0]
L[-3:-6:-1]= [7, 6, 5]
L[-3:-100:0]= exceptions.ValueError : slice step cannot be zero
L[-3:-6:0.5]= exceptions.TypeError : slice indices must be integers
L[-3:34.5:1]= exceptions.TypeError : slice indices must be integers
L[2.0:-6:-1]= exceptions.TypeError : slice indices must be integers

Sep 9 '05 #41

This discussion thread is closed

Replies have been disabled for this discussion.