473,804 Members | 3,894 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Possible improvement to slice opperations.


After considering several alternatives and trying out a few ideas with a
modified list object Bengt Richter posted, (Thank You), I think I've
found a way to make slice operation (especially far end indexing)
symmetrical and more consistent.

So to find out if this is indeed a possibility, it would be nice to get
a few opinions at this point. So blast away... or hopefully tell me
what you like about it instead. ;-)

(Any suggestions or contributions to make it better would be appreciated.)

Cheers,
Ron Adam

"""
IMPROVED SLICING
=============== =

Slicing is one of the best features of Python in my opinion, but
when you try to use negative index's and or negative step increments
it can be tricky and lead to unexpected results.

This topic has come up fairly often on comp.lang.pytho n, and often
times, the responses include:

* Beginners should avoid negative extended slices.

* Slices with negative step increments are for advanced
python programmers.

* It's not broke if you look at it in a different way.

* You should do it a different way.

All of these and various responses similar to them are unsatisfactory in
my opinion and it's a good indicator it's not as good as it could be.
Indexing and slice operations are vary common in Python and I don't
think we should be discouraging people from learning and using them.
COMPATIBILITY
-------------
Because the following suggested changes will break current code,
it probably can not be implemented prior to Python 3000.

+ Direct indexing with both positive and negative values
returns the same items as they do now.

+ Extended slices with all positive and or empty default
values remain unchanged.

- Extended slices with negative values return values that
have less items than currently.

- Slices with negative step values return entirely different
results.
REVERSE ORDER STEPPING
----------------------
When negative steps are used, a slice operation
does the following. (or the equivalent)

1. reverse the list
2. cut the reversed sequence using start and stop
3. iterate forward using the absolute value of step.

* This order results in an inverse selection and I believe should be
considered a bug.

Changing the order in the following way results in a much
more predictable pattern which is both easier to understand and use.

1. cut sequence using start and stop.
2 reverse the order of the results.
3. iterate forward using the absolute value of step.
CURRENT INDEXING
----------------

Given a range [a,b,c]:

Positive indexing

| a | b | c |
+---+---+---+
0 1 2 3

Current negative indexing.

| a | b | c |
+---+---+---+
-3 -2 -1 -0
When a single index is used the item to the
right of the index for both positive and
negative index's is returned.

With slices, the items between start, and
stop index's are returned.

Accessing a range at the end of a list numerically
becomes inconvenient when negative index's are used
as the '-0'th position can not be specified numerically
with negative values.
ONES BASED NEGATIVE INDEXING
----------------------------
Making negative index's Ones based, and selecting
individual item to the left of negative index's would enable
addressing the end of the list numerically.

Ones based negative index's.

| a | b | c |
+---+---+---+
-4 -3 -2 -1

Then:

a[-1] -> c # item left of index, same result as now.

a[-3:-2] -> b # item between index's

a[-1:-1] = [d] # insert item at the end.

USE OF '~' IN PLACE OF NEGATIVE INDEX'S
---------------------------------------

The '~' is the binary not symbol which when used
with integers returns the two's compliment. This
works nice with indexing from the end of a list
because convieniently ~0 == -1.

This creates a numerical symmetry between positive
indexing and '~' "End of list" indexing.

a[0] -> first item in the list.
a[~0] -> last item in the list.

a[0:~0] -> whole list.

a[1:~1] -> center, one position from both ends.

* Note: using '~' works as described here in place of single negative
index's in current versions of Python. It does not work as described
here for extended slices.

"""

# TEST LIST CLASS.
"""
A list class to Test '~' end of list indexing.

* This class modifies the slice before returning
a value. The final implementation may do this by
modifying slice objects directly or the underlying
C code of sequences.

"""

class nxlist(object):

def __init__(self, value):
self.value = value

def normslc(self, slc):
start,stop,step = slc.start, slc.stop, slc.step
if type(start) == int and start<0:
start = len(self.value) +start+1
if type(stop) == int and stop<0:
stop = len(self.value) +stop+1
return slice(start,sto p,step)

def __getitem__(sel f, i):
tp = i.__class__
if tp == int:
if i>=0:
return self.value[i]
else:
return self.value[ len(self.value) +i ]
if tp == slice:
slc = self.normslc(i)
value = self.value[slc.start:slc.s top]
if type(i.step) == int and i.step<0:
value.reverse()
slc = slice(None,None ,-i.step)
else:
slc = slice(None,None ,i.step)
return value[slc]

#def __setitem__(sel f, i, v):
#Not emplimented yet.

def __repr__(self): return 'nxlist(%r)'%se lf.value
a = nxlist(range(10 ))
print a

testdata = [
('a[0]'),
('a[1]'),
('a[~0]'),
('a[~1]'),
('a[:]'),
('a[:~1]'),
('a[~1:]'),
('a[::]'),
('a[0:~0]'),
('a[1:~1]'),
('a[~1:1]'),
('a[::-2]'),
('a[:3]'),
('a[3:]'),
('a[~3:]'),
('a[:~3]'),
('a[:3:-1]'),
('a[3::-1]'),
('a[~3::-1]'),
('a[:~3:-1]'),
('a[:3:-2]'),
('a[3::-2]'),
('a[~3::-2]'),
('a[:~3:-2]'),
]

for n, s in enumerate(testd ata):
print '%r. %s = %r' % (n,s,eval(s))
"""
nxlist([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
0. a[0] = 0
1. a[1] = 1
2. a[~0] = 9
3. a[~1] = 8
4. a[:] = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
5. a[:~1] = [0, 1, 2, 3, 4, 5, 6, 7, 8]
6. a[~1:] = [9]
7. a[::] = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
8. a[0:~0] = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
9. a[1:~1] = [1, 2, 3, 4, 5, 6, 7, 8]
10. a[~1:1] = []
11. a[::-2] = [9, 7, 5, 3, 1]
12. a[:3] = [0, 1, 2]
13. a[3:] = [3, 4, 5, 6, 7, 8, 9]
14. a[~3:] = [7, 8, 9]
15. a[:~3] = [0, 1, 2, 3, 4, 5, 6]
16. a[:3:-1] = [2, 1, 0]
17. a[3::-1] = [9, 8, 7, 6, 5, 4, 3]
18. a[~3::-1] = [9, 8, 7]
19. a[:~3:-1] = [6, 5, 4, 3, 2, 1, 0]
20. a[:3:-2] = [2, 0]
21. a[3::-2] = [9, 7, 5, 3]
22. a[~3::-2] = [9, 7]
23. a[:~3:-2] = [6, 4, 2, 0]
"""
r = range(10)
a = nxlist(r)

print r[~0],a[~0] # ok
print r[~3:],a[~3:] # one off
print r[~3::-1],a[~3::-1] # other side
print r[~3::-2],a[~3::-2] # other side
"""
Comparisons of negative indexing and '~'
indexing with same values.

current, proposed

9 9
[6, 7, 8, 9] [7, 8, 9]
[6, 5, 4, 3, 2, 1, 0] [9, 8, 7]
[6, 4, 2, 0] [9, 7]
"""

Sep 4 '05
40 2624
Michael Hudson wrote:
Ron Adam <rr*@ronadam.co m> writes:

With current slicing and a negative step...

[ 1 2 3 4 5 6 7 8 9 ]
-9 -8 -7 -6 -5 -4 -3 -2 -1 -0

r[-3:] -> [7, 8, 9] # as expected
r[-3::-1] -> [7, 6, 5, 4, 3, 2, 1, 0] # surprise

The seven is include in both cases, so it's not a true inverse
selection either.


Did you read what Magnus said: "a is the index for the first item to
include"? How could r[-3::x] for any x not include the 7?

Cheers,
mwh


Yes, and I agreed this is how it works. The point above was the 7 was
outside the "in-between" index's. Not that it shouldn't be included.
And yes, I wasn't looking at it quite correctly as well, but my point is
still valid in the context it was given.

I originally posted that negative index's were one's based, but then I
got multiple responses that I was looking at it incorrectly (using the
wrong model) and I should look at index's as being the places between
the items. (Or on the left side of the items) This is also mentioned in
the tutorial as follows.
from 2.4.1 Docs, Section 3.1.2, Strings:
----------------------------------------
The best way to remember how slices work is to think of the indices as
pointing between characters, with the left edge of the first character
numbered 0. Then the right edge of the last character of a string of n
characters has index n, for example:

+---+---+---+---+---+
| H | e | l | p | A |
+---+---+---+---+---+
0 1 2 3 4 5
-5 -4 -3 -2 -1

The first row of numbers gives the position of the indices 0...5 in the
string; the second row gives the corresponding negative indices. The
slice from i to j consists of all characters between the edges labeled i
and j, respectively.
-----------------------------------------
But you need a different pair of index's for this to work with negative
step values as you do with positive step values. (or switch to right
side indexing.)

So I tried to find a consistent way for in-between index's to work and
solution I found "and asked if it might be a possibility" wasn't
desirable. (And I was told again I was using the wrong model.) ;-)

So we get back to how it actually works, (base 1 for negative index's),
Or more precisely .. (len(L)+i) indexing.) which is a little more
complex visually, but I think is better to teach it correctly from the
start and avoid the confusions that the in-between index's lead to
later. And also avoid the confusion of having two different way of
looking at it.

Here's the correct way....
From 2.4.1 Docs, section 2.3.6, Sequence Types:
-----------------------------------------------
s[i] i'th item of s, origin 0 (3)
s[i:j] slice of s from i to j (3), (4)
s[i:j:k] slice of s from i to j with step k (3), (5)

(3)
If i or j is negative, the index is relative to the end of the string:
len(s) + i or len(s) + j is substituted. But note that -0 is still 0.

(4)
The slice of s from i to j is defined as the sequence of items with
index k such that i <= k < j. If i or j is greater than len(s), use
len(s). If i is omitted, use 0. If j is omitted, use len(s). If i is
greater than or equal to j, the slice is empty.

(5)
The slice of s from i to j with step k is defined as the sequence of
items with index x = i + n*k such that . In other words, the indices are
i, i+k, i+2*k, i+3*k and so on, stopping when j is reached (but never
including j). If i or j is greater than len(s), use len(s). If i or j
are omitted then they become ``end'' values (which end depends on the
sign of k). Note, k cannot be zero.
-------------------------------------
This could be a bit clearer.

Notice this doesn't say to use index's between items. Looking at the
index's as being either on the left or right creates the 'apparent'
indexing inconsistency I was referring to. It looks good at first, but
breaks down later, so the examples using in-between index's in the docs
and tutorials should probably be rewritten to show how it actually works
right from the start. Something like the following examples with a walk
through would do.

['a', 'b', 'c', 'd', 'e', 'f', 'g']
0 1 2 3 4 5 6 <--- i
-7 -6 -5 -4 -3 -2 -1 <--- len(L)+i

example: i = -7 -> len(L) + -7 -> 7 + -7 -> 0

L[3] = 3
L[:3] = ['a','b','c'] # L[0:3]
L[3:] = ['d','e','f','g'] # L[3:7]

L[-3] -> 'e'
L[:-3] -> ['a','b','c','d'] # L[-7:-3]
L[-3:] -> ['e','f','g'] # L[-3:len(L)] (use len(L), not 0)

L[-3::-1] -> ['e','d','c','b' ,'a'] # L[-3:-8:-1]
L[:-3:-1] -> ['g','f'] # L[-1:-3:-1]

L[3::-1] -> ['d','c','b','a'] # L[3:-Len(L)-1:-1](-1 wont' work here)
L[:3:-1] -> ['g','f','e'] # L[0:3:-1]

L[3:4:1] i-> ['d']
L[3:2:-1] -> ['d']

L[-4:-3: 1] -> ['e']
L[-4:-5:-1] -> ['e']

(*) The problems stituations are the cases where start and stop are on
each side of the -1/0 boundary. This causes difficulty reaching end
points in some situations and also requires having to do extra bounds
checking if you move the stop index around.

So maybe we can improve things in some situations when that occurs?
Or at least try. So here's another (last by me) attempt. The following
will break current code, but it is a stable and predictable method.

It has both advantages and disadvantages over the current method. (as
usual)

+ Bound checking the stop index isn't needed to avoid situations
where it crosses -1/0 . (works in both directions)

+ The order is always preserved from the starting position. No ugly
switches if the stop in incrimented past -1/0.

+ Only need to check start position if you are moving it.

- can't index from both ends using both positive and negative
indexing at the same time.
With the below function, L[1:-1] does not remove an item off each end,
but instead returns [] as the stop is less than the start. So you have
to do it in two slices.

r = L[1:][:-1]

Or use all positive indexing.

r = L[1:len(L)-1]

Or all negative indexing.

r = r[-len(L):-1]

This is because in order to resolve the -1/0 indexing situations you
have to disallow indexing from both sides at the same time. I would
guess that a lot of people wouldn't want to give that up now that
they've gotten used to it. (? shrug)
Anyway, this might give someone else an idea or two. ;-)

Cheers,
Ron


Given: L[i:j:k] as start,stop,step .

And the following diagrams:

|--------i------0-------->
j<=i [i......] j>0
<------+0-----i---------|
j<-1 [......i] j>=i
-1

This function could probably be cleaned up a bit. I think there may be
several possible short cuts that would speed this up and reduce the
number of comparisons.

def Slice(L, start=None, stop=None, step=1):
if step == None:
step = 1
if start != None \
and stop != None \
and (type(start) != int \
or type(stop) != int \
or type(step) != int):
raise TypeError("slic e indices must be integers")
if step == 0:
raise ValueError("sli ce step cannot be zero")
if step >= 1:
if start == None:
start = 0
if stop == None:
stop = len(L)
if start < -len(L):
start = -len(L)
if start > len(L) or stop <= start:
return []
if start < 0 and stop > 0:
start = start+len(L)
stop = stop+len(L)
if stop > len(L):
stop = len(L)
elif step <= -1:
if start == None:
start = -1
if stop == None:
stop = -len(L)-1
if start >= len(L):
start = len(L)-1
if stop >= start or start < -len(L):
return []
if start >= 0 and stop < 0 :
start = start - len(L)
stop = stop - len(L)
if stop < -len(L)-1:
stop = -len(L)-1
new = []
for i in range(start,sto p,step):
new.append(L[i])
return new
L = range(10)
data = [
(0,-0,1),
(None, None, None),
(None, None, 1),
(3, 6, 1),
(3, 100, 1),
(100,6,-1),
(3, -1, -1),
(-3, 0, 1), # this works
(5, -1, -1), # this too
(None, -100, -1),
(-100, 100, 1),
(6, 3, -1),
(-3, None, 1),
(-6, 5, 1),
(-50, -6, 1),
(-3, -100, -1),
(-3, -6, -1),
(-3, -100, 0),
(-3, -6, .5),
(-3, 34.5, 1),
(2.0, -6, -1),
]
import sys
for start,stop,step in data:
print 'L[%r:%r:%r]='%(start,stop, step),
try:
print Slice(L,start,s top,step)
except:
print sys.exc_type, ":", sys.exc_value
print

---------- Output ------------

L[0:0:1]= []
L[None:None:None]= [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
L[None:None:1]= [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
L[3:6:1]= [3, 4, 5]
L[3:100:1]= [3, 4, 5, 6, 7, 8, 9]
L[100:6:-1]= [9, 8, 7]
L[3:-1:-1]= [3, 2, 1, 0]
L[-3:0:1]= [7, 8, 9]
L[5:-1:-1]= [5, 4, 3, 2, 1, 0]
L[None:-100:-1]= [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
L[-100:100:1]= [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
L[6:3:-1]= [6, 5, 4]
L[-3:None:1]= [7, 8, 9]
L[-6:5:1]= [4, 5, 6, 7, 8, 9]
L[-50:-6:1]= [0, 1, 2, 3]
L[-3:-100:-1]= [7, 6, 5, 4, 3, 2, 1, 0]
L[-3:-6:-1]= [7, 6, 5]
L[-3:-100:0]= exceptions.Valu eError : slice step cannot be zero
L[-3:-6:0.5]= exceptions.Type Error : slice indices must be integers
L[-3:34.5:1]= exceptions.Type Error : slice indices must be integers
L[2.0:-6:-1]= exceptions.Type Error : slice indices must be integers

Sep 9 '05 #41

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

15
2495
by: Roberto A. F. De Almeida | last post by:
I found that when using negative indices, the slice object passed to __getitem__ depends on the number of slices. An example to clarify: class a: def __getitem__(self, index): return index >>> b = a() >>> print b Traceback (most recent call last):
4
2936
by: F. Da Costa | last post by:
Hi, I was wondering whether someone could enlighten me as to the reason why the slice does not work in IE when the arr is passed in properly. Checked the values in the srcArr and they are correct so no problems there. Gecko works as expected. Prior to entering the function I can slice the array being entered so I wouldn't expect an "Unexpected call to method or property access" (in IE 6). I guess its something silly but as of yet i'm...
108
6476
by: Bryan Olson | last post by:
The Python slice type has one method 'indices', and reportedly: This method takes a single integer argument /length/ and computes information about the extended slice that the slice object would describe if applied to a sequence of length items. It returns a tuple of three integers; respectively these are the /start/ and /stop/ indices and the /step/ or stride length of the slice. Missing or out-of-bounds indices are handled in a manner...
23
2345
by: Antoon Pardon | last post by:
Now slices are objects in python, I was wondering if slice notation will be usable outside subscribtion in the future. Will it ever be possible to write things like: a = 4:9 for key, value in tree.items('alfa.': 'beta.'): -- Antoon Pardon
2
7256
by: smichr | last post by:
It seems to me that the indices() method for slices is could be improved. Right now it gives back concrete indices for a range of length n. That is, it does not return any None values. Using an example from clpy about this the indices for a 'None, None, -2' slice for a range of length 10 are given as '9, -1, -2'. The problem is that these concrete values cannot be fed back into a slice so that a slice will extract the same elements that...
3
2993
by: Bas | last post by:
Hi, stupid question, but would it be possible to somehow merge xrange (which is supposed to replace range in py3k) and slice? Both have very similar start, stop and step arguments and both are lightweight objects to indicate a range. But you can't do a and 'for i in slice(10,20)'. The only difference is see is some behavior with infinities (e.g. object gives a slice(3,maxint) inside _getitem_ , but I think this should not be a large...
0
9710
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
10593
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10340
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
10085
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
9163
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
7626
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6858
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
2
3830
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
3000
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.