424,830 Members | 1,688 Online
Need help? Post your question and get tips & solutions from a community of 424,830 IT Pros & Developers. It's quick & easy.

# slice notation as values?

 P: n/a Now slices are objects in python, I was wondering if slice notation will be usable outside subscribtion in the future. Will it ever be possible to write things like: a = 4:9 for key, value in tree.items('alfa.': 'beta.'): -- Antoon Pardon Dec 9 '05 #1
23 Replies

 P: n/a Antoon Pardon wrote: Now slices are objects in python, I was wondering if slice notation will be usable outside subscribtion in the future. Will it ever be possible to write things like: a = 4:9 for key, value in tree.items('alfa.': 'beta.'): Do you mean for key, value in tree.items()['alfa.': 'beta.']: What would this mean? regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ Dec 9 '05 #2

 P: n/a Op 2005-12-09, Steve Holden schreef : Antoon Pardon wrote: Now slices are objects in python, I was wondering if slice notation will be usable outside subscribtion in the future. Will it ever be possible to write things like: a = 4:9 for key, value in tree.items('alfa.': 'beta.'): Do you mean for key, value in tree.items()['alfa.': 'beta.']: No, the slice is meant to be a parameter to the method. It would be a more convenient way to write: for key, value in tree.items(slice('alfa.', 'beta.')) -- Antoon Pardon Dec 9 '05 #3

 P: n/a Antoon Pardon wrote: Will it ever be possible to write things like: a = 4:9 for key, value in tree.items('alfa.': 'beta.'): The first of these works fine, except you need to use the correct syntax: a = slice(4,9) range(10)[a] [4, 5, 6, 7, 8] The second also works fine, provide tree is a type which supports it and you rewrite the call as tree.items(slice('alfa','beta.')) or perhaps tree['alfa':'beta.'].items(). To support slicing directly on a dictionary you could do: class sliceable(dict): def __getitem__(self, item): if isinstance(item, slice): return self.__class__((k,v) for (k,v) in self.iteritems() if item.start <= k < item.stop) return dict.__getitem__(self, item) d = sliceable({'alpha': 1, 'aaa': 2, 'beta': 3, 'bee': 4 }) d['alpha':'beta'] {'alpha': 1, 'bee': 4} d['alpha':'beta.'] {'alpha': 1, 'beta': 3, 'bee': 4} for key, value in d['alpha':'beta.'].items(): print key, value alpha 1 beta 3 bee 4 It seems unlikely that this will make it into the builtin dict type, but you never know. Dec 9 '05 #4

 P: n/a Op 2005-12-09, Duncan Booth schreef : Antoon Pardon wrote: Will it ever be possible to write things like: a = 4:9 for key, value in tree.items('alfa.': 'beta.'): The first of these works fine, except you need to use the correct syntax: Sure, I know that. But why a different syntax? If we have lst = range(10), we can write lst[slice(3,7)] instead of lst[3:7] Now my impression is that should we only have the upper notation, slices would be less usefull, because it would make using them more cumbersome. I think that having this easy notation for slices available in more general circumstances, would make the use of them in other situations more easy too. a = slice(4,9) range(10)[a] [4, 5, 6, 7, 8] The second also works fine, provide tree is a type which supports it and you rewrite the call as tree.items(slice('alfa','beta.')) or perhaps tree['alfa':'beta.'].items(). To support slicing directly on a dictionary you could do: class sliceable(dict): def __getitem__(self, item): if isinstance(item, slice): return self.__class__((k,v) for (k,v) in self.iteritems() if item.start <= k < item.stop) return dict.__getitem__(self, item) d = sliceable({'alpha': 1, 'aaa': 2, 'beta': 3, 'bee': 4 }) d['alpha':'beta'] {'alpha': 1, 'bee': 4} d['alpha':'beta.'] {'alpha': 1, 'beta': 3, 'bee': 4} for key, value in d['alpha':'beta.'].items(): print key, value alpha 1 beta 3 bee 4 It seems unlikely that this will make it into the builtin dict type, but you never know. It doesn't need to be in the buildin dict type, I just would think it could make the interface to my own treedict type more intuitive. In my treedict the keys are accessible in order. If you have dictionary with strings as keys doing: for key in tree: will give you the keys in alfabetical order. Doing for key in tree['a':'b']: will give you all the keys that start with an 'a'. Now there are a number of methods with similar results. keys, values, items and there iter variants iterkeys, itervalues and iteritems. These methods are 'sliceable' too and I think the possibilty of giving such a method a slice as parameter, with the same notation as in a subscript, would be the clearest way for the user to provide slice information. If the user can write: for key in tree['a':'b']: Why shouldn't he be able to write: for key, value in tree.iteritems('a':'b'): -- Antoon Pardon Dec 9 '05 #5

 P: n/a Antoon Pardon asked: If we have lst = range(10), we can write lst[slice(3,7)] instead of lst[3:7] Now my impression is that should we only have the upper notation, slices would be less usefull, because it would make using them more cumbersome. Quite right, but the syntax for the slice only applies inside the square brackets and there would be big problems making it work outside a subscript. If you allowed a:b you get syntactic ambiguities: e.g. is this a slice or a complete if statement: if a:b If you allow a slice on its own but require the square brackets still to be there then you don't (I think) get any syntax ambiguities, but the result looks like it should be some weird list comprehension so its just too confusing: myslice = [a:b] I think the case for freestanding slices is there, but they aren't common enough to justify special syntax. If the user can write: for key in tree['a':'b']: Why shouldn't he be able to write: for key, value in tree.iteritems('a':'b'): import this .... There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. .... If the user can write for key in tree['a':'b']: then he can write: for key in tree['a':'b'].iteritems(): so why add a second way to do that? Dec 9 '05 #6

 P: n/a Op 2005-12-09, Duncan Booth schreef : Antoon Pardon asked: If we have lst = range(10), we can write lst[slice(3,7)] instead of lst[3:7] Now my impression is that should we only have the upper notation, slices would be less usefull, because it would make using them more cumbersome. Quite right, but the syntax for the slice only applies inside the square brackets and there would be big problems making it work outside a subscript. If you allowed a:b you get syntactic ambiguities: e.g. is this a slice or a complete if statement: if a:b If you allow a slice on its own but require the square brackets still to be there then you don't (I think) get any syntax ambiguities, but the result looks like it should be some weird list comprehension so its just too confusing: myslice = [a:b] I don't see the problem. The syntax for tuples can create syntactic ambiguities too. Take the following statement: f(a,b) Does the call have two parameters or one that is a tuple? In practice one uses parenthesis to disambigue the stament. So in the above case it would be a complete if statement. And myslice = a:b wouldn't cause a problem. I think the case for freestanding slices is there, but they aren't common enough to justify special syntax. If the user can write: for key in tree['a':'b']: Why shouldn't he be able to write: for key, value in tree.iteritems('a':'b'): import this ... There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. ... If the user can write for key in tree['a':'b']: then he can write: for key in tree['a':'b'].iteritems(): No he can't. tree['a':'b'] would provide a list of keys that all start with an 'a'. Such a list doesn't have an iteritems method. It wouldn't even contain the information to construct items. -- Antoon Pardon Dec 9 '05 #7

 P: n/a Antoon Pardon wrote: If the user can write for key in tree['a':'b']: then he can write: for key in tree['a':'b'].iteritems(): No he can't. tree['a':'b'] would provide a list of keys that all start with an 'a'. Such a list doesn't have an iteritems method. It wouldn't even contain the information to construct items. Why would it produce a list? Slicing a *list* produces a list, slicing a tuple produces a tuple, slicing a string produces a string. I would expect slicing any other type would also return you a new object of the same type. Thats what the code sample I posted earlier does. Dec 9 '05 #8

 P: n/a On 2005-12-09, Duncan Booth wrote: Antoon Pardon wrote: If the user can write for key in tree['a':'b']: then he can write: for key in tree['a':'b'].iteritems(): No he can't. tree['a':'b'] would provide a list of keys that all start with an 'a'. Such a list doesn't have an iteritems method. It wouldn't even contain the information to construct items. Why would it produce a list? Correction, it produces an iterator. Slicing a *list* produces a list, slicing a tuple produces a tuple, slicing a string produces a string. I would expect slicing any other type would also return you a new object of the same type. But iterating over a list, tuple or string, give you the elements of a list, tuple or string one by one. Iterating over a dict doesn't give you the elements one by one, but only the keys. In general I use slices over a tree because I only want to iterate over a specific subdomain of the keys. I'm not iterested in make a tree over the subdomain. Making such a subtree would be an enormous waste of resources. So even if we agree that tree['a':'b'] should create a subtree. Then I still would want a way to iterate over a subdomain of the keys in the tree without creating a subtree. Yes this would create more than one way to do the same thing. But this is already so with builtin dicts where a number of methods give you different ways to do things, apparantly because according to circumstances one is more efficient than the other. So lets agree that tree['a':'b'] would produce a subtree. Then I still would prefer the possibility to do something like: for key in tree.iterkeys('a':'b') Instead of having to write for key in tree['a':'b'].iterkeys() Sure I can now do it like this: for key in tree.iterkeys('a','b') But the way default arguments work, prevents you from having this work in an analague way as a slice. With slice notation you could have the following two cases: for key in tree.iterkeys('a':) for key in tree.iterkeys(:'b') But you can't do for key in tree.iterkeys('a',) or more regrettably, you can't do: for key in tree.iterkeys(,'b') I also think that other functions could benefit. For instance suppose you want to iterate over every second element in a list. Sure you can use an extended slice or use some kind of while. But why not extend enumerate to include an optional slice parameter, so you could do it as follows: for el in enumerate(lst,::2) If you have an iterable, you sometime want to iterate over only a part of it. The only way now to do so seems to be to either use a while loop or create a second iterable over your limited domain and iterate over the subiterable. I think it would be a an improvement if an iterator could be constructed to just go over the subdomain within your iterable. I also think that the most natural way to define that subdomain would be by using slice notation. -- Antoon Pardon Dec 10 '05 #9

 P: n/a Antoon Pardon wrote: So lets agree that tree['a':'b'] would produce a subtree. Then I still would prefer the possibility to do something like: for key in tree.iterkeys('a':'b') Instead of having to write for key in tree['a':'b'].iterkeys() Sure I can now do it like this: for key in tree.iterkeys('a','b') But the way default arguments work, prevents you from having this work in an analague way as a slice. How so? Can't you just pass the *args to the slice contstructor? E.g.:: def iterkeys(self, *args): keyslice = slice(*args) ... Then you can use the slice object just as you would have otherwise. STeVe Dec 10 '05 #10

 P: n/a Antoon Pardon wrote: In general I use slices over a tree because I only want to iterate over a specific subdomain of the keys. I'm not iterested in make a tree over the subdomain. Making such a subtree would be an enormous waste of resources. Probably not unless you have really large data structures. If you have something like a dbhash which would be inefficient to copy then both the original view of the data structure and the slices could share the same data. Creating a slice doesn't *have* to copy anything just so long as the semantics are clear. With slice notation you could have the following two cases: for key in tree.iterkeys('a':) for key in tree.iterkeys(:'b') x['a':] is short for x['a':None] x[:'b'] is short for x[None:'b'] But you can't do for key in tree.iterkeys('a',) or more regrettably, you can't do: for key in tree.iterkeys(,'b') If your datatype defines iterkeys to accept start and end arguments then you can do: for key in tree.iterkeys('a',None) for key in tree.iterkeys(None,'b') which is directly equivalent to the slices, or you can do: for key in tree.iterkeys(start='a') for key in tree.iterkeys(stop='b') which is more explicit. I also think that other functions could benefit. For instance suppose you want to iterate over every second element in a list. Sure you can use an extended slice or use some kind of while. But why not extend enumerate to include an optional slice parameter, so you could do it as follows: for el in enumerate(lst,::2) 'Why not'? Because it makes for a more complicated interface for something you can already do quite easily. Dec 10 '05 #11

 P: n/a On 2005-12-10, Steven Bethard wrote: Antoon Pardon wrote: So lets agree that tree['a':'b'] would produce a subtree. Then I still would prefer the possibility to do something like: for key in tree.iterkeys('a':'b') Instead of having to write for key in tree['a':'b'].iterkeys() Sure I can now do it like this: for key in tree.iterkeys('a','b') But the way default arguments work, prevents you from having this work in an analague way as a slice. How so? Can't you just pass the *args to the slice contstructor? E.g.:: def iterkeys(self, *args): keyslice = slice(*args) ... Then you can use the slice object just as you would have otherwise. This doesn't work for a number of reasons, 1) slice() Traceback (most recent call last): File "", line 1, in ? TypeError: slice expected at least 1 arguments, got 0 2) It doens't give a clear way to indicate the following kind of slice: tree.iterkeys('a':). Because of the follwing: slice('a') slice(None, 'a', None) which would be equivallent to tree.iterkeys(:'a') -- Antoon Pardon Dec 10 '05 #12

 P: n/a On 2005-12-10, Duncan Booth wrote: Antoon Pardon wrote: In general I use slices over a tree because I only want to iterate over a specific subdomain of the keys. I'm not iterested in make a tree over the subdomain. Making such a subtree would be an enormous waste of resources. Probably not unless you have really large data structures. If you have something like a dbhash which would be inefficient to copy then both the original view of the data structure and the slices could share the same data. Creating a slice doesn't *have* to copy anything just so long as the semantics are clear. With slice notation you could have the following two cases: for key in tree.iterkeys('a':) for key in tree.iterkeys(:'b') x['a':] is short for x['a':None] x[:'b'] is short for x[None:'b'] That is beside the point. The user doesn't have to know that in order to use slices. In point of fact I think that if tomorrow they changed the default to something different not a single program would break. But you can't do for key in tree.iterkeys('a',) or more regrettably, you can't do: for key in tree.iterkeys(,'b') If your datatype defines iterkeys to accept start and end arguments then you can do: for key in tree.iterkeys('a',None) for key in tree.iterkeys(None,'b') which is directly equivalent to the slices, or you can do: for key in tree.iterkeys(start='a') for key in tree.iterkeys(stop='b') which is more explicit. Yes we could do all that. The question remains why we should burden the user with all this extra information he has to know now in order to use this method, while there is a clear notation he can use without all this. The user doesn't has to type: lst[5:None] or lst[None:7], Neither has he to type something like lst[start=5] or lst[stop=7] There are circumstances when the user needs to provide slice information to a function. Why not allow him to use the same notation as he can use in subscribtion. What reason is there to limit a specific notation for a value to specific circumstances. To me this looks as arbitrary as would bracket notation not be allowed as an argument but that you would be obligated to use list((a,b,...)) in calls instead of [a,b,...] It wouldn't prevent you to do anything from what you can do now with python, it would only make a number of things unnecesary cumbersome. So yes, my proposal will not allow you to do anything you can;t already do now. It would just allow you to do a number of things in a less cumbersome way. I also think that other functions could benefit. For instance suppose you want to iterate over every second element in a list. Sure you can use an extended slice or use some kind of while. But why not extend enumerate to include an optional slice parameter, so you could do it as follows: for el in enumerate(lst,::2) 'Why not'? Because it makes for a more complicated interface for something you can already do quite easily. Do you think so? This IMO should provide (0,lst[0]), (2,lst[2]), (4,lst[4]) ... I haven't found a way to do this easily. Except for something like: start = 0: while start < len(lst): yield start, lst[start] start += 2 But if you accept this, then there was no need for enumerate in the first place. So eager to learn something new, how do you do this quite easily? -- Antoon Pardon Dec 10 '05 #13

 P: n/a Antoon Pardon wrote: On 2005-12-10, Duncan Booth wrote: [snip] I also think that other functions could benefit. For instance suppose you want to iterate over every second element in a list. Sure you can use an extended slice or use some kind of while. But why not extend enumerate to include an optional slice parameter, so you could do it as follows: for el in enumerate(lst,::2) 'Why not'? Because it makes for a more complicated interface for something you can already do quite easily. Do you think so? This IMO should provide (0,lst[0]), (2,lst[2]), (4,lst[4]) ... I haven't found a way to do this easily. Except for something like: start = 0: while start < len(lst): yield start, lst[start] start += 2 But if you accept this, then there was no need for enumerate in the first place. So eager to learn something new, how do you do this quite easily? lst = ['ham','eggs','bacon','spam','foo','bar','baz'] list(enumerate(lst))[::2] [(0, 'ham'), (2, 'bacon'), (4, 'foo'), (6, 'baz')] No changes to the language necessary. Dec 10 '05 #14

 P: n/a Antoon Pardon wrote: Will it ever be possible to write things like: a = 4:9 I made a silly recipe to do something like this a while ago, not that I'd recommend using it. But I also think it wouldn't be too far-fetched to allow slice creation using a syntax like the above... http://aspn.activestate.com/ASPN/Coo.../Recipe/415500 -- Brian Beck Adventurer of the First Order Dec 10 '05 #15

 P: n/a Antoon Pardon wrote: On 2005-12-10, Steven Bethard wrote:Antoon Pardon wrote:So lets agree that tree['a':'b'] would produce a subtree. ThenI still would prefer the possibility to do something like: for key in tree.iterkeys('a':'b')Instead of having to write for key in tree['a':'b'].iterkeys()Sure I can now do it like this: for key in tree.iterkeys('a','b')But the way default arguments work, prevents you from havingthis work in an analague way as a slice.How so? Can't you just pass the *args to the slice contstructor? E.g.:: def iterkeys(self, *args): keyslice = slice(*args) ...Then you can use the slice object just as you would have otherwise. This doesn't work for a number of reasons, 1)slice() Traceback (most recent call last): File "", line 1, in ? TypeError: slice expected at least 1 arguments, got 0 I wasn't sure whether or not the slice argument was optional. Apparently it's intended to be, so you have to make one special case: def iterkeys(self, *args): keyslice = args and slice(*args) or slice(None, None, None) 2) It doens't give a clear way to indicate the following kind of slice: tree.iterkeys('a':). Because of the follwing:slice('a') slice(None, 'a', None) which would be equivallent to tree.iterkeys(:'a') Well, it certainly gives a way to indicate it: tree.iterkeys(None, 'a') Whether or not it's a "clear" way is too subjective of a topic for me to get into. That's best left to Guido[1]. My point is that it *does* work, and covers (or can be slightly altered to cover) all the functionality you want. That doesn't mean you have to like the API for it, of course. STeVe [1] By which I mean that you should submit a PEP on the idea, and let Guido decide which way is prettier. Just be sure to give all the equivalent examples - i.e. calling the slice constructor with the appropriate arguments. Dec 10 '05 #16

 P: n/a On 10 Dec 2005 12:07:12 -0800, "Devan L" wrote: Antoon Pardon wrote: On 2005-12-10, Duncan Booth wrote:[snip] >> I also think that other functions could benefit. For instance suppose >> you want to iterate over every second element in a list. Sure you >> can use an extended slice or use some kind of while. But why not >> extend enumerate to include an optional slice parameter, so you could >> do it as follows: >> >> for el in enumerate(lst,::2) > > 'Why not'? Because it makes for a more complicated interface for something > you can already do quite easily. Do you think so? This IMO should provide (0,lst[0]), (2,lst[2]), (4,lst[4]) ... I haven't found a way to do this easily. Except for something like: start = 0: while start < len(lst): yield start, lst[start] start += 2 But if you accept this, then there was no need for enumerate in the first place. So eager to learn something new, how do you do this quite easily? lst = ['ham','eggs','bacon','spam','foo','bar','baz'] list(enumerate(lst))[::2][(0, 'ham'), (2, 'bacon'), (4, 'foo'), (6, 'baz')]No changes to the language necessary. Or, without creating the full list intermediately, lst = ['ham','eggs','bacon','spam','foo','bar','baz'] import itertools list(itertools.islice(enumerate(lst), 0, None, 2)) [(0, 'ham'), (2, 'bacon'), (4, 'foo'), (6, 'baz')] Regards, Bengt Richter Dec 11 '05 #17

 P: n/a Brian Beck wrote: Antoon Pardon wrote: Will it ever be possible to write things like: a = 4:9 I made a silly recipe to do something like this a while ago, not that I'd recommend using it. But I also think it wouldn't be too far-fetched to allow slice creation using a syntax like the above... http://aspn.activestate.com/ASPN/Coo.../Recipe/415500 Another possibility would be to make the slice type itself sliceable, then you could write things like: a = slice[4:9] a slice(4, 9, None) Sample implementation: class MetaSlice(object): def __getitem__(cls, item): return item def __init__(self, *args, **kw): return super(MetaSlice,self).__init__(self, *args, **kw) class Slice(slice): __metaclass__=MetaSlice Slice[2:3] slice(2, 3, None) Slice[:3] slice(None, 3, None) Slice[:3:-1] slice(None, 3, -1) Dec 11 '05 #18

 P: n/a Op 2005-12-10, Devan L schreef : Antoon Pardon wrote: On 2005-12-10, Duncan Booth wrote: [snip] >> I also think that other functions could benefit. For instance suppose >> you want to iterate over every second element in a list. Sure you >> can use an extended slice or use some kind of while. But why not >> extend enumerate to include an optional slice parameter, so you could >> do it as follows: >> >> for el in enumerate(lst,::2) > > 'Why not'? Because it makes for a more complicated interface for something > you can already do quite easily. Do you think so? This IMO should provide (0,lst[0]), (2,lst[2]), (4,lst[4]) ... I haven't found a way to do this easily. Except for something like: start = 0: while start < len(lst): yield start, lst[start] start += 2 But if you accept this, then there was no need for enumerate in the first place. So eager to learn something new, how do you do this quite easily? lst = ['ham','eggs','bacon','spam','foo','bar','baz'] list(enumerate(lst))[::2] [(0, 'ham'), (2, 'bacon'), (4, 'foo'), (6, 'baz')] It is not about what is needed, but about convenience. Now let me see, in order to just iterate over the even elements of a list with the index of the element, you turned an iterator into a list, which you use to create an other list which you will finaly iterate over. If this is the proposed answer, I wonder why iterators were introduced in the first place. I thought iterator were went to avoid the need to construct and copy list when all you want is iterate and when I ask how to get a specific iterator you come with a construct that makes rather heavily use of list constructions. -- Antoon Pardon Dec 12 '05 #19