By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
449,264 Members | 1,769 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 449,264 IT Pros & Developers. It's quick & easy.

Sequence iterators with __index__

P: n/a
I think it would be useful if iterators on sequences had the __index__
method so that they could be used to slice sequences. I was writing a
class and wanted to return a list iterator to callers. I then wanted
to let callers slice from an iterator's position, but that isn't
supported without creating a custom iterator class.

Are there reasons for not supporting this generally? I realize not all
iterators would have the __index__ method, but that seems ok.

In Python 3, maybe this could be called a SequenceIterator

-Brad
Jun 27 '08 #1
Share this Question
Share on Google+
7 Replies


P: n/a
On Jun 24, 3:29*pm, schickb <schi...@gmail.comwrote:
I think it would be useful if iterators on sequences had the __index__
method so that they could be used to slice sequences. I was writing a
class and wanted to return a list iterator to callers. *I then wanted
to let callers slice from an iterator's position, but that isn't
supported without creating a custom iterator class.

Are there reasons for not supporting this generally? I realize not all
iterators would have the __index__ method, but that seems ok.

In Python 3, maybe this could be called a SequenceIterator

-Brad
Could you post an example of what you are talking about? I'm not
getting it. In any case, the first step is writing a PEP.
http://www.python.org/dev/peps/

Matt
Jun 27 '08 #2

P: n/a
On Jun 24, 3:45*pm, Matimus <mccre...@gmail.comwrote:
>
I think it would be useful if iterators on sequences had the __index__
method so that they could be used to slice sequences. I was writing a
class and wanted to return a list iterator to callers. *I then wanted
to let callers slice from an iterator's position, but that isn't
supported without creating a custom iterator class.

Could you post an example of what you are talking about? I'm not
getting it.
Interactive mock-up:
>>a = ['x','y','z']
it = iter(a)
a[it:]
['x', 'y', 'z']
>>it.next()
'x'
>>a[it:]
['y', 'z']
>>a[:it]
['x']
>>it.next()
'y'
>>a[it:]
['z']

This lets you use sequence iterators more general position indicators.
Currently if you want to track a position and slice from a tracked
position you must do it manually with an integer index. It's not
difficult, but given that sequence iterators already do that already
it seems redundant (and of course more error prone).

In any case, the first step is writing a PEP.http://www.python.org/dev/peps/
Ok thanks, but I do want some idea of interest level before spending a
bunch of time on this.

-Brad
Jun 27 '08 #3

P: n/a


schickb wrote:
On Jun 24, 3:45 pm, Matimus <mccre...@gmail.comwrote:
>>I think it would be useful if iterators on sequences had the __index__
method so that they could be used to slice sequences. I was writing a
class and wanted to return a list iterator to callers. I then wanted
to let callers slice from an iterator's position, but that isn't
supported without creating a custom iterator class.
Creating custom classes is what the class statement is for. See below.
>Could you post an example of what you are talking about? I'm not
getting it.

Interactive mock-up:
>>>a = ['x','y','z']
it = iter(a)
a[it:]
['x', 'y', 'z']
>>>it.next()
'x'
>>>a[it:]
['y', 'z']
>>>a[:it]
['x']
>>>it.next()
'y'
>>>a[it:]
['z']

This lets you use sequence iterators more general position indicators.
Currently if you want to track a position and slice from a tracked
position you must do it manually with an integer index. It's not
difficult, but given that sequence iterators already do that already
it seems redundant (and of course more error prone).
Python's iterator protocol is intentionally simple and general.
Wanting to slice while iterating is a *very* specialized usage.
In any case:
A. If the iterator uses in incrementing index to iterate, you want access.
B. Using an iterator as an integer will strike most people as
conceptually bizarre; it will never be accepted.
C. Doing so is unnecessary since the internal index can just as easily
be exposed as an integer attribute called 'index' or, more generally,
'count'.
a[it.count:] looks *much* better.
D. You can easily add .index or .count to any iterator you write. The
iterator protocol is a minimum rather than maximum specification.
E. You can easily wrap any iterable/iterator in an iterator class that
provides .count for *any* iteration process. Slicing is not the only
possible use of such an attribute.

class indexerator():
def __inti__(self, iterable):
self.it = iter(iterable)
self.count = 0
def __iter__(self): return self
def __next__(self): # 3.0
tem = self.it.next()
self.count += 1
return tem

..count is always the number of items returned. It is also the index of
the next item to be returned if (but only if() the base iterable exists
and is an indexable sequence.
F. Even this should be unnecessary for most usages. Built-in function
enumerate(iterable) generates count,item pairs in much the same manner:
>In any case, the first step is writing a PEP.http://www.python.org/dev/peps/
Discussion usually comes first.
Ok thanks, but I do want some idea of interest level before spending a
bunch of time on this.
Even the developers sometimes take this precaution. Many ideas never
make it to the PEP stage. I see no need for a change in Python here.

Terry Jan Reedy

Jun 27 '08 #4

P: n/a
On Jun 24, 5:46 pm, Terry Reedy <tjre...@udel.eduwrote:
>
Wanting to slice while iterating is a *very* specialized usage.
I disagree because iterators mark positions, which for sequences are
just offsets. And slicing is all about offsets. Here is a quote from
the already implemented PEP 357:

"Currently integers and long integers play a special role in slicing
in that they are the only objects allowed in slice syntax. In other
words, if X is an object implementing the sequence protocol, then
X[obj1:obj2] is only valid if obj1 and obj2 are both integers or long
integers. There is no way for obj1 and obj2 to tell Python that they
could be reasonably used as indexes into a sequence. This is an
unnecessary limitation."

But this isn't just about slicing. I'd like sequence iterators to be
usable as simple indexes as well; like a[it] (which __index__ would
also provide).
In any case:
A. If the iterator uses in incrementing index to iterate, you want access.
B. Using an iterator as an integer will strike most people as
conceptually bizarre; it will never be accepted.
It's not meant to be used as an integer. It's meant to be used as a
position in the sequence, which iterators already are. The fact that
the position is represented as an integer is not that important
(except to python). I'll grant you that it is conceptually strange
that you could use an iterator on one sequence as an index into
another.
C. Doing so is unnecessary since the internal index can just as easily
be exposed as an integer attribute called 'index' or, more generally,
'count'.
a[it.count:] looks *much* better.
D. You can easily add .index or .count to any iterator you write. The
iterator protocol is a minimum rather than maximum specification.
Following that line of reasoning, the __index__ special method
shouldn't really exist at all. Your arguments would suggest that NumPy
shouldn't use __index__ either because:
a[ushort.index] "looks *much* better".
E. You can easily wrap any iterable/iterator in an iterator class that
provides .count for *any* iteration process.
Sure, and that is why I mentioned this in my original post. But the
idea is to avoid redundant code and data in the case of sequences, and
make it a standard feature.
F. Even this should be unnecessary for most usages. Built-in function
enumerate(iterable) generates count,item pairs in much the same manner:
I am not aware of a way to get the current position out of an
enumerate object without advancing it (or creating a custom wrapper).
If the special __index__ method was added it might be interesting ;)
But iterators are already a clean and abstract position marker, and
for sequences it seems surprising to me that they can't really be used
as such.

-Brad
Jun 27 '08 #5

P: n/a
On Jun 25, 12:11*am, schickb <schi...@gmail.comwrote:
>
But this isn't just about slicing. I'd like sequence iterators to be
usable as simple indexes as well; like a[it] (which __index__ would
also provide).
It occurred to me that this wouldn't need to be limited to sequence
iterators. Although somewhat of a misnomer, __index__ could just a
well return the current key for mapping iterators. Type checking would
then be specific to the context, rather than hard-coded to integer.
Perhaps __position__ would have been a better name.

The generalized idea here is that iterators identify positions in
collections, so why shouldn't they be usable where collections accept
such identifiers?
>>m = {'a':1, 'b':2}
it = iter(m)
m[it]
1
>>it.next()
m[it]
2

These are trivial examples, but there are lots of uses for abstract
position indicators. From what I've seen, iterators are currently used
almost exclusively as temporary objects in loops. But perhaps if they
had a bit more functionality they could serve a wider purpose.

-Brad
Jun 27 '08 #6

P: n/a
On Jun 24, 4:19*pm, schickb <schi...@gmail.comwrote:
On Jun 24, 3:45*pm, Matimus <mccre...@gmail.comwrote:
I think it would be useful if iterators on sequences had the __index__
method so that they could be used to slice sequences. I was writing a
class and wanted to return a list iterator to callers. *I then wanted
to let callers slice from an iterator's position, but that isn't
supported without creating a custom iterator class.
Could you post an example of what you are talking about? I'm not
getting it.

Interactive mock-up:
>a = ['x','y','z']
it = iter(a)
a[it:]
['x', 'y', 'z']
>it.next()
'x'
>a[it:]
['y', 'z']
>a[:it]
['x']
>it.next()
'y'
>a[it:]

['z']

This lets you use sequence iterators more general position indicators.
Currently if you want to track a position and slice from a tracked
position you must do it manually with an integer index. It's not
difficult, but given that sequence iterators already do that already
it seems redundant (and of course more error prone).
In any case, the first step is writing a PEP.http://www.python.org/dev/peps/

Ok thanks, but I do want some idea of interest level before spending a
bunch of time on this.

-Brad
I have no problem with being able to query the position (state) of an
iterator without changing its state. I think using the iterator itself
as the index or part of a slice in the original sequence is non-
obvious and also less useful than just a new method to query state.
"Explicit is better than Implicit". I would rather see just
`it.index()` or `it.state()` than the new specialized behavior implied
by `it.__index__()`. I'm leaning towards `state` because sequences
already have an `index` method, and two methods of the same name with
different behaviors may be confusing. This gives you essentially the
same ability, and the code seems a little more obvious IMHO.
>>a = ['x','y','z']
it = iter(a)
a[it.state():]
['x', 'y', 'z']
>>it.next()
'x'
>>a[it.state():]
['y', 'z']
>>a[:it.state()]
['x']
>>it.next()
'y'
>>a[it.state():]
['z']
>>it.state()
2
Matt
Jun 27 '08 #7

P: n/a
On Jun 24, 4:19 pm, schickb <schi...@gmail.comwrote:
On Jun 24, 3:45 pm, Matimus <mccre...@gmail.comwrote:
I think it would be useful if iterators on sequences had the __index__
method so that they could be used to slice sequences. I was writing a
class and wanted to return a list iterator to callers. I then wanted
to let callers slice from an iterator's position, but that isn't
supported without creating a custom iterator class.
Could you post an example of what you are talking about? I'm not
getting it.

Interactive mock-up:
>a = ['x','y','z']
it = iter(a)
a[it:]
['x', 'y', 'z']
>it.next()
'x'
>a[it:]
['y', 'z']
>a[:it]
['x']
>it.next()
'y'
>a[it:]

['z']

This lets you use sequence iterators more general position indicators.
Currently if you want to track a position and slice from a tracked
position you must do it manually with an integer index. It's not
difficult, but given that sequence iterators already do that already
it seems redundant (and of course more error prone).
In any case, the first step is writing a PEP.http://www.python.org/dev/peps/

Ok thanks, but I do want some idea of interest level before spending a
bunch of time on this.

-Brad

Brad,

enumerate() seems to solve this problem for me....
>>
a = ['x','y','z']
ea = enumerate(a)
index, value = ea.next()
index
0
>value
'x'
>index, value = ea.next()
a[index:]
['y', 'z']
>>
putting this bit of code in a thin class wrapper should be useful to
keep the two data objects in sync. adding a reset() method allows you
to rebuild the enumerate object as often as needed.

Hope this helps.

--Alan
Jun 27 '08 #8

This discussion thread is closed

Replies have been disabled for this discussion.