Question about exausted iterators

[Christophe]

Is there a good reason why when you try to take an element from an
already exausted iterator, it throws StopIteration instead of some other
exception ? I've lost quite some times already because I was using a lot
of iterators and I forgot that that specific function parameter was one.

Exemple :

def f(i): .... print list(i)
.... print list(i)
.... f(iter(range(2) )) [0, 1]
[]

This is using Python 2.4.2

[Terry Reedy]

"Christophe " <ch************ *@free.fr> wrote in message
news:44******** **************@ news.free.fr...

Is there a good reason why when you try to take an element from an
already exausted iterator, it throws StopIteration instead of some other
exception ?
Yes.
..
..
To distinguish the control message "I am done yielding values, as per the
code specification (so don't bother calling me again)." from error messages
that say "Something is wrong, I cannot yield values and give up." In other
words, to distinguish expected correct behavior from unexpected incorrect
behavior. This is essential for the normal and correct use of iterators.
I've lost quite some times already because I was using a lot
of iterators and I forgot that that specific function parameter was one.
I think you mean 'specific function argument' for a parameter which could
be any iterable.
Exemple :

Example

def f(i): ... print list(i)
... print list(i)
... f(iter(range(2) ))

[0, 1]
[]

As per specification.
I am guessing that you want the first list() call to terminate normally and
return a list, which requires exhausted i to raise StopIteration, while you
want the second list() to not terminate but raise an exception, which
requires exhausted i to raise something other than StopIteration. Tough.

One solution is call list(i) exactly once:

def f(i):
li = list(i)
print li
print li

Another is to document f as requiring that i be a non-iterator reiterable
iterable and only pass correct arguments.

A third is to add a line like
if iter(i) is i: raise TypeError("inpu t appears to be iterator")

This is not quite exact since it will improperly exclude self-iterator
reiterables (which, I believe, no builtin is) and improperly pass
non-reiterable non-iterator iterables (at least some file objects). But it
might work for all your cases.

Terry Jan Reedy

[George Sakkis]

Christophe wrote:

Is there a good reason why when you try to take an element from an
already exausted iterator, it throws StopIteration instead of some other
exception ? I've lost quite some times already because I was using a lot
of iterators and I forgot that that specific function parameter was one.

Exemple :

>>> def f(i): ... print list(i)
... print list(i)
... >>> f(iter(range(2) ))

[0, 1]
[]

Whether trying to iterate over an exhausted iterator should be treated
differently is appication dependent. In most cases, you don't really
care to distinguish between an iterator that yields no elements and an
iterator that did yield some elements before but it has been exhausted.
If you do care, you can roll your own iterator wrapper:
class ExhaustibleIter ator(object):
def __init__(self, iterable):
self._next = getattr(iterabl e, 'next', iter(iterable). next)
self._exhausted = False

def next(self):
if self._exhausted :
raise ExhaustedIterat orException()
try: return self._next()
except StopIteration:
self._exhausted = True
raise

def __iter__(self):
return self

class ExhaustedIterat orException(Exc eption):
pass
And then in your function:
def f(i):
i = ExhaustibleIter ator(i)
print list(i)
print list(i)
HTH,
George

[Christophe]

Terry Reedy a écrit :

"Christophe " <ch************ *@free.fr> wrote in message
news:44******** **************@ news.free.fr...

Is there a good reason why when you try to take an element from an
already exausted iterator, it throws StopIteration instead of some other
exception ?

Yes.
.
.
To distinguish the control message "I am done yielding values, as per the
code specification (so don't bother calling me again)." from error messages
that say "Something is wrong, I cannot yield values and give up." In other
words, to distinguish expected correct behavior from unexpected incorrect
behavior. This is essential for the normal and correct use of iterators.

You talk about expected behaviour and my expected behaviour is that an
iterator should not be usable once it has raised StopIteration once.

>def f(i):

... print list(i)
... print list(i)
...

>f(iter(ran ge(2)))

[0, 1]
[]

As per specification.

Specifications sometimes have "bugs" too.
I am guessing that you want the first list() call to terminate normally and
return a list, which requires exhausted i to raise StopIteration, while you
want the second list() to not terminate but raise an exception, which
requires exhausted i to raise something other than StopIteration. Tough.
Exactly. This would be a sane way to handle it.
One solution is call list(i) exactly once:

def f(i):
li = list(i)
print li
print li

Ok, call me stupid if you want but I know perfectly well the "solution"
to that problem ! Come on, I was showing example code of an horrible
gotcha on using iterators.

Instead of saying that all works as intended could you be a little
helpful and tell me why it was intended in such an obviously broken way
instead ?

[looping]

Christophe wrote:

Ok, call me stupid if you want but I know perfectly well the "solution"
to that problem ! Come on, I was showing example code of an horrible
gotcha on using iterators.

OK, your are stupid ;-)
Why asking questions when you don't want to listen answers ?



Instead of saying that all works as intended could you be a little
helpful and tell me why it was intended in such an obviously broken way
instead ?

Why an exausted iterator must return an Exception (other than
StopIteration of course) ?
Well an exausted iterator could be seen like an empty string or an
empty list (or tons of others things), so you expect the code
for car in "":
print car
to return an Exception because it's empty ???
It's your job to check the iterator when it need to be.

Regards.
Dom

[Christophe]

looping a écrit :

Christophe wrote:

Ok, call me stupid if you want but I know perfectly well the "solution"
to that problem ! Come on, I was showing example code of an horrible
gotcha on using iterators.
OK, your are stupid ;-)
Why asking questions when you don't want to listen answers ?

Because I'm still waiting for a valid answer to my question. The answer
"Because it has been coded like that" or is not a valid one.

Instead of saying that all works as intended could you be a little
helpful and tell me why it was intended in such an obviously broken way
instead ?

Why an exausted iterator must return an Exception (other than
StopIteration of course) ?

Because it's exausted. Because it has been for me a frequent cause of
bugs and because I have yet to see a valid use case for such behaviour.
Well an exausted iterator could be seen like an empty string or an
empty list (or tons of others things), so you expect the code
for car in "":
print car
to return an Exception because it's empty ???
Of course not.
It's your job to check the iterator when it need to be.

It's my job to avoid coding bugs, it's the language job to avoid placing
pitfalls everywhere I go.

I must confess I have a strong opinion on that point. Not long ago I
started working on some fresh code where I decided to use a lot of
iterators and set instead of list if possible. That behaviour has caused
me to lose quite some time tracking bugs.

[Fredrik Lundh]

Christophe wrote:

Because I'm still waiting for a valid answer to my question. The answer
"Because it has been coded like that" or is not a valid one.

it's been coded like that because that's what the specification says:

http://www.python.org/dev/peps/pep-0234/

</F>

[Christophe]

Fredrik Lundh a écrit :

Christophe wrote:

Because I'm still waiting for a valid answer to my question. The
answer "Because it has been coded like that" or is not a valid one.

it's been coded like that because that's what the specification says:

http://www.python.org/dev/peps/pep-0234/

I didn't though I had to mention that "Because the spec has been writen
like that" wasn't a valid answer either.

[Fredrik Lundh]

Christophe wrote:

Because I'm still waiting for a valid answer to my question. The
answer "Because it has been coded like that" or is not a valid one.

it's been coded like that because that's what the specification says:

http://www.python.org/dev/peps/pep-0234/

I didn't though I had to mention that "Because the spec has been writen
like that" wasn't a valid answer either.

so what is a valid answer?

</F>

[Diez B. Roggisch]

Christophe wrote:

Fredrik Lundh a écrit :

Christophe wrote:

Because I'm still waiting for a valid answer to my question. The
answer "Because it has been coded like that" or is not a valid one.

it's been coded like that because that's what the specification says:

http://www.python.org/dev/peps/pep-0234/

I didn't though I had to mention that "Because the spec has been writen
like that" wasn't a valid answer either.

The important thing is: it _is_ specified. And what about code like this:
iterable = produce_some_it erable()

for item in iterable:
if some_condition( item)
break
do_something()

for item in iterable:
do_something_wi th_the_rest()
If it weren't for StopIteration raised if the iterable was exhausted, you'd
have to clutter that code with something like

try:
for item in iterable:
do_something_wi th_the_rest()
except IteratorExhaust ed:
pass

What makes you say that this is better than the above? Just because _you_
had some cornercases that others seems not to have (at least that
frequently, I personally can't remember I've ever bitten by it) isn't a
valid reason to _not_ do it as python does.

Besides that: it would be a major change of semantics of iterators that I
seriously doubt it would make it into anything before P3K. So - somewhat a
moot point to discuss here I'd say.

Diez