435,473 Members | 3,657 Online
Need help? Post your question and get tips & solutions from a community of 435,473 IT Pros & Developers. It's quick & easy.

# iterator clone

 P: n/a Whats is the way to clone "independent" iterator? I can't use tee(), because I don't know how many "independent" iterators I need. copy and deepcopy doesn't work... --pavel Jul 13 '08 #1
13 Replies

 P: n/a Yosifov Pavel wrote: Whats is the way to clone "independent" iterator? I can't use tee(), because I don't know how many "independent" iterators I need. copy and deepcopy doesn't work... There is no general way. For "short" sequences you can store the items in a list which is also the worst-case behaviour of tee(). What are you trying to do? Peter Jul 13 '08 #2

 P: n/a On 13 ÉÀÌ, 14:12, Peter Otten <__pete...@web.dewrote: Yosifov Pavel wrote: Whats is the way to clone "independent" iterator? I can't use tee(), because I don't know how many "independent" iterators I need. copy and deepcopy doesn't work... There is no general way. For "short" sequences you can store the items ina list which is also the worst-case behaviour of tee(). What are you trying to do? Peter I try to generate iterators (iterator of iterators). Peter, you are right! Thank you. For example, it's possible to use something like this: def cloneiter( it ): """return (clonable,clone)""" return tee(it) and usage: clonable,seq1 = cloneiter(seq) ...iter over seq1... then clone again: clonable,seq2 = cloneiter(clonable) ...iter over seq2... Or in class: class ReIter: def __init__( self, it ): self._it = it def __iter__( self ): self._it,ret = tee(self._it) return ret and usage: ri = ReIter(seq) ...iter over ri... ...again iter over ri... ...and again... But I think (I'm sure!) it's deficiency of Python iterators! They are not very good... --Pavel Jul 13 '08 #3

 P: n/a Yosifov Pavel wrote: On 13 Ð¸ÑŽÐ», 14:12, Peter Otten <__pete...@web.dewrote: >Yosifov Pavel wrote: Whats is the way to clone "independent" iterator? I can't use tee(), because I don't know how many "independent" iterators I need. copy and deepcopy doesn't work... There is no general way. For "short" sequences you can store the items ina list which is also the worst-case behaviour of tee().What are you trying to do?Peter I try to generate iterators (iterator of iterators). Peter, you are right! Thank you. For example, it's possible to use something like this: def cloneiter( it ): """return (clonable,clone)""" return tee(it) [snip] That is too abstract, sorry. What concrete problem are you trying to solve with your cloned iterators? There might be a way to rearrange your setup in a way that doesn't need them. But I think (I'm sure!) it's deficiency of Python iterators! They are not very good... Well, I think Python's iterators, especially the generators, are beautiful. More importantly, I think there is no general way to make iterators copyable, regardless of the programming language. The problem is that most of the useful ones depend on external state. Peter Jul 13 '08 #4

 P: n/a Well, I think Python's iterators, especially the generators, are beautiful. More importantly, I think there is no general way to make iterators copyable, regardless of the programming language. The problem is that most of the useful ones depend on external state. Peter Hmm, but tee() de facto do it (clone iterator) and ignore side-effects of iterator ("external" state). And tee() create independent **internal** state of iterator (current position). But **external** state - is headache of programmer. So, iterator/generator have to be method for copy itself (the tee() implementation) or be "re- startable". Why not? Concrete problem was to generate iterators (iterator of slices). It was solved with ReIter. --Best regards, --pavel Jul 14 '08 #5

 P: n/a On Sun, 13 Jul 2008 18:51:19 -0700, Yosifov Pavel wrote: >Well, I think Python's iterators, especially the generators, are beautiful.More importantly, I think there is no general way to make iteratorscopyable, regardless of the programming language. The problem is that mostof the useful ones depend on external state. Hmm, but tee() de facto do it (clone iterator) and ignore side-effects of iterator ("external" state). And tee() create independent **internal** state of iterator (current position). `tee()` doesn't copy the iterator or its internal state but just caches it's results, so you can iterate over them again. That makes only sense if you expect to use the two iterators in a way they don't get much out of sync. If your usage pattern is "consume iterator 1 fully, and then re-iterate with iterator 2" `tee()` has no advantage over building a list of all results of the original iterator and iterate over that twice. `tee()` would be building this list anyway. But **external** state - is headache of programmer. So, iterator/generator have to be method for copy itself (the tee() implementation) or be "re- startable". Why not? Because it's often not possible without generating a list with all results, and the advantage of a low memory footprint is lost. Ciao, Marc 'BlackJack' Rintsch Jul 14 '08 #6

 P: n/a `tee()` doesn't copy the iterator or its internal state but just caches it's results, so you can iterate over them again. That makes only sense if you expect to use the two iterators in a way they don't get much out of sync. If your usage pattern is "consume iterator 1 fully, and then re-iterate with iterator 2" `tee()` has no advantage over building a list of all results of the original iterator and iterate over that twice. `tee()` would be building this list anyway. It's interesting and a concrete answer. Thanks a lot. Because it's often not possible without generating a list with all results, and the advantage of a low memory footprint is lost. Ciao, Marc 'BlackJack' Rintsch Seems like "monada". But I think is possible to determine when there is a bounded external state (side-effects) or not, may be is needed some new class-protocol for it... or something else. Or another way: iterators may be re-iterable always, but if programmer need to point to the extra- (external) state, he has to raise some a special exception in __iter)) method... OK, it's only fantasies about language design :-) --pavel Jul 14 '08 #7

 P: n/a On 13 juil, 12:05, Yosifov Pavel

 P: n/a On 13 Jul., 08:53, Yosifov Pavel

 P: n/a On 14 ÉÀÌ, 23:36, "bruno.desthuilli...@gmail.com"

 P: n/a On 15 Jul., 08:16, Yosifov Pavel

 P: n/a Kay, can you show example of such generator? ReIter, for example, work with usual generators. But for "big" iterator, I think is no any good solutions. IMHO we can discern 2 types of iterators: re-startable (based on internal Python objects) and not re-startable (with an external state, side- effects)... Best regards, Pavel Jul 16 '08 #12

 P: n/a On Tue, 15 Jul 2008 19:54:30 -0700, Yosifov Pavel wrote: Kay, can you show example of such generator? ReIter, for example, work with usual generators. But for "big" iterator, I think is no any good solutions. IMHO we can discern 2 types of iterators: re-startable (based on internal Python objects) and not re-startable (with an external state, side- effects)... Has nothing to do with internal vs. external. Examples: ``itertools.count(1)``, ``itertools.cycle(iterable)``, or def fib(): a, b = 0, 1 while True: yield a a, b = b, a + b Ciao, Marc 'BlackJack' Rintsch Jul 16 '08 #13

 P: n/a On 16 ÉÀÌ, 11:32, Marc 'BlackJack' Rintsch

### This discussion thread is closed

Replies have been disabled for this discussion.