Converting a flat list to a list of tuples

[metiu uitem]

Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

I was thinking of something along the lines of:
for (key,number) in list:
print key, number

but it's not working...

Thank you

[Duncan Booth]

metiu uitem wrote:

Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:

aList = ['a', 1, 'b', 2, 'c', 3]
it = iter(aList)
zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

[metiu uitem]

Thanks for the answer... yes the example was wrong!

[Steven D'Aprano]

On Tue, 22 Nov 2005 02:57:14 -0800, metiu uitem wrote:

Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

def split_and_combine(L):
newL = []
for i in range(len(L)//2):
newL.append( [L[2*i], L[2*i+1]] )
return newL

Another possibility is a list comprehension:

L = ['a', 1, 'b', 2, 'c', 3]
[[L[i], L[i+1]] for i in range(len(L)) if i%2 == 0]

Personally, I think that's just about as complex as a single list
comprehension should get. Otherwise it is too easy to create
unmaintainable code.

It is much easier (and probably faster) if you arrange matters so that you
have two lists at the start:

zip( ['a', 'b', 'c'], [1, 2, 3] )

returns [('a', 1), ('b', 2), ('c', 3)]

If you absolutely need the inner tuples to be lists, use a list
comprehension afterwards:
[list(t) for t in zip(['a', 'b', 'c'], [1, 2, 3])]
--
Steven.

[Fredrik Lundh]

"metiu uitem" wrote:

Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

simplest possible (works in all Python versions):

L = ['a', 1, 'b', 2, 'c', 3]

out = []
for i in range(0, len(L), 2):
out.append(L[i:i+2])

or, on one line (works in all modern versions):

out = [L[i:i+2] for i in range(0, len(L), 2)]

or, from the slightly-silly-department:

out = map(list, zip(L[0::2], L[1::2]))

or, using the standard grouping pydiom:

out = []; item = []
for i in L:
item.append(i)
if len(item) == 2:
out.append(item)
item = []
if item:
out.append(item)

etc.

</F>

[bonono]

Duncan Booth wrote:

metiu uitem wrote:

Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:

aList = ['a', 1, 'b', 2, 'c', 3]
it = iter(aList)
zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

brilliant.

[Laurent Rahuel]

metiu uitem wrote:

Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

I was thinking of something along the lines of:
for (key,number) in list:
print key, number

but it's not working...

Thank you

Hi,

newList = zip(aList[::2], aList[1::2])
newList
[('a', 1), ('b', 2), ('c', 3)]

Regards,

Laurent.

[Steven D'Aprano]

On Tue, 22 Nov 2005 11:11:23 +0000, Duncan Booth wrote:

aList = ['a', 1, 'b', 2, 'c', 3]
it = iter(aList)
zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

I'm not sure if I should fall to my knees in admiration of a Cool Hack,
or recoil in horror at a Bogus Kludge :-)

The code looks like it should return [('a', 'a'), (1, 1), ('b', 'b'), (2,
2), ('c', 'c'), (3, 3)] but of course it does not: the arguments for zip
are not independent.

I guess it is more of a Neat Trick, with a dash of Gotcha For The Unwary.

--
Steven.

[Fredrik Lundh]

Duncan Booth wrote:

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:

aList = ['a', 1, 'b', 2, 'c', 3]
it = iter(aList)
zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

yesterday, we got locals()["_[1]"]. and now this ?

is "relying on undefined behaviour" perhaps the new black ?

and people are impressed? it's like my old Z80 days, when
some folks thought it was truly amazing that call(11) printed
the raw contents of the entire memory to the screen...

</F>

[André Malo]

* Duncan Booth <du**********@invalid.invalid> wrote:

metiu uitem wrote:

Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:

aList = ['a', 1, 'b', 2, 'c', 3]
it = iter(aList)
zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

Though it looks nice, it's an implementation dependant solution. What if
someone changes zip to fetch the second item first?

nd

[bonono]

André Malo wrote:

* Duncan Booth <du**********@invalid.invalid> wrote:

metiu uitem wrote:

Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:

>> aList = ['a', 1, 'b', 2, 'c', 3]
>> it = iter(aList)
>> zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

Though it looks nice, it's an implementation dependant solution. What if
someone changes zip to fetch the second item first?

I believe someone should change the behaviour in the next release(is
that 2.4.3 or 2.5?), then it will give us the hard lesson :-)

Just saying it is no good is not going to stop people from doing it.

[David Isaac]

"Duncan Booth" <du**********@invalid.invalid> wrote in message
news:Xn*************************@127.0.0.1...

aList = ['a', 1, 'b', 2, 'c', 3]
it = iter(aList)
zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

That behavior is currently an accident.
http://sourceforge.net/tracker/?grou...il&aid=1121416
Alan Isaac

[George Sakkis]

"Laurent Rahuel" wrote:

Hi,

newList = zip(aList[::2], aList[1::2])
newList
[('a', 1), ('b', 2), ('c', 3)]

Regards,

Laurent

Or if aList can get very large and/or the conversion has to be
performed many times:

from itertools import islice
newList = zip(islice(aList,0,None,2), islice(aList,1,None,2))

George

[Bengt Richter]

On 22 Nov 2005 11:11:23 GMT, Duncan Booth <du**********@invalid.invalid> wrote:

metiu uitem wrote:

Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:

aList = ['a', 1, 'b', 2, 'c', 3]
it = iter(aList)
zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

Thank you for that. That is cool ;-)

Regards,
Bengt Richter

[Bengt Richter]

On Tue, 22 Nov 2005 13:26:45 +0100, "Fredrik Lundh" <fr*****@pythonware.com> wrote:

Duncan Booth wrote:

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:

>>> aList = ['a', 1, 'b', 2, 'c', 3]
>>> it = iter(aList)
>>> zip(it, it) [('a', 1), ('b', 2), ('c', 3)]

yesterday, we got locals()["_[1]"]. and now this ?

I don't really think those are comparable.
is "relying on undefined behaviour" perhaps the new black ? Is it really undefined? If so, IMO it should be defined to
do what it apparently does.
and people are impressed? it's like my old Z80 days, when
some folks thought it was truly amazing that call(11) printed
the raw contents of the entire memory to the screen...

You really don't think it was cool? Or could be well defined? ;-)

Hm, actually, something tells me I've seen some variation of this before,
but I can't think of the context off hand.

Regards,
Bengt Richter

[Bengt Richter]

On Tue, 22 Nov 2005 13:37:06 +0100, =?ISO-8859-1?Q?Andr=E9?= Malo <au********@g-kein-spam.com> wrote:

* Duncan Booth <du**********@invalid.invalid> wrote:

metiu uitem wrote:

> Say you have a flat list:
> ['a', 1, 'b', 2, 'c', 3]
>
> How do you efficiently get
> [['a', 1], ['b', 2], ['c', 3]]

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:

>>> aList = ['a', 1, 'b', 2, 'c', 3]
>>> it = iter(aList)
>>> zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

Though it looks nice, it's an implementation dependant solution. What if
someone changes zip to fetch the second item first?

That would be a counter-intuitive thing to do. Most things go left->right
in order as the default assumption.

Regards,
Bengt Richter

[Bengt Richter]

On Tue, 22 Nov 2005 14:28:56 GMT, "David Isaac" <ai*****@verizon.net> wrote:

"Duncan Booth" <du**********@invalid.invalid> wrote in message
news:Xn*************************@127.0.0.1...

>>> aList = ['a', 1, 'b', 2, 'c', 3]
>>> it = iter(aList)
>>> zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

That behavior is currently an accident.
http://sourceforge.net/tracker/?grou...il&aid=1121416
Alan Isaac

That says
"""
ii. The other problem is easier to explain by example.
Let it=iter([1,2,3,4]).
What is the result of zip(*[it]*2)?
The current answer is: [(1,2),(3,4)],
but it is impossible to determine this from the docs,
which would allow [(1,3),(2,4)] instead (or indeed
other possibilities).
"""
IMO left->right is useful enough to warrant making it defined behaviour,
not an accident. Isn't it(),it() well defined for a given iterator?
So is the question whether zip will access its referenced input iterators
in some peculiar order? Is the order of zip(a,b) accesses to a and b
undefined? If, so, IMO it's reasonable to make it defined as if

def zip(*args): ... return list(tuple([it.next() for it in its])
... for its in [[iter(a) for a in args]]
... for _ in iter(lambda:0,1))
... aList = ['a', 1, 'b', 2, 'c', 3]
it = iter(aList)
zip(it, it) [('a', 1), ('b', 2), ('c', 3)] it = iter(aList)
zip(it, it, it) [('a', 1, 'b'), (2, 'c', 3)] it = iter(aList)
zip(it) [('a',), (1,), ('b',), (2,), ('c',), (3,)] zip(range(3), range(4)) [(0, 0), (1, 1), (2, 2)] zip(range(4), range(3))

[ (0, 0), (1, 1), (2, 2)]

(I just hacked this out, so maybe it's not bullet-proof, but the point is,
I think there's no reason not to define the behaviour of zip to cycle
through its arguments in the intuitive way).

Regards,
Bengt Richter

[Dave Hansen]

On Tue, 22 Nov 2005 20:12:52 GMT in comp.lang.python, bo**@oz.net
(Bengt Richter) wrote:

On Tue, 22 Nov 2005 13:26:45 +0100, "Fredrik Lundh" <fr*****@pythonware.com> wrote:

Duncan Booth wrote:

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:

>>> aList = ['a', 1, 'b', 2, 'c', 3]
>>> it = iter(aList)
>>> zip(it, it)
[('a', 1), ('b', 2), ('c', 3)]

[...]
Hm, actually, something tells me I've seen some variation of this before,
but I can't think of the context off hand.

Earlier this fall I posted a question about iterating over a sequence
while modifying it. Google should bring it up.

This strikes me as the same idea, only inside-out...

Regards,
-=Dave

--
Change is inevitable, progress is not.

[Bengt Richter]

On 22 Nov 2005 07:42:31 -0800, "George Sakkis" <gs*****@rutgers.edu> wrote:

"Laurent Rahuel" wrote:

Hi,

newList = zip(aList[::2], aList[1::2])
newList
[('a', 1), ('b', 2), ('c', 3)]

Regards,

Laurent

Or if aList can get very large and/or the conversion has to be
performed many times:

from itertools import islice
newList = zip(islice(aList,0,None,2), islice(aList,1,None,2))

Or, if you want to include fractional groups at the end

aList = ['a', 1, 'b', 2, 'c', 3]
from itertools import groupby
def grouper(n): ... def git():
... while True:
... for _ in xrange(n): yield 0
... for _ in xrange(n): yield 1
... git = git()
... def grouper(_): return git.next()
... return grouper
... [tuple(g) for _, g in groupby(aList, grouper(2))] [('a', 1), ('b', 2), ('c', 3)] [tuple(g) for _, g in groupby(aList, grouper(3))] [('a', 1, 'b'), (2, 'c', 3)] [tuple(g) for _, g in groupby(aList, grouper(4))]

[('a', 1, 'b', 2), ('c', 3)]

Regards,
Bengt Richter

[bonono]

Bengt Richter wrote:

On Tue, 22 Nov 2005 13:37:06 +0100, =?ISO-8859-1?Q?Andr=E9?= Malo <au********@g-kein-spam.com> wrote:

* Duncan Booth <du**********@invalid.invalid> wrote:

metiu uitem wrote:

> Say you have a flat list:
> ['a', 1, 'b', 2, 'c', 3]
>
> How do you efficiently get
> [['a', 1], ['b', 2], ['c', 3]]

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:

>>> aList = ['a', 1, 'b', 2, 'c', 3]
>>> it = iter(aList)
>>> zip(it, it)
[('a', 1), ('b', 2), ('c', 3)]

Though it looks nice, it's an implementation dependant solution. What if
someone changes zip to fetch the second item first?

That would be a counter-intuitive thing to do. Most things go left->right
in order as the default assumption.

I have to admit that the poster was right as there is nothing stop the
implementor to do it this way and still gives you the defined result of
zip(). One scenario I can think of is that it can be implemented to run
in parallel, taking "n" results from "n" streams at the same time and
when all n arrived, pump the resultinging tuple out and repeat.

Why it needs to be done this way or is it desirable is another story.

And I doubt the current behaviour will go away, unless they really want
to break some codes to make the point.

[bonono]

Bengt Richter wrote:

On 22 Nov 2005 07:42:31 -0800, "George Sakkis" <gs*****@rutgers.edu> wrote:

"Laurent Rahuel" wrote:

Hi,

newList = zip(aList[::2], aList[1::2])
newList
[('a', 1), ('b', 2), ('c', 3)]

Regards,

Laurent

Or if aList can get very large and/or the conversion has to be
performed many times:

from itertools import islice
newList = zip(islice(aList,0,None,2), islice(aList,1,None,2))

Or, if you want to include fractional groups at the end

>>> aList = ['a', 1, 'b', 2, 'c', 3]
>>> from itertools import groupby
>>> def grouper(n): ... def git():
... while True:
... for _ in xrange(n): yield 0
... for _ in xrange(n): yield 1
... git = git()
... def grouper(_): return git.next()
... return grouper
... >>> [tuple(g) for _, g in groupby(aList, grouper(2))] [('a', 1), ('b', 2), ('c', 3)] >>> [tuple(g) for _, g in groupby(aList, grouper(3))] [('a', 1, 'b'), (2, 'c', 3)] >>> [tuple(g) for _, g in groupby(aList, grouper(4))]

[('a', 1, 'b', 2), ('c', 3)]

Personally, I would like to see it as [('a',1,'b',2), ('c',3,
None,None)], as a list of tuple of equal length is easier to be dealt
with.

i = iter(aList)
zip(i,chain(i,repeat(None)),
chain(i,repeat(None)),chain(i,repeat(None)))

[Bengt Richter]

On 22 Nov 2005 16:32:25 -0800, "bo****@gmail.com" <bo****@gmail.com> wrote:

Bengt Richter wrote:

On 22 Nov 2005 07:42:31 -0800, "George Sakkis" <gs*****@rutgers.edu> wrote:

>"Laurent Rahuel" wrote:
>
>> Hi,
>>
>> newList = zip(aList[::2], aList[1::2])
>> newList
>> [('a', 1), ('b', 2), ('c', 3)]
>>
>> Regards,
>>
>> Laurent
>
>Or if aList can get very large and/or the conversion has to be
>performed many times:
>
>from itertools import islice
>newList = zip(islice(aList,0,None,2), islice(aList,1,None,2))
>

Or, if you want to include fractional groups at the end

>>> aList = ['a', 1, 'b', 2, 'c', 3]
>>> from itertools import groupby
>>> def grouper(n):

... def git():
... while True:
... for _ in xrange(n): yield 0
... for _ in xrange(n): yield 1
... git = git()
... def grouper(_): return git.next()
... return grouper
...

>>> [tuple(g) for _, g in groupby(aList, grouper(2))]

[('a', 1), ('b', 2), ('c', 3)]

>>> [tuple(g) for _, g in groupby(aList, grouper(3))]

[('a', 1, 'b'), (2, 'c', 3)]

>>> [tuple(g) for _, g in groupby(aList, grouper(4))]

[('a', 1, 'b', 2), ('c', 3)]

Personally, I would like to see it as [('a',1,'b',2), ('c',3,
None,None)], as a list of tuple of equal length is easier to be dealt
with.

i = iter(aList)
zip(i,chain(i,repeat(None)),
chain(i,repeat(None)),chain(i,repeat(None)))

Yes, but OTOH you might like to loop and catch ValueError when/if the last tuple
doesn't unpack properly. Then you don't have to worry about None vs a sentinel :-)

Regards,
Bengt Richter

[bonono]

Bengt Richter wrote:

On 22 Nov 2005 16:32:25 -0800, "bo****@gmail.com" <bo****@gmail.com> wrote:

Bengt Richter wrote:

On 22 Nov 2005 07:42:31 -0800, "George Sakkis" <gs*****@rutgers.edu> wrote:

>"Laurent Rahuel" wrote:
>
>> Hi,
>>
>> newList = zip(aList[::2], aList[1::2])
>> newList
>> [('a', 1), ('b', 2), ('c', 3)]
>>
>> Regards,
>>
>> Laurent
>
>Or if aList can get very large and/or the conversion has to be
>performed many times:
>
>from itertools import islice
>newList = zip(islice(aList,0,None,2), islice(aList,1,None,2))
>
Or, if you want to include fractional groups at the end

>>> aList = ['a', 1, 'b', 2, 'c', 3]
>>> from itertools import groupby
>>> def grouper(n):
... def git():
... while True:
... for _ in xrange(n): yield 0
... for _ in xrange(n): yield 1
... git = git()
... def grouper(_): return git.next()
... return grouper
...
>>> [tuple(g) for _, g in groupby(aList, grouper(2))]
[('a', 1), ('b', 2), ('c', 3)]
>>> [tuple(g) for _, g in groupby(aList, grouper(3))]
[('a', 1, 'b'), (2, 'c', 3)]
>>> [tuple(g) for _, g in groupby(aList, grouper(4))]
[('a', 1, 'b', 2), ('c', 3)]

Personally, I would like to see it as [('a',1,'b',2), ('c',3,
None,None)], as a list of tuple of equal length is easier to be dealt
with.

i = iter(aList)
zip(i,chain(i,repeat(None)),
chain(i,repeat(None)),chain(i,repeat(None)))

Yes, but OTOH you might like to loop and catch ValueError when/if the last tuple
doesn't unpack properly. Then you don't have to worry about None vs a sentinel :-)

That to me is a matter of style. I in general prefer the no
catch/except way of coding, the filter/map way(or the unix pipe way).
Not that one is better than another.

But I think your solution for this case is a bit hard to swallow. I
would rather do it the more imperative way of :

def split(s,n):
a=[]
c=0
for x in s:
a.append(x)
c+=1
if c%n == 0:
yield a
a=[]
if a !=[]: yield a

list(split(aList,4))

[Duncan Booth]

Bengt Richter wrote:

On Tue, 22 Nov 2005 13:26:45 +0100, "Fredrik Lundh"
<fr*****@pythonware.com> wrote:

Duncan Booth wrote:

>>> it = iter(aList)
>>> zip(it, it)
[('a', 1), ('b', 2), ('c', 3)] <snip>

is "relying on undefined behaviour" perhaps the new black ?

Is it really undefined? If so, IMO it should be defined to
do what it apparently does.

<snip>
Hm, actually, something tells me I've seen some variation of this
before, but I can't think of the context off hand.

Yes, the subject does come up occasionally. Perhaps you are thinking
of this thread:

http://groups.google.co.uk/group/com...33c7333d3863ce

In that thread, I was the one arguing that the behaviour was undefined.
My memory was that I was forced to back down on that one, but looking back
at the thread I now think it was only itertools.izip I was forced to admit
defines its behaviour as working that way.

More googling will show that it was proposed that zip should
be defined as working with this, but that proposal was rejected. See:

http://groups.google.co.uk/group/com...a3d3b6d1a9fcbd

So scratch my original suggestion and substitute this for defined behaviour:

it = iter(aList)
list(itertools.izip(it, it))

[('a', 1), ('b', 2), ('c', 3)]

[Fredrik Lundh]

Bengt Richter wrote:

Though it looks nice, it's an implementation dependant solution. What if
someone changes zip to fetch the second item first?

That would be a counter-intuitive thing to do. Most things go left->right
in order as the default assumption.

it's not only the order that matters, but also the number of items
read from the source iterators on each iteration.

</F>

[Peter Otten]

bo****@gmail.com wrote:

Personally, I would like to see it as [('a',1,'b',2), ('c',3,
None,None)],**as*a*list*of*tuple*of*equal*length*is*easier*to *be*dealt
with.

i = iter(aList)
zip(i,chain(i,repeat(None)),
chain(i,repeat(None)),chain(i,repeat(None)))

Here's some more:

it = iter(range(5))
map(None, it, it) [(0, 1), (2, 3), (4, None)]
N = 3
it = chain(range(10), repeat("MISSING", N-1))
zip(*(it,)*N)

[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, 'MISSING', 'MISSING')]

Peter

[Bengt Richter]

On Wed, 23 Nov 2005 09:54:46 +0100, "Fredrik Lundh" <fr*****@pythonware.com> wrote:

Bengt Richter wrote:

>Though it looks nice, it's an implementation dependant solution. What if
>someone changes zip to fetch the second item first?

That would be a counter-intuitive thing to do. Most things go left->right
in order as the default assumption.

it's not only the order that matters, but also the number of items
read from the source iterators on each iteration.

Not sure I understand.

Are you thinking of something like lines from a file, where there might be
chunky buffering? ISTM that wouldn't matter if the same next method was called.
Here we have multiple references to the same iterator. Isn't e.g. buiding
a plain tuple defined with evaluation one element at a time left to right?
So an iterator it = xrange(4) can't know that it's being used in a context
like (it.next(), it.next()), so why should zip be any different? Zip _is_ building
tuples after all, and it's perfectly clear where they are coming from (or am
I missing something?) Why not left to right like a normal tuple?

Regards,
Bengt Richter

[bonono]

Bengt Richter wrote:

it's not only the order that matters, but also the number of items
read from the source iterators on each iteration.

Not sure I understand.

Are you thinking of something like lines from a file, where there might be
chunky buffering? ISTM that wouldn't matter if the same next method was called.
Here we have multiple references to the same iterator. Isn't e.g. buiding
a plain tuple defined with evaluation one element at a time left to right?
So an iterator it = xrange(4) can't know that it's being used in a context
like (it.next(), it.next()), so why should zip be any different? Zip _is_ building
tuples after all, and it's perfectly clear where they are coming from (or am
I missing something?) Why not left to right like a normal tuple?

The implementor of zip() may select to buffer the iterables so instead
of it.next(), it may loop it for a number of tmes, or emit multiple
threads making it async and all those kind of thing.

However, I would say this is highly unlikely or like a extremely remote
scenario to prove that this usage is wrong and we are bad boys ;-)

[Fredrik Lundh]

Bengt Richter wrote:

Are you thinking of something like lines from a file, where there might be
chunky buffering? ISTM that wouldn't matter if the same next method was called.
Here we have multiple references to the same iterator. Isn't e.g. buiding
a plain tuple defined with evaluation one element at a time left to right?

yeah, but what says that the iterator has to be called during tuple construction?

while 1:
for each sequence:
# optimize cache behaviour!
grab up to N items from each iterator
M = length of shortest output list
for i in range(M):
build tuple and append
if M != N:
break

</F>

[Simon Brunning]

On 22/11/05, Bengt Richter <bo**@oz.net> wrote:

That would be a counter-intuitive thing to do. Most things go left->right
in order as the default assumption.

+1

--
Cheers,
Simon B,
si***@brunningonline.net,
http://www.brunningonline.net/simon/blog/

[Peter Otten]

Fredrik Lundh wrote:

Bengt Richter wrote:

Are you thinking of something like lines from a file, where there might
be chunky buffering? ISTM that wouldn't matter if the same next method
was called. Here we have multiple references to the same iterator. Isn't
e.g. buiding a plain tuple defined with evaluation one element at a time
left to right?

yeah, but what says that the iterator has to be called during tuple
construction?

while 1:
for each sequence:
# optimize cache behaviour!
grab up to N items from each iterator
M = length of shortest output list
for i in range(M):
build tuple and append
if M != N:
break

Wouldn't every attempt to introduce such an optimization be shot down by the
likes of

def exponential():
for i in xrange(sys.maxint):
time.sleep(2**i)
yield "whatever"

def const():
for i in xrange(5): yield i

zip(exponential(), const())

To say it another way, aren't the problems that can be created by not
specifying zip() behaviour in a way that allows the zip(it, it) trick worse
than those you want to prevent?

Peter

[Bengt Richter]

On Wed, 23 Nov 2005 13:23:21 +0100, "Fredrik Lundh" <fr*****@pythonware.com> wrote:

Bengt Richter wrote:

Are you thinking of something like lines from a file, where there might be
chunky buffering? ISTM that wouldn't matter if the same next method was called.
Here we have multiple references to the same iterator. Isn't e.g. buiding
a plain tuple defined with evaluation one element at a time left to right?

yeah, but what says that the iterator has to be called during tuple construction?

while 1:
for each sequence:
# optimize cache behaviour!
grab up to N items from each iterator
M = length of shortest output list
for i in range(M):
build tuple and append
if M != N:
break

I'd say it's ok to cache some iterator output, but the cache buffer
must be associated with the iterator like cachedict[id(it)]. Then
the loop would only pump up the same cache buffer, since all iterators
would refer to the same buffer, and the loop that builds the tuple would
draw one from "each" buffer unknowingly drawing from the same buffer,
since they would be all found by cachedict[id(it)] which would be the same.
This would preserve the identity and logical sequentiality of the data
stream(s). To buffer separately on the unverified assumption that they
are different iterators seems like a buggy implementation to me ;-)
(optimizing use of total available buffering space is another matter,
not to mention handling StopIteration in the buffer loading)

ISTM to me the easiest just to define the intuitive behaviour, and let
implementers buffer accordingly if they want to.

Regards,
Bengt Richter