469,934 Members | 1,750 Online

# Converting a flat list to a list of tuples

Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

I was thinking of something along the lines of:
for (key,number) in list:
print key, number

but it's not working...

Thank you

Nov 22 '05 #1
31 2182
metiu uitem wrote:
Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:
aList = ['a', 1, 'b', 2, 'c', 3]
it = iter(aList)
zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

Nov 22 '05 #2
Thanks for the answer... yes the example was wrong!

Nov 22 '05 #3
On Tue, 22 Nov 2005 02:57:14 -0800, metiu uitem wrote:
Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

def split_and_combine(L):
newL = []
for i in range(len(L)//2):
newL.append( [L[2*i], L[2*i+1]] )
return newL

Another possibility is a list comprehension:

L = ['a', 1, 'b', 2, 'c', 3]
[[L[i], L[i+1]] for i in range(len(L)) if i%2 == 0]

Personally, I think that's just about as complex as a single list
comprehension should get. Otherwise it is too easy to create
unmaintainable code.

It is much easier (and probably faster) if you arrange matters so that you
have two lists at the start:

zip( ['a', 'b', 'c'], [1, 2, 3] )

returns [('a', 1), ('b', 2), ('c', 3)]

If you absolutely need the inner tuples to be lists, use a list
comprehension afterwards:
[list(t) for t in zip(['a', 'b', 'c'], [1, 2, 3])]
--
Steven.

Nov 22 '05 #4
"metiu uitem" wrote:
Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

simplest possible (works in all Python versions):

L = ['a', 1, 'b', 2, 'c', 3]

out = []
for i in range(0, len(L), 2):
out.append(L[i:i+2])

or, on one line (works in all modern versions):

out = [L[i:i+2] for i in range(0, len(L), 2)]

or, from the slightly-silly-department:

out = map(list, zip(L[0::2], L[1::2]))

or, using the standard grouping pydiom:

out = []; item = []
for i in L:
item.append(i)
if len(item) == 2:
out.append(item)
item = []
if item:
out.append(item)

etc.

</F>

Nov 22 '05 #5

Duncan Booth wrote:
metiu uitem wrote:
Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:
aList = ['a', 1, 'b', 2, 'c', 3]
it = iter(aList)
zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

brilliant.

Nov 22 '05 #6
metiu uitem wrote:
Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

I was thinking of something along the lines of:
for (key,number) in list:
print key, number

but it's not working...

Thank you

Hi,

newList = zip(aList[::2], aList[1::2])
newList
[('a', 1), ('b', 2), ('c', 3)]

Regards,

Laurent.
Nov 22 '05 #7
On Tue, 22 Nov 2005 11:11:23 +0000, Duncan Booth wrote:
aList = ['a', 1, 'b', 2, 'c', 3]
it = iter(aList)
zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

I'm not sure if I should fall to my knees in admiration of a Cool Hack,
or recoil in horror at a Bogus Kludge :-)

The code looks like it should return [('a', 'a'), (1, 1), ('b', 'b'), (2,
2), ('c', 'c'), (3, 3)] but of course it does not: the arguments for zip
are not independent.

I guess it is more of a Neat Trick, with a dash of Gotcha For The Unwary.

--
Steven.

Nov 22 '05 #8
Duncan Booth wrote:
That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:
aList = ['a', 1, 'b', 2, 'c', 3]
it = iter(aList)
zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

yesterday, we got locals()["_[1]"]. and now this ?

is "relying on undefined behaviour" perhaps the new black ?

and people are impressed? it's like my old Z80 days, when
some folks thought it was truly amazing that call(11) printed
the raw contents of the entire memory to the screen...

</F>

Nov 22 '05 #9
* Duncan Booth <du**********@invalid.invalid> wrote:
metiu uitem wrote:
Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:
aList = ['a', 1, 'b', 2, 'c', 3]
it = iter(aList)
zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

Though it looks nice, it's an implementation dependant solution. What if
someone changes zip to fetch the second item first?

nd
Nov 22 '05 #10

André Malo wrote:
* Duncan Booth <du**********@invalid.invalid> wrote:
metiu uitem wrote:
Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:
>> aList = ['a', 1, 'b', 2, 'c', 3]
>> it = iter(aList)
>> zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

Though it looks nice, it's an implementation dependant solution. What if
someone changes zip to fetch the second item first?

I believe someone should change the behaviour in the next release(is
that 2.4.3 or 2.5?), then it will give us the hard lesson :-)

Just saying it is no good is not going to stop people from doing it.

Nov 22 '05 #11

"Duncan Booth" <du**********@invalid.invalid> wrote in message
news:Xn*************************@127.0.0.1...
aList = ['a', 1, 'b', 2, 'c', 3]
it = iter(aList)
zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

That behavior is currently an accident.
http://sourceforge.net/tracker/?grou...il&aid=1121416
Alan Isaac
Nov 22 '05 #12
"Laurent Rahuel" wrote:
Hi,

newList = zip(aList[::2], aList[1::2])
newList
[('a', 1), ('b', 2), ('c', 3)]

Regards,

Laurent

Or if aList can get very large and/or the conversion has to be
performed many times:

from itertools import islice
newList = zip(islice(aList,0,None,2), islice(aList,1,None,2))

George

Nov 22 '05 #13
On 22 Nov 2005 11:11:23 GMT, Duncan Booth <du**********@invalid.invalid> wrote:
metiu uitem wrote:
Say you have a flat list:
['a', 1, 'b', 2, 'c', 3]

How do you efficiently get
[['a', 1], ['b', 2], ['c', 3]]

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:
aList = ['a', 1, 'b', 2, 'c', 3]
it = iter(aList)
zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

Thank you for that. That is cool ;-)

Regards,
Bengt Richter
Nov 22 '05 #14
On Tue, 22 Nov 2005 13:26:45 +0100, "Fredrik Lundh" <fr*****@pythonware.com> wrote:
Duncan Booth wrote:
That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:
>>> aList = ['a', 1, 'b', 2, 'c', 3]
>>> it = iter(aList)
>>> zip(it, it) [('a', 1), ('b', 2), ('c', 3)]

yesterday, we got locals()["_[1]"]. and now this ?

I don't really think those are comparable.
is "relying on undefined behaviour" perhaps the new black ? Is it really undefined? If so, IMO it should be defined to
do what it apparently does.
and people are impressed? it's like my old Z80 days, when
some folks thought it was truly amazing that call(11) printed
the raw contents of the entire memory to the screen...

You really don't think it was cool? Or could be well defined? ;-)

Hm, actually, something tells me I've seen some variation of this before,
but I can't think of the context off hand.

Regards,
Bengt Richter
Nov 22 '05 #15
On Tue, 22 Nov 2005 13:37:06 +0100, =?ISO-8859-1?Q?Andr=E9?= Malo <au********@g-kein-spam.com> wrote:
* Duncan Booth <du**********@invalid.invalid> wrote:
metiu uitem wrote:
> Say you have a flat list:
> ['a', 1, 'b', 2, 'c', 3]
>
> How do you efficiently get
> [['a', 1], ['b', 2], ['c', 3]]

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:
>>> aList = ['a', 1, 'b', 2, 'c', 3]
>>> it = iter(aList)
>>> zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

Though it looks nice, it's an implementation dependant solution. What if
someone changes zip to fetch the second item first?

That would be a counter-intuitive thing to do. Most things go left->right
in order as the default assumption.

Regards,
Bengt Richter
Nov 22 '05 #16
On Tue, 22 Nov 2005 14:28:56 GMT, "David Isaac" <ai*****@verizon.net> wrote:

"Duncan Booth" <du**********@invalid.invalid> wrote in message
news:Xn*************************@127.0.0.1...
>>> aList = ['a', 1, 'b', 2, 'c', 3]
>>> it = iter(aList)
>>> zip(it, it)

[('a', 1), ('b', 2), ('c', 3)]

That behavior is currently an accident.
http://sourceforge.net/tracker/?grou...il&aid=1121416
Alan Isaac

That says
"""
ii. The other problem is easier to explain by example.
Let it=iter([1,2,3,4]).
What is the result of zip(*[it]*2)?
but it is impossible to determine this from the docs,
which would allow [(1,3),(2,4)] instead (or indeed
other possibilities).
"""
IMO left->right is useful enough to warrant making it defined behaviour,
not an accident. Isn't it(),it() well defined for a given iterator?
So is the question whether zip will access its referenced input iterators
in some peculiar order? Is the order of zip(a,b) accesses to a and b
undefined? If, so, IMO it's reasonable to make it defined as if
def zip(*args): ... return list(tuple([it.next() for it in its])
... for its in [[iter(a) for a in args]]
... for _ in iter(lambda:0,1))
... aList = ['a', 1, 'b', 2, 'c', 3]
it = iter(aList)
zip(it, it) [('a', 1), ('b', 2), ('c', 3)] it = iter(aList)
zip(it, it, it) [('a', 1, 'b'), (2, 'c', 3)] it = iter(aList)
zip(it) [('a',), (1,), ('b',), (2,), ('c',), (3,)] zip(range(3), range(4)) [(0, 0), (1, 1), (2, 2)] zip(range(4), range(3))

[ (0, 0), (1, 1), (2, 2)]

(I just hacked this out, so maybe it's not bullet-proof, but the point is,
I think there's no reason not to define the behaviour of zip to cycle
through its arguments in the intuitive way).

Regards,
Bengt Richter
Nov 22 '05 #17
On Tue, 22 Nov 2005 20:12:52 GMT in comp.lang.python, bo**@oz.net
(Bengt Richter) wrote:
On Tue, 22 Nov 2005 13:26:45 +0100, "Fredrik Lundh" <fr*****@pythonware.com> wrote:
Duncan Booth wrote:
That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:

>>> aList = ['a', 1, 'b', 2, 'c', 3]
>>> it = iter(aList)
>>> zip(it, it)
[('a', 1), ('b', 2), ('c', 3)]

[...]
Hm, actually, something tells me I've seen some variation of this before,
but I can't think of the context off hand.

Earlier this fall I posted a question about iterating over a sequence
while modifying it. Google should bring it up.

This strikes me as the same idea, only inside-out...

Regards,
-=Dave

--
Change is inevitable, progress is not.
Nov 22 '05 #18
On 22 Nov 2005 07:42:31 -0800, "George Sakkis" <gs*****@rutgers.edu> wrote:
"Laurent Rahuel" wrote:
Hi,

newList = zip(aList[::2], aList[1::2])
newList
[('a', 1), ('b', 2), ('c', 3)]

Regards,

Laurent

Or if aList can get very large and/or the conversion has to be
performed many times:

from itertools import islice
newList = zip(islice(aList,0,None,2), islice(aList,1,None,2))

Or, if you want to include fractional groups at the end
aList = ['a', 1, 'b', 2, 'c', 3]
from itertools import groupby
def grouper(n): ... def git():
... while True:
... for _ in xrange(n): yield 0
... for _ in xrange(n): yield 1
... git = git()
... def grouper(_): return git.next()
... return grouper
... [tuple(g) for _, g in groupby(aList, grouper(2))] [('a', 1), ('b', 2), ('c', 3)] [tuple(g) for _, g in groupby(aList, grouper(3))] [('a', 1, 'b'), (2, 'c', 3)] [tuple(g) for _, g in groupby(aList, grouper(4))]

[('a', 1, 'b', 2), ('c', 3)]

Regards,
Bengt Richter
Nov 22 '05 #19

Bengt Richter wrote:
On Tue, 22 Nov 2005 13:37:06 +0100, =?ISO-8859-1?Q?Andr=E9?= Malo <au********@g-kein-spam.com> wrote:
* Duncan Booth <du**********@invalid.invalid> wrote:
metiu uitem wrote:

> Say you have a flat list:
> ['a', 1, 'b', 2, 'c', 3]
>
> How do you efficiently get
> [['a', 1], ['b', 2], ['c', 3]]

That's funny, I thought your subject line said 'list of tuples'. I'll
answer the question in the subject rather than the question in the body:

>>> aList = ['a', 1, 'b', 2, 'c', 3]
>>> it = iter(aList)
>>> zip(it, it)
[('a', 1), ('b', 2), ('c', 3)]

Though it looks nice, it's an implementation dependant solution. What if
someone changes zip to fetch the second item first?

That would be a counter-intuitive thing to do. Most things go left->right
in order as the default assumption.

I have to admit that the poster was right as there is nothing stop the
implementor to do it this way and still gives you the defined result of
zip(). One scenario I can think of is that it can be implemented to run
in parallel, taking "n" results from "n" streams at the same time and
when all n arrived, pump the resultinging tuple out and repeat.

Why it needs to be done this way or is it desirable is another story.

And I doubt the current behaviour will go away, unless they really want
to break some codes to make the point.

Nov 23 '05 #20

Bengt Richter wrote:
On 22 Nov 2005 07:42:31 -0800, "George Sakkis" <gs*****@rutgers.edu> wrote:
"Laurent Rahuel" wrote:
Hi,

newList = zip(aList[::2], aList[1::2])
newList
[('a', 1), ('b', 2), ('c', 3)]

Regards,

Laurent

Or if aList can get very large and/or the conversion has to be
performed many times:

from itertools import islice
newList = zip(islice(aList,0,None,2), islice(aList,1,None,2))

Or, if you want to include fractional groups at the end
>>> aList = ['a', 1, 'b', 2, 'c', 3]
>>> from itertools import groupby
>>> def grouper(n): ... def git():
... while True:
... for _ in xrange(n): yield 0
... for _ in xrange(n): yield 1
... git = git()
... def grouper(_): return git.next()
... return grouper
... >>> [tuple(g) for _, g in groupby(aList, grouper(2))] [('a', 1), ('b', 2), ('c', 3)] >>> [tuple(g) for _, g in groupby(aList, grouper(3))] [('a', 1, 'b'), (2, 'c', 3)] >>> [tuple(g) for _, g in groupby(aList, grouper(4))]

[('a', 1, 'b', 2), ('c', 3)]

Personally, I would like to see it as [('a',1,'b',2), ('c',3,
None,None)], as a list of tuple of equal length is easier to be dealt
with.

i = iter(aList)
zip(i,chain(i,repeat(None)),
chain(i,repeat(None)),chain(i,repeat(None)))

Nov 23 '05 #21
On 22 Nov 2005 16:32:25 -0800, "bo****@gmail.com" <bo****@gmail.com> wrote:

Bengt Richter wrote:
On 22 Nov 2005 07:42:31 -0800, "George Sakkis" <gs*****@rutgers.edu> wrote:
>"Laurent Rahuel" wrote:
>
>> Hi,
>>
>> newList = zip(aList[::2], aList[1::2])
>> newList
>> [('a', 1), ('b', 2), ('c', 3)]
>>
>> Regards,
>>
>> Laurent
>
>Or if aList can get very large and/or the conversion has to be
>performed many times:
>
>from itertools import islice
>newList = zip(islice(aList,0,None,2), islice(aList,1,None,2))
>

Or, if you want to include fractional groups at the end
>>> aList = ['a', 1, 'b', 2, 'c', 3]
>>> from itertools import groupby
>>> def grouper(n):

... def git():
... while True:
... for _ in xrange(n): yield 0
... for _ in xrange(n): yield 1
... git = git()
... def grouper(_): return git.next()
... return grouper
...
>>> [tuple(g) for _, g in groupby(aList, grouper(2))]

[('a', 1), ('b', 2), ('c', 3)]
>>> [tuple(g) for _, g in groupby(aList, grouper(3))]

[('a', 1, 'b'), (2, 'c', 3)]
>>> [tuple(g) for _, g in groupby(aList, grouper(4))]

[('a', 1, 'b', 2), ('c', 3)]

Personally, I would like to see it as [('a',1,'b',2), ('c',3,
None,None)], as a list of tuple of equal length is easier to be dealt
with.

i = iter(aList)
zip(i,chain(i,repeat(None)),
chain(i,repeat(None)),chain(i,repeat(None)))

Yes, but OTOH you might like to loop and catch ValueError when/if the last tuple
doesn't unpack properly. Then you don't have to worry about None vs a sentinel :-)

Regards,
Bengt Richter
Nov 23 '05 #22

Bengt Richter wrote:
On 22 Nov 2005 16:32:25 -0800, "bo****@gmail.com" <bo****@gmail.com> wrote:

Bengt Richter wrote:
On 22 Nov 2005 07:42:31 -0800, "George Sakkis" <gs*****@rutgers.edu> wrote:

>"Laurent Rahuel" wrote:
>
>> Hi,
>>
>> newList = zip(aList[::2], aList[1::2])
>> newList
>> [('a', 1), ('b', 2), ('c', 3)]
>>
>> Regards,
>>
>> Laurent
>
>Or if aList can get very large and/or the conversion has to be
>performed many times:
>
>from itertools import islice
>newList = zip(islice(aList,0,None,2), islice(aList,1,None,2))
>
Or, if you want to include fractional groups at the end

>>> aList = ['a', 1, 'b', 2, 'c', 3]
>>> from itertools import groupby
>>> def grouper(n):
... def git():
... while True:
... for _ in xrange(n): yield 0
... for _ in xrange(n): yield 1
... git = git()
... def grouper(_): return git.next()
... return grouper
...
>>> [tuple(g) for _, g in groupby(aList, grouper(2))]
[('a', 1), ('b', 2), ('c', 3)]
>>> [tuple(g) for _, g in groupby(aList, grouper(3))]
[('a', 1, 'b'), (2, 'c', 3)]
>>> [tuple(g) for _, g in groupby(aList, grouper(4))]
[('a', 1, 'b', 2), ('c', 3)]

Personally, I would like to see it as [('a',1,'b',2), ('c',3,
None,None)], as a list of tuple of equal length is easier to be dealt
with.

i = iter(aList)
zip(i,chain(i,repeat(None)),
chain(i,repeat(None)),chain(i,repeat(None)))

Yes, but OTOH you might like to loop and catch ValueError when/if the last tuple
doesn't unpack properly. Then you don't have to worry about None vs a sentinel :-)

That to me is a matter of style. I in general prefer the no
catch/except way of coding, the filter/map way(or the unix pipe way).
Not that one is better than another.

But I think your solution for this case is a bit hard to swallow. I
would rather do it the more imperative way of :

def split(s,n):
a=[]
c=0
for x in s:
a.append(x)
c+=1
if c%n == 0:
yield a
a=[]
if a !=[]: yield a

list(split(aList,4))

Nov 23 '05 #23
Bengt Richter wrote:
On Tue, 22 Nov 2005 13:26:45 +0100, "Fredrik Lundh"
<fr*****@pythonware.com> wrote:
Duncan Booth wrote:
>>> it = iter(aList)
>>> zip(it, it)
[('a', 1), ('b', 2), ('c', 3)] <snip>
is "relying on undefined behaviour" perhaps the new black ?

Is it really undefined? If so, IMO it should be defined to
do what it apparently does.

<snip>
Hm, actually, something tells me I've seen some variation of this
before, but I can't think of the context off hand.

Yes, the subject does come up occasionally. Perhaps you are thinking

In that thread, I was the one arguing that the behaviour was undefined.
My memory was that I was forced to back down on that one, but looking back
at the thread I now think it was only itertools.izip I was forced to admit
defines its behaviour as working that way.

More googling will show that it was proposed that zip should
be defined as working with this, but that proposal was rejected. See:

So scratch my original suggestion and substitute this for defined behaviour:
it = iter(aList)
list(itertools.izip(it, it))

[('a', 1), ('b', 2), ('c', 3)]

Nov 23 '05 #24
Bengt Richter wrote:
Though it looks nice, it's an implementation dependant solution. What if
someone changes zip to fetch the second item first?

That would be a counter-intuitive thing to do. Most things go left->right
in order as the default assumption.

it's not only the order that matters, but also the number of items
read from the source iterators on each iteration.

</F>

Nov 23 '05 #25
bo****@gmail.com wrote:
Personally, I would like to see it as [('a',1,'b',2), ('c',3,
None,None)],**as*a*list*of*tuple*of*equal*length*is*easier*to *be*dealt
with.

i = iter(aList)
zip(i,chain(i,repeat(None)),
chain(i,repeat(None)),chain(i,repeat(None)))

Here's some more:
it = iter(range(5))
map(None, it, it) [(0, 1), (2, 3), (4, None)]
N = 3
it = chain(range(10), repeat("MISSING", N-1))
zip(*(it,)*N)

[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, 'MISSING', 'MISSING')]

Peter

Nov 23 '05 #26
On Wed, 23 Nov 2005 09:54:46 +0100, "Fredrik Lundh" <fr*****@pythonware.com> wrote:
Bengt Richter wrote:
>Though it looks nice, it's an implementation dependant solution. What if
>someone changes zip to fetch the second item first?

That would be a counter-intuitive thing to do. Most things go left->right
in order as the default assumption.

it's not only the order that matters, but also the number of items
read from the source iterators on each iteration.

Not sure I understand.

Are you thinking of something like lines from a file, where there might be
chunky buffering? ISTM that wouldn't matter if the same next method was called.
Here we have multiple references to the same iterator. Isn't e.g. buiding
a plain tuple defined with evaluation one element at a time left to right?
So an iterator it = xrange(4) can't know that it's being used in a context
like (it.next(), it.next()), so why should zip be any different? Zip _is_ building
tuples after all, and it's perfectly clear where they are coming from (or am
I missing something?) Why not left to right like a normal tuple?

Regards,
Bengt Richter
Nov 23 '05 #27

Bengt Richter wrote:
it's not only the order that matters, but also the number of items
read from the source iterators on each iteration.

Not sure I understand.

Are you thinking of something like lines from a file, where there might be
chunky buffering? ISTM that wouldn't matter if the same next method was called.
Here we have multiple references to the same iterator. Isn't e.g. buiding
a plain tuple defined with evaluation one element at a time left to right?
So an iterator it = xrange(4) can't know that it's being used in a context
like (it.next(), it.next()), so why should zip be any different? Zip _is_ building
tuples after all, and it's perfectly clear where they are coming from (or am
I missing something?) Why not left to right like a normal tuple?

The implementor of zip() may select to buffer the iterables so instead
of it.next(), it may loop it for a number of tmes, or emit multiple
threads making it async and all those kind of thing.

However, I would say this is highly unlikely or like a extremely remote
scenario to prove that this usage is wrong and we are bad boys ;-)

Nov 23 '05 #28
Bengt Richter wrote:
Are you thinking of something like lines from a file, where there might be
chunky buffering? ISTM that wouldn't matter if the same next method was called.
Here we have multiple references to the same iterator. Isn't e.g. buiding
a plain tuple defined with evaluation one element at a time left to right?

yeah, but what says that the iterator has to be called during tuple construction?

while 1:
for each sequence:
# optimize cache behaviour!
grab up to N items from each iterator
M = length of shortest output list
for i in range(M):
build tuple and append
if M != N:
break

</F>

Nov 23 '05 #29
On 22/11/05, Bengt Richter <bo**@oz.net> wrote:
That would be a counter-intuitive thing to do. Most things go left->right
in order as the default assumption.

+1

--
Cheers,
Simon B,
si***@brunningonline.net,
http://www.brunningonline.net/simon/blog/
Nov 23 '05 #30
Fredrik Lundh wrote:
Bengt Richter wrote:
Are you thinking of something like lines from a file, where there might
be chunky buffering? ISTM that wouldn't matter if the same next method
was called. Here we have multiple references to the same iterator. Isn't
e.g. buiding a plain tuple defined with evaluation one element at a time
left to right?

yeah, but what says that the iterator has to be called during tuple
construction?

while 1:
for each sequence:
# optimize cache behaviour!
grab up to N items from each iterator
M = length of shortest output list
for i in range(M):
build tuple and append
if M != N:
break

Wouldn't every attempt to introduce such an optimization be shot down by the
likes of

def exponential():
for i in xrange(sys.maxint):
time.sleep(2**i)
yield "whatever"

def const():
for i in xrange(5): yield i

zip(exponential(), const())

To say it another way, aren't the problems that can be created by not
specifying zip() behaviour in a way that allows the zip(it, it) trick worse
than those you want to prevent?

Peter

Nov 23 '05 #31
On Wed, 23 Nov 2005 13:23:21 +0100, "Fredrik Lundh" <fr*****@pythonware.com> wrote:
Bengt Richter wrote:
Are you thinking of something like lines from a file, where there might be
chunky buffering? ISTM that wouldn't matter if the same next method was called.
Here we have multiple references to the same iterator. Isn't e.g. buiding
a plain tuple defined with evaluation one element at a time left to right?

yeah, but what says that the iterator has to be called during tuple construction?

while 1:
for each sequence:
# optimize cache behaviour!
grab up to N items from each iterator
M = length of shortest output list
for i in range(M):
build tuple and append
if M != N:
break

I'd say it's ok to cache some iterator output, but the cache buffer
must be associated with the iterator like cachedict[id(it)]. Then
the loop would only pump up the same cache buffer, since all iterators
would refer to the same buffer, and the loop that builds the tuple would
draw one from "each" buffer unknowingly drawing from the same buffer,
since they would be all found by cachedict[id(it)] which would be the same.
This would preserve the identity and logical sequentiality of the data
stream(s). To buffer separately on the unverified assumption that they
are different iterators seems like a buggy implementation to me ;-)
(optimizing use of total available buffering space is another matter,

ISTM to me the easiest just to define the intuitive behaviour, and let
implementers buffer accordingly if they want to.

Regards,
Bengt Richter
Nov 24 '05 #32

### This discussion thread is closed

Replies have been disabled for this discussion.