Iterating over several lists at once

[Gal Diskin]

Hi,
I am writing a code that needs to iterate over 3 lists at the same
time, i.e something like this:

for x1 in l1:
for x2 in l2:
for x3 in l3:
print "do something with", x1, x2, x3

What I need to do is go over all n-tuples where the first argument is
from the first list, the second from the second list, and so on...
I was wondering if one could write this more easily in some manner
using only 1 for loop.
What I mean is something like this:

for (x1,x2,x3) in (l1,l2,l3):
print "do something with", x1, x2, x3

Or maybe like this:

for x1 in l1, x2 in l2, x3 in l3:
print "do something with", x1, x2, x3

However, this code obviously doesn't work...
I'd be very happy to receive ideas about how to do this in one loop and
with minimal initialization (if at all required).

Thanks in advance,
Gal

[Fredrik Lundh]

"Gal Diskin" wrote:

I am writing a code that needs to iterate over 3 lists at the same
time, i.e something like this:

for x1 in l1:
for x2 in l2:
for x3 in l3:
print "do something with", x1, x2, x3

What I need to do is go over all n-tuples where the first argument is
from the first list, the second from the second list, and so on...

I was wondering if one could write this more easily in some manner
using only 1 for loop.
What I mean is something like this:

for (x1,x2,x3) in (l1,l2,l3):
print "do something with", x1, x2, x3

how about

for x1, x2, x3 in func(l1, l2, l3):
print x1, x2, x3

where func is defined as, say,

def func(l1, l2, l3):
return ((x1, x2, x3) for x1 in l1 for x2 in l2 for x3 in l3)

or if you prefer

def helper(l1, l2, l3):
for x1 in l1:
for x2 in l2:
for x3 in l3:
yield x1, x2, x3

</F>

[Roberto Bonvallet]

Gal Diskin wrote:

Hi,
I am writing a code that needs to iterate over 3 lists at the same
time, i.e something like this:

for x1 in l1:
for x2 in l2:
for x3 in l3:
print "do something with", x1, x2, x3

What's wrong with this?

[...]

I'd be very happy to receive ideas about how to do this in one loop and
with minimal initialization (if at all required).

def cartesian_product(l1, l2, l3):
for i in l1:
for j in l2:
for k in l3:
yield (i, j, k)

for (i, j, k) in cartesian_product(l1, l2, l3):
print "do something with", i, j, k

--
Roberto Bonvallet

[Paul Rubin]

"Gal Diskin" <ga********@gmail.comwrites:

I am writing a code that needs to iterate over 3 lists at the same
time, i.e something like this:

for x1 in l1:
for x2 in l2:
for x3 in l3:
print "do something with", x1, x2, x3

This does look a little kludgy (untested):

for x1,x2,x3 in ((x1,x2,x3) for x1 in l1 for x2 in l2 for x3 in l3):
print "do something with", x1, x2, x3

[Peter Otten]

Gal Diskin wrote:

I am writing a code that needs to iterate over 3 lists at the same
time, i.e something like this:

for x1 in l1:
for x2 in l2:
for x3 in l3:
print "do something with", x1, x2, x3

I was wondering if one could write this more easily in some manner
using only 1 for loop.

def nested_loops(*args):
assert args
if len(args) == 1:
for item in args[0]:
yield (item,)
else:
gap = len(args) // 2
for left in nested_loops(*args[:gap]):
for right in nested_loops(*args[gap:]):
yield left + right

if __name__ == "__main__":
for i, k in nested_loops("abc", "12"):
print i, k
for i, j, k in nested_loops("ab", "123", "xyz"):
print i, j, k

Be prepared for a significant performance hit.

Peter

PS: Did anybody say macro? No? I must be hallucinating...

[Gal Diskin]

Nothing seriously wrong, but it's not too elegent. Especially when the
number of lists you want to iterate over gets bigger (especially
because of the indentation in python). As you noticed (an phrased
better than me), what I was wondering is if there is a way to iterate
over the cartesian product, but without actually doing all n for loops
but using a single "for" loop.

Thanks for replying me.
On Dec 13, 3:58 pm, Roberto Bonvallet <[e-mail]>
wrote:

Gal Diskin wrote:

Hi,
I am writing a code that needs to iterate over 3 lists at the same
time, i.e something like this:

for x1 in l1:
for x2 in l2:
for x3 in l3:
print "do something with", x1, x2, x3What's wrong with this?

[...]

I'd be very happy to receive ideas about how to do this in one loop and
with minimal initialization (if at all required).def cartesian_product(l1, l2, l3):

for i in l1:
for j in l2:
for k in l3:
yield (i, j, k)

for (i, j, k) in cartesian_product(l1, l2, l3):
print "do something with", i, j, k

--
Roberto Bonvallet

[Kay Schluehr]

Gal Diskin schrieb:

Hi,
I am writing a code that needs to iterate over 3 lists at the same
time, i.e something like this:

for x1 in l1:
for x2 in l2:
for x3 in l3:
print "do something with", x1, x2, x3

What I need to do is go over all n-tuples where the first argument is
from the first list, the second from the second list, and so on...

Heard about recursion?

def collect(L,*lists):
if not lists:
return [(x,) for x in L]
collection = []
for x in L:
for y in collect(lists[0],*lists[1:]):
collection.append((x,)+y)
return collection

for item in collect( l1, l2, l3):
func(*item)

Here is the same in generator form

def collect(L,*lists):
if not lists:
for x in L:
yield (x,)
else:
for x in L:
for y in collect(lists[0],*lists[1:]):
yield (x,)+y

( o.k - it required two nested for-loops in each implementation :)

[Gal Diskin]

Thanks, that's an improvment (your first way).
But I still wish I could find an even shorter (or more elegent) way of
doing it. (Well, I guess if I expect every wish I have to come true I
should at least wish for something more valuable.)

Thanks again,
Gal

On Dec 13, 3:58 pm, "Fredrik Lundh" <[e-mail]:

"Gal Diskin" wrote:

I am writing a code that needs to iterate over 3 lists at the same
time, i.e something like this:

for x1 in l1:
for x2 in l2:
for x3 in l3:
print "do something with", x1, x2, x3

What I need to do is go over all n-tuples where the first argument is
from the first list, the second from the second list, and so on...

I was wondering if one could write this more easily in some manner
using only 1 for loop.
What I mean is something like this:

for (x1,x2,x3) in (l1,l2,l3):
print "do something with", x1, x2, x3how about

for x1, x2, x3 in func(l1, l2, l3):
print x1, x2, x3

where func is defined as, say,

def func(l1, l2, l3):
return ((x1, x2, x3) for x1 in l1 for x2 in l2 for x3 in l3)

or if you prefer

def helper(l1, l2, l3):
for x1 in l1:
for x2 in l2:
for x3 in l3:
yield x1, x2, x3

</F>

[Maksim Kasimov]

Hi,

if you "needs to iterate over 3 lists at the same" and according your example

for (x1,x2,x3) in (l1,l2,l3):
print "do something with", x1, x2, x3

i has guessed that you need this (may be i was wrong):
a = (1,2,3, 1)
b = (4,5,6)
c = (7,8,9, 2, 3)

for x, y, z in zip(a, b, c):
print x, y, z
or this
for x, y, z in map(None, a, b, c):
print x,y,z
Try both examples with tuples that have various length, they have difference


Gal Diskin wrote:

Hi,
I am writing a code that needs to iterate over 3 lists at the same
time, i.e something like this:

for x1 in l1:
for x2 in l2:
for x3 in l3:
print "do something with", x1, x2, x3

What I need to do is go over all n-tuples where the first argument is
from the first list, the second from the second list, and so on...
I was wondering if one could write this more easily in some manner
using only 1 for loop.
What I mean is something like this:

for (x1,x2,x3) in (l1,l2,l3):
print "do something with", x1, x2, x3

Or maybe like this:

for x1 in l1, x2 in l2, x3 in l3:
print "do something with", x1, x2, x3

However, this code obviously doesn't work...
I'd be very happy to receive ideas about how to do this in one loop and
with minimal initialization (if at all required).

Thanks in advance,
Gal

--
Maksim Kasimov

[Carl Banks]

Gal Diskin wrote:

On Dec 13, 3:58 pm, Roberto Bonvallet <[e-mail]>
wrote:

Gal Diskin wrote:

Hi,
I am writing a code that needs to iterate over 3 lists at the same
time, i.e something like this:

for x1 in l1:
for x2 in l2:
for x3 in l3:
print "do something with", x1, x2, x3What's wrong with this?

[...]

I'd be very happy to receive ideas about how to do this in one loop and
with minimal initialization (if at all required).def cartesian_product(l1, l2, l3):

for i in l1:
for j in l2:
for k in l3:
yield (i, j, k)

for (i, j, k) in cartesian_product(l1, l2, l3):
print "do something with", i, j, k

Nothing seriously wrong, but it's not too elegent.

I wonder why you think that. Many people here would consider it the
height of elegance to take some complicated logic, writing a function
encompassing the complicated parts, and ending up with simpler logic.
That's what happens here.

The cartesian_product solution above is not much longer than this
syntax you proposed:

for (i,j,k) in (a,b,c):
use(i,j,k)

(You can make the function name shorter if you want.) What is it that
you find inelegant about this solution? Is it that you don't like
extra function sitting there? Just put it in a module and import it.

Especially when the
number of lists you want to iterate over gets bigger (especially
because of the indentation in python).

The function can be extended to allow arbitrary arguments. Here's a
non-minmal recursive version.

def cartesian_product(*args):
if len(args) 1:
for item in args[0]:
for rest in cartesian_product(*args[1:]):
yield (item,) + rest
elif len(args) == 1:
for item in args[0]:
yield (item,)
else:
yield ()

As you noticed (an phrased
better than me), what I was wondering is if there is a way to iterate
over the cartesian product, but without actually doing all n for loops
but using a single "for" loop.

Even if Python had a special syntax for it, it would still be looping
internally.

And there's almost no chance of there ever being a special syntax for
it. First of all, it's very straightforward to do with functional
solutions, as we've shown here. Second, compare the case of zip.
Iteration in parallel (such as you would do with zip) is a lot more
common than nested iteration (cartesian_product), but even parallel
iteration doesn't have a special syntax, and th BDFL had declared that
won't in Python 3.0.

The best you could hope for is a built in function like zip, and that's
doubtful. OTOH, something like this has decent shot of appearing in a
standard functional or iterator module of some sort.

Thanks for replying me.

You're welcome in advance. Also, one note about comp.lang.python
and/or python-list: it's conventional here to put replies beneath the
quoted text. This is for the benefit of future readers, so they don't
have read conversations in reverse order.
Carl Banks

[at]

Sorry for breaking into this thread, but I agree completely that any
unnecessary indentations should be avoided. For the same reason I advocate
that the following syntax should work:

for x in some_list if some_condition:
... code ...

in stead of

for x in some_list
if some_condition:
... code ...

All the best!

@

PS: maybe using 'sets' can help you out for a particular problem.
Gal Diskin wrote:

Nothing seriously wrong, but it's not too elegent. Especially when the
number of lists you want to iterate over gets bigger (especially
because of the indentation in python). As you noticed (an phrased
better than me), what I was wondering is if there is a way to iterate
over the cartesian product, but without actually doing all n for loops
but using a single "for" loop.

Thanks for replying me.
On Dec 13, 3:58 pm, Roberto Bonvallet <[e-mail]>
wrote:

>Gal Diskin wrote:

Hi,
I am writing a code that needs to iterate over 3 lists at the same
time, i.e something like this:

for x1 in l1:
for x2 in l2:
for x3 in l3:
print "do something with", x1, x2, x3What's wrong with this?

[...]

I'd be very happy to receive ideas about how to do this in one loop and
with minimal initialization (if at all required).def
cartesian_product(l1, l2, l3):

for i in l1:
for j in l2:
for k in l3:
yield (i, j, k)

for (i, j, k) in cartesian_product(l1, l2, l3):
print "do something with", i, j, k

--
Roberto Bonvallet

[Mike Erickson]

* at ([e-mail]) wrote:

Sorry for breaking into this thread, but I agree completely that any
unnecessary indentations should be avoided. For the same reason I advocate
that the following syntax should work:

for x in some_list if some_condition:
... code ...

in stead of

for x in some_list
if some_condition:
... code ...

It is possible to avoid the extra level of indentaion, but I think it's
much less readable than the 2-level verbose expresion:

>>a

[1, 2, 3, 4, 5, 6, 7]

>>for odd in (num for num in a if num % 2 == 1):

.... print odd
....
1
3
5
7

there is also continue, which I think is a good compromise:

>>for num in a:

.... if num % 2 == 0:
.... continue
.... print num
....
1
3
5
7

HTH (and not lead astray),

mike

[John Henry]

Carl Banks wrote:
<snip>

>
The function can be extended to allow arbitrary arguments. Here's a
non-minmal recursive version.

def cartesian_product(*args):
if len(args) 1:
for item in args[0]:
for rest in cartesian_product(*args[1:]):
yield (item,) + rest
elif len(args) == 1:
for item in args[0]:
yield (item,)
else:
yield ()

Very nice.

[Michael Spencer]

John Henry wrote:

Carl Banks wrote:
<snip>

>The function can be extended to allow arbitrary arguments. Here's a
non-minmal recursive version.

def cartesian_product(*args):
if len(args) 1:
for item in args[0]:
for rest in cartesian_product(*args[1:]):
yield (item,) + rest
elif len(args) == 1:
for item in args[0]:
yield (item,)
else:
yield ()

Very nice.

another implementation of cartesian_product using 'reduce' to add
mystery ;-)

def star(outer, inner):
for rest in outer:
for item in inner:
yield rest + (item,)

def cartesian_product(*args):
return reduce(star, args, ((),))

>>list(cartesian_product("01","01","01"))

[('0', '0', '0'), ('0', '0', '1'), ('0', '1', '0'), ('0', '1', '1'),
('1', '0', '0'), ('1', '0', '1'), ('1', '1', '0'), ('1', '1', '1')]

>>>

Michael

[John Henry]

Is this suppose to be a brain teaser or something?

Michael Spencer wrote:

John Henry wrote:

Carl Banks wrote:
<snip>

The function can be extended to allow arbitrary arguments. Here's a
non-minmal recursive version.

def cartesian_product(*args):
if len(args) 1:
for item in args[0]:
for rest in cartesian_product(*args[1:]):
yield (item,) + rest
elif len(args) == 1:
for item in args[0]:
yield (item,)
else:
yield ()

Very nice.

another implementation of cartesian_product using 'reduce' to add
mystery ;-)

def star(outer, inner):
for rest in outer:
for item in inner:
yield rest + (item,)

def cartesian_product(*args):
return reduce(star, args, ((),))

>>list(cartesian_product("01","01","01"))

[('0', '0', '0'), ('0', '0', '1'), ('0', '1', '0'), ('0', '1', '1'),
('1', '0', '0'), ('1', '0', '1'), ('1', '1', '0'), ('1', '1', '1')]

>>>

Michael

[Gal Diskin]

On Dec 13, 3:47 pm, "Gal Diskin" <[e-mail]:

Hi,
I am writing a code that needs to iterate over 3 lists at the same
time, i.e something like this:

for x1 in l1:
for x2 in l2:
for x3 in l3:
print "do something with", x1, x2, x3

What I need to do is go over all n-tuples where the first argument is
from the first list, the second from the second list, and so on...

I was wondering if one could write this more easily in some manner
using only 1 for loop.
What I mean is something like this:

for (x1,x2,x3) in (l1,l2,l3):
print "do something with", x1, x2, x3

Or maybe like this:

for x1 in l1, x2 in l2, x3 in l3:
print "do something with", x1, x2, x3

However, this code obviously doesn't work...

I'd be very happy to receive ideas about how to do this in one loop and
with minimal initialization (if at all required).

Thanks in advance,Gal

Sorry for bumping this thread up. I just got back from a vacation and I
had to say thanks to all the people that took the time to answer me.

To Maksim - that's not what I meant, I want to iterate over the
cartesian product of the lists (all ordered tuples, _not_ the tuple
created from the first item in each list, then the tubple created from
the second item in each list... as your example does).

Thanks,
Gal

[Erik Johnson]

"Gal Diskin" <ga********@gmail.comwrote in message
news:11**********************@f1g2000cwa.googlegro ups.com...

>

On Dec 13, 3:47 pm, "Gal Diskin" <[e-mail]:

Hi,
I am writing a code that needs to iterate over 3 lists at the same
time, i.e something like this:

for x1 in l1:
for x2 in l2:
for x3 in l3:
print "do something with", x1, x2, x3

What I need to do is go over all n-tuples where the first argument is
from the first list, the second from the second list, and so on...

I don't see your previous article. Not sure if you are still looking for
a solution, but here's one:

>>[(x, y, z) for x in [1,2,3] for y in list('abc') for z in list('XY')]

[(1, 'a', 'X'), (1, 'a', 'Y'), (1, 'b', 'X'), (1, 'b', 'Y'), (1, 'c', 'X'),
(1, 'c', 'Y'), (2, 'a', 'X'), (2, 'a', 'Y'), (2, 'b', 'X'), (2, 'b', 'Y'),
(2, 'c', 'X'), (2, 'c', 'Y'), (3, 'a', 'X'), (3, 'a', 'Y'), (3, 'b', 'X'),
(3, 'b', 'Y'), (3, 'c', 'X'), (3, 'c', 'Y')]

This is a bit more compact, but I don't see anything wrong with what you
have.

[Roy Smith]

In article <11**********************@16g2000cwy.googlegroups. com>,
"Gal Diskin" <ga********@gmail.comwrote:

Hi,
I am writing a code that needs to iterate over 3 lists at the same
time, i.e something like this:

for x1 in l1:
for x2 in l2:
for x3 in l3:
print "do something with", x1, x2, x3

What I need to do is go over all n-tuples where the first argument is
from the first list, the second from the second list, and so on...

Take a look at
http://mail.python.org/pipermail/pyt...er/104983.html

or

http://aspn.activestate.com/ASPN/Coo.../Recipe/159975

There's nothing magic about either -- fundamentally, you're still doing an
N^3 operation and it's going to be slow. You might want to stop and think
if there's some better algorithm than an exhaustive search of the entire
domain space for whatever it is that you're trying to do.