i=2; lst=[i**=2 while i<1000]

Hello NG,

I am wondering if there were proposals or previous disscussions in this
NG considering using 'while' in comprehension lists

# pseudo code
i=2
lst=[i**=2 while i<1000]

of course this could be easily rewritten into
i=2
lst=[]
while i<1000:
i**=2
lst.append(i)



usually I would prefer one liners like
lines=[line.strip() for line in file("foo").readlines() if line]
they make the purpose clear
so if 'while' were allowed in comprehension list this would allow
to write concise code

to the example above, I tried a workaround
i=2
lst=[i**=2 for _ in iter(lambda:_<1000, False)]

but this failes with SyntaxError because of i**=2
and must be also rewritten into
[color=blue][color=green][color=darkred]
>>> i=2
>>> def f():[/color][/color][/color]
.... global i
.... i**=2
.... return i
....[color=blue][color=green][color=darkred]
>>> lst=[f() for _ in iter(lambda:i<1000, False)]
>>> lst[/color][/color][/color]
[4, 16, 256, 65536]

I think this loses compared with
i=2
lst=[i**=2 while i<1000]

Regards, Daniel

re: i=2; lst=[i**=2 while i<1000]

Daniel Schüle wrote:[color=blue]
> Hello NG,
>
> I am wondering if there were proposals or previous disscussions in this
> NG considering using 'while' in comprehension lists
>
> # pseudo code
> i=2
> lst=[i**=2 while i<1000]
>
> of course this could be easily rewritten into
> i=2
> lst=[]
> while i<1000:
> i**=2
> lst.append(i)[/color]

That would loop endlessly since you don't increment i.
You can use i**=2 for i in range(1000) instead

re: i=2; lst=[i**=2 while i<1000]

D H wrote:[color=blue]
> Daniel Schüle wrote:[color=green]
> > Hello NG,
> >
> > I am wondering if there were proposals or previous disscussions in this
> > NG considering using 'while' in comprehension lists
> >
> > # pseudo code
> > i=2
> > lst=[i**=2 while i<1000]
> >
> > of course this could be easily rewritten into
> > i=2
> > lst=[]
> > while i<1000:
> > i**=2
> > lst.append(i)[/color]
>
> That would loop endlessly since you don't increment i.
> You can use i**=2 for i in range(1000) instead[/color]

I don't think one can use assignment in list comprehension or generator
expression. The limitation is very much like lambda.

re: i=2; lst=[i**=2 while i<1000]

bonono@gmail.com wrote:[color=blue][color=green]
>>You can use i**=2 for i in range(1000) instead[/color]
>
>
> I don't think one can use assignment in list comprehension or generator
> expression. The limitation is very much like lambda.
>[/color]

i**2

re: i=2; lst=[i**=2 while i<1000]

Daniel Schüle wrote:
[color=blue]
> I am wondering if there were proposals or previous disscussions in this
> NG considering using 'while' in comprehension lists
>
> # pseudo code
> i=2
> lst=[i**=2 while i<1000]
>
> of course this could be easily rewritten into
> i=2
> lst=[]
> while i<1000:
> i**=2
> lst.append(i)
>[/color]

Neither of these loops would terminate until memory is exhausted. Do you
have a use case for a 'while' in a list comprehension which would
terminate?

re: i=2; lst=[i**=2 while i<1000]

D H wrote:[color=blue]
> bonono@gmail.com wrote:
>[color=green][color=darkred]
>>> You can use i**=2 for i in range(1000) instead[/color]
>>
>>
>>
>> I don't think one can use assignment in list comprehension or generator
>> expression. The limitation is very much like lambda.
>>[/color]
>
> i**2[/color]

lst=[i**2 for i in range(1000)]

you will get a list with 1000 items
[0,1,4,9 ... ]

is not the same as

i,lst=2,[]
while i<1000:
i**=2
lst.append(i)

here you get [4,16,256,65536]
only 4 items

Regards, Daniel

re: i=2; lst=[i**=2 while i<1000]

hi,

[...]
[color=blue][color=green]
>># pseudo code
>>i=2
>>lst=[i**=2 while i<1000]
>>
>>of course this could be easily rewritten into
>>i=2
>>lst=[]
>>while i<1000:
>> i**=2
>> lst.append(i)
>>[/color]
>
>
> Neither of these loops would terminate until memory is exhausted. Do you
> have a use case for a 'while' in a list comprehension which would
> terminate?[/color]

unless I am missing something obvious, I can not see why the loop should
not terminate
sure pseudo code is not executable but the other one works
while tests the boolean expression first then decides whether to execute
the body or not, in particular no next-iterator is involved(??)
as it would be in
lst=range(5)
for i in lst:
del lst[0]


Regards, Daniel

re: i=2; lst=[i**=2 while i<1000]

Daniel Schüle wrote:[color=blue]
> D H wrote:[color=green]
> > bonono@gmail.com wrote:
> >[color=darkred]
> >>> You can use i**=2 for i in range(1000) instead
> >>
> >>
> >>
> >> I don't think one can use assignment in list comprehension or generator
> >> expression. The limitation is very much like lambda.
> >>[/color]
> >
> > i**2[/color]
>
> lst=[i**2 for i in range(1000)]
>
> you will get a list with 1000 items
> [0,1,4,9 ... ]
>
> is not the same as
>
> i,lst=2,[]
> while i<1000:
> i**=2
> lst.append(i)
>
> here you get [4,16,256,65536]
> only 4 items
>[/color]
You want a combination of takewhile and scanl and filter. For this
particular snippet, I think a simple loop is cleaner. I am bias towards
one-liner but for this case, explicit loop beats it. Just define a
short function.

re: i=2; lst=[i**=2 while i<1000]

Daniel Schüle wrote:
[color=blue]
> hi,
>
> [...]
>[color=green][color=darkred]
>>># pseudo code
>>>i=2
>>>lst=[i**=2 while i<1000]
>>>
>>>of course this could be easily rewritten into
>>>i=2
>>>lst=[]
>>>while i<1000:
>>> i**=2
>>> lst.append(i)
>>>[/color]
>>
>>
>> Neither of these loops would terminate until memory is exhausted. Do
>> you have a use case for a 'while' in a list comprehension which would
>> terminate?[/color]
>
> unless I am missing something obvious, I can not see why the loop
> should not terminate
> sure pseudo code is not executable but the other one works
> while tests the boolean expression first then decides whether to
> execute the body or not, in particular no next-iterator is
> involved(??) as it would be in
> lst=range(5)
> for i in lst:
> del lst[0]
>[/color]
Yes, I wasn't paying attention. The pseudo-code fails because you would
need to support assignment and generating a value inside the loop, but the
simple loop does work.

The simple case in the pseudo-coded loop isn't nearly general enough, what
if you wanted a tuple, or i**2+1 as each loop value. You would need syntax
which generate more complex list values e.g:

lst = [ (i, i**2) while i < 1000; i**=2 ]

One current way to write this would be:

def squaring(start, end):
i = start
while i < end:
yield i
i = i**2

Then you can do:

lst = [(i, i**2) for i in squaring(1, 1000) ]

which has the advantage of pulling all the complex logic out of the list
comprehension.

re: i=2; lst=[i**=2 while i<1000]

Daniel Schüle wrote:[color=blue]
> I am wondering if there were proposals or previous disscussions in this
> NG considering using 'while' in comprehension lists
>
> # pseudo code
> i=2
> lst=[i**=2 while i<1000][/color]

I haven't had much need for anything like this. Can't you rewrite with
a list comprehension something like::
[color=blue][color=green][color=darkred]
>>> [4**(2**i) for i in xrange(math.log(1000, 4))][/color][/color][/color]
[4, 16, 256, 65536]

?

STeVe

re: i=2; lst=[i**=2 while i<1000]

Duncan Booth wrote:[color=blue]
> Daniel Schüle wrote:
>[color=green]
> > hi,
> >
> > [...]
> >[color=darkred]
> >>># pseudo code
> >>>i=2
> >>>lst=[i**=2 while i<1000]
> >>>
> >>>of course this could be easily rewritten into
> >>>i=2
> >>>lst=[]
> >>>while i<1000:
> >>> i**=2
> >>> lst.append(i)
> >>>
> >>
> >>
> >> Neither of these loops would terminate until memory is exhausted. Do
> >> you have a use case for a 'while' in a list comprehension which would
> >> terminate?[/color]
> >
> > unless I am missing something obvious, I can not see why the loop
> > should not terminate
> > sure pseudo code is not executable but the other one works
> > while tests the boolean expression first then decides whether to
> > execute the body or not, in particular no next-iterator is
> > involved(??) as it would be in
> > lst=range(5)
> > for i in lst:
> > del lst[0]
> >[/color]
> Yes, I wasn't paying attention. The pseudo-code fails because you would
> need to support assignment and generating a value inside the loop, but the
> simple loop does work.
>
> The simple case in the pseudo-coded loop isn't nearly general enough, what
> if you wanted a tuple, or i**2+1 as each loop value. You would need syntax
> which generate more complex list values e.g:
>
> lst = [ (i, i**2) while i < 1000; i**=2 ]
>
> One current way to write this would be:
>
> def squaring(start, end):
> i = start
> while i < end:
> yield i
> i = i**2
>
> Then you can do:
>
> lst = [(i, i**2) for i in squaring(1, 1000) ]
>
> which has the advantage of pulling all the complex logic out of the list
> comprehension.[/color]

If one really wants a very messy one-liner, it is possible

import operator
x=[2]
lst=list(((operator.setitem(x,0,x[0]**2),x[0])[1] for _ in
xrange(10000000) if x[0] < 1000 or iter([]).next()))

re: i=2; lst=[i**=2 while i<1000]

Daniel Schüle wrote:[color=blue]
> hi,
>
> [...]
>
>[color=green][color=darkred]
>>># pseudo code
>>>i=2
>>>lst=[i**=2 while i<1000]
>>>
>>>of course this could be easily rewritten into
>>>i=2
>>>lst=[]
>>>while i<1000:
>>> i**=2
>>> lst.append(i)
>>>[/color]
>>
>>
>>Neither of these loops would terminate until memory is exhausted. Do you
>>have a use case for a 'while' in a list comprehension which would
>>terminate?[/color]
>
>
> unless I am missing something obvious, I can not see why the loop should
> not terminate[/color]

In that case, kindly explain how the condition i<1000 can become false
when it starts at 2 and never changes! [In other words: you *are*
missing something obvious].
[color=blue]
> sure pseudo code is not executable but the other one works
> while tests the boolean expression first then decides whether to execute
> the body or not, in particular no next-iterator is involved(??)
> as it would be in
> lst=range(5)
> for i in lst:
> del lst[0]
>[/color]
Indeed. But the test condition is initially true, and can never become
false, so the loop is endless. It will probably eventually terminate by
throwing a MemoryError exception when lst and its element values use up
all available space.

Don't you have an interpreter you could run the code in to verify that
it does indeed loop interminably? You seem to be assuming that the
expression i**2 changes the value of i. It doesn't.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006 www.python.org/pycon/

re: i=2; lst=[i**=2 while i<1000]

On Tue, 2005-12-06 at 10:44, Steve Holden wrote:[color=blue]
> Daniel Schüle wrote:[color=green][color=darkred]
> >>>i=2
> >>>lst=[]
> >>>while i<1000:
> >>> i**=2
> >>> lst.append(i)
> >>>
> >>[/color]
> >
> > unless I am missing something obvious, I can not see why the loop should
> > not terminate[/color]
>
> In that case, kindly explain how the condition i<1000 can become false
> when it starts at 2 and never changes! [In other words: you *are*
> missing something obvious].
>
> Don't you have an interpreter you could run the code in to verify that
> it does indeed loop interminably? You seem to be assuming that the
> expression i**2 changes the value of i. It doesn't.[/color]

Note that the OP wrote i**=2, not i**2.

-Carsten

re: i=2; lst=[i**=2 while i<1000]

Steve Holden wrote:
[color=blue][color=green][color=darkred]
>>>>lst=[i**=2 while i<1000]
>>>>
>>>>of course this could be easily rewritten into
>>>>i=2
>>>>lst=[]
>>>>while i<1000:
>>>> i**=2
>>>> lst.append(i)
>>>>
>>>[/color][/color][/color]
....[color=blue]
> Don't you have an interpreter you could run the code in to verify that
> it does indeed loop interminably? You seem to be assuming that the
> expression i**2 changes the value of i. It doesn't.[/color]

I think that like me you read 'i**2' when the OP actually wrote 'i**=2'.
i**=2 will change the value of i in the conventional while loop. In the
list comprehension it is of course a syntax error even if you assume list
comprehensions are expanded to allow the while.

re: i=2; lst=[i**=2 while i<1000]

On 2005-12-06, Steve Holden wrote:[color=blue]
> Daniel Schüle wrote:[color=green]
>> hi,
>>
>> [...]
>>
>>[color=darkred]
>>>># pseudo code
>>>>i=2
>>>>lst=[i**=2 while i<1000]
>>>>
>>>>of course this could be easily rewritten into
>>>>i=2
>>>>lst=[]
>>>>while i<1000:
>>>> i**=2
>>>> lst.append(i)
>>>>
>>>
>>>
>>>Neither of these loops would terminate until memory is exhausted. Do you
>>>have a use case for a 'while' in a list comprehension which would
>>>terminate?[/color]
>>
>>
>> unless I am missing something obvious, I can not see why the loop should
>> not terminate[/color]
>
> In that case, kindly explain how the condition i<1000 can become false
> when it starts at 2 and never changes! [In other words: you *are*
> missing something obvious].[/color]

What does i**=2 do if not change i?
[color=blue][color=green][color=darkred]
>>> i=2
>>> lst=[]
>>> while i<1000:[/color][/color][/color]
.... i**=2
.... lst.append(i)
....[color=blue][color=green][color=darkred]
>>> lst[/color][/color][/color]
[4, 16, 256, 65536]


--
Chris F.A. Johnson, author | <http://cfaj.freeshell.org>
Shell Scripting Recipes: | My code in this post, if any,
A Problem-Solution Approach | is released under the
2005, Apress | GNU General Public Licence

re: i=2; lst=[i**=2 while i<1000]

On 6 Dec 2005 07:41:45 -0800, bonono@gmail.com wrote:[color=blue]
>
>If one really wants a very messy one-liner, it is possible
>
>import operator
>x=3D[2]
>lst=3Dlist(((operator.setitem(x,0,x[0]**2),x[0])[1] for _ in
>xrange(10000000) if x[0] < 1000 or iter([]).next()))
>[/color]
Or[color=blue][color=green][color=darkred]
>>> list(iter(lambda b=[2]:b.append(b[0]**2) or b[0]<1000 and b.pop(0) or None, None))[/color][/color][/color]
[2, 4, 16, 256]

Regards,
Bengt Richter

re: i=2; lst=[i**=2 while i<1000]

Bengt Richter wrote:[color=blue]
> On 6 Dec 2005 07:41:45 -0800, bonono@gmail.com wrote:[color=green]
> >
> >If one really wants a very messy one-liner, it is possible
> >
> >import operator
> >x=3D[2]
> >lst=3Dlist(((operator.setitem(x,0,x[0]**2),x[0])[1] for _ in
> >xrange(10000000) if x[0] < 1000 or iter([]).next()))
> >[/color]
> Or[color=green][color=darkred]
> >>> list(iter(lambda b=[2]:b.append(b[0]**2) or b[0]<1000 and b.pop(0) or None, None))[/color][/color]
> [2, 4, 16, 256]
>[/color]
out of curiosity, what stops the iterator ?

re: i=2; lst=[i**=2 while i<1000]

Carsten Haese wrote:[color=blue]
> On Tue, 2005-12-06 at 10:44, Steve Holden wrote:
>[color=green]
>>Daniel Schüle wrote:
>>[color=darkred]
>>>>>i=2
>>>>>lst=[]
>>>>>while i<1000:
>>>>> i**=2
>>>>> lst.append(i)
>>>>>
>>>>
>>>unless I am missing something obvious, I can not see why the loop should
>>>not terminate[/color]
>>
>>In that case, kindly explain how the condition i<1000 can become false
>>when it starts at 2 and never changes! [In other words: you *are*
>>missing something obvious].
>>
>>Don't you have an interpreter you could run the code in to verify that
>>it does indeed loop interminably? You seem to be assuming that the
>>expression i**2 changes the value of i. It doesn't.[/color]
>
>
> Note that the OP wrote i**=2, not i**2.
>[/color]
Oops. Thanks :-)

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006 www.python.org/pycon/

re: i=2; lst=[i**=2 while i<1000]

On Tue, 06 Dec 2005 15:44:33 +0000, Steve Holden <steve@holdenweb.com> wrote:
[color=blue]
>Daniel Schüle wrote:[color=green]
>> hi,
>>
>> [...]
>>
>>[color=darkred]
>>>># pseudo code
>>>>i=2
>>>>lst=[i**=2 while i<1000]
>>>>
>>>>of course this could be easily rewritten into
>>>>i=2
>>>>lst=[]
>>>>while i<1000:
>>>> i**=2
>>>> lst.append(i)
>>>>
>>>
>>>
>>>Neither of these loops would terminate until memory is exhausted. Do you
>>>have a use case for a 'while' in a list comprehension which would
>>>terminate?[/color]
>>
>>
>> unless I am missing something obvious, I can not see why the loop should
>> not terminate[/color]
>
>In that case, kindly explain how the condition i<1000 can become false
>when it starts at 2 and never changes! [In other words: you *are*
>missing something obvious].
>[color=green]
>> sure pseudo code is not executable but the other one works
>> while tests the boolean expression first then decides whether to execute
>> the body or not, in particular no next-iterator is involved(??)
>> as it would be in
>> lst=range(5)
>> for i in lst:
>> del lst[0]
>>[/color]
>Indeed. But the test condition is initially true, and can never become
>false, so the loop is endless. It will probably eventually terminate by
>throwing a MemoryError exception when lst and its element values use up
>all available space.
>
>Don't you have an interpreter you could run the code in to verify that
>it does indeed loop interminably? You seem to be assuming that the
>expression i**2 changes the value of i. It doesn't.
>[/color]
No, but i**=2 does. Are you two talking about the same code?
[color=blue][color=green][color=darkred]
>>> i=2
>>> lst=[]
>>> while i<1000:[/color][/color][/color]
... i**=2
... lst.append(i)
...[color=blue][color=green][color=darkred]
>>> lst[/color][/color][/color]
[4, 16, 256, 65536]

Regards,
Bengt Richter

re: i=2; lst=[i**=2 while i<1000]

bonono@gmail.com wrote:
[color=blue][color=green][color=darkred]
>>>>>*list(iter(lambda*b=[2]:b.append(b[0]**2)*or*b[0]<1000*and*b.pop(0)*or[/color]
>>None,*None)) [2,*4,*16,*256]
>>[/color]
> out of curiosity, what stops the iterator ?[/color]

"""
Help on built-in function iter in module __builtin__:

iter(...)
iter(collection) -> iterator
iter(callable, sentinel) -> iterator

Get an iterator from an object. In the first form, the argument must
supply its own iterator, or be a sequence.
In the second form, the callable is called until it returns the
sentinel.
"""

The lambda is used as the callable, None as the sentinel.

Peter

re: i=2; lst=[i**=2 while i<1000]

<bonono@gmail.com> wrote:[color=blue]
>Bengt Richter wrote:[color=green][color=darkred]
>> >>> list(iter(lambda b=[2]:b.append(b[0]**2) or b[0]<1000 and b.pop(0) or None, None))[/color]
>> [2, 4, 16, 256][/color]
>out of curiosity, what stops the iterator ?[/color]

<http://docs.python.org/lib/built-in-funcs.html>:

iter(o, sentinel) [ ... ]
The iterator created in this case will call o with no arguments for
each call to its next() method; if the value returned is equal to
sentinel, StopIteration will be raised, otherwise the value will be
returned.

In this case, o is:

lambda b=[2]:b.append(b[0]**2) or b[0]<1000 and b.pop(0) or None

which returns None when b[0]>=1000 (None or (False and _) or None
evaluates to the last None).

--
\S -- siona@chiark.greenend.org.uk -- http://www.chaos.org.uk/~sion/
___ | "Frankly I have no feelings towards penguins one way or the other"
\X/ | -- Arthur C. Clarke
her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump

re: i=2; lst=[i**=2 while i<1000]

Daniel Schüle wrote:[color=blue]
> Hello NG,
>
> I am wondering if there were proposals or previous disscussions in this
> NG considering using 'while' in comprehension lists
>
> # pseudo code
> i=2
> lst=[i**=2 while i<1000]
>[/color]

You are actually describing two features that list comps don't natively support
- while-based termination, and calculating based on prior values of output. Of
course there are work-arounds for both, which others have shown. Here's another
approach:

The while-based termination can be easily achieved using itertools.takewhile, e.g.,:
[color=blue][color=green][color=darkred]
>>> list(itertools.takewhile(lambda x: x < 10, range(100)))[/color][/color][/color]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9][color=blue][color=green][color=darkred]
>>>[/color][/color][/color]

the harder piece is to access the prior value. One way is like this:

def chasetail(start, func):
from itertools import tee
def mygen():
yield start
for i in (func(i) for i in iterators[0]):
yield i
iterators = tee(mygen())
return iterators[1]

the trick is to create two independent iterators, using itertools.tee, one of
which is consumed internally in the func(i) for i in iterators[0] generator
expression, the other is returned to use code.
[color=blue][color=green][color=darkred]
>>> it = chasetail(2, lambda x: x*x) #careful - this won't terminate
>>> it.next()[/color][/color][/color]
2[color=blue][color=green][color=darkred]
>>> it.next()[/color][/color][/color]
4[color=blue][color=green][color=darkred]
>>> it.next()[/color][/color][/color]
16[color=blue][color=green][color=darkred]
>>> it.next()[/color][/color][/color]
256[color=blue][color=green][color=darkred]
>>> it.next()[/color][/color][/color]
65536[color=blue][color=green][color=darkred]
>>>[/color][/color][/color]

Then you can combine these two approaches to get something semantically like
what you wanted in the first place (although not as pretty ;-)
[color=blue][color=green][color=darkred]
>>> list(itertools.takewhile(lambda x: x < 1000, chasetail(2, lambda x: x*x)))[/color][/color][/color]
[2, 4, 16, 256][color=blue][color=green][color=darkred]
>>>[/color][/color][/color]



If you like this sort of thing, you might want to generalize the concept with a
Stream class. Here's minimal implementation:

import itertools as it

class Stream(object):
"""An extendable stream, that provides a separate iterator
(using itertools.tee) on every iteration request"""

def __init__(self, *iterables):
self.queue = list(iterables)
self.itertee = it.tee(self._chain(self.queue))[0]

def _chain(self, queue):
while queue:
for i in self.queue.pop(0):
self.head = i
yield i

def extend(self,other):
self.queue.append(other)

def __iter__(self):
"""Normal iteration over the iterables in self.queue in turn"""
return self.itertee.__copy__()


then, you can write your squaring algorithm as:
[color=blue][color=green][color=darkred]
>>> s= Stream([2])
>>> s.extend(it.takewhile(lambda x: x < 1000, (i**2 for i in s)))
>>> list(s)[/color][/color][/color]
[2, 4, 16, 256]


Michael