473,554 Members | 2,934 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

PEP 289: Generator Expressions (please comment)

Peter Norvig's creative thinking triggered renewed interest in PEP 289.
That led to a number of contributors helping to re-work the pep details
into a form that has been well received on the python-dev list:

http://www.python.org/peps/pep-0289.html

In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)
min(d.temperatu re()*9/5 + 32 for d in days)
Set(word.lower( ) for word in text.split() if len(word) < 5)
dict( (k, somefunc(k)) for k in keylist )
dotproduct = sum(x*y for x,y in itertools.izip( xvec, yvec))
bestplayer, bestscore = max( (p.score, p.name) for p in players )

Each of the above runs without creating a full list in memory,
which saves allocation time, conserves resources, and exploits
cache locality.

The new form is highly expressive and greatly enhances the utility
of the many python functions and methods that accept iterable arguments.
Raymond Hettinger
Jul 18 '05
72 4343
py****@rcn.com (Raymond Hettinger) writes:
In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)


This is a little confusing. Consider:

1) [x*x for x in roots]
2) (x*x for x in roots)

#2 looks like it should be a "sequence comprehension" and not a
generator expression. I'd go back to your earlier proposal of
using a yield keyword if you want a generator expression:

sum(yield x for x in roots)

Note there's no builtin function "sum". It's always annoyed me that
operator.add always takes exactly two arguments, so you have to use
reduce. I'd like to be able to say operator.add(2, 3,4), or in the
generator expression case, operator.add(yi eld x*x for x in roots).
Jul 18 '05 #11
rzed wrote:
I like it! How about in conjunction with the print statement?
print "%s\n" % (s for s in strlist)
??


Presuming you want this to result in a series of print statements, this
sounds like a very bad idea. All the PEP proposed is a new way of
creating an anonymous generator that uses a syntax similar to that of
list comprehensions; it doesn't actually change the way generators
themselves are already handled by the language. Consider:
def f(): .... i, j = 0, 1
.... yield i
.... yield j
.... while True:
.... i, j = j, i + j
.... yield j
.... g = f() # g is now a generator
print g <generator object at 0x402e3bcc> print "%s" % g # this doesn't print the Fibonacci sequence

<generator object at 0x402e3bcc>

What you're suggesting would grievously sacrifice consistency at the
expense of a tiny little bit of convenience. It isn't worth it.

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
__ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
/ \ You and I / We've seen it all / Chasing our hearts' desire
\__/ The Russian and Florence, _Chess_
Jul 18 '05 #12
"rzed" <rz*****@ntelos .net> wrote in news:1066956387 .364126@cache1:

"Raymond Hettinger" <py****@rcn.com > wrote in message
news:5d******** *************** ***@posting.goo gle.com...
Peter Norvig's creative thinking triggered renewed interest in PEP
289. That led to a number of contributors helping to re-work the pep
details into a form that has been well received on the python-dev
list:

http://www.python.org/peps/pep-0289.html

In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)
min(d.temperatu re()*9/5 + 32 for d in days)
Set(word.lower( ) for word in text.split() if len(word) < 5)
dict( (k, somefunc(k)) for k in keylist )
dotproduct = sum(x*y for x,y in itertools.izip( xvec, yvec))
bestplayer, bestscore = max( (p.score, p.name) for p in players )

Each of the above runs without creating a full list in memory,
which saves allocation time, conserves resources, and exploits
cache locality.

The new form is highly expressive and greatly enhances the utility
of the many python functions and methods that accept iterable
arguments.


I like it! How about in conjunction with the print statement?
print "%s\n" % (s for s in strlist)


Wouldn't work in general, i.e. in your version there isn't a way to
supply multiple elements. For a (completely?) general solution see the
Lisp FORMAT function, the ~{ @{ etc. format specifiers
(http://www.lisp.org/HyperSpec/Body/sec_22-3-7-4.html). Perhaps not the
most Pythonic? <grin>

-- bjorn
Jul 18 '05 #13
Christoph Becker-Freyseng <we*******@beyo nd-thoughts.com> wrote in
news:ma******** *************** ************@py thon.org:

[...]


BTW: Python is getting more and more stuff. So I fear that some code
might get harder to read for newbies (not knowing all the new stuff).

Would it be possible to make an expression-/statement-parser that can
analyse a code snippet (one line of python-code maybe more) and tell the
user what syntax-constructs were used.

This might be a big help when reading code and if you don't know some
special construct.

e.g.:
newstuff.py:
...
sum(x*x for x in roots)
...
??? Mmmm What's that
pyscryer "sum(x*x for x in roots)"
<Function-Call>(<Generato r Expression>)

Then you can search the Internet to close your knowledge gap (you can't
search for "sum(x*x for x in roots)", but "python Generator Expression"
might be succesful)


Something like:

compiler.parse( "[x*x for x in roots]")

(not completely what you want, but close :-)

-- bjorn
Jul 18 '05 #14
Paul Rubin <http://ph****@NOSPAM.i nvalid> wrote in
news:7x******** ****@ruckus.bro uhaha.com:
py****@rcn.com (Raymond Hettinger) writes:
In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)

[..]
Note there's no builtin function "sum". It's always annoyed me that
operator.add always takes exactly two arguments, so you have to use
reduce.

[..]

time to upgrade?

f:\>python
Python 2.3 (#46, Jul 29 2003, 18:54:32) [MSC v.1200 32 bit (Intel)] on
win32
Type "help", "copyright" , "credits" or "license" for more information.
sum(range(10))

45

-- bjorn
Jul 18 '05 #15
rzed wrote:
I like it! How about in conjunction with the print statement?
print "%s\n" % (s for s in strlist)


It's not better than

for s in strlist:
print s

or I didn't understand your intent.

Mike


Jul 18 '05 #16


Paul Rubin wrote:
py****@rcn.com (Raymond Hettinger) writes:
In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)

This is a little confusing. Consider:

1) [x*x for x in roots]
2) (x*x for x in roots)

#2 looks like it should be a "sequence comprehension" and not a
generator expression. I'd go back to your earlier proposal of
using a yield keyword if you want a generator expression:

sum(yield x for x in roots)

I too prefer the above use of yield because then you can always
associate generators with the yield keyword.

Pad.

Jul 18 '05 #17
Donald 'Paddy' McCarthy wrote:
I too prefer the above use of yield because then you can always
associate generators with the yield keyword.


I don't, because using 'yield' here would lead to greater confusion when
using generator expressions in generator functions.
yield [x for x in y] # Yields a list
yield x for x in y # Creates and discards a generator
yield yield x for x in y # Yields an iterator
--
Rainer Deyke - ra*****@eldwood .com - http://eldwood.com
Jul 18 '05 #18
Raymond Hettinger wrote:
Peter Norvig's creative thinking triggered renewed interest in PEP 289.
That led to a number of contributors helping to re-work the pep details
into a form that has been well received on the python-dev list:

http://www.python.org/peps/pep-0289.html

In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)
min(d.temperatu re()*9/5 + 32 for d in days)
Set(word.lower( ) for word in text.split() if len(word) < 5)
dict( (k, somefunc(k)) for k in keylist )
dotproduct = sum(x*y for x,y in itertools.izip( xvec, yvec))
bestplayer, bestscore = max( (p.score, p.name) for p in players )
Although most people on python-dev and here seems to like this PEP I
think it generally decreases readability. The above constructs seem
heavy and difficult to parse and thus i am afraid that they interfere
with the perceived simplicity of Python's syntax.

Sometimes it seems that introducing new syntax happens much easier
than improving or adding stdlib-modules.
Each of the above runs without creating a full list in memory,
which saves allocation time, conserves resources, and exploits
cache locality.


I do like these properties, though :-)

At least I hope that generator expressions will only be introduced
via a __future__ statement in Python 2.4 much like it happened with
'yield' and generators. Maybe the PEP should have an "implementa tion
plan" chapter mentioning such details?

cheers,

holger

Jul 18 '05 #19
Holger Krekel <py**@devel.tri llke.net> writes:
At least I hope that generator expressions will only be introduced
via a __future__ statement in Python 2.4 much like it happened with


Why? Does it break backwards compatibility somehow?

As far as readability goes, genexps seem to be pretty much as
readable/unreadable as list comprehensions. To me, BTW, the idea looks
great. However, I'm not completely sure about allowing them to be
creatad w/o parens in function calls, seems a bit too magical for my
taste.

--
Ville Vainio http://www.students.tut.fi/~vainio24
Jul 18 '05 #20

This thread has been closed and replies have been disabled. Please start a new discussion.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.