473,418 Members | 2,323 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,418 software developers and data experts.

PEP 289: Generator Expressions (please comment)

Peter Norvig's creative thinking triggered renewed interest in PEP 289.
That led to a number of contributors helping to re-work the pep details
into a form that has been well received on the python-dev list:

http://www.python.org/peps/pep-0289.html

In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)
min(d.temperature()*9/5 + 32 for d in days)
Set(word.lower() for word in text.split() if len(word) < 5)
dict( (k, somefunc(k)) for k in keylist )
dotproduct = sum(x*y for x,y in itertools.izip(xvec, yvec))
bestplayer, bestscore = max( (p.score, p.name) for p in players )

Each of the above runs without creating a full list in memory,
which saves allocation time, conserves resources, and exploits
cache locality.

The new form is highly expressive and greatly enhances the utility
of the many python functions and methods that accept iterable arguments.
Raymond Hettinger
Jul 18 '05 #1
72 4322
Raymond Hettinger wrote:
bestplayer, bestscore = max( (p.score, p.name) for p in players )

Each of the above runs without creating a full list in memory,
which saves allocation time, conserves resources, and exploits
cache locality.


Why should generator comprehension be useful only for arguments?

counter proposal:
g[(p.score, p.name) for p in players ]

This works similar to r'\' for raw strings and u'x' for unicode strings.

This has the advantage that it can be more easily extended:
d[(p.name, p.score) for p in players] creates a dictionary
s[(p.score, p.name) for p in players] creates a sorted list
and
p[$*-=%/sd/!] allows to embed perl code

Daniel

Jul 18 '05 #2
Daniel Dittmar wrote:

Raymond Hettinger wrote:
bestplayer, bestscore = max( (p.score, p.name) for p in players )

Each of the above runs without creating a full list in memory,
which saves allocation time, conserves resources, and exploits
cache locality.


Why should generator comprehension be useful only for arguments?

counter proposal:
g[(p.score, p.name) for p in players ]

This works similar to r'\' for raw strings and u'x' for unicode strings.


Except that r'' can mean nothing else, syntactically, while g[something]
could be a dictionary lookup using <something> as the key.
Jul 18 '05 #3
"Daniel Dittmar" <da****@dittmar.net> wrote in message
news:bn**********@news.eusc.inter.net...
Why should generator comprehension be useful only for arguments?


It is not limited to just function arguments.
It is a first class citizen. You can use it for anything you want.
Raymond Hettinger

Jul 18 '05 #4
On 23 Oct 2003 12:58:31 -0700, py****@rcn.com (Raymond Hettinger)
wrote:
Peter Norvig's creative thinking triggered renewed interest in PEP 289.
That led to a number of contributors helping to re-work the pep details
into a form that has been well received on the python-dev list:


I like it. Simple and effective.
--
Steve Horne

steve at ninereeds dot fsnet dot co dot uk
Jul 18 '05 #5
Raymond Hettinger wrote:
"Daniel Dittmar" <da****@dittmar.net> wrote in message
news:bn**********@news.eusc.inter.net...
Why should generator comprehension be useful only for arguments?


It is not limited to just function arguments.
It is a first class citizen. You can use it for anything you want.


At the very least, it would require parentheses if you want to create a list
containing exactly one such generator.

[(x for x in y)] # List containing generator
[x for x in y] # List comprehension
--
Rainer Deyke - ra*****@eldwood.com - http://eldwood.com
Jul 18 '05 #6
[Rainer Deyke]
At the very least, it would require parentheses if you want to create a list
containing exactly one such generator.

[(x for x in y)] # List containing generator
[x for x in y] # List comprehension


Yes, that's what the pep call for.
Raymond Hettinger
Jul 18 '05 #7

"Raymond Hettinger" <py****@rcn.com> wrote in message
news:5d**************************@posting.google.c om...
Peter Norvig's creative thinking triggered renewed interest in PEP 289.
That led to a number of contributors helping to re-work the pep details
into a form that has been well received on the python-dev list:

http://www.python.org/peps/pep-0289.html

In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)
min(d.temperature()*9/5 + 32 for d in days)
Set(word.lower() for word in text.split() if len(word) < 5)
dict( (k, somefunc(k)) for k in keylist )
dotproduct = sum(x*y for x,y in itertools.izip(xvec, yvec))
bestplayer, bestscore = max( (p.score, p.name) for p in players )

Each of the above runs without creating a full list in memory,
which saves allocation time, conserves resources, and exploits
cache locality.

The new form is highly expressive and greatly enhances the utility
of the many python functions and methods that accept iterable arguments.
I like it.

The final example in point 2 is incorrect. It currently says:

g = (x**2 for i in range(10))

x and i should be changed to match.

John Roth



John Roth

Raymond Hettinger

Jul 18 '05 #8

"Raymond Hettinger" <py****@rcn.com> wrote in message
news:5d**************************@posting.google.c om...
Peter Norvig's creative thinking triggered renewed interest in PEP 289.
That led to a number of contributors helping to re-work the pep details
into a form that has been well received on the python-dev list:

http://www.python.org/peps/pep-0289.html

In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)
min(d.temperature()*9/5 + 32 for d in days)
Set(word.lower() for word in text.split() if len(word) < 5)
dict( (k, somefunc(k)) for k in keylist )
dotproduct = sum(x*y for x,y in itertools.izip(xvec, yvec))
bestplayer, bestscore = max( (p.score, p.name) for p in players )

Each of the above runs without creating a full list in memory,
which saves allocation time, conserves resources, and exploits
cache locality.

The new form is highly expressive and greatly enhances the utility
of the many python functions and methods that accept iterable arguments.


I like it! How about in conjunction with the print statement?
print "%s\n" % (s for s in strlist)
??
--
rzed

Jul 18 '05 #9
Raymond Hettinger wrote:
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)

+3.1415

Especially I like the boost of use of many python-standard-functions --
like min, max, ... :-)
Furthermore the syntax is quite clear and readable.
However I'm a bit worried if some could get into trouble because of the
difference between using Generator Expressions in function-call with one
argument and a call with more than one arguments.
Christoph Becker-Freyseng
BTW: Python is getting more and more stuff. So I fear that some code
might get harder to read for newbies (not knowing all the new stuff).

Would it be possible to make an expression-/statement-parser that can
analyse a code snippet (one line of python-code maybe more) and tell the
user what syntax-constructs were used.

This might be a big help when reading code and if you don't know some
special construct.

e.g.:
newstuff.py:
...
sum(x*x for x in roots)
...
??? Mmmm What's that
pyscryer "sum(x*x for x in roots)"
<Function-Call>(<Generator Expression>)

Then you can search the Internet to close your knowledge gap (you can't
search for "sum(x*x for x in roots)", but "python Generator Expression"
might be succesful)

Jul 18 '05 #10
py****@rcn.com (Raymond Hettinger) writes:
In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)


This is a little confusing. Consider:

1) [x*x for x in roots]
2) (x*x for x in roots)

#2 looks like it should be a "sequence comprehension" and not a
generator expression. I'd go back to your earlier proposal of
using a yield keyword if you want a generator expression:

sum(yield x for x in roots)

Note there's no builtin function "sum". It's always annoyed me that
operator.add always takes exactly two arguments, so you have to use
reduce. I'd like to be able to say operator.add(2,3,4), or in the
generator expression case, operator.add(yield x*x for x in roots).
Jul 18 '05 #11
rzed wrote:
I like it! How about in conjunction with the print statement?
print "%s\n" % (s for s in strlist)
??


Presuming you want this to result in a series of print statements, this
sounds like a very bad idea. All the PEP proposed is a new way of
creating an anonymous generator that uses a syntax similar to that of
list comprehensions; it doesn't actually change the way generators
themselves are already handled by the language. Consider:
def f(): .... i, j = 0, 1
.... yield i
.... yield j
.... while True:
.... i, j = j, i + j
.... yield j
.... g = f() # g is now a generator
print g <generator object at 0x402e3bcc> print "%s" % g # this doesn't print the Fibonacci sequence

<generator object at 0x402e3bcc>

What you're suggesting would grievously sacrifice consistency at the
expense of a tiny little bit of convenience. It isn't worth it.

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
__ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
/ \ You and I / We've seen it all / Chasing our hearts' desire
\__/ The Russian and Florence, _Chess_
Jul 18 '05 #12
"rzed" <rz*****@ntelos.net> wrote in news:1066956387.364126@cache1:

"Raymond Hettinger" <py****@rcn.com> wrote in message
news:5d**************************@posting.google.c om...
Peter Norvig's creative thinking triggered renewed interest in PEP
289. That led to a number of contributors helping to re-work the pep
details into a form that has been well received on the python-dev
list:

http://www.python.org/peps/pep-0289.html

In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)
min(d.temperature()*9/5 + 32 for d in days)
Set(word.lower() for word in text.split() if len(word) < 5)
dict( (k, somefunc(k)) for k in keylist )
dotproduct = sum(x*y for x,y in itertools.izip(xvec, yvec))
bestplayer, bestscore = max( (p.score, p.name) for p in players )

Each of the above runs without creating a full list in memory,
which saves allocation time, conserves resources, and exploits
cache locality.

The new form is highly expressive and greatly enhances the utility
of the many python functions and methods that accept iterable
arguments.


I like it! How about in conjunction with the print statement?
print "%s\n" % (s for s in strlist)


Wouldn't work in general, i.e. in your version there isn't a way to
supply multiple elements. For a (completely?) general solution see the
Lisp FORMAT function, the ~{ @{ etc. format specifiers
(http://www.lisp.org/HyperSpec/Body/sec_22-3-7-4.html). Perhaps not the
most Pythonic? <grin>

-- bjorn
Jul 18 '05 #13
Christoph Becker-Freyseng <we*******@beyond-thoughts.com> wrote in
news:ma***********************************@python. org:

[...]


BTW: Python is getting more and more stuff. So I fear that some code
might get harder to read for newbies (not knowing all the new stuff).

Would it be possible to make an expression-/statement-parser that can
analyse a code snippet (one line of python-code maybe more) and tell the
user what syntax-constructs were used.

This might be a big help when reading code and if you don't know some
special construct.

e.g.:
newstuff.py:
...
sum(x*x for x in roots)
...
??? Mmmm What's that
pyscryer "sum(x*x for x in roots)"
<Function-Call>(<Generator Expression>)

Then you can search the Internet to close your knowledge gap (you can't
search for "sum(x*x for x in roots)", but "python Generator Expression"
might be succesful)


Something like:

compiler.parse("[x*x for x in roots]")

(not completely what you want, but close :-)

-- bjorn
Jul 18 '05 #14
Paul Rubin <http://ph****@NOSPAM.invalid> wrote in
news:7x************@ruckus.brouhaha.com:
py****@rcn.com (Raymond Hettinger) writes:
In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)

[..]
Note there's no builtin function "sum". It's always annoyed me that
operator.add always takes exactly two arguments, so you have to use
reduce.

[..]

time to upgrade?

f:\>python
Python 2.3 (#46, Jul 29 2003, 18:54:32) [MSC v.1200 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
sum(range(10))

45

-- bjorn
Jul 18 '05 #15
rzed wrote:
I like it! How about in conjunction with the print statement?
print "%s\n" % (s for s in strlist)


It's not better than

for s in strlist:
print s

or I didn't understand your intent.

Mike


Jul 18 '05 #16


Paul Rubin wrote:
py****@rcn.com (Raymond Hettinger) writes:
In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)

This is a little confusing. Consider:

1) [x*x for x in roots]
2) (x*x for x in roots)

#2 looks like it should be a "sequence comprehension" and not a
generator expression. I'd go back to your earlier proposal of
using a yield keyword if you want a generator expression:

sum(yield x for x in roots)

I too prefer the above use of yield because then you can always
associate generators with the yield keyword.

Pad.

Jul 18 '05 #17
Donald 'Paddy' McCarthy wrote:
I too prefer the above use of yield because then you can always
associate generators with the yield keyword.


I don't, because using 'yield' here would lead to greater confusion when
using generator expressions in generator functions.
yield [x for x in y] # Yields a list
yield x for x in y # Creates and discards a generator
yield yield x for x in y # Yields an iterator
--
Rainer Deyke - ra*****@eldwood.com - http://eldwood.com
Jul 18 '05 #18
Raymond Hettinger wrote:
Peter Norvig's creative thinking triggered renewed interest in PEP 289.
That led to a number of contributors helping to re-work the pep details
into a form that has been well received on the python-dev list:

http://www.python.org/peps/pep-0289.html

In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)
min(d.temperature()*9/5 + 32 for d in days)
Set(word.lower() for word in text.split() if len(word) < 5)
dict( (k, somefunc(k)) for k in keylist )
dotproduct = sum(x*y for x,y in itertools.izip(xvec, yvec))
bestplayer, bestscore = max( (p.score, p.name) for p in players )
Although most people on python-dev and here seems to like this PEP I
think it generally decreases readability. The above constructs seem
heavy and difficult to parse and thus i am afraid that they interfere
with the perceived simplicity of Python's syntax.

Sometimes it seems that introducing new syntax happens much easier
than improving or adding stdlib-modules.
Each of the above runs without creating a full list in memory,
which saves allocation time, conserves resources, and exploits
cache locality.


I do like these properties, though :-)

At least I hope that generator expressions will only be introduced
via a __future__ statement in Python 2.4 much like it happened with
'yield' and generators. Maybe the PEP should have an "implementation
plan" chapter mentioning such details?

cheers,

holger

Jul 18 '05 #19
Holger Krekel <py**@devel.trillke.net> writes:
At least I hope that generator expressions will only be introduced
via a __future__ statement in Python 2.4 much like it happened with


Why? Does it break backwards compatibility somehow?

As far as readability goes, genexps seem to be pretty much as
readable/unreadable as list comprehensions. To me, BTW, the idea looks
great. However, I'm not completely sure about allowing them to be
creatad w/o parens in function calls, seems a bit too magical for my
taste.

--
Ville Vainio http://www.students.tut.fi/~vainio24
Jul 18 '05 #20
Ville Vainio wrote:
Holger Krekel <py**@devel.trillke.net> writes:
At least I hope that generator expressions will only be introduced
via a __future__ statement in Python 2.4 much like it happened with
Why? Does it break backwards compatibility somehow?


OK ok. Probably only backwards readability :-)
As far as readability goes, genexps seem to be pretty much as
readable/unreadable as list comprehensions.


Yes, i don't like those much, either. But then i am one of this strange
minority which likes map, filter, lambda and passing functions/callables all
around.

But actually generators i do appreciate a lot so i'll probably not only
get used to generator expressions but will enjoy them (more than list
comprehensions at least, i guess).

cheers,

holger

Jul 18 '05 #21

"Raymond Hettinger" <py****@rcn.com> wrote in message
news:5d**************************@posting.google.c om...
Peter Norvig's creative thinking triggered renewed interest in PEP 289.
That led to a number of contributors helping to re-work the pep details
into a form that has been well received on the python-dev list:

http://www.python.org/peps/pep-0289.html

This seems a well thought out and useful enhancement to me.
Personally I like the syntax as proposed and would vote yes if this was the
kind of thing we got votes on!
Jul 18 '05 #22
Raymond Hettinger:

http://www.python.org/peps/pep-0289.html


I like it. This made me realize that list comprehension sytax is a
special case that never really needed to be special-cased.

In fact, list comprehensions themselves become redundant syntactic
sugar under the new proposal:

[foo(x) for x in bar] ==> list(foo(x) for x in bar)

Chris Perkins
Jul 18 '05 #23
+1

Raymond Hettinger wrote:
Peter Norvig's creative thinking triggered renewed interest in PEP 289.
That led to a number of contributors helping to re-work the pep details
into a form that has been well received on the python-dev list:


Jul 18 '05 #24
I too prefer the above use of yield because then you can always
associate generators with the yield keyword.


Rainer> I don't, because using 'yield' here would lead to greater
Rainer> confusion when using generator expressions in generator
Rainer> functions.

There's also the mental confusion about what the yield is doing. Is it
yielding a value back from the function containing the generator expression
or yielding values from the generator expression to the current function?
One of the earliest proposals on python-dev used the yield keyword. I fell
into this exact hole. It took a message or two from others participating in
the thread to extricate myself.

Skip

Jul 18 '05 #25

Ville> However, I'm not completely sure about allowing them to be
Ville> creatad w/o parens in function calls, seems a bit too magical for
Ville> my taste.

Think of parens and genex's the same way you'd think of parens and tuples.
Creating tuples often doesn't require parens either, but sometimes does in
cases where the syntax would be ambiguous). As arguments to function calls,
parens would be required to separate generator expressions from other
arguments. In the one arg case they aren't required (though you're free to
add them if it warms the cockles of your heart ;-).

Skip

Jul 18 '05 #26
sum(x*x for x in roots)
min(d.temperature()*9/5 + 32 for d in days)
Set(word.lower() for word in text.split() if len(word) < 5)
dict( (k, somefunc(k)) for k in keylist )
dotproduct = sum(x*y for x,y in itertools.izip(xvec, yvec))
bestplayer, bestscore = max( (p.score, p.name) for p in players )
Holger> Although most people on python-dev and here seems to like this
Holger> PEP I think it generally decreases readability. The above
Holger> constructs seem heavy and difficult to parse and thus i am
Holger> afraid that they interfere with the perceived simplicity of
Holger> Python's syntax.

Of course, you can do all of these things today by just turning the args
into list comprehensions:

sum([x*x for x in roots])
min([d.temperature()*9/5 + 32 for d in days])
Set([word.lower() for word in text.split() if len(word) < 5])
dict([(k, somefunc(k)) for k in keylist])
dotproduct = sum([x*y for x,y in itertools.izip(xvec, yvec)])
bestplayer, bestscore = max([(p.score, p.name) for p in players])

[regarding the proposed generator expressions, not my listcomp examples] Each of the above runs without creating a full list in memory, which
saves allocation time, conserves resources, and exploits cache
locality.


Which can be a sticking point for using list comprehensions. With generator
expressions in place, list comprehensions become just syntactic sugar for

list(generator expression)

Alex Martelli demonstrated some performance improvements (about 2-to-1 I
think) of generator expressions over the equivalent list comprehensions.

Holger> At least I hope that generator expressions will only be
Holger> introduced via a __future__ statement in Python 2.4 much like it
Holger> happened with 'yield' and generators. Maybe the PEP should have
Holger> an "implementation plan" chapter mentioning such details?

I think the plan is to make them available in 2.4. I don't think there's
any plan for a __future__ import because they won't change the semantics of
any existing constructs.

Skip

Jul 18 '05 #27
Raymond Hettinger revives pep 289:
Peter Norvig's creative thinking triggered renewed interest in PEP 289.
That led to a number of contributors helping to re-work the pep details
into a form that has been well received on the python-dev list:

http://www.python.org/peps/pep-0289.html

In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:


What do you get from:
type(x*x for x in roots)

I assume you'd get something like genexp that behaves like an iterator and
that the consuming function accesses items until StopIteration(StopGenxing?
;) is raised, and that a genexp is otherwise a first class object.

From the PEP:
"All free variable bindings are captured at the time this function is
defined, and passed into it using default argument values...[snip]...In
fact, to date, no examples have been found of code where it would be better
to use the execution-time instead of the definition-time value of a free
variable."

Does this imply then that instead of:
dumprec = "\n".join('%s : %s' % (fld.name, fld.val) for fld in rec)
for rec in file: print dumprec

one would need to write some variation of:
for rec in file:
print "\n".join('%s : %s' % (fld.name, fld.val) for fld in rec)

or:
def dumprec(rec):
return "\n".join('%s : %s' % (fld.name, fld.val) for fld in rec)
for rec in file: print dumprec(rec)

I find the first construct easier on the eyes, and it fulfills on the
promise of 'generator expression' as opposed to 'iterable generated
expression'. Although it does feel much more macro-ish...

Overall, +1.
Emile van Sebille
em***@fenx.com
Jul 18 '05 #28
sdd
John Burton wrote:
...{about PEP 289)...

This seems a well thought out and useful enhancement to me.
Personally I like the syntax as proposed and would vote yes
if this was the kind of thing we got votes on!


Ahhh, you do get a vote, but more by reason than numbers.
The PSF is watching this discussion with bated breath.
This whole discussion in c.l.p is part of examining the
merits of the PEP.

Don't tell anyone that I mentioned the inner workings of
the PSF cabal, or ..................................~}~~}

Jul 18 '05 #29
ch************@hotmail.com (Chris Perkins) wrote in message news:<45**************************@posting.google. com>...
Raymond Hettinger:

http://www.python.org/peps/pep-0289.html


I like it. This made me realize that list comprehension sytax is a
special case that never really needed to be special-cased.

In fact, list comprehensions themselves become redundant syntactic
sugar under the new proposal:

[foo(x) for x in bar] ==> list(foo(x) for x in bar)

Chris Perkins


.... and here I proclaim list comprehension deprecated!

;) M.
Jul 18 '05 #30
"Emile van Sebille" <em***@fenx.com> writes:
[...]
What do you get from:
type(x*x for x in roots)

[...]

<type 'function'>?
John
Jul 18 '05 #31
In article <87************@pobox.com>, jj*@pobox.com (John J. Lee)
wrote:
"Emile van Sebille" <em***@fenx.com> writes:
[...]
What do you get from:
type(x*x for x in roots)

[...]

<type 'function'>?


<type 'generator'> actually:
def foo(): yield 1 .... type(foo) <type 'function'> type(foo()) <type 'generator'>


Just
Jul 18 '05 #32
On 23 Oct 2003 12:58:31 -0700, py****@rcn.com (Raymond Hettinger) wrote:
Peter Norvig's creative thinking triggered renewed interest in PEP 289.
That led to a number of contributors helping to re-work the pep details
into a form that has been well received on the python-dev list:

http://www.python.org/peps/pep-0289.html

In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)
min(d.temperature()*9/5 + 32 for d in days)
Set(word.lower() for word in text.split() if len(word) < 5)
dict( (k, somefunc(k)) for k in keylist )
dotproduct = sum(x*y for x,y in itertools.izip(xvec, yvec))
bestplayer, bestscore = max( (p.score, p.name) for p in players )

Each of the above runs without creating a full list in memory,
which saves allocation time, conserves resources, and exploits
cache locality.

The new form is highly expressive and greatly enhances the utility
of the many python functions and methods that accept iterable arguments.

+1

Regards,
Bengt Richter
Jul 18 '05 #33

daniels> Don't tell anyone that I mentioned the inner workings of the
daniels> PSF cabal, or ..................................~}~~}

You're confusing the PSF (which does exist) with the PSU (whi

Jul 18 '05 #34
Hi,
Rainer Deyke wrote:

At the very least, it would require parentheses if you want to create a list
containing exactly one such generator.

[(x for x in y)] # List containing generator
[x for x in y] # List comprehension


personally the parentheses seem a little strange to me, or put another
way, I feel that simple parentheses around "everything" should not
change the meaning of the code.

For example, see current tuple syntax:
def p(x): .... print type(x)
.... p(5) <type 'int'> p((5)) <type 'int'> p((5,))

<type 'tuple'>
As the code shows, just putting parentheses around the value 5 doesn't
make it a tuple.
Another point is that I think it's one of Python's advantages to not
require extensive parentheses e. g. for 'if' 'while' 'for' etc.

Unlike in C you can just write

if a > 7:
print "Large enough."
So requiring parentheses around the only item for a list with one
generator in it seems a little inconsistent in my eyes.

Maybe an alternative would be to have similar to tuples:

[x for x in y, ] # List containing generator
[x for x in y] # List comprehension
Just my 2c

Werner

Jul 18 '05 #35
Holger Krekel wrote (answering Raymond Hettinger):
...
sum(x*x for x in roots) ... bestplayer, bestscore = max( (p.score, p.name) for p in players )

Actually I think RH meant "bestscore, bestplayer = ..." here...
Although most people on python-dev and here seems to like this PEP I
think it generally decreases readability. The above constructs seem
heavy and difficult to parse and thus i am afraid that they interfere
with the perceived simplicity of Python's syntax.
Let's see: today, I could code:
sum([x*x for x in roots])
In 2.4, I will be able to code instead:
sum(x*x for x in roots)
with some performance benefits. Could you please explain how using
the lighter-weight simple parentheses rather than today's somewhat
goofy ([ ... ]) bracketing "generally decreases readability"...?

Similarly, today's:
bestscore, bestplayer = max([ (p.score, p.name) for p in players ])
I'll be able to code as a somewhat faster:
bestscore, bestplayer = max( (p.score, p.name) for p in players )
Again, I don't see how the proposed new construct is any more "heavy
and difficult to parse" than today's equivalent -- on the contrary,
it seems somewhat lighter and easier to parse, to me.

It may be a different issue (as you appear, from a later message, to
be particularly enamoured of Python's current 'fp' corner) to claim
that, say, yesteryear's (and still-usable):

reduce(operator.add, map(lambda x: x*x, roots))

"increases readability" compared to the proposed

sum(x*x for x in roots)

It boggles my mind to try thinking of the latter as "heavy and
difficult to parse" compared to the former, to be honest. And
when I'm thinking of "the sum of the squares of the roots", I
DO find the direct expression of this as sum(x*x for x in roots)
to be most immediate and obvious compared to spelling it out
as
total = 0
for x in roots:
total = total + x*x
which feels more like a series of detailed instructions on
how to implement that "sum of squares", while the "sum(x*x"...
DOES feel like the simplest way to say "sum of squares".

Sometimes it seems that introducing new syntax happens much easier
than improving or adding stdlib-modules.
What a weird optical illusion, hm? In Python 2.3 *NO* new syntax
was introduced (the EXISTING syntax for extended slices is now
usable on more types, but that's not "introducing" any NEW syntax
whatsoever), while HUNDREDS of changes were done that boil down
to "improving or adding standard library modules". So, it's
self-evidently obvious that the "happens much easier" perception
is pure illusion. In 2.4, _some_ syntax novelties are likely to
be allowed -- most PEPs under consideration are already in the
PEP repository or will be there shortly, of course (e.g., syntax
to support some variation of PEP 310, possibly closer to your
own interesting experiments that you reported back in February).
But once again there is absolutely no doubt that many more
changes will be to standard library modules than will "introduce
new syntax".

At least I hope that generator expressions will only be introduced
via a __future__ statement in Python 2.4 much like it happened with
'yield' and generators. Maybe the PEP should have an "implementation
plan" chapter mentioning such details?


I don't think __future__ has EVER been used for changes that do
not introduce backwards incompatibilities, nor do I see why that
excellent tradition should be broken for this PEP specifically.
Alex

Jul 18 '05 #36
Werner Schiendl wrote:
...
personally the parentheses seem a little strange to me, or put another
way, I feel that simple parentheses around "everything" should not
change the meaning of the code. ... As the code shows, just putting parentheses around the value 5 doesn't
make it a tuple.
No, but, in all cases where the lack of parentheses would make the
syntax unclear, you do need parentheses even today to denote tuples.
Cfr:
[1,2,3]
versus
[(1,2,3)]
a list of three items, versus a list of one tuple.
Maybe an alternative would be to have similar to tuples:
The PEP *IS* "similar to tuples": parentheses are mandatory around a
generator expression wherever their lack would make things unclear
(and that's basically everywhere, except where the genexp is the
only argument to a function call). Hard to imagine a simpler rule.

The idea of "adding a comma" like this:
[x for x in y, ] # List containing generator


feels totally arbitrary and ad-hoc to me, since there is NO other
natural connection between genexps and commas (quite differently
from tuples). Moreover, it would not help in the least in other
VERY similar and typical cases, e.g.:

[x for x in y, z,] # ok, now what...?

today this means [y, z]. Would you break backwards compatibility
by having it mean a [<genexp>] instead? Or asking for a SECOND
trailing comma to indicate the latter? And what about a genexp
followed by z, and, ... ?

No, really, this whole "trailing comma to indicate a genexp in one
very special case" idea is sort of insane. Compare with:

[x for x in y, z,] # same as [y, z]

[(x for x in y, z,)] # the [<genexp>] case

[(x for x in y), z,] # the [<genexp>, z] case

what could possibly be clearer? Not to mention the parens rule
is simple, universally applicable, breaks no backwards compat,
AND is far closer to the general case for tuples ("parens when
needed for clarity") than the "singleton tuple" case you seem
to have focused on.
Alex

Jul 18 '05 #37
Daniel Dittmar wrote:
Raymond Hettinger wrote:
bestplayer, bestscore = max( (p.score, p.name) for p in players )

Each of the above runs without creating a full list in memory,
which saves allocation time, conserves resources, and exploits
cache locality.
Why should generator comprehension be useful only for arguments?


Nobody ever said it would be. RH's examples were all function
calls because that's an "obvious" use case.

counter proposal:
g[(p.score, p.name) for p in players ]
Looks like it's indexing g.
This works similar to r'\' for raw strings and u'x' for unicode strings.
Nope, it's ambiguous, while <single letter> adjacent to '...' isn't.
This has the advantage that it can be more easily extended:
d[(p.name, p.score) for p in players] creates a dictionary
dict((p.name, p.score) for p in players)

will do it in a much more obvious way.
s[(p.score, p.name) for p in players] creates a sorted list
list.sorted((p.score, p.name) for p in players)

will do it in a much more obvious way, and also allow you to
specify such optional niceties as (e.g.) "key=" (in 2.4 --
and, btw, thanks to all of RH's work on the issue!).
and
p[$*-=%/sd/!] allows to embed perl code


If you want perl, you know where to find it.
Alex

Jul 18 '05 #38
Alex Martelli wrote:
No, but, in all cases where the lack of parentheses would make the
syntax unclear, you do need parentheses even today to denote tuples.
Cfr:
[1,2,3]
versus
[(1,2,3)]
a list of three items, versus a list of one tuple.
ok, but that's the whole point (1,2,3) *is* a tuple.

but

[(1),(2),(3)]

is the same than

[1,2,3]

and

[(1)]

is the same than

[1]

if you want a 1-tuple in a list, you've to write

[(1,)]

Maybe an alternative would be to have similar to tuples:

The PEP *IS* "similar to tuples": parentheses are mandatory around a
generator expression wherever their lack would make things unclear
(and that's basically everywhere, except where the genexp is the
only argument to a function call). Hard to imagine a simpler rule.


With "similar to tuples" I meant the kind of indication that I want
a list with exactly one item that is a generator.
The idea of "adding a comma" like this:

[x for x in y, ] # List containing generator

feels totally arbitrary and ad-hoc to me, since there is NO other
natural connection between genexps and commas (quite differently
from tuples). Moreover, it would not help in the least in other
VERY similar and typical cases, e.g.:

[x for x in y, z,] # ok, now what...?


That's a reasonable point of course.

To be honest, I didn't consider that as I always put the items to
iterate over in an explicit list or tuple or have 'em already inside one.

There is no way to judge what's meant in that case, it's a little
unfortunate that this *is* currently allowed.

So within a list literal, the parentheses are required *sigh* since the
default would need to be that following values are the values to iterate
over.
No, really, this whole "trailing comma to indicate a genexp in one
very special case" idea is sort of insane. Compare with:

[x for x in y, z,] # same as [y, z]

[(x for x in y, z,)] # the [<genexp>] case

[(x for x in y), z,] # the [<genexp>, z] case

what could possibly be clearer? Not to mention the parens rule
is simple, universally applicable, breaks no backwards compat,
AND is far closer to the general case for tuples ("parens when
needed for clarity") than the "singleton tuple" case you seem
to have focused on.


The trailing comma rule *is* there in the language already.

The problem with my idea is with the possibility of a tuple without
parentheses in a list comprehension (as discussed above).
Still I do not like the follwing example from the specification:

g = (x**2 for x in range(10))

Whatfor are the parens?

To me this looks like requiring parens for

a = b + c
best regards

Werner

Jul 18 '05 #39
Werner Schiendl <n1*******************@neverbox.com> wrote in
news:3f******@brateggebdc5.br-automation.co.at:
For example, see current tuple syntax:
def p(x): ... print type(x)
... p(5)<type 'int'> p((5))<type 'int'> p((5,))<type 'tuple'>
As the code shows, just putting parentheses around the value 5 doesn't
make it a tuple.


Just putting parentheses around something can change the type, if you pick
the right example:
def p(x): print type(x)

p(5,) <type 'int'> p((5,)) <type 'tuple'>
And even closer to the list comprehension vs list containing generator
example:
[5,] [5] [(5,)] [(5,)]


--
Duncan Booth du****@rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?
Jul 18 '05 #40
Alex Martelli wrote:
counter proposal:
g[(p.score, p.name) for p in players ]


Looks like it's indexing g.
This works similar to r'\' for raw strings and u'x' for unicode
strings.


Nope, it's ambiguous, while <single letter> adjacent to '...' isn't.


Although I admit my proposal is flawed, the principle behind is is sound,
namely that syntax constructs should be recognisable at the start.
p[$*-=%/sd/!] allows to embed perl code


If you want perl, you know where to find it.


See how difficult it is even for you to recognize something if it isn't
introduced by the proper markers, in this case <sarkasm>?

Daniel

Jul 18 '05 #41
Alex Martelli wrote:
Although most people on python-dev and here seems to like this PEP I
think it generally decreases readability. The above constructs seem
heavy and difficult to parse and thus i am afraid that they interfere
with the perceived simplicity of Python's syntax.


Let's see: today, I could code:
sum([x*x for x in roots])
In 2.4, I will be able to code instead:
sum(x*x for x in roots)


Yes, Raymond send me some private mail already and i clarified
that i don't like list comprehension syntax as well :-)

Actually i do like generator expressions *more* than list comprehensions
so i agree to your points that gen expressions do improve on the (rather
specific) list comprehensions.

Nevertheless, i have the impression that this kind of syntax (be it list
comprehension or generator expressions) somewhat decreases the
chances for "immediate" understanding of python code without knowing
the language much. It's likely that you need to parse generator
expressions in detail especially because they don't provide sharp
visual clues like indented blocks do. I do consider the "visual clues"
to be a strength of current python although you can always provide
examples with a language's syntax which are not readable.

Generator expressions will probably raise the barrier to understanding
python code without having appropriate knowledge but that may very well
prove to be worth it. We'll probably find out soon so i think we don't
need to argue a lot about it now. IOW, i am not completly opposed
to the PEP just a bit skeptical. And, btw, i appreciated very much,
that Guido made it clear from the start that Python 2.3 would be a
maintenance release with no changes to syntax. I am probably just
astonished at these huge syntax-threads all over python-dev :-)

have fun,

holger

Jul 18 '05 #42
On Monday 27 October 2003 04:52 pm, Holger Krekel wrote:
Alex Martelli wrote:
Although most people on python-dev and here seems to like this PEP I
think it generally decreases readability. The above constructs seem
heavy and difficult to parse and thus i am afraid that they interfere
with the perceived simplicity of Python's syntax.
Let's see: today, I could code:
sum([x*x for x in roots])
In 2.4, I will be able to code instead:
sum(x*x for x in roots)


Yes, Raymond send me some private mail already and i clarified
that i don't like list comprehension syntax as well :-)

Actually i do like generator expressions *more* than list comprehensions
so i agree to your points that gen expressions do improve on the (rather
specific) list comprehensions.


And therefore, since you ain't gonna take list comprehensions away,
you should be in favour, not against, generator expressions.

Personally, I agree with David Ascher's evaluation in the intro to Chapter 1
of "Python Cookbook" (end of p.2): "List comprehensions have clearly
become wildly successful". I have seen non-programmers just learning
programming and Python together "GET" list comprehensions AT ONCE,
particularly if they had ever seen typical set notation as used in maths,
but not only under such conditions. I do agree with you that generator
expressions are a further improvement (except in name -- sigh -- I wish
"iterator expressions" had prevailed instead!-).

Nevertheless, i have the impression that this kind of syntax (be it list
comprehension or generator expressions) somewhat decreases the
chances for "immediate" understanding of python code without knowing
the language much. It's likely that you need to parse generator
I've had substantial feedback from articles on Python I had written,
particularly on Italian magazines, and much of it was from people who
indeed don't know the language much (or at all). That feedback tells
me that list comprehensions were absolutely nothing like the obstacle
you fear, confirming my experiences with newbies. The things that DID
give non-Python-knowers _serious_ trouble were rather e.g.:

while <condition>:
<body>
else:
<something else>

NOBODY who doesn't have a good grasp of Python gets "immediate
understanding" of this -- *not one person* out of all I've talked about
this. A large majority imagines that the "else:" clause is executed if
the <condition> NEVER holds, the remaining minority just doesn't
get it at all, NOBODY guesses that this may have to do with whether
a "break" inside the <body> has or hasn't executed.
expressions in detail especially because they don't provide sharp
visual clues like indented blocks do. I do consider the "visual clues"
to be a strength of current python although you can always provide
examples with a language's syntax which are not readable.
Oh, the above example is quite readable and rich with visual clues --
it's just that its _semantics_ are totally anti-intuitive to all readers who
haven't studied Python pretty well. On the contrary, and I wish there
was a way to make a bet on this and settle it objectively, I claim e.g.:

print sum(x*x for x in thelist if x>10)

will be "immediately read" as "print the sum of the squares of those
elements of 'thelist' that are larger than 10" by a majority of the
readers (if anything, understandable confusion will come about the
fact that "print" has nothing whatsoever to do with the *printer*
that one may or may not have attached to one's computer, but
rather will normally display stuff on a terminal-window... the mental
disconnection of "print" from "printer" is NOT intuitive at all -- it just
SEEMS that way after one has been programming long enough!).

Of course, more complicated expressions will be harder to read --
but that's quite independent of whether those expressions will
be generator expressions or other kinds.

Generator expressions will probably raise the barrier to understanding
python code without having appropriate knowledge but that may very well
prove to be worth it. We'll probably find out soon so i think we don't
need to argue a lot about it now. IOW, i am not completly opposed
to the PEP just a bit skeptical.
We'll find out in, I suspect, well over a year (when 2.4 final is released
and gets widespread). And we'll "find out" on a purely anecdotal and
experiential basis, just as we have "found out" about list comprehensions
over the last few years, unless somebody has money burning in their
pocket to finance a systematic study, which I seriously doubt:-).
And, btw, i appreciated very much,
that Guido made it clear from the start that Python 2.3 would be a
maintenance release with no changes to syntax. I am probably just
So did I! Great move.
astonished at these huge syntax-threads all over python-dev :-)


I was anything but astonished -- a bunch of language enthusiasts
stopped from any _productive_ discussion of syntax changes for,
what, over two years (since the last alpha of 2.2 until now)...?! It's
not amazing that we'd be bubbling over now we have a chance...:-).

Guido (with support from the usual adverse-to-any-change brigade)
will no doubt exert his usual veto right with the usual firmness, so
all of our threads are little more than a venting out of too-long
repressed energy, anyway:-).
Alex
Jul 18 '05 #43
Bjorn Pettersen wrote:
Christoph Becker-Freyseng <we*******@beyond-thoughts.com> wrote in
news:ma***********************************@python. org:

[...]

Something like:

compiler.parse("[x*x for x in roots]")

(not completely what you want, but close :-)

-- bjorn


:-)

Unfortunately I don't have time right now to play around with Pythons
Compiler/Parser as my Biophysics-study is quite time consuming now ...

cu Christoph Becker-Freyseng
(I've been offline for some days due to an failure of telekom.de)
Jul 18 '05 #44
Daniel Dittmar wrote:
Alex Martelli wrote:
counter proposal:
g[(p.score, p.name) for p in players ]


Looks like it's indexing g.
This works similar to r'\' for raw strings and u'x' for unicode
strings.


Nope, it's ambiguous, while <single letter> adjacent to '...' isn't.


Although I admit my proposal is flawed, the principle behind is is sound,
namely that syntax constructs should be recognisable at the start.


Avoiding lookahead is a good principle, but not an absolute.

For example, when I see a literal that starts with the characters:

1233445

I do NOT start grumbling that, since it violates the Dittmar Principle,
I'm desperate by not yet knowing after reading the first few characters
whether it's going to be an integer literal, such as

123344567

or a float literal, such as

1233445.67

Your "should" is just too weak to make this notation other than
perfectly fine. A language designed strictly by your principle, thus
forcing initial stropping of digit sequences to distinguish e.g.
integer literals from float ones, would be a syntax disaster.

p[$*-=%/sd/!] allows to embed perl code


If you want perl, you know where to find it.


See how difficult it is even for you to recognize something if it isn't
introduced by the proper markers, in this case <sarkasm>?


"Introduced" matters not a whit (just like the fact that, in English,
"sarcasm" is spelled with a c and not with a k). If you had chosen to
indicate your dubious sarcasm (against your own proposal?!) by a
trailing parenthetical such as "(just kidding)" I would not have
wasted the time to respond. Don't take the _complete_ lack of any
clarifying denotation (which is definitely a stupid choice) obscuring
the message, as any proof that the clarifying denotation must be
BEFORE rather than AFTER -- it just doesn't follow. As your proposal
DOES look distinctly perlish (cfr q/foo/ vs qq/foo/ etc), it's quite
reasonable to form the working hypothesis that you LIKE perl syntax,
rather than being engaged in sarcasm against your own proposals (which
is a form of split personality one rarely comes upon).
Alex

Jul 18 '05 #45
Werner Schiendl wrote:
Alex Martelli wrote:
No, but, in all cases where the lack of parentheses would make the
syntax unclear, you do need parentheses even today to denote tuples.
Cfr:
[1,2,3]
versus
[(1,2,3)]
a list of three items, versus a list of one tuple.
ok, but that's the whole point (1,2,3) *is* a tuple.


It's a tuple in parentheses.

The parentheses are only necessary when the syntax would otherwise
be unclear, and THAT is the whole point.
x = 1, 2, 3
type(x)

<type 'tuple'>

See? "Look ma, no parentheses", because none are needed in the
assignment to x -- the tuple, by itself, is parentheses-less,
though people often (understandably) put redundant parentheses
around it anyway, just in case (so do repr and str when you
apply them to a tuple object, again for clarity).

Normally, tuples are indicated by commas; but when commas could
mean something else (list displays, function calls), then you
ALSO put parentheses around the tuple and thus clarify things
(you also must use parentheses to indicate an EMPTY tuple,
because there is no comma involved in that case).

if you want a 1-tuple in a list, you've to write

[(1,)]
Yes, and if you write [1,] WITHOUT the parentheses, that's
a list of one number, NOT a list of one tuple -- exactly my
point.
With "similar to tuples" I meant the kind of indication that I want
a list with exactly one item that is a generator.
So you parenthesize the one item to disambiguate, just like you
would parenthesize the one item to disambiguate if the one item
was a tuple such as
1,
rather than a genexp such as
x*x for x in foo

I.e., the proposed syntax, requiring parentheses, IS strictly
similar to the identical requirement for parentheses in tuples
(when either a tuple or a genexp happens to be the one item
in a list display).

To assign a one-item tuple to x you can write

x = 23,

the comma is what makes the tuple, and parentheses have nothing
to do with the case. You can of course choose to add redundant
parentheses, if you think they improve readability, just like
you can generally include redundant parentheses in just about
all kinds of expressions for exactly the same reason (e.g. you
fear a reader would not feel secure about operator priorities).

The difference between tuples and the proposed syntax for
genexps is that genexp are going to _require_ parentheses
around them -- unless the parentheses already "happen" to be
there, i.e., in the frequent case in which the genexp is the
only argument to a function or type call. Basically, that's
because, even though the _compiler_ would have no problems,
Guido judges that a _human reader_ might, in parsing

x = y for y in foo

as meaning

x = (y for y in foo)

and NOT the incorrect

(x = y) for y in foo

So the parentheses are mandated to make it clear beyond any
doubt that the former, NOT the latter, parse is meant.

The trailing comma rule *is* there in the language already.
A trailing comma is allowed in any "comma-separated list".
A tuple display is a comma-separated list.
Therefore, a tuple display allows a trailing comma.

[A SINGLETON tuple requires a trailing comma because commas
are what DEFINES that a tuple display is occurring -- except
in certain contexts such as list displays and function calls,
where the commas have another meaning, and for an empty
tuple, where commas are -- perhaps arbitrarily -- disallowed]

A genexp is not a comma-separated list, and has nothing to do
with comma-separated lists, therefor it would be entirely
arbitrary and capricious to inject commas in its syntax AT ALL.

Still I do not like the follwing example from the specification:

g = (x**2 for x in range(10))

Whatfor are the parens?
See above: to avoid humans misparsing the unparenthesized form
as the incorrect (g = x**2) for ...

To me this looks like requiring parens for

a = b + c


If there was any risk of humans misparsing this as the
incorrect (a = b) + c -- then parentheses around the RHS
might be prudent. Fortunately, in this case there is no
such risk. Generator expressions are novel enough that
Guido judges the risk would exist in their case, and most
particularly since assigning a genexp to a variable will be
extremely rare usage (could you point to a few compelling
use cases for it...?) -- therefore it's absolutely NOT a
bad idea to force the author of code for that rare case to
type a couple of extra characters to make his intentions
absolutely crystal-clear to human readers, who can easily
be unfamiliar with that rare case and will thus be helped.

The "a = b + c" case is exactly the opposite: it IS very
common, therefore [1] typical readers don't need help
parsing it because it's familiar to them AND [2] forcing
parentheses in a very common case would be cumbersome
(while it is not to force them in a very rare case).
Alex

Jul 18 '05 #46
Raymond Hettinger wrote:

Peter Norvig's creative thinking triggered renewed interest in PEP 289.
That led to a number of contributors helping to re-work the pep details
into a form that has been well received on the python-dev list:

http://www.python.org/peps/pep-0289.html


+1

But to add to the naming argument, I'd like them called comprehension generators.

ta,
Tom
Jul 18 '05 #47
Alex Martelli wrote:
reasonable to form the working hypothesis that you LIKE perl syntax,
rather than being engaged in sarcasm against your own proposals (which
is a form of split personality one rarely comes upon).


I don't like list comprehension when compared to Smalltalk code blocks,
so I was really looking for a way to keep any additional 'comprehension'
variants limited to one syntax rule. But it seems as if the current
trend in Python is toward succinctness. To paraphrase one poster: 'It is
documented, therefor it is explicit'. I try to keep quiet, enjoy the
parts of Python I like and wait for the tide to turn.

Daniel

Jul 18 '05 #48
Alex Martelli wrote:
Werner Schiendl wrote:
Alex Martelli wrote:
No, but, in all cases where the lack of parentheses would make the
syntax unclear, you do need parentheses even today to denote tuples.
Cfr:
[1,2,3]
versus
[(1,2,3)]
a list of three items, versus a list of one tuple.
ok, but that's the whole point (1,2,3) *is* a tuple.


It's a tuple in parentheses.


I've never bought the argument that "comma's make the tuple." All the
explanations I've seen seem to be a posteriori rationalizations by
people who are being a bit too "clever." (... no offense intended Alex.)

I base my interpretation of this on the fact that (as I understand it)
the comma explanation was brought up *after* the relevant semantics of
Python were coded.

To me, tuple packing/unpacking is a special case, not a general feature
of tuple notation. I claim as evidence the recent c.l.p thread about
"for d[n] in mylist:", etc. the ability to use a comma separated list in
such situations is a special case - it isn't *really* a tuple, the fact
that it is a tuple is an implementation detail. (Note that it may be a
*required* implementation detail, but that's neither here or there ...)

Of course, I'll change my opinion if Guido comes out and says that he
consciously intended all along for tuples to be defined by commas, and
the parenthesis were just there to disambiguate - but my guess is that
it's instead convergent evolution of separate ad hoc devices.
The parentheses are only necessary when the syntax would otherwise
be unclear, and THAT is the whole point.
"In the face of ambiguity, refuse the temptation to guess."

I'd say that by allowing for some instances the ability for tuples to go
around naked, the unparenthesized form is *still* ambiguous. The
argument "Well, for this case it can't be a tuple, because if it *was* a
tuple then it would be ambiguous" makes my head hurt. It seems a little
disingenuous to define language semantics by defining what something
*isn't*.
The difference between tuples and the proposed syntax for
genexps is that genexp are going to _require_ parentheses
around them -- unless the parentheses already "happen" to be
there, i.e., in the frequent case in which the genexp is the
only argument to a function or type call. Basically, that's
because, even though the _compiler_ would have no problems,
Guido judges that a _human reader_ might.


I was going to write a big rant on the fact that I have difficulties in
reading the genex as a "chunk", and how there should be some sort of
bracketing required, but Guido used/uses/will use his time machine to
deflate 9/10 of my argument.

So let me see if I have this correct:

(Long Answer)

Genex's are required to have parenthesis around them except when they
would yield double parenthesis i.e. ((...)) - and the only place where
this is relevant is in function calls with one argument, or with
creating a tuple. (As genex's as base class lists and as a parameter
list would be illegal.) But with creating a tuple, you have to have a
comma to create a singleton, so that case doesn't matter either.

As a parenthetical (sic.) note, the only other cases of 'double
brackets' are {(...)} and [(...)], as python doesn't use <...> as
brackets. The first is illegal anyway - you can't create a dictionary
with just a genex. The second is covered in the PEP - that is:

"""
[x for x in S] # This is a list comprehension.
[(x for x in S)] # This is a list containing one generator
# expression.
"""
*and* the former is depreciated in favor of list(<genex>).

(Short Answer)

Parenthesis are required around genex's except when they are the *only*
parameter to a function.

(Tuples?)

My only other concern is with tuples without parenthesis in the single
parameter to a function case, as they might be confused with multiple
parameter lists.

As I understand, the genex syntax is:

expression1 "for" expression2 "in" expression3a
or
expression1 "for" expression2 "in" expression3b "if" expression4

If expression1 is a tuple, it is required to have parenthesis. (Current
behavior of list comprehensions.)

If expression2 is a tuple, it can have parenthesis absent. (As it is
delimited between for and in.) Is this the same for genex's?

If expression3a is a tuple, it will be required to have parenthesis in
genex's. (They are currently optional for list comps.)

What about a tuple for expression3b? Are parenthesis required here?

Although it doesn't really make much sense, expression4 can be a tuple.
The PEP doesn't say, but am I correct in assuming that in such a
condition, expression4 would require parenthesis? (Parenthesis are
required for list comps, so I'm guessing the answer is yes.)

(Current Final short answer)

Parenthesis are required around genex's except when they are the *only*
parameter to a function. If either the first or last expression in a
genex is a tuple, they must have parenthesis around them.

....

I was against genex's without brackets, but with the requirements Guido
added, I'm not quite sure about it. My guess is that it's a "time will
tell" situation. I'll pass the buck to Guido.

-Rocco

Jul 18 '05 #49
PEP 289 proposes stuff like
sum(x*x for x in roots)
min(d.temperature()*9/5 + 32 for d in days)
Set(word.lower() for word in text.split() if len(word) < 5)
dict( (k, somefunc(k)) for k in keylist )
dotproduct = sum(x*y for x,y in itertools.izip(xvec, yvec))
bestplayer, bestscore = max( (p.score, p.name) for p in players )

and it seems that it's been accepted.

I meekly accept the judgement of BDFL, but have a sneaking suspicion
that we're on the way to re-inventing the LOOP macro here!

I can see the appeal, but fear where it might lead us ...

cheers

Chris Wright
Jul 18 '05 #50

This thread has been closed and replies have been disabled. Please start a new discussion.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.