map/filter/reduce/lambda opinions and background unscientificmini-survey

"iK" <Ik@sdsfsfd.com> writes:

Seems like he wants python programmers to solve their problems all in the
same way. While that is great for corporate slaves it is terrible for the
creative programmer.
No, he wants Python to be Pythonic. TMTOWTDI is not Pythonic.
Python is quickly becoming the visual basic of the 21 century. If you want
to have fun while getting some work done you need to look elsewhere. It's a
shame...

If you'd rather spend your time figuring out which of multiple ways to
do things is the best for the job at hand than producing code, there's
a language that makes TMTOWTDI a way of life.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

Jul 19 '05 #9

Tom Anderson wrote:

So, if you're a pythonista who loves map and lambda, and disagrees with
Guido, what's your background? Functional or not?

I'm familiar with several function languages but haven't used them
extensively; I was primarily a C++ programmer before I found Python. I
definitely use lambda, map, filter, and reduce, and will miss them when
they're gone.

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Heaven ne'er helps the man who will not act.
-- Sophocles

Jul 19 '05 #10

Ron Adam

Tom Anderson wrote:

So, if you're a pythonista who loves map and lambda, and disagrees with
Guido, what's your background? Functional or not?

I find map too limiting, so won't miss it. I'm +0 on removing lambda
only because I'm unsure that there's always a better alternative.

So what would be a good example of a lambda that couldn't be replaced?

Cheers,
Ron
BTW... I'm striving to be Pythonic. ;-)

Jul 19 '05 #11

Sean McIlroy

Tom Anderson wrote:
<snip>

So, if you're a pythonista who loves map and lambda, and disagrees with
Guido, what's your background? Functional or not?

glad you asked. personally i don't know lisp (or scheme), but now i've
decided to learn it, because eventually it will no longer be possible
in python to pass functions as arguments or return them as values. the
education sig will have to change its motto to "computer programming
for every C-programmer". until then hangers-on like myself can use
home-grown substitutes for the functional constructs (examples below),
but in my opinion the best thing is to migrate as soon as possible. the
real programmers are squeezing us out. now is the time to abandon
python for an intelligent language (macros! real conditional evaluation
instead of the and/or kluge!)

def LISTCOMP(f,s,g):
reval = []
for x in s:
if g(x):
reval.append(f(x))
return reval

def LAMBDA(arguments,value):
symbols = arguments.split(',')
def reval(*args):
for i in range(len(args)):
locals()[symbols[i]] = args[i]
return eval(value)
return reval

def MAP(f,s):
return LISTCOMP(f,s,LAMBDA('x','True'))

def FILTER(f,s):
return type(s)(LISTCOMP(LAMBDA('x','x'),s,f))

def REDUCE(f,s,t):
if not s: return t
return f(s[0],REDUCE(f,s[1:],t))

Jul 19 '05 #12

Terry Hancock

On Friday 01 July 2005 03:36 pm, Ron Adam wrote:

I find map too limiting, so won't miss it. I'm +0 on removing lambda
only because I'm unsure that there's always a better alternative.
Seems like some new idioms would have to be coined, like:

def my_function(a1, a2):
def _(a,b): return a+b
call_a_lib_w_callback(callback=_)

doesn't seem too bad, and defeats the "wasting time thinking of names"
argument.
So what would be a good example of a lambda that couldn't be replaced?

I suspect the hardest would be building a list of functions. Something
like:

powers = [lambda a, i=i: a**i for i in range(10)]

which you might be able to make like this:

powers = []
for i in range(10):
def _(a,i=i): return a**i
powers.append(_)

which works and is understandable, but a bit less concise.

The main obstacle to the lambda style here is that def statements
are not expressions. I think that's been proposed as an alternative,
too -- make def return a value so you could say:

powers = [def _(a,i=i): return a**i for i in range(10)]

Personally, I think this is understandable, and given that lambda
is to be pulled, a nice substitute (I would say it is easier to read
than the current lambda syntax, and easier for a newbie to
understand).

But it would probably encourage some bad habits, such as:

myfunc = def _(a,b):
print a,b
return a+b

which looks too much like Javascript, to me, where there are
about three different common idioms for defining a
function (IIRC). :-/

--
Terry Hancock ( hancock at anansispaceworks.com )
Anansi Spaceworks http://www.anansispaceworks.com

Jul 19 '05 #13

John Roth

"Tom Anderson" <tw**@urchin.earth.li> wrote in message
news:Pi*******************************@urchin.eart h.li...

Comrades,

During our current discussion of the fate of functional constructs in
python, someone brought up Guido's bull on the matter:

http://www.artima.com/weblogs/viewpost.jsp?thread=98196

He says he's going to dispose of map, filter, reduce and lambda. He's
going to give us product, any and all, though, which is nice of him.

What really struck me, though, is the last line of the abstract:

"I expect tons of disagreement in the feedback, all from ex-Lisp-or-Scheme
folks. :-)"

I disagree strongly with Guido's proposals, and i am not an
ex-Lisp, -Scheme or -any-other-functional-language programmer; my only
other real language is Java. I wonder if i'm an outlier.

So, if you're a pythonista who loves map and lambda, and disagrees with
Guido, what's your background? Functional or not?
As far as biases is concerned: my background is assembler.
I've never learned Lisp (although I did learn Forth at one time.)

I think I could note that originally there were five of the things,
including apply, which was cleanly replaced by the * and **
notation in function definitions and calls.

Map, filter and reduce are three things one can do with lists
(or in fact more general sequences).
List comprehensions cleanly replace filter. They don't
quite replace map, and they definitely don't replace reduce.
Claiming that sum etc. do the same job is the whimper of
someone who doesn't want to openly disagree with Guido.

Lambda is a horse of a quite different color. There are times
when an inline definition is the best thing one can have for
notational expressiveness. While lambda has a lot of
problems, Guido seems to be resisting replacing it with
something that can actually do the job.

Having to define an external function or class doesn't
always improve expressiveness. Sometimes it reduces
it. To quote Sean O'Lochlain: "Sometimes the best
symbol for a sharp knife is a sharp knife." [1]

John Roth

[1] In one of Randall Garrett's Lord Darcy stories.

tom

--
Batman always wins

Jul 19 '05 #14

Peter Hansen

Sean McIlroy wrote:

personally i don't know lisp (or scheme), but now i've
decided to learn it, because eventually it will no longer be possible
in python to pass functions as arguments or return them as values. the
education sig will have to change its motto to "computer programming
for every C-programmer".

Huh? Where did that come from? Functions are objects in Python and
I've not heard the least discussion about this being changed, until now.

Sean, what gave you the impression this would change?

-Peter

Jul 19 '05 #15

Paul McGuire

I have never written a line of Lisp or Scheme, so it took me a while to
grok "lambda" as a synonym for "expression to be evaluated later". I
then thought it was similar to Smalltalk's functional closures, since
you can define local arguments at the beginning of the block, and then
write the body of the block using those args. But then I saw that it
was only a subset of that capability, since one may only implement an
expression in a lambda, not a full code block.

Even with those limitations, I've found lambda to be a nice compact
form for specifying callbacks. It is especially helpful in pyparsing,
where I define a mechanism for the programmer to specify a parse action
to be performed, which can modify the matched tokens. Here is one that
is very compact:

quotedString.setParseResults( lambda s,loc,toks: toks[0][1:-1] )

Parse actions take 3 arguments: the original string being parsed, the
location of the beginning of the match, and a ParseResults object
containing the matched tokens (ParseResults objects can act as a list,
dict, or object with attributes). The purpose of this parse action is
to remove the opening and closing quotation marks from the matched
quoted string. One of the things I especially like about this
simplicity of parse actions is that there is no need for checking
whether toks is an empty list, or if the first and last characters are
quotation marks before stripping them - quotedStrings call their parse
actions *only* with a single element list, and only with the first
element containing a string with opening and closing quotes. Still, in
anticipation of the demise of lambda, and because this function is
frequently needed, I've added it as a built-in helper function to
pyparsing, called removeQuotes. But lambda is very simple and
immediate for defining such simple transforms, and there is no need to
go track down where a named function definition may be found. (Yes, I
*could* stop in my tracks just prior to this statement and define this
one-line function, but then this interrupts the flow of my grammar
definition.)

It seems to me that lambda's built-in limitation of *only* supporting
expressions, rather than complete code blocks, has led to people
applying their boundless creativity to trying to cram conditional logic
into bewildering and failure-prone boolean expressions, and it reminds
me of some of the C macro coding that I had to sift through about 20
years ago.

Not coincidentally, I think the lambda limitation is the true origin of
may of the requests we read on c.l.py for the proper syntax for
Python's version of the ternary ?: operator as in "how do I write
(x>10? a : b) in Python", which is invariably followed by a post such
as, "don't bother with that, just do "(x>10 and a or b)", which is then
usually followed with, "but watch out in case a evaluates to False..."

So personally, I like lambdas even if I am not found of the keyword
"lambda". Maybe we could replace "lambda" with "@" or "$"?

-- Paul

Jul 19 '05 #16

Sean McIlroy

Peter Hansen wrote:
<snip>

Sean, what gave you the impression this would change?

just inductive reasoning. i've been wrong before (like anyone who makes
that claim), and i'm a former python enthusiast, so my judgement must
be colored to some extent by bitterness. maybe they have solid reasons
for scrapping the functional constructs. but to me it seems like
they're eliminating them just because they offend the sensibilities of
C-programmers. (i mean those stereotypical C-programmers, baffled by
recursion and the like, who don't want to be reproached with the fact
of their mathematical illiteracy.) if that's the case then list
comprehensions and/or "first class functions" are likely to be the next
target. even if they're not, it's pretty clear that python is leaving
its multiparadigmatic origins behind. "do it our way," the pundits are
effectively saying, "or get out". for my part, i'm getting out.

Jul 19 '05 #17

Chris Smith

>>>>> "Devan" == Devan L <de****@gmail.com> writes:

Devan> None of them are really indispensible. Map and filter cab
Devan> be replaced with list comprehensions. reduce is redundant
Devan> except when multiplying a series; there's a sum function
Devan> for a reason. Lambda looks cleaner in some cases, but you
Devan> don't gain any functionality.

Devan> What really struck me, though, is the last line of the
Devan> abstract:

Devan> "I expect tons of disagreement in the feedback, all from
Devan> ex-Lisp-or-Scheme folks. :-)"

Devan> Guido wrote somewhere that the original map, filter, and
Devan> reduce came from a lisp hacker who missed them.

My question is, why not move them into, say, a "functional" library,
so that legacy code can be handled via an import, and those heads
preferring to think that way can be satisfied, and those little corner
cases not handled by the newer, sweller syntaxes can still be managed?
IOW, just ripping them out of the core and leaving everyone in the
lurch doesn't seem too pythonic to me.
Best,
Chris

Jul 19 '05 #18

Sean McIlroy wrote:

if that's the case then list
comprehensions and/or "first class functions" are likely to be the next
target.

Slippery slope arguments are logical fallacies, you know.

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Can I walk with you / Through your life
-- India Arie

Jul 19 '05 #19

Robert Kern

Chris Smith wrote:

My question is, why not move them into, say, a "functional" library,
so that legacy code can be handled via an import, and those heads
preferring to think that way can be satisfied, and those little corner
cases not handled by the newer, sweller syntaxes can still be managed?
IOW, just ripping them out of the core and leaving everyone in the
lurch doesn't seem too pythonic to me.

Python 3000 will be breaking more backwards compatibility than
map/reduce/filter. That legacy code won't work anyways.

I predict, though, that one of the first 3rd party modules to come out
for Python 3000 will be such a library.

--
Robert Kern
rk***@ucsd.edu

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

Jul 19 '05 #20

Robert Kern

Sean McIlroy wrote:

Peter Hansen wrote:
<snip>
Sean, what gave you the impression this would change?
just inductive reasoning. i've been wrong before (like anyone who makes
that claim), and i'm a former python enthusiast, so my judgement must
be colored to some extent by bitterness. maybe they have solid reasons
for scrapping the functional constructs. but to me it seems like
they're eliminating them just because they offend the sensibilities of
C-programmers.

This is incorrect.
(i mean those stereotypical C-programmers, baffled by
recursion and the like, who don't want to be reproached with the fact
of their mathematical illiteracy.) if that's the case then list
comprehensions and/or "first class functions" are likely to be the next
target.
map and filter are being removed *because of* list comprehensions. Did
you even read Guido's articles about this issue? Your understanding of
why these changes are planned is incorrect; consequently your projection
based on that understanding is not on firm footing.
even if they're not, it's pretty clear that python is leaving
its multiparadigmatic origins behind. "do it our way," the pundits are
effectively saying, "or get out". for my part, i'm getting out.

If that's what you want to do, no one is going to stop you. But please
do it quietly.

--
Robert Kern
rk***@ucsd.edu

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

Jul 19 '05 #21

"Sean McIlroy" <se**********@yahoo.com> writes:

Peter Hansen wrote:
<snip>
Sean, what gave you the impression this would change?

if that's the case then list comprehensions and/or "first class
functions" are likely to be the next target.

The existence of list comprehensions are the reason that these
functions are going away, so they aren't likely to be next. It's all
part of "There should be one-- and preferably only one --obvious way
to do it."

Personally, I hope they wind up in a "functional" module, so you can add
"from functional import *" to the top of your scripts, and keep doing
exactly what you've been doing.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

Jul 19 '05 #22

Ron Adam

Terry Hancock wrote:

On Friday 01 July 2005 03:36 pm, Ron Adam wrote:
I find map too limiting, so won't miss it. I'm +0 on removing lambda
only because I'm unsure that there's always a better alternative.

Seems like some new idioms would have to be coined, like:

def my_function(a1, a2):
def _(a,b): return a+b
call_a_lib_w_callback(callback=_)

doesn't seem too bad, and defeats the "wasting time thinking of names"
argument.

Usually the time is regained later when you need to go back and figure
out what it is that the lambda is doing. Not so easy for beginners.

A hard to understand process that is easy to do, is not easier than an
easy to understand process that is a bit harder to do.

So what would be a good example of a lambda that couldn't be replaced?

I suspect the hardest would be building a list of functions. Something
like:

powers = [lambda a, i=i: a**i for i in range(10)]

which you might be able to make like this:

powers = []
for i in range(10):
def _(a,i=i): return a**i
powers.append(_)

which works and is understandable, but a bit less concise.

This would be a more direct translation I think:

def put_func(i):
def power_of_i(a):
return a**i
return power_of_i
power = [put_func(i) for i in range(10)]

I think it's also clearer what it does. I had to type the lambda
version into the shell to be sure I understood it. I think that's what
we want to avoid.
The main obstacle to the lambda style here is that def statements
are not expressions. I think that's been proposed as an alternative,
too -- make def return a value so you could say:

powers = [def _(a,i=i): return a**i for i in range(10)]
Wouldn't it be:

powers = [(def _(a): return a**i) for i in range(10)]

The parens around the def make it clearer I think.

That would be pretty much just renaming lambda and changing a syntax a
tad. I get the feeling that the continual desire to change the syntax
of lambda is one of the reasons for getting rid of it. I'm not sure any
of the suggestions will fix that. Although I prefer this version over
the current lambda.

Instead of reusing 'def', resurrecting 'let' as a keyword might be an
option.

powers = [ (let a return a**i) for i in range(10) ]
I just now thought this up, but I like it a lot as an alternative syntax
to lambda. :-)

Personally, I think this is understandable, and given that lambda
is to be pulled, a nice substitute (I would say it is easier to read
than the current lambda syntax, and easier for a newbie to
understand).

But it would probably encourage some bad habits, such as:

myfunc = def _(a,b):
print a,b
return a+b

which looks too much like Javascript, to me, where there are
about three different common idioms for defining a
function (IIRC). :-/

I don't think it would be used that way... Very often anyway.

Still none of these examples make for good use cases for keeping lambda.
And I think that's whats needed first, then the new syntax can be decided.

Ron

Jul 19 '05 #23

Dennis Lee Bieber

On Fri, 01 Jul 2005 19:24:09 -0700, Robert Kern <rk***@ucsd.edu>
declaimed the following in comp.lang.python:

I predict, though, that one of the first 3rd party modules to come out
for Python 3000 will be such a library.
Should be a breeze, considering the work that had to go into
making a ComeFrom module...

-- ================================================== ============ <
wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
wu******@dm.net | Bestiaria Support Staff <
================================================== ============ <
Home Page: <http://www.dm.net/~wulfraed/> <
Overflow Page: <http://wlfraed.home.netcom.com/> <

Jul 19 '05 #24

Donn Cave

Quoth Tom Anderson <tw**@urchin.earth.li>:
....
| I disagree strongly with Guido's proposals, and i am not an ex-Lisp,
| -Scheme or -any-other-functional-language programmer; my only other real
| language is Java. I wonder if i'm an outlier.
|
| So, if you're a pythonista who loves map and lambda, and disagrees with
| Guido, what's your background? Functional or not?

Dysfunctional, I reckon.

I think I disagree with the question more than the answer.

First, map and lambda are two different things, and it's reasonable
to approve of one and abhor the other. Especially if you have a
background in a functional language where lambda works like it should.
On the other hand, the list comprehension gimmick that replaces some
of the "higher order functions" is borrowed from Haskell, as you probably
know, so it isn't exactly alien to functional programming. Prelude.hs
defines map: map f xs = [ f x | x <- xs ]

Secondly, if there's anything I detest about the Python development
model, it is the tendency to focus on gimmicks. For 2.X, elimination
of these features would be an atrocity, a gratuitous change that would
break programs - but I don't think anyone who counts has seriously
proposed to do that. With 3.X, we are talking about a different language.
May not ever even get off the ground, but if it does, it's supposed to
be distinctly different, and we need to know a lot more about it before
we can reasonably worry about trivial details like whether map is going
to be there.

I personally think real FP is seriously hot stuff, but I think Python
is a lousy way to do it, with or without map. I suppose there's a
remote possibility that 3.X will change all that. Or more likely,
there will by then be a really attractive FP language, maybe out
of the "links" initiative by Wadler et al.

Donn Cave, do**@drizzle.com

Jul 19 '05 #25

John Roth

"Robert Kern" <rk***@ucsd.edu> wrote in message
news:ma***************************************@pyt hon.org...

map and filter are being removed *because of* list comprehensions. Did you
even read Guido's articles about this issue? Your understanding of why
these changes are planned is incorrect; consequently your projection based
on that understanding is not on firm footing.
May I most respectfully point out that you've got it backwards.
Part of the justification for list comprehensions was that they could
be used to replace map and filter.

The jihad against the functional constructs has been going on for a
long time, and list comprehensions are only one piece of it.

John Roth
--
Robert Kern
rk***@ucsd.edu

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

Jul 19 '05 #26

Tom Anderson

On Fri, 1 Jul 2005, Ivan Van Laningham wrote:

Personally, I find that Lisp & its derivatives put your head in a very
weird place. Even weirder than PostScript/Forth/RPN, when you come
right down to it.

+1 QOTW!

tom

--
REMOVE AND DESTROY

Jul 19 '05 #27

Robert Kern

John Roth wrote:

"Robert Kern" <rk***@ucsd.edu> wrote in message
news:ma***************************************@pyt hon.org...
map and filter are being removed *because of* list comprehensions. Did you
even read Guido's articles about this issue? Your understanding of why
these changes are planned is incorrect; consequently your projection based
on that understanding is not on firm footing.

May I most respectfully point out that you've got it backwards.
Part of the justification for list comprehensions was that they could
be used to replace map and filter.

That is essentially what I said. List comprehensions were made to
replace map and filter. Now that they are here, Guido wants to remove
map and filter to keep OOWTDI.

My response was incomplete, not incorrect.

--
Robert Kern
rk***@ucsd.edu

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

Jul 19 '05 #28

Jamey Cribbs

Tom Anderson wrote:

So, if you're a pythonista who loves map and lambda, and disagrees with
Guido, what's your background? Functional or not?

I have no functional language background. Until recently, I had no use
for programming "expression to be evaluated later" or "deferred
expressions" or whatever else they are being called.

Where I came to see the awesomeness of "deferred expressions" was a few
months ago when I started a major rewrite of KirbyBase for Ruby. I
wanted to make the Ruby version of KirbyBase take advantage of the
strengths of the language. Another Ruby programmer, Hal Fulton, was
helping me by constantly pushing me to make KirbyBase more Ruby-ish.
One thing he kept pushing was to be able to specify select querys using
Ruby's "deferred expression" mechanism, code blocks (before anyone
starts yelling, I know that Ruby code blocks are *much* more than just
"deferred expressions"; I'm just using that descriptor here for the sake
of this discussion).

Code blocks allow you to wrap up any Ruby code and pass it to a method
and have it executed within that method. It is more powerful than
lambda, because you can have multiple statements in the code block and
you can do assignment within the code block. This allowed me to rewrite
KirbyBase so that you can do a select like this:

plane_tbl.select { |r| r.country == 'USA' and r.speed > 350 }

Now, this is cool, but you can do this using lambda in Python. Where
Ruby code blocks really shine is that you can also do this:

plane_tbl.update {|r| r.name == 'P-51'}.set {|r|
r.speed = 405
r.range = 1210
}

I have one code block that I pass to the update method which says
"Select all planes with name equal to P-51". Then, I pass a code block
to the set method which assigns new values to the speed and range fields
for those records (i.e. P-51) that were selected in the update method.
This is something you can't do with lambda.

Now, I think I can duplicate the same functionality of Ruby code blocks
by using Python functions, but it is not going to be as pretty.

So, even though lambda is not as powerful as Ruby code blocks, I was
still bummed to read that it is going away, because it is better than
nothing.

Hopefully, Guido will reconsider or, even better, give us something even
more powerful.

Jamey Cribbs

Jul 19 '05 #29

Jamey Cribbs <jc*****@twmi.rr.com> writes:

Code blocks allow you to wrap up any Ruby code and pass it to a method
and have it executed within that method. It is more powerful than
lambda, because you can have multiple statements in the code block and
you can do assignment within the code block.

Just FYI, the inability to have statements - even multiple statements
- in a lambda is what people are talking about when they talk about
the limitations of lambda. It's a common thing for someone with a
background that includes a proper lambda to trip over when they first
start programming in Python. It's not that uncommon for newcommers to
trip over it with "print".

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

Jul 19 '05 #30

On Fri, 01 Jul 2005 09:13:58 -0700, mcherm wrote:

Lambda serves a very specific purpose: declaring small, in-place
functions which are no bigger than a single expression. I do this often
enough that I DO want special syntax for it. But I'll admit that I
wish "lambda" were about 5 or 6 characters shorter
As in an empty string? :-)
and didn't have such an obscure name.

Lambda is no more an obscure name than "function", "decorator", "closure",
"class", or "module". The first time you come across it, you don't know
what it means. Then you learn what it means, and then you know.

--
Steve

Jul 19 '05 #31

On Fri, 01 Jul 2005 13:42:10 -0400, Mike Meyer wrote:

"iK" <Ik@sdsfsfd.com> writes:
Seems like he wants python programmers to solve their problems all in the
same way. While that is great for corporate slaves it is terrible for the
creative programmer.

No, he wants Python to be Pythonic. TMTOWTDI is not Pythonic.

Too Many T--- Only Way To Do It?

There Might Tangle One Way To Do It?

T--- M--- Two Obvious Ways To Do It?

Nope, sorry, still not getting it.

Python is quickly becoming the visual basic of the 21 century. If you want
to have fun while getting some work done you need to look elsewhere. It's a
shame...

If you'd rather spend your time figuring out which of multiple ways to
do things is the best for the job at hand than producing code, there's
a language that makes TMTOWTDI a way of life.

Figuring out which of multiple ways to do things is the best for the job
at hand _is_ part of producing code. There will always be multiple ways to
do the job. For starters, there is the choice, which language should I
use? Top-Down or Bottom-Up design? Test-driven or not? For loop or list
comprehension or generator? Procedural programming or object-oriented or a
mixture of both? Singleton or Borg design pattern? Test your data first or
deal with the exceptions when they happen? And so on.

Only One Obvious Way makes a nice slogan, but it is easy to turn a
flexible language like Python into a straight-jacket where there is Only
One Way To Do It Regards Of Whether It Is The Best For The Job On Hand Or
Not. Not such a short and concise slogan.

Now that Python has list comps, should for loops be removed from the
language? Why did Python bother introducing list comps when there is
nothing they can do that a for loop can't?

Functional programming using map etc does require a slightly different way
of thinking about programming than does procedural programming, just as
object-oriented needs a different way of thinking than spaghetti-coding
using GOTOs. Different ways of thinking about programming should be
encouraged, not discouraged. Even the much-maligned GOTO has its modern
usage case: the Exception.

If map/filter/reduce have to be removed from the built-ins, and I don't
think they should, I'd prefer for them to be moved into a module rather
than dropped altogether. Provided list comps are made as fast as map and
filter, then at the cost of readability they can be replaced by list
comps. But reduce can't be written as a list comp, only as a relatively
complex for loop at a HUGE loss of readability -- and I've never used
Lisp or Scheme in my life. I'm surely not the only one.

--
Steven

Jul 19 '05 #32

On Fri, 01 Jul 2005 19:15:46 -0700, Erik Max Francis wrote:

Sean McIlroy wrote:
if that's the case then list
comprehensions and/or "first class functions" are likely to be the next
target.

Slippery slope arguments are logical fallacies, you know.

Not if you are actually standing on a slippery slope. But seriously, no,
they aren't. The slippery slope argument is _not_ "X is happening now, so
Y will happen no matter what we do". That would be a fallacy.

The argument is actually "X is happening now. If X continues to happen
into the future, Y is the logical consequence of that process. If we wish
to avoid Y, we must stop X". And that is not a fallacy in general
(although of course it could be, if there is no causal relationship
between X and Y).

In this particular case, I suspect Sean is wrong. Guido seems to like list
comprehensions. Unless I'm mistaken (not for the first time) I think he
actually introduced them to the language. They won't be going anywhere
anytime soon.
--
Steven

Jul 19 '05 #33

Devan L

Claiming that sum etc. do the same job is the whimper of
someone who doesn't want to openly disagree with Guido.

Could you give an example where sum cannot do the job(besides the
previously mentioned product situation?

Also, map is easily replaced.
map(f1, sequence) == [f1(element) for element in sequence]

Jul 19 '05 #34

Peter Hansen

Steven D'Aprano wrote:

On Fri, 01 Jul 2005 13:42:10 -0400, Mike Meyer wrote:
No, he wants Python to be Pythonic. TMTOWTDI is not Pythonic.

Too Many T--- Only Way To Do It?

There Might Tangle One Way To Do It?

T--- M--- Two Obvious Ways To Do It?

Nope, sorry, still not getting it.

If you were serious, Google would be a real good friend here, since the
answer is in its first search result... without even having to click on
the link! Heck, it even points you to the web site: http://tmtowtdi.com :-)

-Peter

Jul 19 '05 #35

On Sat, 02 Jul 2005 20:26:31 -0700, Devan L wrote:

Claiming that sum etc. do the same job is the whimper of
someone who doesn't want to openly disagree with Guido.

Could you give an example where sum cannot do the job(besides the
previously mentioned product situation?
There is an infinite number of potential lambdas, and therefore an
infinite number of uses for reduce.

sum only handles a single case, lambda x,y: x+y

product adds a second case: lambda x,y: x*y

So sum and product together cover precisely 2/infinity, or zero percent,
of all possible uses of reduce.

Also, map is easily replaced.
map(f1, sequence) == [f1(element) for element in sequence]

Three mental tokens ( map, f1, sequence ) versus seven ( [], f1, element,
for, element, in, sequence ).

Also, map can take any number of sequences:

map(f1, seq1, seq2, seq3, seq4, ...)
--

Steven.

Jul 19 '05 #36

Steven D'Aprano wrote:

comps. But reduce can't be written as a list comp, only as a relatively
complex for loop at a HUGE loss of readability -- and I've never used
Lisp or Scheme in my life. I'm surely not the only one.

See my reply to your other post for a more detailed explanation, but I
don't think that the for-loop solution is much less readable at all, and
the additional complexity involved is simply setting the initial value
and result for the accumulator. The for-loop solution is even more
flexible, because it can include anonymous code blocks and not just
expressions.

One caevat that I just noticed, though -- with the for-solution, you do
need to be careful about whether you're using a generator or list if you
do not set an explicit initial value (and instead use the first value of
'sequence' as the start). The difference is:
_accum = g.next()
for i in g: _accum = stuff(_accum,i)

versus
_accum = g[0]
for i in g[1:]: _accum = stuff(_accum,i)

The difference is because generators don't support subscripts, while
lists don't support .next() iteration. Unless I'm missing something in
the language (entirely possible), this suggests a missing feature for
same-syntax iteration over the two types.

Jul 19 '05 #37

On Sun, 03 Jul 2005 08:14:28 -0700, Scott David Daniels wrote:

egbert wrote:
On Sat, Jul 02, 2005 at 08:26:31PM -0700, Devan L wrote:
Also, map is easily replaced.
map(f1, sequence) == [f1(element) for element in sequence]
How do you replace
map(f1,sequence1, sequence2)
especially if the sequences are of unequal length ?

I didn't see it mentioned yet as a candidate for limbo,
but the same question goes for:
zip(sequence1,sequence2)

OK, you guys are picking on what reduce "cannot" do.
The first is [f1(*args) for args in itertools.izip(iter1, iter2)]

And now we get messier and messier... Compare these two idioms:

"Map function f1 to each pair of items from seq1 and seq2."

"Build a list comprehension by calling function f1 with the unpacked list
that you get from a list built by zipping seq1 and seq2 together in pairs."

Good thing that removing reduce is supposed to make code easier to
understand, right?

How to _you_ use map to avoid making all the intermediate structures?
I don't understand the question. Presumably the sequences already exist.
That's not the point.
I never saw anything about making zip go away. It is easy to explain.
I don't find map any less clear than zip.

Except for the arbitrary choice that zip truncates unequal sequences while
map doesn't, zip is completely redundant:

def my_zip(*seqs):
return map(lambda *t: t, *seqs)

Zip is just a special case of map. I find it disturbing that Guido is
happy to fill Python with special case built-ins like sum, zip and
(proposed) product while wanting to cut out more general purpose solutions.
[snip] If you want functional programming in python, you have at least
three big problems:

1) Python has side effect like mad, so order of evaluation matters.
Not if you *just* use functional operations.

Not that I would ever do that. The point isn't to turn Python into a
purely functional language, but to give Python developers access to
functional tools for when it is appropriate to use them.
2) Python's essential function call is not a single-argument
function which might be a tuple, it is a multi-argument function
which is not evaluated in the same way.
And I'm sure that makes a difference to the functional programming
purists. But not to me.
3) Python doesn't have a full set of functional primitives.
Fold-right is one example, K-combinator is another, .... Why pick on
reduce as-is to keep? There is another slippery slope argument
going up the slope adding functional primitives.

My car isn't amphibious, so I can't go everywhere with it. Should I throw
it away just because I can't drive under water?

No, of course not. Just because Python isn't a purely functional language
doesn't mean that we should reject what functional idioms (like list
comps, and zip, and reduce) it does have.

Personally, I'd like to learn more about about fold-right and
K-combinator, rather than dump reduce and map.

Frankly, I find this entire discussion very surreal. Reduce etc *work*,
right now. They have worked for years. If people don't like them, nobody
is forcing them to use them. Python is being pushed into directions which
are *far* harder to understand than map and reduce (currying, decorators,
etc) and people don't complain about those. And yet something as simple
and basic as map is supposed to give them trouble? These are the same
people who clamoured for zip, which is just a special case of map?
--
Steven.

Jul 21 '05 #38

Carl Banks wrote:

Listcomps et al. cannot do everything map, lambda, filter, and reduce
did. Listcomps are inferior for functional programming. But, you see,
functional is not the point. Streamlining procedural programs is the
point, and I'd say listcomps do that far better, and without all the
baroque syntax (from the procedural point of view).

I've heard this said a couple times now -- how can listcomps not
completely replace map and filter?

I'd think that:
mapped = [f(i) for i in seq]
filtered = [i for i in seq if f(i)]

The only map case that doesn't cleanly reduce is for multiple sequences
of different length -- map will extend to the longest one (padding the
others with None), while zip (izip) truncates sequences at the shortest.
This suggests an extension to (i)zip, possibly (i)lzip ['longest zip']
that does None padding in the same way that map does.

Reduce can be rewritten easily (if an initial value is supplied) as a
for loop:
_accum = initial
for j in seq: _accum=f(_accum,j)
result = _accum

(two lines if the result variable can also be used as the accumulator --
this would be undesirable of assigning to that can trigger, say, a
property function call)

Lambdas, I agree, can't be replaced easily, and they're the feature I'd
probably be least happy to see go, even though I haven't used them very
much.

Jul 21 '05 #39

Scott David Daniels wrote:

egbert wrote:
How do you replace
map(f1,sequence1, sequence2)
especially if the sequences are of unequal length ?

I didn't see it mentioned yet as a candidate for limbo,
but the same question goes for:
zip(sequence1,sequence2)

OK, you guys are picking on what reduce "cannot" do.
The first is [f1(*args) for args in itertools.izip(iter1, iter2)]
How to _you_ use map to avoid making all the intermediate structures?

Not quite -- zip an izip terminate at the shortest sequence, map extends
the shortest with Nones. This is resolvable by addition of an lzip (and
ilzip) function in Python 2.5 or something.

And egbert's Chicken Littling with the suggestion that 'zip' will be
removed.

Jul 21 '05 #40

Peter Hansen

Steven D'Aprano wrote:

Frankly, I find this entire discussion very surreal. Reduce etc *work*,
right now. They have worked for years. If people don't like them, nobody
is forcing them to use them. Python is being pushed into directions which
are *far* harder to understand than map and reduce (currying, decorators,
etc) and people don't complain about those.

I find it surreal too, for a different reason.

Python *works*, right now. It has worked for years. If people don't
like the direction it's going, nobody is forcing them to upgrade to the
new version (which is not imminent anyway).

In the unlikely event that the latest and greatest Python in, what, five
years or more?, is so alien that one can't handle it, one has the right
to fork Python and maintain a tried-and-true-and-still-including-reduce-
-filter-and-map version of it, or even just to stick with the most
recent version which still has those features. And that's assuming it's
not acceptable (for whatever bizarre reason I can't imagine) to use the
inevitable third-party extension that will provide them anyway.

I wonder if some of those who seem most concerned are actually more
worried about losing the free support of a team of expert developers as
those developers evolve their vision of the language, than about losing
access to something as minor as reduce().

-Peter

Jul 21 '05 #41

Ron Adam

Erik Max Francis wrote:

Ron Adam wrote:
I'm just estimating, but I think that is the gist of adding those two
in exchange for reduce. Not that they will replace all of reduce use
cases, but that sum and product cover most situations and can be
implemented more efficiently than using reduce or a for loop to do the
same thing. The other situations can easily be done using for loops,
so it's really not much of a loss.

I really don't understand this reasoning. You essentially grant the
position that reduce has a purpose, but you still seem to approve
removing it. Let's grant your whole point and say that 90% of the use
cases for reduce are covered by sum and product, and the other 10% are
used by eggheads and are of almost no interest to programmers. But it
still serves a purpose, and a useful one. That it's not of immediate
use to anyone is an argument for moving it into a functional module
(something I would have no serious objection to, though I don't see its
necessity), not for removing it altogether! Why would you remove the
functionality that already exists _and is being used_ just because? What
harm does it do, vs. the benefit of leaving it in?

There are really two separate issues here.

First on removing reduce:

1. There is no reason why reduce can't be put in a functional module or
you can write the equivalent yourself. It's not that hard to do, so it
isn't that big of a deal to not have it as a built in.

2. Reduce calls a function on every item in the list, so it's
performance isn't much better than the equivalent code using a for-loop.

*** (note, that list.sort() has the same problem. I would support
replacing it with a sort that uses an optional 'order-list' as a sort
key. I think it's performance could be increased a great deal by
removing the function call reference. ***
Second, the addition of sum & product:

1. Sum, and less so Product, are fairly common operations so they have
plenty of use case arguments for including them.

2. They don't need to call a pre-defined function between every item, so
they can be completely handled internally by C code. They will be much
much faster than equivalent code using reduce or a for-loop. This
represents a speed increase for every program that totals or subtotals a
list, or finds a product of a set.

But removing reduce is just removing
functionality for no other reason, it seems, than spite.

No, not for spite. It's more a matter of increasing the over all
performance and usefulness of Python without making it more complicated.
In order to add new stuff that is better thought out, some things
will need to be removed or else the language will continue to grow and
be another visual basic.

Having sum and product built in has a clear advantage in both
performance and potential frequency of use, where as reduce doesn't have
the same performance advantage and most poeple don't use it anyway, so
why have it built in if sum and product are? Why not just code it as a
function and put it in your own module?

def reduce( f, seq):
x = 0
for y in seq:
x = f(x,y)
return x

But I suspect that most people would just do what I currently do and
write the for-loop to do what they want directly instead of using lambda
in reduce.

x = 1
for y in seq:
x = x**y

If performance is needed while using reduce with very large lists or
arrays, using the numeric module would be a much better solution.

http://www-128.ibm.com/developerwork...y/l-cpnum.html

Cheers,
Ron

Jul 21 '05 #42

Carl Banks

Steven D'Aprano wrote:

On Sun, 03 Jul 2005 08:14:28 -0700, Scott David Daniels wrote:
egbert wrote:
On Sat, Jul 02, 2005 at 08:26:31PM -0700, Devan L wrote:

Also, map is easily replaced.
map(f1, sequence) == [f1(element) for element in sequence]

How do you replace
map(f1,sequence1, sequence2)
especially if the sequences are of unequal length ?

I didn't see it mentioned yet as a candidate for limbo,
but the same question goes for:
zip(sequence1,sequence2)
OK, you guys are picking on what reduce "cannot" do.
The first is [f1(*args) for args in itertools.izip(iter1, iter2)]

And now we get messier and messier... Compare these two idioms:

"Map function f1 to each pair of items from seq1 and seq2."

"Build a list comprehension by calling function f1 with the unpacked list
that you get from a list built by zipping seq1 and seq2 together in pairs."

The shamelessness with which you inflated the verbosity of the latter
is hilarious.

Good thing that removing reduce is supposed to make code easier to
understand, right?

It was a bad example. I would say most people don't usually just call
a function in the list comp, because, frankly, they don't have to. A
realistic list comp would look something like this in a real program:

[ x**2 + y**2 for (x,y) in izip(xlist,ylist) ]

Now there's no longer much advantage in conciseness for the map version
(seeing that you'd have to define a function to pass to map), and this
is more readable.
--
CARL BANKS

Jul 21 '05 #43

Carl Banks

Christopher Subich wrote:

Carl Banks wrote:
Listcomps et al. cannot do everything map, lambda, filter, and reduce
did. Listcomps are inferior for functional programming. But, you see,
functional is not the point. Streamlining procedural programs is the
point, and I'd say listcomps do that far better, and without all the
baroque syntax (from the procedural point of view).

I've heard this said a couple times now -- how can listcomps not
completely replace map and filter?

If you're doing heavy functional programming, listcomps are
tremendously unwieldy compared to map et al.
--
CARL BANKS

Jul 21 '05 #44

On Sun, 03 Jul 2005 19:31:02 +0000, Ron Adam wrote:

First on removing reduce:

1. There is no reason why reduce can't be put in a functional module
Don't disagree with that.
or
you can write the equivalent yourself. It's not that hard to do, so it
isn't that big of a deal to not have it as a built in.
Same goes for sum. Same goes for product, which doesn't have that many
common usages apart from calculating the geometric mean, and let's face
it, most developers don't even know what the geometric mean _is_.

If you look back at past discussions about sum, you will see that there is
plenty of disagreement about how it should work when given non-numeric
arguments, eg strings, lists, etc. So it isn't so clear what sum should do.
2. Reduce calls a function on every item in the list, so it's
performance isn't much better than the equivalent code using a for-loop.
That is an optimization issue. Especially when used with the operator
module, reduce and map can be significantly faster than for loops.
*** (note, that list.sort() has the same problem. I would support
replacing it with a sort that uses an optional 'order-list' as a sort
key. I think it's performance could be increased a great deal by
removing the function call reference. ***
Second, the addition of sum & product:

1. Sum, and less so Product, are fairly common operations so they have
plenty of use case arguments for including them.
Disagree about product, although given that sum is in the language, it
doesn't hurt to put product as well for completion and those few usages.
2. They don't need to call a pre-defined function between every item, so
they can be completely handled internally by C code. They will be much
much faster than equivalent code using reduce or a for-loop. This
represents a speed increase for every program that totals or subtotals a
list, or finds a product of a set.
I don't object to adding sum and product to the language. I don't object
to adding zip. I don't object to list comps. Functional, er, functions
are a good thing. We should have more of them, not less.

But removing reduce is just removing
functionality for no other reason, it seems, than spite.

No, not for spite. It's more a matter of increasing the over all
performance and usefulness of Python without making it more complicated.
In order to add new stuff that is better thought out, some things
will need to be removed or else the language will continue to grow and
be another visual basic.

Another slippery slope argument.
Having sum and product built in has a clear advantage in both
performance and potential frequency of use, where as reduce doesn't have
the same performance advantage and most poeple don't use it anyway, so
why have it built in if sum and product are?
Because it is already there.
Why not just code it as a
function and put it in your own module?
Yes, let's all re-invent the wheel in every module! Why bother having a
print statement, when it is so easy to write your own:

def myprint(obj):
sys.stdout.write(str(obj))

Best of all, you can customize print to do anything you like, _and_ it is
a function.
def reduce( f, seq):
x = 0
for y in seq:
x = f(x,y)
return x
Because that is far less readable, and you take a performance hit.
But I suspect that most people would just do what I currently do and
write the for-loop to do what they want directly instead of using lambda
in reduce.

That's your choice. I'm not suggesting we remove for loops and force you
to use reduce. Or even list comps.
--
Steven.

Jul 21 '05 #45

Carl Banks wrote:

Christopher Subich wrote:
I've heard this said a couple times now -- how can listcomps not
completely replace map and filter?

If you're doing heavy functional programming, listcomps are
tremendously unwieldy compared to map et al.

Interesting; could you post an example of this? Whenever I try to think
of that, I come up with unwieldly syntax for the functional case. In
purely functional code the results of map/filter/etc would probably be
directly used as arguments to other functions, which might make the
calls longer than I'd consider pretty. This is especially true with
lots of lambda-ing to declare temporary expressions.

Jul 21 '05 #46

Steven Bethard

Christopher Subich wrote:

One caevat that I just noticed, though -- with the for-solution, you do
need to be careful about whether you're using a generator or list if you
do not set an explicit initial value (and instead use the first value of
'sequence' as the start). The difference is:
_accum = g.next()
for i in g: _accum = stuff(_accum,i)

versus
_accum = g[0]
for i in g[1:]: _accum = stuff(_accum,i)

If you want to be general for all iterables (list, generators, etc), you
can write the code like:

itr = iter(g)
_accum = itr.next()
for i in itr:
_accum = stuff(_accum, i)

STeVe

Jul 21 '05 #47

Christopher Subich wrote:

Interesting; could you post an example of this? Whenever I try to think
of that, I come up with unwieldly syntax for the functional case. In
purely functional code the results of map/filter/etc would probably be
directly used as arguments to other functions, which might make the
calls longer than I'd consider pretty. This is especially true with
lots of lambda-ing to declare temporary expressions.

I personally think that map looks clearer than a list comprehension for
a simple function call, e.g.

map(str, sequence)

vs.

[str(x) for x in sequence]

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
In Heaven all the interesting people are missing.
-- Friedrich Nietzsche

Jul 21 '05 #48

Steven D'Aprano <st***@REMOVETHIScyber.com.au> writes:

I don't object to adding sum and product to the language. I don't object
to adding zip. I don't object to list comps. Functional, er, functions
are a good thing. We should have more of them, not less.

Yes, but where should they go? Adding functions in the standard
library is one thing. Adding builtins is another. Builtins make every
python process heavier. This may not matter on your desktop, but
Python gets used in embedded applications as well, and it does
there. Builtins also clutter the namespace. Nothing really wrong with
that, but it's unappealing.

I'd say that removing functions is a bad thing. On the other hand, I'd
say moving them from builtins to the standard library when Python has
functionality that covers most of the use cases for them is a good
thing.

The latter has occured for map, filter, and reduce. Lambda I'm not so
sure of, but it gets swept up with the same broom. Moving the first
three into a library module seems like a good idea. I'm not sure about
removing lambda. Removing map, filter and reduce remove most of my
use cases for it. But not all of them.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

Jul 21 '05 #49