Comrades,
During our current discussion of the fate of functional constructs in
python, someone brought up Guido's bull on the matter: http://www.artima.com/weblogs/viewpost.jsp?thread=98196
He says he's going to dispose of map, filter, reduce and lambda. He's
going to give us product, any and all, though, which is nice of him.
What really struck me, though, is the last line of the abstract:
"I expect tons of disagreement in the feedback, all from ex-Lisp-or-Scheme
folks. :-)"
I disagree strongly with Guido's proposals, and i am not an ex-Lisp,
-Scheme or -any-other-functional-language programmer; my only other real
language is Java. I wonder if i'm an outlier.
So, if you're a pythonista who loves map and lambda, and disagrees with
Guido, what's your background? Functional or not?
tom
--
Batman always wins
Jul 19 '05
181  8583 
Steven D'Aprano wrote: On Sun, 03 Jul 2005 19:31:02 +0000, Ron Adam wrote:
First on removing reduce:
1. There is no reason why reduce can't be put in a functional module
Don't disagree with that.
or
you can write the equivalent yourself. It's not that hard to do, so it
isn't that big of a deal to not have it as a built in.
Same goes for sum. Same goes for product, ...
Each item needs to stand on it's own. It's a much stronger argument for
removing something because something else fulfills it's need and is
easier or faster to use than just saying we need x because we have y.
In this case sum and product fulfill 90% (estimate of course) of reduces
use cases. It may actually be as high as 99% for all I know. Or it may
be less. Anyone care to try and put a real measurement on it?
which doesn't have that many common usages apart from calculating the geometric mean, and let's face
it, most developers don't even know what the geometric mean _is_.
I'm neutral on adding product myself.
If you look back at past discussions about sum, you will see that there is
plenty of disagreement about how it should work when given non-numeric
arguments, eg strings, lists, etc. So it isn't so clear what sum should do.
Testing shows sum() to be over twice as fast as either using reduce or a
for-loop. I think the disagreements will be sorted out.
2. Reduce calls a function on every item in the list, so it's
performance isn't much better than the equivalent code using a for-loop.
That is an optimization issue. Especially when used with the operator
module, reduce and map can be significantly faster than for loops.
I tried it... it made about a 1% improvement in the builtin reduce and
an equal improvement in the function that used the for loop.
The inline for loop also performed about the same.
See below..
*** (note, that list.sort() has the same problem. I would support
replacing it with a sort that uses an optional 'order-list' as a sort
key. I think it's performance could be increased a great deal by
removing the function call reference. ***
Second, the addition of sum & product:
1. Sum, and less so Product, are fairly common operations so they have
plenty of use case arguments for including them.
Disagree about product, although given that sum is in the language, it
doesn't hurt to put product as well for completion and those few usages.
I'm not convinced about product either, but if I were to review my
statistics textbooks, I could probably find more uses for it. I suspect
that there may be a few common uses for it that are frequent enough to
make it worth adding. But it might be better in a module.
2. They don't need to call a pre-defined function between every item, so
they can be completely handled internally by C code. They will be much
much faster than equivalent code using reduce or a for-loop. This
represents a speed increase for every program that totals or subtotals a
list, or finds a product of a set.
I don't object to adding sum and product to the language. I don't object
to adding zip. I don't object to list comps. Functional, er, functions
are a good thing. We should have more of them, not less.
Yes, we should have lots of functions to use, in the library, but not
necessarily in builtins.
But removing reduce is just removing
functionality for no other reason, it seems, than spite.
No, not for spite. It's more a matter of increasing the over all
performance and usefulness of Python without making it more complicated.
In order to add new stuff that is better thought out, some things
will need to be removed or else the language will continue to grow and
be another visual basic.
Another slippery slope argument.
Do you disagree or agree? Or are you undecided?
Having sum and product built in has a clear advantage in both
performance and potential frequency of use, where as reduce doesn't have
the same performance advantage and most poeple don't use it anyway, so
why have it built in if sum and product are?
Because it is already there.
Hmm.. I know a few folks, Good people, but they keep everything to the
point of not being able to find anything because they have so much.
They can always think of reasons to keep things, "It's worth something",
"it means something to me", "I'm going to fix it", "I'm going to sell
it", "I might need it". etc..
"Because it is already there" sound like one of those type of reasons.
Why not just code it as a
function and put it in your own module?
Yes, let's all re-invent the wheel in every module! Why bother having a
print statement, when it is so easy to write your own:
def myprint(obj):
sys.stdout.write(str(obj))
Yes, Guido wants to make print a function in Python 3000. The good
thing about this is you can call your function just 'p' and save some
typing.
p("hello world")
Actually, I think i/o functions should be grouped in an interface
module. That way you choose the interface that best fits your need. It
may have a print if it's a console, or it may have a widget if it's a gui.
Best of all, you can customize print to do anything you like, _and_ it is
a function.
def reduce( f, seq):
x = 0
for y in seq:
x = f(x,y)
return x
Because that is far less readable, and you take a performance hit.
They come out pretty close as far as I can tell.
def reduce_f( f, seq):
x = seq[0]
for y in seq[1:]:
x = f(x,y)
return x
import time
t = time.time()
r2 = reduce(lambda x,y: x*y, range(1,10000))
t2 = time.time()-t
print 'reduce builtin:', t2
t = time.time()
r1 = reduce_f(lambda x,y: x*y, range(1,10000))
t2 = time.time()-t
print 'reduce_f: ', t2
if r1!=r2: print "results not equal"
reduce builtin: 0.156000137329
reduce_f: 0.155999898911
reduce builtin: 0.15700006485
reduce_f: 0.155999898911
reduce builtin: 0.141000032425
reduce_f: 0.155999898911
But I suspect that most people would just do what I currently do and
write the for-loop to do what they want directly instead of using lambda
in reduce.
That's your choice. I'm not suggesting we remove for loops and force you
to use reduce. Or even list comps.
Just don't force me to use decorators! ;-)
Nah, they're ok too, but it did take me a little while to understand
their finer points.
Cheers,
Ron
Ron Adam wrote: Each item needs to stand on it's own. It's a much stronger argument for
removing something because something else fulfills it's need and is
easier or faster to use than just saying we need x because we have y.
In this case sum and product fulfill 90% (estimate of course) of reduces
use cases. It may actually be as high as 99% for all I know. Or it may
be less. Anyone care to try and put a real measurement on it?
Well, reduce covers 100% of them, and it's one function, and it's
already there.
--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
I'm not jumping in / A wave that just passes
-- Sandra St. Victor
Erik Max Francis wrote: Ron Adam wrote:
Each item needs to stand on it's own. It's a much stronger argument
for removing something because something else fulfills it's need and
is easier or faster to use than just saying we need x because we have y.
In this case sum and product fulfill 90% (estimate of course) of
reduces use cases. It may actually be as high as 99% for all I know.
Or it may be less. Anyone care to try and put a real measurement on it?
Well, reduce covers 100% of them, and it's one function, and it's
already there.
So you are saying that anything that has a 1% use case should be
included as a builtin function?
I think I can find a few hundred other functions in the library that are
used more than ten times as often as reduce. Should those be builtins too?
This is a practical over purity issue, so what are the practical reasons
for keeping it. "It's already there" isn't a practical reason. And it
covers 100% of it's own potential use cases, is circular logic without a
real underlying basis.
Cheers,
Ron
Ron Adam wrote: So you are saying that anything that has a 1% use case should be
included as a builtin function?
I think I can find a few hundred other functions in the library that are
used more than ten times as often as reduce. Should those be builtins too?
This is a practical over purity issue, so what are the practical reasons
for keeping it. "It's already there" isn't a practical reason. And it
covers 100% of it's own potential use cases, is circular logic without a
real underlying basis.
But the Python 3000 plan, at least what we've heard of it so far, isn't
to move it to a standard library module. It's to remove it altogether,
replacing it with sum and product. Since sum and product don't cover
all the uses cases for reduce, this is a case of taking one function
that handles all the required use cases and replacing it with _two_
functions that don't. Since it's doubling the footprint of the reduce
functionality, arguments about avoiding pollution are red herrings.
--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Every astronaut who goes up knows the risks he or she faces.
-- Sally Ride
Carl Banks wrote: The shamelessness with which you inflated the verbosity of the latter
is hilarious.
[snip]
[ x**2 + y**2 for (x,y) in izip(xlist,ylist) ]
Now there's no longer much advantage in conciseness for the map version
(seeing that you'd have to define a function to pass to map), and this
is more readable.
and then, five minutes later in another post, wrote:
If you're doing heavy functional programming,
listcomps are tremendously unwieldy compared to
map et al.
Having a dollar each way I see :-)
--
Steven.
Mike Meyer wrote: Steven D'Aprano <st***@REMOVETHIScyber.com.au> writes:
I don't object to adding sum and product to the language. I don't object
to adding zip. I don't object to list comps. Functional, er, functions
are a good thing. We should have more of them, not less.
Yes, but where should they go? Adding functions in the standard
library is one thing. Adding builtins is another. Builtins make every
python process heavier. This may not matter on your desktop, but
Python gets used in embedded applications as well, and it does
there. Builtins also clutter the namespace. Nothing really wrong with
that, but it's unappealing.
I'd say that removing functions is a bad thing. On the other hand, I'd
say moving them from builtins to the standard library when Python has
functionality that covers most of the use cases for them is a good
thing.
The latter has occured for map, filter, and reduce. Lambda I'm not so
sure of, but it gets swept up with the same broom. Moving the first
three into a library module seems like a good idea. I'm not sure about
removing lambda. Removing map, filter and reduce remove most of my
use cases for it. But not all of them.
Metoobe!!!
Practicality beats purity: I am perfectly happy to have
list comps in the language and fast, efficient
functional programming tools in a module. I'm even
happy to see sum and product and zip as builtins, even
though logically they belong with map and reduce in the
(hypothetical) functional module.
I know Python isn't, and never will be, a purely
functional language. But being able to use some
functional techniques is good, and shouldn't be lost.
--
Steven.
Erik Max Francis wrote: Ron Adam wrote:
So you are saying that anything that has a 1% use case should be
included as a builtin function?
I think I can find a few hundred other functions in the library that are
used more than ten times as often as reduce. Should those be builtins too?
This is a practical over purity issue, so what are the practical reasons
for keeping it. "It's already there" isn't a practical reason. And it
covers 100% of it's own potential use cases, is circular logic without a
real underlying basis.
But the Python 3000 plan, at least what we've heard of it so far, isn't
to move it to a standard library module. It's to remove it altogether,
replacing it with sum and product. Since sum and product don't cover
all the uses cases for reduce, this is a case of taking one function
that handles all the required use cases and replacing it with _two_
functions that don't. Since it's doubling the footprint of the reduce
functionality, arguments about avoiding pollution are red herrings.
Four, in fact. sum(), product(), any(), and all().
The problem with this discussion is that no one is saying anything
new[1]. We've heard all of the arguments for and against removing these
functions. Over and over and over again. They're not changing anyone's
mind, not yours, not mine, and definitely not Guido's.
And it's not even like Python 3000 is around the corner or in any stage
of concrete planning. Adding our voices to the chorus *now* won't make a
bit of difference, nor should it. The size of the chorus doesn't matter;
Python isn't developed by votes. We tried that once before; it didn't
work so well.
When planning for Python 3000 really gets going, when Guido gets a
year's sabbatical to work on it, when we can see how
map/filter/reduce/lambda fit into the whole scheme of how this new
language works, *then* is the time to be having these discussions. Right
now, all we're doing is making each other bitter and angry for no good
reason.
[1] Okay, there was that guy who predicted that list comprehensions and
first-class functions were the next to go. That was new. But also wrong.
I think we can discount that.
--
Robert Kern rk***@ucsd.edu
"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter
Steven D'Aprano wrote: Carl Banks wrote:
The shamelessness with which you inflated the verbosity of the latter
is hilarious.
[snip]
[ x**2 + y**2 for (x,y) in izip(xlist,ylist) ]
Now there's no longer much advantage in conciseness for the map version
(seeing that you'd have to define a function to pass to map), and this
is more readable.
and then, five minutes later in another post, wrote:
> If you're doing heavy functional programming,
> listcomps are tremendously unwieldy compared to
> map et al.
Having a dollar each way I see :-)
Don't think so. The verbosity I spoke of was your describing the code
snippets in English, not the conciseness of the example. map and
friends are more concise than listcomps, I wasn't arguing that, except
that for the typical Pythonic use of listcomps it isn't much. One
listcomp or one call to map is not "heavily functional."
--
CARL BANKS
Christopher Subich wrote: Carl Banks wrote:
Christopher Subich wrote:I've heard this said a couple times now -- how can listcomps not
completely replace map and filter?
If you're doing heavy functional programming, listcomps are
tremendously unwieldy compared to map et al.
Interesting; could you post an example of this? Whenever I try to think
of that, I come up with unwieldly syntax for the functional case. In
purely functional code the results of map/filter/etc would probably be
directly used as arguments to other functions, which might make the
calls longer than I'd consider pretty. This is especially true with
lots of lambda-ing to declare temporary expressions.
I suspect you're misunderstanding what I mean by heavily functional.
You appear to see maps and listcomps merely as a shortcut for a for
loop. You're comparing the map shortcut and the listcomp shortcut and
seeing which one's less verbose. In a mostly procedural program which
uses functional constructs in isolation, listcomps are going to win
most of those battles.
Heavily functional programming is a different mindset altogether. In
heavily functional programming, things like maps and filters and
function applications are actually what you're thinking about. map
isn't an indirect way to do a for loop; it's a direct way to do a map.
When your mind is focused on "applying a function to each member of
this list and returning a list of the results" as opposed to
"convenient shortcut to a for loop", map is going to be far superior to
a listcomp. And when you're doing dozens and dozens of maps over a
large purely functional program, you don't want to write out a listcomp
every single time you want to do it.
--
CARL BANKS
On Sunday 03 July 2005 07:05 pm, Erik Max Francis wrote: I personally think that map looks clearer than a list comprehension for
a simple function call, e.g.
I have to disagree
map(str, sequence)
This says "call a function 'map' on 'str' and 'sequence'"
Which, syntactically, is not terribly informative.
I have to remember:
* "str" is actually a callable
* "map" is a mathematical concept of linking one thing to another. What
things? "str to sequence"? No! Wrong guess. "str" is the "mapping function",
and the result is the thing sequence is to be linked to.
Now, sure, I know all this, and I learned what "map" did from the manual,
but it's not especially easy to remember.
This on the other hand,
[str(x) for x in sequence]
is practically plain English:
"call the function "str" on x, for every x in sequence"
Other than chopping out a few words, and using the () operator instead
of "call", it's hard to imagine this being any closer to exactly what you
would say to describe the operation. And for most of us, English comes
easier than Computer Science jargon.
--
Terry Hancock ( hancock at anansispaceworks.com )
Anansi Spaceworks http://www.anansispaceworks.com
On Sun, 3 Jul 2005, Robert Kern wrote: Erik Max Francis wrote: Ron Adam wrote:
So you are saying that anything that has a 1% use case should be included
as a builtin function?
I think I can find a few hundred other functions in the library that are
used more than ten times as often as reduce. Should those be builtins
too?
This is a practical over purity issue, so what are the practical reasons
for keeping it. "It's already there" isn't a practical reason. And it
covers 100% of it's own potential use cases, is circular logic without a
real underlying basis.
But the Python 3000 plan, at least what we've heard of it so far, isn't
to move it to a standard library module. It's to remove it altogether,
replacing it with sum and product. Since sum and product don't cover
all the uses cases for reduce, this is a case of taking one function
that handles all the required use cases and replacing it with _two_
functions that don't. Since it's doubling the footprint of the reduce
functionality, arguments about avoiding pollution are red herrings.
Four, in fact. sum(), product(), any(), and all().
I'll just chip in and say i'd quite like a flatten(), too; at the moment,
i have one like this:
def flatten(ll):
return reduce(lambda a, l: a.extend(l), ll, [])
A builtin, which had fast special-case code for then the argument is a
list of lists (rather than an iterable of iterables), would be nice, since
this is a reasonably big use of reduce for me.
How would one do that as a list comp, by the way? I'm really not very good
with them yet.
[1] Okay, there was that guy who predicted that list comprehensions and
first-class functions were the next to go. That was new. But also wrong.
I think we can discount that.
True. Guido will only get rid of those after he's got rid of lowercase
letters in identifiers.
tom
--
A military-industrial illusion of democracy
Terry Hancock wrote: On Sunday 03 July 2005 07:05 pm, Erik Max Francis wrote:I personally think that map looks clearer than a list comprehension for
a simple function call
This on the other hand, [str(x) for x in sequence]
is practically plain English:
"call the function "str" on x, for every x in sequence"
Other than chopping out a few words, and using the () operator instead
of "call", it's hard to imagine this being any closer to exactly what you
would say to describe the operation. And for most of us, English comes
easier than Computer Science jargon.
And with a better choice of names than "x", that line becomes even more
self-documenting.
[str(parrot) for parrot in sequence], for example, tells you much more
about what is going on than str(x) does.
Exactly what, I have no idea... but it says _so_ much more. ;-)
-Peter
"Tom Anderson" <tw**@urchin.earth.li> wrote: I'll just chip in and say i'd quite like a flatten(), too; at the moment,
i have one like this:
def flatten(ll):
return reduce(lambda a, l: a.extend(l), ll, [])
This doesn't work; a.extend() returns None, not the extended list a: seq = [[1,2],[3],[],[4,[5,6]]]
flatten(seq)
AttributeError: 'NoneType' object has no attribute 'extend'
This works for 1-level flattening:
def flatten(ll):
return reduce(lambda a, l: a.extend(l) or a, ll, [])
flatten(seq)
[1, 2, 3, 4, [5, 6]]
And finally for recursive flattening:
def flatten(seq):
return reduce(_accum, seq, [])
def _accum(seq, x):
if isinstance(x,list):
seq.extend(flatten(x))
else:
seq.append(x)
return seq
flatten(seq)
[1, 2, 3, 4, 5, 6]
George
Peter Hansen wrote: [str(parrot) for parrot in sequence], for example, tells you much more
about what is going on than str(x) does.
Exactly what, I have no idea... but it says _so_ much more. ;-)
Yarr! Avast! Etc!
Carl Banks wrote: I suspect you're misunderstanding what I mean by heavily functional.
<snip> Heavily functional programming is a different mindset altogether. In
heavily functional programming, things like maps and filters and
function applications are actually what you're thinking about. map
isn't an indirect way to do a for loop; it's a direct way to do a map.
That's true; I'm more comfortable with procedural programming in
general, but I had a few classes that used LISP and understand what
you're talking about.
That said, Python itself is mostly a procedural language, with the
functional tools really being bolted on[1]. When we're talking about
Py3K, I think we're really talking about a redesign and rethink of
pretty much the entire language -- with list and generator
comprehensions, for procedural programming the need for map and lambda
goes away. Reduce isn't directly replaced, of course, but a for-loop
implementation (for procedural programming) is clearer, more powerful,
more explicit, and possibly faster.
That said, I very much like the idea of putting map and filter in a
functional module. For applications like functional-style programming
where map/etc are clearer, that keeps them in the library for efficient
use, yet it leaves the native language with OO(g)WTDI [Only One (good)
Way to Do It].
[1] -- lambda excepted. I think it's kind of cute, in a baby-mammal
kind of way.
Erik Max Francis wrote: Ron Adam wrote:
In this case sum and product fulfill 90% (estimate of course) of
reduces use cases. It may actually be as high as 99% for all I know.
Or it may be less. Anyone care to try and put a real measurement on it?
Well, reduce covers 100% of them, and it's one function, and it's
already there.
And it's almost two times slower:
$ python -m timeit -s "x = xrange(1000)" "sum(x)"
10000 loops, best of 3: 92.5 usec per loop
$ python -m timeit -s "from operator import add; x = xrange(1000)"
"reduce(add, x)"
10000 loops, best of 3: 157 usec per loop
And that's only if I have the sense to import from operator:
$ python -m timeit -s "x = xrange(1000); add = lambda x, y: x + y"
"reduce(add, x)"
1000 loops, best of 3: 587 usec per loop
Note that the simple for-loop beats the case where you define your own
function because it doesn't have the repeated overhead of function calls
(which are expensive in Python):
$ python -m timeit -s "sum = 0; x = xrange(1000)" "for i in x: sum += i"
10000 loops, best of 3: 291 usec per loop
What would really help here is if you could identify the cases where you
think reduce is really a gain. A lot of them are actually not good
practice in Python because of the function-call overhead. However, I'm
willing to be convinced otherwise with a few good examples.
STeVe
Steven Bethard wrote: And it's almost two times slower:
That's because you're not using operator.add.
--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Virtue has never been as respectable as money.
-- Mark Twain
Christopher Subich wrote: That said, Python itself is mostly a procedural language, with the
functional tools really being bolted on[1].
[etc., snip]
Yeah, that's pretty much what I said in the first place.
--
CARL BANKS
George Sakkis wrote: And finally for recursive flattening:
def flatten(seq):
return reduce(_accum, seq, [])
def _accum(seq, x):
if isinstance(x,list):
seq.extend(flatten(x))
else:
seq.append(x)
return seq
flatten(seq)
[1, 2, 3, 4, 5, 6]
George
How about this for a non recursive flatten.
def flatten(seq):
s = []
while seq:
while isinstance(seq[0],list):
seq = seq[0]+seq[1:]
s.append(seq.pop(0))
return s
seq = [[1,2],[3],[],[4,[5,6]]]
flatten(seq)
Ron
Erik Max Francis wrote: Steven Bethard wrote:
And it's almost two times slower:
That's because you're not using operator.add.
Huh? Please re-read my post. That's exactly what I used.
STeVe
Steven D'Aprano writes: Lambda is no more an obscure name than "function", "decorator", "closure",
"class", or "module". The first time you come across it, you don't know
what it means. Then you learn what it means, and then you know.
I believe you've made two errors here. First of all, "lambda" is part
of the Python language, while "function", "decorator", "closure", and
"module" are not. The fact that some words which are NOT part of the
Python language are obscure has no bearing whatsoever on "lambda". The
obnubilation created by this comparison is an obscure word also, but,
it isn't relevent to the Python language.
The second error is that I believe most english speakers COULD provide
a definition for the fairly common words "function", "class", and
"decorator". The exact meaning of "class" might not be what they expect
at first, but exposure to any object oriented language would make the
concept quickly familiar. But "lambda" has a very clear meaning... it's
a letter of the greek alphabet. The connection between that letter and
anonymous functions is tenuous at best, and fails the test of making
Python read like "executable pseudocode".
-- Michael Chermside
On Mon, 4 Jul 2005, George Sakkis wrote: "Tom Anderson" <tw**@urchin.earth.li> wrote:
I'll just chip in and say i'd quite like a flatten(), too; at the moment,
i have one like this:
def flatten(ll):
return reduce(lambda a, l: a.extend(l), ll, [])
This doesn't work; a.extend() returns None, not the extended list a:
Ah, no, very good, i was hoping someone would notice that. Well done.
I think my lambda looks more like lambda a, b: a + b, but i realised the
other day that extend could make it more efficient, and didn't think it
through properly.
tom
--
The revolution will not be televised. The revolution will be live.
On Tue, 05 Jul 2005 05:03:32 -0700, mcherm wrote: Steven D'Aprano writes: Lambda is no more an obscure name than "function", "decorator", "closure",
"class", or "module". The first time you come across it, you don't know
what it means. Then you learn what it means, and then you know.
I believe you've made two errors here. First of all, "lambda" is part
of the Python language, while "function", "decorator", "closure", and
"module" are not.
Sorry, but you are mistaken. "lambda" is a _reserved_ word in the
Python language, while "function" etc are not, but they are certainly
part of the language. Try explaining what def and import do without
using the words "function" or "module". Maybe you can do it, using
circumlocutions, but it isn't easy, and costs clarity.
[snip] The second error is that I believe most english speakers COULD provide
a definition for the fairly common words "function", "class", and
"decorator". The exact meaning of "class" might not be what they expect
at first,
Function, in the sense of a mathematical function, I agree. Class as in
the thing you go to at school and decorator as in the person who advises
you what colour curtains to have, certainly. But in the Python sense? No.
Especially not decorator, which I believe most _programmers_ would have
trouble explaining, let alone non-programmer English speakers. I know I do.
but exposure to any object oriented language would make the
concept quickly familiar.
Just as exposure to functional languages would make lambda very familiar.
But "lambda" has a very clear meaning... it's
a letter of the greek alphabet. The connection between that letter and
anonymous functions is tenuous at best, and fails the test of making
Python read like "executable pseudocode".
Think back to when you were a schoolboy at your first day of school.
Unless you had a very unusual upbringing, you probably had never heard the
word "function" before. There is nothing about the word "function" that
brings to mind "a mathematical entity which transforms a variable into a
different variable", let alone "a programming subroutine that returns a
result". (Or for that matter, "the purpose which a person or thing is
for", as in the function of a spanner is to tighten nuts on bolts.)
You had to learn that word, discover what it means, and then it becomes
familiar. You don't notice the process only because it happened so long
ago, at an age that your brain was operating in "language acquisition
mode" and picking up vocabulary at an incredible rate.
There is nothing about the word "string" that especially brings to mind
"an array of bytes representing characters". The analogy of "string of
characters" to "string of beads" breaks down as soon as you have multiple
lines of text. And as for float, that's what boats do, heaven only knows
what it has to do with numbers.
(Yes, I know what it has to do with numbers. But that is something I had
to learn, and even now I still have difficulty because I expect floats to
operate like mathematical real numbers, and they don't.)
And dare I say it, what do constricting snakes have to do with programming?
I won't say that the anonymous function meaning of lambda comes to my mind
before the Greek letter, but it isn't very far behind, and rapidly
catching up. (I use lambda a lot more than I speak Greek.) It wouldn't
surprise me if one day I think of Python programming before the Greek
letter, just as the world aleph brings to my mind the sense of infinity
before the sense of it being a Hebrew letter.
--
Steven.
On Mon, 4 Jul 2005, Ron Adam wrote: George Sakkis wrote:
And finally for recursive flattening:
def flatten(seq):
return reduce(_accum, seq, [])
def _accum(seq, x):
if isinstance(x,list):
seq.extend(flatten(x))
else:
seq.append(x)
return seq
> flatten(seq)
[1, 2, 3, 4, 5, 6]
How about this for a non recursive flatten.
def flatten(seq):
s = []
while seq:
while isinstance(seq[0],list):
seq = seq[0]+seq[1:]
s.append(seq.pop(0))
return s
seq = [[1,2],[3],[],[4,[5,6]]]
flatten(seq)
The trouble with these is that they make a lot of temporary lists -
George's version does it with the recursive calls to flatten, and Ron's
with the slicing and concatenating. How about a version which never makes
new lists, only appends the base list? We can use recursion to root
through the lists ...
def isiterable(x):
return hasattr(x, "__iter__") # close enough for government work
def visit(fn, x): # perhaps better called applytoall
if (isiterable(x)):
for y in x: visit(fn, y)
else:
fn(x)
def flatten(seq):
a = []
def appendtoa(x):
a.append(x)
visit(appendtoa, seq)
return a
If you hate recursion, you can write a non-recursive version of visit;
you'll have to manage a stack of lists yourself, though. Something like:
def visit(fn, x):
if (not isiterable(x)): x = (x,)
stack = [None] # stack of iterators
cur = iter(x) # current iterator
while (cur != None):
try:
thing = cur.next()
if (isiterable(thing)):
stack.append(cur)
cur = iter(thing)
else:
fn(thing)
except StopIteration:
cur = stack.pop()
There might be a cleverer way to do this.
tom
--
The revolution will not be televised. The revolution will be live.
On Tuesday 05 July 2005 08:17 am, Steven D'Aprano wrote: Sorry, but you are mistaken. "lambda" is a _reserved_ word in the
Python language, while "function" etc are not, but they are certainly
part of the language. Try explaining what def and import do without
using the words "function" or "module". Maybe you can do it, using
circumlocutions, but it isn't easy, and costs clarity.
This is still a relevant distinction. One relevant point is that I am
perfectly free to use, say, the Spanish or Chinese word to describe
"module" or "function", but the keywords "def" and "import" will
remain the same.
The second error is that I believe most english speakers COULD provide
a definition for the fairly common words "function", "class", and
"decorator". The exact meaning of "class" might not be what they expect
at first,
Function, in the sense of a mathematical function, I agree. Class as in
the thing you go to at school and decorator as in the person who advises
you what colour curtains to have, certainly. But in the Python sense? No.
Especially not decorator, which I believe most _programmers_ would have
trouble explaining, let alone non-programmer English speakers. I know I do.
The "Python sense" is not arbitrary. There are very direct visual or logical
(i.e. INTUITIVE) connections between these words' *English* meanings (not
just mathematical, either) and their meanings in Python.
A "Function" is (in English) (kdict-gcide*):
1.) The act of executing or performing any duty, office, or
calling; performance....
2.) The appropriate action of any special organ or
part of an animal or vegetable organism...
[...]
5.) (math) A quantity so connected with another quantity,
that if any alteration be made in the latter there will be
a consequent alteration in the former.
The programming use is probably *closer* to the English meaning
than the math jargon meaning (except in the sense of "functional
programming" which leans more on the math meaning.
Similarly, "decorate" is 'make more attractive by adding ornament, colour, etc.'
In Python, a "decorator" applies a wrapper to a function to provide it with
some additional functionality. The function definition is furthermore
"decorated" with the decorator declaration to give it more meaning, which
is arguably more aesthetically pleasing (otherwise, why not literally wrap
with a function after defining the function?). These meanings are very
connected with the English definition of the word.
"Class" can, of course, mean a room in which you teach classes, but again
Webster's will certainly provide meaning much closer to the programming
term:
1. A group of individuals ranked together as possessing
common characteristics...
[2 is the class of students sense]
3. A comprehensive division of animate or inanimate objects,
grouped together on account of their common
characteristics
4. A set; a kind or description, species or variety.
Meanings 1,3, & 4 are all arguably intimately connected to the OOP meaning,
especially meaning #3 which even mentions "objects". (And I won't bother
to point out that the English meaning of "object" is tied closely to what it
means in programming).
A similar argument can be made for "object", "module", "script", and
even "method" and "program".
Now, if you are armed ONLY with the English definition, you will possibly
run into some trouble, because the programming usage is a *specialization*
of the term -- we strictly take only *one* of the English definitions to apply,
and we narrow its meaning a bit. "Objects" in computer science never means
"the purpose of the program" nor does it ever refer to "a small brick", even
though the English word can mean both of those things. But it's not exactly
a shocker that a programming term is going to apply to things you can find
in a program, so I don't think we're stumping the newbie with such terms.
"lambda" has no such advantage. Here's the *entire* gcide definition:
Lambda \Lamb"da\, n. [NL., fr. Gr. la`mbda.]
1. The name of the Greek letter [Lambda], [lambda],
corresponding with the English letter L, l.
[1913 Webster]
2. (Anat.) The point of junction of the sagittal and lambdoid
sutures of the skull.
[1913 Webster]
3. (Phys.) A subatomic particle carrying no charge, having a
mass equal to 2183 times that of an electron; it decays
rapidly, typically forming a nucleon and a pion. --MW10
[PJC]
Lambda moth (Zool.), a moth so called from a mark on its
wings, resembling the Greek letter lambda ([Lambda]).
[1913 Webster]
but exposure to any object oriented language would make the
concept quickly familiar.
Just as exposure to functional languages would make lambda very familiar.
Yeah, well, there *is* an entry in the "Online Dictionary of Computing":
LAMBDA
A version of typed lambda-calculus, used to describe
semantic domains.
["Outline of a Mathematical Theory of Computation",
D.S. Scott, TM PRG-2, PRG, Oxford U, 1971].
If even this means what "lambda" does in Python, I would be surprised,
certainly it doesn't mean a whole lot to me.
Think back to when you were a schoolboy at your first day of school.
Unless you had a very unusual upbringing, you probably had never heard the
word "function" before.
Total BS. I knew the word "function" in it's English language sense, probably
by the time I was 6. I *know* my kids know it.
There is nothing about the word "function" that
brings to mind "a mathematical entity which transforms a variable into a
different variable",
They'll get this in about the 6th grade, IIRC.
let alone "a programming subroutine that returns a
result".
It also *does something*, which is what my first understanding of a "function"
was. Side-effects would seem to be perfectly natural to anyone with an
English language background, I guess. ;-)
(Or for that matter, "the purpose which a person or thing is
for", as in the function of a spanner is to tighten nuts on bolts.)
You are really stretching if you think kids (let alone average adults) don't
know this meaning of the word "function".
You had to learn that word, discover what it means, and then it becomes
familiar. You don't notice the process only because it happened so long
ago, at an age that your brain was operating in "language acquisition
mode" and picking up vocabulary at an incredible rate.
If you're arguing that language is acquired rather than innate, you
are bludgeoning an obvious point. The point is that *jargon* should
ideally derive in a natural way from commonly-used language, if
we want it to be easy to acquire for people who don't learn programming
between the ages of 1 and 5 as we learn our native languages. Even
in the 21st century, I think this includes just about all of us. ;-)
There is nothing about the word "string" that especially brings to mind
"an array of bytes representing characters". The analogy of "string of
characters" to "string of beads" breaks down as soon as you have multiple
lines of text.
Ah, but that's useful. "Strings" AREN'T "multiple lines of text" in the
computer's memory, are they? '\n' is just another bead. The "multiple
lines" is a representation, or way of laying out the beads. Very useful
distinction, and immediately driven by the choice of analogy.
I'm going to use that, thanks. ;-)
And as for float, that's what boats do, heaven only knows
what it has to do with numbers.
Ah, yes. Here, it is clear that Fortran whips Python on readability, it
calls them "reals". The only problem is that real mathematicians probably
had conniption fits about the fact that "real" variables actually represent
"rational" numbers.
(Yes, I know what it has to do with numbers. But that is something I had
to learn, and even now I still have difficulty because I expect floats to
operate like mathematical real numbers, and they don't.)
And dare I say it, what do constricting snakes have to do with programming?
Nothing. Then again *we* know that "Python" hasn't anything to do
with snakes, either. ;-D
I won't say that the anonymous function meaning of lambda comes to my mind
before the Greek letter, but it isn't very far behind, and rapidly
catching up. (I use lambda a lot more than I speak Greek.) It wouldn't
surprise me if one day I think of Python programming before the Greek
letter, just as the world aleph brings to my mind the sense of infinity
before the sense of it being a Hebrew letter.
Then it is clearly *not you* who should be served by the naming scheme.
Anyone so deeply trained and experienced should be expected to adapt,
you have the wherewithall to do so. It is the new user for whom the clarity
of the jargon is so important.
Personally, I find the term "anonymous function" to be a whole lot clearer
than "lambda" or "lambda function". Indeed, if asked what "lambda" means,
my reply is it's a "stupid name for an anonymous function", and if my
listener is less savvy "for a function that doesn't have a name, because you
only use it once".
Having said that, I too will miss the *concept* of an anonymous function,
although I wouldn't mind at all if its name changed, or if it were somehow
integrated into the "def" keyword's usage. Using backticks or some other
syntax delimiter also sounds promising, although we're sort of running out
of them. ;-)
--
*An unfortunate acronym for "Gnu Collaborative International Dictionary
of English".
--
Terry Hancock ( hancock at anansispaceworks.com )
Anansi Spaceworks http://www.anansispaceworks.com mc****@gmail.com wrote: concept quickly familiar. But "lambda" has a very clear meaning... it's
a letter of the greek alphabet. The connection between that letter and
anonymous functions is tenuous at best, and fails the test of making
Python read like "executable pseudocode".
But 'lambda' does have a very clear meaning in the realm of functional
programming, and it means precisely (mostly) what it means in Python: an
anonymous function.
It might not be the -best- possible name, but anyone who's had a
computer science education should have had a class that introduced basic
functional programming topics (even if only for academic interest), and
so they should be familiar with the keyword name.
If not, then it's just a magic word. Kind of like 'def'.
Up until a few years ago, I ran the computer science department at a
high-school. I provided support for the English teachers who taught
*all* students -- but they taught things like the use of a word
processor or the internet, and never covered the meaning of "lambda". I
taught a computer applications course which was taken by only small
fraction of the students (<10%) but there I taught things like the use
of photo-editing software, creating web sites, and the use of simple
databases; I never covered the meaning of "lambda". I also taught the
programming class (taken by only a dozen or so students per graduating
class) -- students learned basic concepts like variables, looping, up
through fancier bits like a couple different sorting algorithms. But I
didn't cover the meaning of "lambda". And I also taught the "AP"
computer course (taken by an average of just 4 students per year!), in
which I explained things like object oriented programming and recursion
and managed to get the students to the level where they could work
together as a group to write a moderately complex program, like a
simple video game. And I didn't teach the meaning of "lambda", nor was
it covered by the "AP" exam, which is supposed to be equivalent to a
single college-level course in computer programming.
So I'd say that it's a pretty obscure name that most people wouldn't
know.
And besides, "def" isn't a "magic" word... it's an abreviation for
"define"... I hope that any student who didn't understand a word as
common as "define" wouldn't have graduated from our school.
-- Michael Chermside
mc****@gmail.com wrote:
[snip description of experience teaching high school students] So I'd say that it's a pretty obscure name that most people wouldn't
know.
It would be hard to argue against that statement; certainly "lambda" in
this context (or probably any) is not a word "most people" would know.
On the other hand, the name itself is probably not very important. I
still remember my first encounter with "lambda" in Python very clearly.
I saw the word, thought "huh? what the heck is that?", then read a
sentence about it that included some comment about its background in
other fields.
"Oh," I said, "pre-existing usage. Whatever." I proceeded to read
about what it did and how to use it.
The name was irrelevant. If the text had said "anonymous functions are
created using the keyword 'tribble' (named for a similar feature in a
fictional Klingon programming language)", I wouldn't have felt any
differently about it. So it makes some sense to a few trekkers... big
furry deal.
What bothered me was the syntax. Arguments without parentheses? What
possessed anyone to put something so inconsistent in the language? No
statements? Dang, that will limit my interest in using them. Oh well,
what's page four of the tutorial got for me next? It shouldn't take
anyone more than ten seconds to integrate "lambda" into their brain and
carry on with useful work.
Really, the name is such a trivial, unimportant part of this whole thing
that it's hardly worth discussing. The syntax is more important, and
the limitations are of definite interest. Not the name.
-Peter
On 2005-07-05, mc****@gmail.com <mc****@gmail.com> wrote: Up until a few years ago, I ran the computer science department at a
high-school. I provided support for the English teachers who taught
*all* students -- but they taught things like the use of a word
processor or the internet,
That's not computer science.
and never covered the meaning of "lambda". I taught a computer
applications course which was taken by only small fraction of
the students (<10%) but there I taught things like the use of
photo-editing software, creating web sites, and the use of
simple databases;
That's not computer science.
I never covered the meaning of "lambda". I also taught the
programming class (taken by only a dozen or so students per
graduating class) -- students learned basic concepts like
variables, looping, up through fancier bits like a couple
different sorting algorithms.
Now you're getting a little closer to computer science.
It sounds like you ran a computer user training department. I
don't think it could be called computer science.
But I didn't cover the meaning of "lambda". And I also taught
the "AP" computer course (taken by an average of just 4
students per year!), in which I explained things like object
oriented programming and recursion and managed to get the
students to the level where they could work together as a
group to write a moderately complex program, like a simple
video game. And I didn't teach the meaning of "lambda", nor
was it covered by the "AP" exam, which is supposed to be
equivalent to a single college-level course in computer
programming.
Computer programming isn't the same thing as computer science.
It's just one of the tools used to do computer science.
So I'd say that it's a pretty obscure name that most people
wouldn't know.
I can't believe that anybody with any computer science
background doesn't know it.
And besides, "def" isn't a "magic" word... it's an abreviation
for "define"... I hope that any student who didn't understand
a word as common as "define" wouldn't have graduated from our
school.
Lamda isn't a magic word either. It comes from lambda
calculus.
--
Grant Edwards grante Yow! Is this where people
at are HOT and NICE and they
visi.com give you TOAST for FREE??
def flatten(iterable):
if not hasattr(iterable, '__iter__'):
return [iterable]
return sum([flatten(element) for element in iterable],[])
Recursion makes things so much shorter.
Grant Edwards enlightened us with: It sounds like you ran a computer user training department. I don't
think it could be called computer science.
Being a computer science student at the University of Amsterdam, I can
tell you that it definitely should not be called "computer science".
I can't believe that anybody with any computer science background
doesn't know it.
I agree.
Sybren
--
The problem with the world is stupidity. Not saying there should be a
capital punishment for stupidity, but why don't we just take the
safety labels off of everything and let the problem solve itself?
Frank Zappa
"Devan L" wrote: def flatten(iterable):
if not hasattr(iterable, '__iter__'):
return [iterable]
return sum([flatten(element) for element in iterable],[])
Recursion makes things so much shorter.
The last line can faster and more compact by:
from itertools import imap
def flatten(iterable):
if not hasattr(iterable, '__iter__'):
return [iterable]
return sum(imap(flatten,iterable),[])
George
On Tue, 5 Jul 2005, Terry Hancock wrote: Having said that, I too will miss the *concept* of an anonymous
function, although I wouldn't mind at all if its name changed, or if it
were somehow integrated into the "def" keyword's usage. Using backticks
or some other syntax delimiter also sounds promising, although we're
sort of running out of them. ;-)
I understand that the backslash is popular in some ivory-tower functional
languages. Currently, a backslash can be used for explicit line joining,
and is illegal elsewhere on a line outside a string literal, so i think
it's available for this. It would be utterly unpythonic to use puntuation
instead of a keyword, and it would make no sense to novices, but it would
scare the crap out of C programmers, which has to be worth something.
tom
--
[Philosophy] is kind of like being driven behind the sofa by Dr Who - scary, but still entertaining. -- Itchyfidget
Hi All--
Tom Anderson wrote:
I understand that the backslash is popular in some ivory-tower functional
languages. Currently, a backslash can be used for explicit line joining,
and is illegal elsewhere on a line outside a string literal, so i think
it's available for this. It would be utterly unpythonic to use puntuation
instead of a keyword, and it would make no sense to novices, but it would
scare the crap out of C programmers, which has to be worth something.
Oh, I don't think so. Don't forget that Perl was written by impatient C
programmers. I think scaring C programmers, like giving engineers too
much information, is really hard to do. Live by the sword, die by the
sword.
Metta,
<int *(*(*foo)())()>-ly y'rs,
Ivan
----------------------------------------------
Ivan Van Laningham
God N Locomotive Works http://www.andi-holmes.com/ http://www.foretec.com/python/worksh...oceedings.html
Army Signal Corps: Cu Chi, Class of '70
Author: Teach Yourself Python in 24 Hours
Tom Anderson wrote: The trouble with these is that they make a lot of temporary lists -
George's version does it with the recursive calls to flatten, and Ron's
with the slicing and concatenating. How about a version which never
makes new lists, only appends the base list? We can use recursion to
root through the lists ...
Ok... How about a non-recursive flatten in place? ;-)
def flatten(seq):
i = 0
while i!=len(seq):
while isinstance(seq[i],list):
seq.__setslice__(i,i+1,seq[i])
i+=1
return seq
seq = [[1,2],[3],[],[4,[5,6]]]
print flatten(seq)
I think I'll be using the __setslice__ method more often.
And the test:
#--------------------------------
# Georges recursive flatten
init_a = """
def flatten(seq):
return reduce(_accum, seq, [])
def _accum(seq, x):
if isinstance(x,list):
seq.extend(flatten(x))
else:
seq.append(x)
return seq
seq = [[1,2],[3],[],[4,[5,6]]]
"""
# Ron's non-recursive
init_b = """
def flatten(seq):
s = []
while seq:
while isinstance(seq[0],list):
seq = seq[0]+seq[1:]
s.append(seq.pop(0))
return s
seq = [[1,2],[3],[],[4,[5,6]]]
"""
# Tom's recursive, no list copies made
init_c = """
def isiterable(x):
return hasattr(x, "__iter__") # close enough for government work
def visit(fn, x): # perhaps better called applytoall
if (isiterable(x)):
for y in x: visit(fn, y)
else:
fn(x)
def flatten(seq):
a = []
def appendtoa(x):
a.append(x)
visit(appendtoa, seq)
return a
seq = [[1,2],[3],[],[4,[5,6]]]
"""
# Devan' smallest recursive
init_d = """
def flatten(iterable):
if not hasattr(iterable, '__iter__'):
return [iterable]
return sum([flatten(element) for element in iterable],[])
seq = [[1,2],[3],[],[4,[5,6]]]
"""
# Ron's non-recursive flatten in place! Much faster too!
init_e = """
def flatten(seq):
i = 0
while i!=len(seq):
while isinstance(seq[i],list):
seq.__setslice__(i,i+1,seq[i])
i+=1
return seq
seq = [[1,2],[3],[],[4,[5,6]]]
"""
import timeit
t = timeit.Timer("flatten(seq)",init_a)
print 'recursive flatten:',t.timeit()
import timeit
t = timeit.Timer("flatten(seq)",init_b)
print 'flatten in place-non recursive:',t.timeit()
import timeit
t = timeit.Timer("flatten(seq)",init_c)
print 'recursive-no copies:',t.timeit()
import timeit
t = timeit.Timer("flatten(seq)",init_d)
print 'smallest recursive:',t.timeit()
import timeit
t = timeit.Timer("flatten(seq)",init_e)
print 'non-recursive flatten in place without copies:',t.timeit()
#--------------------------------------------
The results on Python 2.3.5: (maybe someone can try it on 2.4)
recursive flatten: 23.6332723852
flatten in place-non recursive: 22.1817641628
recursive-no copies: 30.909762833
smallest recursive: 35.2678756658
non-recursive flatten in place without copies: 7.8551944451
A 300% improvement!!!
This shows the value of avoiding copies, recursion, and extra function
calls.
Cheers,
Ron Adam Ok... How about a non-recursive flatten in place? ;-)
def flatten(seq):
i = 0
while i!=len(seq):
while isinstance(seq[i],list):
seq.__setslice__(i,i+1,seq[i])
i+=1
return seq
seq = [[1,2],[3],[],[4,[5,6]]]
print flatten(seq)
I think I'll be using the __setslice__ method more often.
This is probably the more correct way to do it. :-)
def flatten(seq):
i = 0
while i!=len(seq):
while isinstance(seq[i],list):
seq[i:i+1]=seq[i]
i+=1
return seq
On Tue, 05 Jul 2005 09:46:41 -0500, Terry Hancock wrote: On Tuesday 05 July 2005 08:17 am, Steven D'Aprano wrote: Sorry, but you are mistaken. "lambda" is a _reserved_ word in the
Python language, while "function" etc are not, but they are certainly
part of the language. Try explaining what def and import do without
using the words "function" or "module". Maybe you can do it, using
circumlocutions, but it isn't easy, and costs clarity.
This is still a relevant distinction. One relevant point is that I am
perfectly free to use, say, the Spanish or Chinese word to describe
"module" or "function", but the keywords "def" and "import" will
remain the same.
No, that isn't relevant. That's just localisation. I'm also free to fork
Python and change the keywords.
[snip] The "Python sense" is not arbitrary. There are very direct visual or
logical (i.e. INTUITIVE) connections between these words' *English*
meanings (not just mathematical, either) and their meanings in Python.
I had NEVER even heard the word "tuple" before learning Python. I spent
weeks mispelling it as "turple", and I finally had to look it up in a
dictionary to see if it was a real English word. Out of the four English
dictionaries in my house, none of them have the word.
Let's dump tuple from the language too, its just "a stupid name for a list
that can't change". Agreed?
[snip] Similarly, "decorate" is 'make more attractive by adding ornament,
colour, etc.' In Python, a "decorator" applies a wrapper to a function
to provide it with some additional functionality.
Which is certainly NOT decoration, since the point of decoration is that
it is not functional! Plain bathroom tiles are functional. Decorating them
with leaves or flowers or patterns is not functional.
[snip] Now, if you are armed ONLY with the English definition, you will
possibly run into some trouble, because the programming usage is a
*specialization* of the term -- we strictly take only *one* of the
English definitions to apply, and we narrow its meaning a bit.
Yes. Every specialization has its own jargon, and computer programming is
no different. One of the things new programmers have to learn is the
jargon.
[snip] "lambda" has no such advantage. Here's the *entire* gcide definition:
Nonsense. All that means is that lambda in the programming sense is too
specialized to make it into most ordinary dictionaries. Just like tuple.
[snip]
Think back to when you were a schoolboy at your first day of school.
Unless you had a very unusual upbringing, you probably had never heard
the word "function" before.
Total BS. I knew the word "function" in it's English language sense,
probably by the time I was 6. I *know* my kids know it.
Okay, maybe my memories of being five are faulty.
But if not six, then five, or four, or three, or two. At _some_time_
function was entirely unknown to you, and you had to just learn it.
[snip]
You had to learn that word, discover what it means, and then it becomes
familiar. You don't notice the process only because it happened so long
ago, at an age that your brain was operating in "language acquisition
mode" and picking up vocabulary at an incredible rate.
If you're arguing that language is acquired rather than innate, you are
bludgeoning an obvious point. The point is that *jargon* should ideally
derive in a natural way from commonly-used language,
That's a bonus, sure. It is an important rule to follow, but not at the
expense of distorting the language:
t = immutable_list(L)
map(anonymous_function x: x+1, L)
versus:
t = tuple(L)
map(lambda x: x+1, L)
if we want it to be
easy to acquire for people who don't learn programming between the ages
of 1 and 5 as we learn our native languages. Even in the 21st century,
I think this includes just about all of us. ;-)
If they can't memorize one or two things, they aren't going to be much
good at programming no matter how easy the language is to use.
There is nothing about the word "string" that especially brings to mind
"an array of bytes representing characters". The analogy of "string of
characters" to "string of beads" breaks down as soon as you have
multiple lines of text.
Ah, but that's useful. "Strings" AREN'T "multiple lines of text" in the
computer's memory, are they? '\n' is just another bead. The "multiple
lines" is a representation, or way of laying out the beads. Very useful
distinction, and immediately driven by the choice of analogy.
Who cares about the implementation details of how the bytes are laid out
in the computer's memory? Unless you are programming in a low level
language like C or assembly, this is entirely irrelevant. You have
characters laid out along lines, and lines laid out in a second dimension.
The natural way to work with text is something like this:
for line in text:
for word in line: # or character
do_something()
[snip]
I won't say that the anonymous function meaning of lambda comes to my
mind before the Greek letter, but it isn't very far behind, and rapidly
catching up. (I use lambda a lot more than I speak Greek.) It wouldn't
surprise me if one day I think of Python programming before the Greek
letter, just as the world aleph brings to my mind the sense of infinity
before the sense of it being a Hebrew letter.
Then it is clearly *not you* who should be served by the naming scheme.
Anyone so deeply trained and experienced should be expected to adapt,
you have the wherewithall to do so. It is the new user for whom the
clarity of the jargon is so important.
Personally, I find the term "anonymous function" to be a whole lot
clearer than "lambda" or "lambda function". Indeed, if asked what
"lambda" means, my reply is it's a "stupid name for an anonymous
function", and if my listener is less savvy "for a function that doesn't
have a name, because you only use it once".
And any half-way experienced Python programmer will say, "What are you
talking about?"
add_one = lambda x: x+1
Yes, I know that the name "add_one" is not the same as the name for a
function when you use def, but the function still has a name. The
difference might be an important difference, but it is not one you care
about unless you are doing introspection.
Having said that, I too will miss the *concept* of an anonymous
function, although I wouldn't mind at all if its name changed, or if it
were somehow integrated into the "def" keyword's usage.
Def would be short for ... defend? defile? defer? defame? default? deflect?
There's always *something* to learn. Why def instead of define? Because
"easy to write" beats "instantly obvious to a beginner", if the word is
used all the time and is easy to memorize.
Using backticks
or some other syntax delimiter also sounds promising,
Ew, perl :-(
although we're sort of running out of them. ;-)
--
Steven.
On Tue, 05 Jul 2005 12:11:47 -0700, mcherm wrote: And besides, "def" isn't a "magic" word... it's an abreviation for
"define"...
Really? I thought it was an abbreviation for "definition". As in,
"definition of MyFunc is..."
I hope that any student who didn't understand a word as
common as "define" wouldn't have graduated from our school.
How about tuple?
--
Steven.
On Tue, 5 Jul 2005, Ron Adam wrote: Tom Anderson wrote:
The trouble with these is that they make a lot of temporary lists -
George's version does it with the recursive calls to flatten, and Ron's
with the slicing and concatenating. How about a version which never
makes new lists, only appends the base list? We can use recursion to
root through the lists ...
Ok... How about a non-recursive flatten in place? ;-)
How about, er, oh, i give up.
def flatten(seq):
i = 0
while i!=len(seq):
while isinstance(seq[i],list):
seq.__setslice__(i,i+1,seq[i])
i+=1
return seq
seq = [[1,2],[3],[],[4,[5,6]]]
print flatten(seq)
I think I'll be using the __setslice__ method more often.
Um, can't you just do slice assignment? Make that line:
seq[i:i+1] = seq[i]
And the test:
Stupendous and timely work!
The results on Python 2.3.5: (maybe someone can try it on 2.4)
recursive flatten: 23.6332723852
flatten in place-non recursive: 22.1817641628
recursive-no copies: 30.909762833
smallest recursive: 35.2678756658
non-recursive flatten in place without copies: 7.8551944451
GAAAAH!
A 300% improvement!!!
This shows the value of avoiding copies, recursion, and extra function
calls.
Specifically, it shows the value of avoiding extra function calls, since
my zerocopy version is slower than the copy-happy single-function
versions.
Also, there are some differences between the functions which constitute
potential hillocks on the playing field - i test flattenability by looking
for an __iter__ method, whereas other implementations mostly ask
"instanceof list?" (less general, and less in the spirit of duck typing,
IMNERHO). I'd argue that my decomposition into functions is this sort of
difference, too - a matter of style (good style!), not algorithm. So,
levelling those differences, and throwing in my non-recursive zerocopy
foolery, here's my take on it ...
# ----
# here be a quick reimplementation of timeit to time function objects
# no exec for me no siree bob
# all you need to know is that timeit(fn) gives you time taken to run fn
import sys
import time
TIMERS = {
"win32": time.clock
}
timer = TIMERS.get(sys.platform, time.time)
def timeit(fn, n=None):
if (n == None):
t = 0.1
n = 1
while (t < 1.0):
n = max(int((n * min((1.0 / t), 10))), (n + 1))
t = _timeit(fn, n)
else:
t = _timeit(fn, n)
return t / n
def _timeit(fn, n):
it = xrange(n)
t0 = timer()
for i in it:
fn()
t1 = timer()
return float((t1 - t0))
# there is real code now
# i've rewritten the functions to use uniform variable names
# and to use isiterable
def isiterable(obj):
return hasattr(obj, "__iter__")
def georges_recursive_flatten(seq):
return reduce(_accum, seq, [])
def _accum(a, item):
if isiterable(item):
a.extend(georges_recursive_flatten(item))
else:
a.append(item)
return a
def rons_nonrecursive_flatten(seq):
a = []
while seq:
while isiterable(seq[0]):
seq = seq[0] + seq[1:]
a.append(seq.pop(0))
return a
def toms_recursive_zerocopy_flatten(seq, a=[]):
if (isiterable(seq)):
for item in seq:
toms_recursive_zerocopy_flatten(item, a)
else:
a.append(seq)
return a
def toms_iterative_zerocopy_flatten(seq):
stack = [None]
cur = iter(seq)
a = []
while (cur != None):
try:
item = cur.next()
if (isiterable(item)):
stack.append(cur)
cur = iter(item)
else:
a.append(item)
except StopIteration:
cur = stack.pop()
return a
def devans_smallest_recursive_flatten(seq):
if (isiterable(seq)):
return sum([devans_smallest_recursive_flatten(item) for item in seq], [])
else:
return [seq]
def rons_nonrecursive_inplace_flatten(seq):
i = 0
while (i != len(seq)):
while (isiterable(seq[i])):
seq[i:(i + 1)] = seq[i] # setslice takes iterators!
i = i + 1
return seq
flattens = [
georges_recursive_flatten,
rons_nonrecursive_flatten,
toms_recursive_zerocopy_flatten,
toms_iterative_zerocopy_flatten,
devans_smallest_recursive_flatten,
rons_nonrecursive_inplace_flatten
]
seq = [[1,2],[3],[],[4,[5,6]]]
def timeflatten(flatten):
return timeit(lambda: flatten(seq))
def funcname(fn):
return fn.func_name
print zip(map(funcname, flattens), map(timeflatten, flattens))
# ----
The output (in python 2.3 on a 1.5 GHz G4 pbook with OS X 10.3) is:
[
('georges_recursive_flatten', 0.00015331475192276888),
('rons_nonrecursive_flatten', 0.00015447115513356376),
('toms_recursive_zerocopy_flatten', 0.00012239551614106925),
('toms_iterative_zerocopy_flatten', 0.00035910996630353429),
('devans_smallest_recursive_flatten', 0.00019606360118084218),
('rons_nonrecursive_inplace_flatten', 5.8524826144294404e-05)
]
So, my zerocopy flatten is, after all, faster than George, you and Devan's
copy-happy versions, although not by much, my iterative version is much,
much slower, and the winner is still your in-place flatten, which beats my
not-too-bad 122 microseconds with a scorching 58!
I guess setslice is a lot faster than i thought. How are python lists
implemented? Presumably not as straightforward arrays, where inserting a
bunch of items at the head would mean moving everything else in the list
along?
We really ought to do this benchmark with a bigger list as input - a few
thousand elements, at least. But that would mean writing a function to
generate random nested lists, and that would mean specifying parameters
for the geometry of its nestedness, and that would mean exploring the
dependence of the performance of each flatten on each parameter, and that
would mean staying up until one, so i'm not going to do that.
tom
--
[Philosophy] is kind of like being driven behind the sofa by Dr Who - scary, but still entertaining. -- Itchyfidget
Grant Edwards wrote: mc****@gmail.com wrote: So I'd say that it's a pretty obscure name that most people
wouldn't know.
I can't believe that anybody with any computer science
background doesn't know it.
Perhaps this reflects on the quality of education in the United States
;) but I managed to get a BS in Computer Science at the University of
Arizona without ever seeing the word lambda (in the programming
languages sense).
However, I only took one class in programming languages, and it was more
of a survey class than a theory class. When I did take a theory class,
here at University of Colorado at Boulder, they did, of course, cover
lambda calculus. But there was at least a year or two in which I would
have considered myself somebody "with any computer science background"
who wasn't familiar with lambda.
OTOH, I fully agree with Peter Hansen: "Really, the name is such a
trivial, unimportant part of this whole thing that it's hardly worth
discussing."
STeVe
"Steven D'Aprano" <st***@REMOVETHIScyber.com.au> wrote: On Tue, 05 Jul 2005 09:46:41 -0500, Terry Hancock wrote:
[snip] Having said that, I too will miss the *concept* of an anonymous
function, although I wouldn't mind at all if its name changed, or if it
were somehow integrated into the "def" keyword's usage.
Def would be short for ... defend? defile? defer? defame? default? deflect?
There's always *something* to learn. Why def instead of define? Because
"easy to write" beats "instantly obvious to a beginner", if the word is
used all the time and is easy to memorize.
Still it's hard to explain why four specific python keywords - def,
del, exec and elif - were chosen to be abbreviated, while all the rest
are full words ( http://docs.python.org/ref/keywords.html). "Ease of
typing" is a joke for an excuse; "finally" is longer than
"define","delete" and equally long to "execute" and "else if" (the
latter has the even more serious flaw of going against TOOWTDI and in
this case practicallity hardly beats purity IMO). In any case, python
was never about minimizing keystrokes; theres another language that
strives for this <wink>.
So, who would object the full-word versions for python 3K ?
def -> define
del -> delete
exec -> execute
elif -> else if
George
"Tom Anderson" <tw**@urchin.earth.li> wrote in message
news:Pi*******************************@urchin.eart h.li... I guess setslice is a lot faster than i thought.
Nope, the example is too short to be meaningful.
How are python lists
implemented? Presumably not as straightforward arrays, where inserting a
bunch of items at the head would mean moving everything else in the list
along?
They *are* extensible arrays, and insertion *does* mean exactly that --
moving everything else along -- although CPython probably does it with C
memcopy(). So the littly puny 'test' example with about 20 total elements
is testing overhead, not large-list behavior. So you are right, *much*
longer lists are needed. You could start, for example, with a list of 1000
lists of 1000 elements (all 0 would be fine). Or maybe 100 of 100 of 100.
For that, I would start with something that appended and extended the
result list. Or try writing a generator iflatten that yielded elements in
order:
def flatten(iterable): return list(iflatten(iterable))
which pushes appending down into the C code of list().
Terry J. Reedy
"Steven Bethard" <st************@gmail.com> wrote in message
news:-8********************@comcast.com... OTOH, I fully agree with Peter Hansen: "Really, the name is such a
trivial, unimportant part of this whole thing that it's hardly worth
discussing."
From a certain viewpoint, I would agree. Yet, the word 'lambda' *is* the
center of most of the fuss. For beginners, it is a minor issue: learn it
and move on. But for some functionalists, it is a major issue. They
'know' that lambda means 'expressionized anonymous function'. And in
lambda calculus, it is the main actor. But in Python, lambda only means
anonymous trivial function. It is only an expressionized convenience
abbreviation for an important but small subset of possible functions. So
for years, such knowledgeable people have called for and proposed various
syntaxes for 'proper lambdas' or 'true lambdas', saying pretty clearly that
what Python has is improper or false. Would there have been so much fuss
if the keyword had been 'fun' and the word 'lambda' had never appeared in
the Python docs? I strongly doubt it.
I also suspect that the years of fuss over Python's lambda being what it is
rather that what it is 'supposed' to be (and is in other languages) but is
not, has encourage Guido to consider just getting rid of it. I personally
might prefer keeping the feature but using a different keyword.
Terry J. Reedy
"George Sakkis" <gs*****@rutgers.edu> wrote in message
news:11*********************@g43g2000cwa.googlegro ups.com... Still it's hard to explain why four specific python keywords - def,
del, exec and elif - were chosen to be abbreviated,
Precedence in other languages and CS usage?
So, who would object the full-word versions for python 3K ?
def -> define
del -> delete
exec -> execute
These three I might prefer to keep.
elif -> else if
This one I dislike and would prefer to write out. I never liked it in
whatever else language I first encountered it and still don't.
Terry J. Reedy
Tom Anderson wrote:
We really ought to do this benchmark with a bigger list as input - a few
thousand elements, at least. But that would mean writing a function to
generate random nested lists, and that would mean specifying parameters
for the geometry of its nestedness, and that would mean exploring the
dependence of the performance of each flatten on each parameter, and
that would mean staying up until one, so i'm not going to do that.
tom
Without getting to picky, would this do?
import random
import time
random.seed(time.time())
def rand_depth_sequence(seq):
for n in range(len(seq)):
start = random.randint(0,len(seq)-2)
end = random.randint(start,start+3)
seq[start:end]=[seq[start:end]]
return seq
seq = rand_depth_seq(range(100))
print seq
[[[[[0, 1], 2, [3, [4, 5, 6]]], [7], [8, [[9, [], 10]]]], [11, [12]],
[[[]]]], [[], [13, 14, 15]], [[[16, 17]]], [18], [[[19], 20, [[21, 22,
[23]], [[24]]]], 25, 26], [], [27, 28], [], 29, [], [30, [31, 32, 33]],
[34], [[35]], 36, [[37, 38, 39], [[40, 41], [[42]]]], [43, 44], 45, 46,
[47, []], [[[48, 49], [50], [51]], 52], [[[53], [54, 55, [56, 57, 58]],
[]], [], []], [[59, 60, 61]], 62, [[63]], [], [64], [[[65]]], [[[66, 67,
68], [69, 70, [71, 72]], [[73, 74], [75, 76]]], [77, 78]], [], 79, 80,
[[81], []], 82, [[[83, [[], 84], [85]], 86, [[87, 88]]], [[[89], 90,
91], [92, [93], [94, 95, 96]]]], [97, [98, 99]]]
Devan L wrote: Claiming that sum etc. do the same job is the whimper of
someone who doesn't want to openly disagree with Guido.
Could you give an example where sum cannot do the job(besides the
previously mentioned product situation?
Here's a couple of examples from my own code:
# from a Banzhaf Power Index calculator
# adds things that aren't numbers
return reduce(operator.add,
(VoteDistributionTable({0: 1, v: 1}) for v in electoral_votes))
# from a custom numeric class
# converts a tuple of digits into a number
mantissa = sign * reduce(lambda a, b: 10 * a + b, mantissa)
Also, map is easily replaced.
map(f1, sequence) == [f1(element) for element in sequence]
There's also the issue of having to rewrite old code.
Dan Bishop wrote: There's also the issue of having to rewrite old code.
It's Python 3000. You will have to rewrite old code regardless if reduce
stays.
--
Robert Kern rk***@ucsd.edu
"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter
Terry Reedy wrote: I also suspect that the years of fuss over Python's lambda being what it is
rather that what it is 'supposed' to be (and is in other languages) but is
not, has encourage Guido to consider just getting rid of it. I personally
might prefer keeping the feature but using a different keyword.
Terry J. Reedy
Yes, I think a different key word would help. My current favorite
alternative is to put it in parentheses similar to list comprehensions
and use "let".
(let x,y return x+y)
Or you could just explain lambda as let, they both begin with 'L', and
then the colon should be read as return.
So lambda x,y: x+y should be read as: let x,y return x+y
I'm in the group that hadn't heard about lambda as a function before
Python even after > twenty years of computer tech experience. I think
presuming it's common knowledge is a mistake.
Cheers, Ron
Terry Reedy wrote: "George Sakkis" <gs*****@rutgers.edu> wrote in messageSo, who would object the full-word versions for python 3K ?
def -> define
del -> delete
exec -> execute
These three I might prefer to keep.
elif -> else if
This one I dislike and would prefer to write out. I never liked it in
whatever else language I first encountered it and still don't.
Terry J. Reedy
Interesting, the exact opposite of what I was thinking.
I don't use del and exec that often, so the long versions are fine to
me. Define is ok for me too because it's usually done only once for
each function or method so I'm not apt to have a lot of defines repeated
in a short space like you would in C declarations.
elif... I was thinking we should keep that one because it's used fairly
often and having two keywords in sequence doesn't seem like it's the
beat way to do it.
Although it could be replaced with an 'andif' and 'orif' pair. The
'andif' would fall though like a C 'case', and the 'orif' would act just
like the 'elif'. Actually this is a completely differnt subject
reguarding flow testing verses value testing. Else and also would be
the coorisponding end pair, but it seemed nobody really liked that idea
when I suggested it a while back. <shrug>
Cheers,
Ron
I was going to drop the lambda discussion, as it has
been going on and on and on, but Terry's comment
strikes me as so wrong that it needs to be challenged.
Terry Reedy wrote: From a certain viewpoint, I would agree. Yet, the word 'lambda' *is* the
center of most of the fuss. For beginners, it is a minor issue: learn it
and move on. But for some functionalists, it is a major issue. They
'know' that lambda means 'expressionized anonymous function'. And in
lambda calculus, it is the main actor. But in Python, lambda only means
anonymous trivial function. It is only an expressionized convenience
abbreviation for an important but small subset of possible functions. So
for years, such knowledgeable people have called for and proposed various
syntaxes for 'proper lambdas' or 'true lambdas', saying pretty clearly that
what Python has is improper or false. Would there have been so much fuss
if the keyword had been 'fun' and the word 'lambda' had never appeared in
the Python docs? I strongly doubt it.
People object to the fact that lambda doesn't allow
statements. They do this, not because they know about
the lambda calculus (I don't!) but because they keep
trying to do things like this:
map(lambda x: if x == 0: 1; else: math.sin(x)/x,
myList)
Hands up who thinks this usage case would disappear if
lambda was called "fun" or "anonymous_function" instead?
I also suspect that the years of fuss over Python's lambda being what it is
rather that what it is 'supposed' to be (and is in other languages) but is
not, has encourage Guido to consider just getting rid of it. I personally
might prefer keeping the feature but using a different keyword.
Best of all would be if lambda was extended to allow
statements, just like a real made-with-def function.
Although, I worry about syntax and readability. But
then I'm not completely comfortable with the existing
lambda syntax either.
And now, I shall say no more on this issue.
--
Steven. This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Stephen C. Waterbury |
last post by:
This seems like it ought to work, according to the
description of reduce(), but it doesn't. Is this
a bug, or am I missing something?
Python 2.3.2 (#1, Oct 20 2003, 01:04:35)
on linux2
Type...
|
by: Ben |
last post by:
Hi all,
I'm trying to figure out how how complex map, filter and reduce work
based on the following piece of code from
http://www-106.ibm.com/developerworks/linux/library/l-prog.html :
...
|
by: Logan |
last post by:
I read in PEP 279 the following comment by GvR:
"filter and map should die and be subsumed into list
comprehensions, not grow more variants."
Actually, I am using 'filter' and 'map' a lot in...
|
by: Thomas Philips |
last post by:
To experiment with filtering, I define a function f(x,k) as follows
>>> def f(x,k=2):
return x%k==0
I can check that it works by typing
>>> f(10,3)
False
Now, I try to filter a range using...
|
by: Charlie Taylor |
last post by:
I find that I use lambda functions mainly for callbacks to things like
integration or root finding routines as follows.
flow = integrate(lambda x: 2.0*pi * d(x)* v(x) * sin(a(x)),xBeg, xEnd)
...
|
by: Cameron Laird |
last post by:
QOTW: "... and to my utter surprise it worked." - Andrew Nagel on
his move from wxPython to programming Tkinter in desperation
"Python has more web application frameworks than keywords." - Skip...
|
by: lars_woetmann |
last post by:
I have a list I filter using another list and I would like this to be
as fast as possible
right now I do like this:
i tried using the method filter:
filter(lambda x: x not in list2, list1)
|
by: charles.hebert |
last post by:
Hi,
Can anybody tell me how to to find the nearest value to zero in a list
?
To do that, i'm using list comprenhension :
Something simpler ?
|
by: cnb |
last post by:
This must be because of implementation right? Shouldn't reduce be
faster since it iterates once over the list?
doesnt sum first construct the list then sum it?
-----------------------
reduce...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: nemocccc |
last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
| |
