reduce() anomaly?

[Stephen C. Waterbury]

This seems like it ought to work, according to the
description of reduce(), but it doesn't. Is this
a bug, or am I missing something?

Python 2.3.2 (#1, Oct 20 2003, 01:04:35)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.

d1 = {'a':1}
d2 = {'b':2}
d3 = {'c':3}
l = [d1, d2, d3]
d4 = reduce(lambda x, y: x.update(y), l) Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 1, in <lambda>
AttributeError: 'NoneType' object has no attribute 'update' d4 = reduce(lambda x, y: x.update(y), l, {})

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 1, in <lambda>
AttributeError: 'NoneType' object has no attribute 'update'

- Steve.

[Dang Griffith]

On Wed, 12 Nov 2003 13:19:15 -0500, Douglas Alan <ne****@mit.edu >
wrote:

I would say that if you didn't get introduced to at least the concept
of functional programming and a hint of how it works, then something
was seriously wrong with your CS education.
|>oug

I've been lurking this thread for a while, but missed the beginning.
Can I ask where you got your CS education (school and year)?
From the conversation, it seems that right or wrong, your education
exceeds those in this group. Where should I send my children
such that they receive a better CS education that I?
--dang

[Terry Reedy]

"Douglas Alan" <ne****@mit.edu > wrote in message
news:lc******** ****@gaffa.mit. edu...

Even without any algebra, any kid can tell you that 1 + 2 is the same as 2 + 1. Replace 1 and 2 by a and b and you get the same result.
Yes, but they are still two *different* ways to to get to that

result. Starting with a and adding b to it, is not the same thing as starting with b and adding a to it.
I disagree. Number addition (like several other functions) is a bag
(multiset) reduction operator that reduces a (possibly empty)
collection of similar items to one of the same type. Abstractly, the
reduction is simultaneous. One appropriate notation for such
operators is a simple labelled loop (in 2D) containing the items to be
collected together. In linear text, such loops can be represented by
matched symbols, such as ( ), [ ], { }, which then represent the
intersection of a closed loop with a 'too-narrow' text band (which
thereby chops off the top and bottom of the loop). When operands
scattered in the loop interior are projected down to a 1+D band for
text representation, they gain an ordering that is an artifact of that
limited representation.

(+ item item ...) and sum(item, item, ...) are both labelled loops
with items arbitrarily but necessarily ordered for linear text
representation.
It is only the commutative law of arithmetic,
as any good second grade student can tell you,
that guarantees that the result will be the same.

Well, second graders are mislead by the representation of an n-ary bag
operation with infix notation that implies binarity and order
relevance. Learning about reduce should help one unlearn these
implications ;-).

Even with infix notation, there is a choice between making the default
interpretation be order relevance or order irrelavance. If
mathematicians had made the second choice, therr would be no need for
a 'commutative law' One would say that a+b == b+a because that is the
way it normally is, while a-b != b-a because of the OrderRelevance Law
of subtraction.

< On the other hand, not all mathematical groups are albelian,
< and consequently, a + b != b + a for all mathematical groups.

Yes, there are order-relevant binary-only operations. Number addition
is not one of them.

[Having written the above, I am less enthralled with the overloading
of '+' and operator.add to also mean ordered concatenation of
sequences. The fact that operator.add becomes O(n**2) (usually
unnecessarily) instead of O(n) for such overloadings, to the point
that sum() specially excludes such overloadings for strings,
reinforces this doubt. Perhaps there should be at least be an
operator.cat that would generalize ''.join(strseq)]

Terry J. Reedy

[Alex Martelli]

Douglas Alan wrote:

Alex Martelli <al***@aleax.it > writes:

no disagreement, reduce is in line with that philosophy sum is a
shortcut and as others have said is less general.

'sum' is _way simpler_: _everybody_ understands what it means to sum a
bunch of numbers, _without_ necessarily having studied computer science.
Your claim is silly. sum() is not *way* simpler than reduce(), and
anyone can be explained reduce() in 10 seconds: "reduce() is just like

As you hedged elsewhere, anyone can be EXPLAINED it, but not anyone
will UNDERSTAND the explanation instantly. Now THAT hedging is way
silly, and of course Shakespeare did it best:

"""
Glendower: I can call spirits from the vasty deep.

Hotspur: Why, so can I, or so can any man; But will they come when you do
call for them?
"""
sum(), only with reduce() you can specify whatever addition function
you would like."
This explanation is misleading by overqualificati on (that "addition" is
the overqualificati on). A non-overqualified explanation would have to
mention that ANY callable that can accept two arguments, and return a
result suitable to be its first argument next, can be passed; and it would
have to touch on what happens when the sequence is empty, and the optional
third argument. Yes, you COULD cram it into 10 seconds -- people won't
UNDERSTAND what you're saying, of course, but clearly you don't care
about such trifles, as you have abundantly shown.

The claim, made by somebody else, that _every_ CS 101 course teaches the
functionality of 'reduce' is not just false, but utterly absurd:
'reduce', 'foldl', and similar higher-order functions, were not taught to
me back when _I_ took my first university exam in CS [it used Fortran as
te main language],

Then you weren't taught Computer Science -- you were taught Fortran
programming. Computer Science teaches general concepts, not specific
languages.

CS teaches general concepts and in all decent CS 101 course it starts
doing so with concrete examples (because THAT is how most people learn
best), ideally with an executable computer language so you can TRY things.
It is quite likely that higher-order functions won't be taught yet in
CS 101, or that only a very few specific cases will be shown and reduce
(or foldr or however you want to call it) will not be one of them.

Your false and utterly absurd claim about the content of _every_ CS 101
course has been totally demolished by now -- by many people, including
CS majors. I'm sure you'd be happy to "undo" it, but you've invested
so much emotional energy on it and made it a lynchpin of your argument
in favour of 'reduce' that I understand that's just too hard. Still,
by standing on such rotten foundations, your argument is reduced (heh)
to near-absurdity...

they were not taught to my son in _his_ equivalent course [it used
Pascal], and are not going to be taught to my daughter in _her_
equivalent course [it uses C].

Then your children were done a great diservice by receiving a poor
education. (Assuming that is that they wanted to learn Computer
Science, and not Programming in Pascal or Programming in C.)

They wanted to learn enough computer science to get on with their
real interests in life -- telecom engineering and financial economics
respectively. They may well take second or third courses on CS issues
in the future. But within the confines of a Bachelor (3-years degree)
in such disciplines, heavily different from computer science but able to
use several CS conceptual and practical tools effectively for other
purposes, I think they were served perfectly well by "Foundation s
of Informatics" (aka CS 101 -- we don't number things like that here,
and "Informatic s" is strongly preferred over "CS", and in particular
it's the preference of local computer scientists) covering a lot of
important concepts AND practical exercises, but not ALL higher-order
functions. Personally I think that they wasted effort in teaching
about pointers, but I guess that's debatable.

Python's purpose is not, and has never been, to maximize the generality
of the constructs it offers.

Whoever said anything about "maximizing generality"? If one's mantra

Your only argument FOR reduce is its maximal generality (and the falsehood
that it's taught in every CS 101 course).
elegant features understood by anyone who has studied Computer Science
If a CS 101 course (passed with top marks) counts as having "studied
Computer Science", then 'reduce' is not such a feature. It's not very
elegant either, btw, but that's a subjective opinion -- the fact that
CS 101 courses mostly do NOT teach reduce/foldl is a FACT. So, by
this reasoning of yours, but based on realities rather than falsehoods,
reduce should go away.
*utterly* wrong manner. You should be assuming that your audience are
the smart people that they are, rather than the idiots you are
assuming them to be.
They're smart _and pragmatical_, NOT interested in (very arguable...)
elegance _for its own sake_, for the most part. So, they ask for GOOD
use cases for reduce, if they're taught it -- and... none can be found
throughout the Standard Python library and everywhere else I've checked
into real programs (written by me or my clients, or downloaded from
the net). Every single use of reduce I've found would be faster,
cleaner, simpler with sum, an explicit loop, or other ways yet.

Keep the language small and simple.

Provide only one way to do an operation.

It is not true these principles are rare among computer languages --
they are quite common. Most such language (like most computer
languages in general) just never obtained any wide usage.

URLs please (or non-URL references if you must). I had never seen
these two principles written down for a general-purpose language
before I met C -- and haven't, since, except for Python. I'll be
quite interested in being able to quote other languages explicitly
adopting them as design criteria.

place at the right time, it had a lightweight implementation, was

The lightweight implementation clearly follows from the two
principles above -- C had very-lightweight, fast, portable
compilers early on for just the same reason.
Alex

[Emile van Sebille]

Alex Martelli:

... no doubt you'd argue for
a .sort() followed by [-1] as "more general" than max...

Well, you must admit that writing pentultimate(a) has it all over
a.sorted()[-2] ;-)

--

Emile van Sebille
em***@fenx.com

[Dang Griffith]

On Fri, 14 Nov 2003 18:07:44 GMT, Dang Griffith
<no*****@noemai l4u.com> wrote:

On Wed, 12 Nov 2003 13:19:15 -0500, Douglas Alan <ne****@mit.edu >
wrote:

I would say that if you didn't get introduced to at least the concept
of functional programming and a hint of how it works, then something
was seriously wrong with your CS education.
|>oug

I've been lurking this thread for a while, but missed the beginning.
Can I ask where you got your CS education (school and year)?
From the conversation, it seems that right or wrong, your education
exceeds those in this group. Where should I send my children
such that they receive a better CS education that I?
--dang

Nevermind--I saw more posts and summarily reached a conclusion,
reducing the alternatives to one.
--dang

[Terry Reedy]

"Alex Martelli" <al***@aleax.it > wrote in message
news:WQ******** ***********@new s1.tin.it...

Terry Reedy wrote:

If the builtins are reduced in 3.0, as I generally would like, I would be fine with moving apply, map, filter, and a repaired version of
reduce to a 'fun'ctional or hof module. But the argument of some
seems to be that this batteries-included language should specifically exclude even that.
A functional module would be neat.

Good. I wrongly thought you might be in the abolish camp.
A great way to enhance the chance that there will be one
would be starting one today (e.g. on sourceforge),
ideally with both pure-Python and C-helped (or pyrex, etc)
implementations ,
and get it reasonably-widely used, debugged, optimized.
Interesting idea. Maybe I will do something with it.
Also, I advocate that 3.0 should have a module or package (call it
"legacy" for example) such that, if you started your program with
some statement such as:

from legacy import *

compatibility with Python 2.4 or thereabouts would be repaired as
much as feasible, to ease running legacy code, and to the expense
of performance, 'neatness' and all such other great things if needed
(e.g., no "repaired" versions or anything -- just compatibility).
Running old code in new interpreters seems not to be much of a problem
yet because of the current policy of not deleting features or (with
pretty minor or obscure exceptions) changing semantics. But if any
deprecations take effect before 3.0, legacy could also be added
before.

I think there should also be an official 'retroxy' module to help run
(a feasible subset of) pythonx.y code in older interpreters. A
retro23 module would define True, False, sum, enumerate and perhaps
other stuff. Accompanying instructions could discuss which older
versions a new module like sets will work with. I am sure this has
been invented privately several times.
One reasonably popular counterproposal to that was to have it as

from __past__ import *

by analogy with today's "from __future__".
which Guido has strongly rejected since it would mean keeping old
stuff in the interpreter forever -- unless it were a synonym for
'import legacy' -- in which case your spelling is better.
I'd also like to make it
easy to get this functionality with a commandline switch, like is
the case today with -Q specifically for _division_ legacy issues.
-L to mean 'import legacy' on startup should be possible.
But mostly, each time I mention that on python-dev, I'm rebuffed with remarks about such issues being way premature today. Somebody's
proposed starting a new list specifically about 3.0, to make sure
remarks and suggestions for it made today are not lost about more
day-to-day python-dev traffic, but I don't think anything's been
done about that yet.

Perhaps Guido is trying to avoid pressure to start before he and the
PSF are ready.

I personally wish 3.0 were out now or the next release. That is what
I more or less expected 2+ years ago when I suggested that the
division switch be delayed until 3.0 -- or rather, suggested that the
first release after 2 years make the switch *and* be called 3.0. I
was not asking to have to import the new division rules for years and
years;-).

Terry

[Thomas Heller]

Alex Martelli <al***@aleax.it > writes:

Terry Reedy wrote:
...

personally argued that reduce() should be removed from the language,
but I do agree that it does not have to be part of "core" Python,
and could easily be relegated to a module.)

If the builtins are reduced in 3.0, as I generally would like, I would
be fine with moving apply, map, filter, and a repaired version of
reduce to a 'fun'ctional or hof module. But the argument of some
seems to be that this batteries-included language should specifically
exclude even that.

A functional module would be neat. A great way to enhance the chance
that there will be one would be starting one today (e.g. on sourceforge),
ideally with both pure-Python and C-helped (or pyrex, etc) implementations ,
and get it reasonably-widely used, debugged, optimized.

I would suggest: don't move the implementation to C too fast or too much
of the code. With pypy on the horizon, this may never be needed (?).

Thomas

[Terry Reedy]

"Alex Martelli" <al***@aleax.it > wrote in message
news:IT******** ************@ne ws2.tin.it...

I'd rather put map/filter/reduce in a 'legacy' module -- warts and
all, e.g. map's horrible habit of None-padding shorter sequences,
EEK -- and have a designed-from-scratch functional module without
the warts.
What a liberating thought. I sometimes forget that the only thing I
am really stuck with when programming in Python is the core syntax (in
the Ref Manual) and that I am free to wrap or replace anything else
(in the Lib Manual). For a project I hope to start soon, I have been
worried about how to work around or explain away the deficiencies of
Python's of reduce(). Instead, I should just write the version I want
for my purposes (especially since speed does not matter). So your
voluminous writing on this has benefited at least one person.
Probably, yes. What functions, as opposed to types, do you
think should absolutely be in the builtins rather than in a separate
module, and (unless it's obvious) why?
I have thought some about this also. Type objects could be put/left
in types, but as constructors, they are too useful and basic not to
have immediately available (unless builtins was reduced to nearly
nothing).
Of course __import__ had better stay or we won't
be able to get anything imported, for example:-).
Does it have to be in builtins for import statements to work?
Direct use of __import__ seems relatively rare.
The attribute-related functions hasattr, getattr,
setattr, and delattr, seem very fundamental
(but I'd want the same status for the item-functions
currently over in operator),
as well as
(at least some of them -- delattr isn't -- but I do think that trying to discriminate too finely would make things too hard to learn)
very frequently used in code.
I had to re-lineate this to parse apart the three thoughts. A static
frequency-of-use analysis for some large code collection would be
interesting. I might write something eventually.
iter I group this with type constructor objects, even though not one
itself len unless made a method, keep in pow [for the crucial 3-arguments case] very specialized and rarely used? move to math as pow3 range (or some preferable variant that returns an iterator) how about iter(3), etc, currently an arg type error?
think of 3 as the sentinal value.
is not typing '0,' worth the wartiness of optional first arg? and zip seem pretty fundamental

agree except for pow

Terry J. Reedy

[Andrew Dalke]

Alex:

Anyway, computational scientists using Python should be using Numeric
(if they aren't, they're sadly misguided).
Or they are like me and work in a subfield where (specialized)
database searches and graph theory is more useful. It was strange
at SciPy with all those people using NumPy and doing CFD and
our little group of computational life sciences people being rather
bored.
Indeed, that's why 'sum' is a commonly used word in English -- exactly
because nobody's every DOING anything like that -- while, of course,
'reduce' is totally obvious -- why, _anybody_ who's ever taken
Chemistry 101 knows it means "to remove oxygen"!

While in Chemistry 102 they learn that it means to add electrons
and can occur in a non-oxygen environment ;)

http://reference.allrefer.com/encycl...oxidreduc.html

Andrew
da***@dalkescie ntific.com

[Terry Reedy]

"Alex Martelli" <al***@aleax.it > wrote in message
news:8Y******** ***********@new s1.tin.it...

Dave Brueck wrote:
...

results = [ func(x) for x in sequence ]
... instead of ...
results = sequence.map(fu nc) ??

Because I find the first much more readable>

I entirely agree with both points.
For this pair, I like the second better. Different aesthetics.
They're even clearer when the contrast is between, e.g.:
results = [ x+23 for x in sequence ]
and:
results = sequence.map(la mbda x: x+23)
where using the HOF approach forces you
to understand (and read) lambda too.

Here I might take the first. 'lambda' is something I feed 'stuck'
with.

Would the hypothetical
results = sequence.map(fu nc x: x+23)
be any better?
How about a very hypothetical (post ``==repr deprecation)
results = sequence..map(` x+23`)
?

Terry J. Reedy