reduce() anomaly?

[Stephen C. Waterbury]

This seems like it ought to work, according to the
description of reduce(), but it doesn't. Is this
a bug, or am I missing something?

Python 2.3.2 (#1, Oct 20 2003, 01:04:35)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.

d1 = {'a':1}
d2 = {'b':2}
d3 = {'c':3}
l = [d1, d2, d3]
d4 = reduce(lambda x, y: x.update(y), l) Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 1, in <lambda>
AttributeError: 'NoneType' object has no attribute 'update' d4 = reduce(lambda x, y: x.update(y), l, {})

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 1, in <lambda>
AttributeError: 'NoneType' object has no attribute 'update'

- Steve.

[Michele Simionato]

Douglas Alan <ne****@mit.edu > wrote in message news:<lc******* *****@gaffa.mit .edu>...

C'mon -- all reduce() is is a generalized sum or product. What's
there to think about? It's as intuitive as can be. And taught in
every CS curiculum. What more does one want out of a function?

|>oug

Others pointed out that 'reduce' is not taught in every CS curriculum
and that many (most?) programmers didn't have a CS curriculum as you
intend it, so let me skip on this point. The real point, as David Eppstein
said, is readability:

reduce(operator .add, seq)

is not readable to many people, even if is readable to you.
That's the only reason why I don't use 'reduce'. I would have preferred
a better 'reduce' rather than a new ad hoc 'sum': for instance something
like

reduce([1,2,3],'+')

in which the sequence goes *before* the operator. It is interesting to
notice that somebody recently suggested in the Scheme newsgroup that

(map function sequence)

was a bad idea and that

(map sequence function)

would be better!

Of course, I do realize that there is no hope of changing 'reduce' or 'map'
at this point; moreover the current trend is to make 'reduce' nearly
useless, so I would not complain about its dead in Python 3.0.
Better no reduce than an unreadable reduce.

Also, I am not happy with 'sum' as it is, but at least it is
dead easy to read.

Michele Simionato

[Michele Simionato]

Douglas Alan <ne****@mit.edu > wrote in message news:<lc******* *****@gaffa.mit .edu>...

The reason for Python's wide acceptance isn't because it is
particularly well-designed compared to other programming languages
that had similar goals of simplicity and minimality (it also isn't
poorly designed compared to any of them -- it is on par with the
better ones) -- the reason for its success is that it was in the right
place at the right time, it had a lightweight implementation, was
well-suited to scripting, and it came with batteries included.

.... and it is free!!! ;)

More seriously, the fact that it has a standard implementation and a
BDFL ensuring the consistency of the language (not committee for Python!)
is also a big plus. Moreover, it got a good documentation, a very active
community and a wonderful newgroup. Also, the time scale between the
submission of a bug (there are very few of them, BTW) and its fixing
is surprisingly short. This is something I value a lot. Finally, the
language is still evolving at fast pace and you feel the sensation
that is has a future. Probably the same things can be said for Ruby
and Perl, so you have a choice if you don't like the Zen of Python ;)

Michele Simionato

[Georgy Pruss]

"Michele Simionato" <mi**@pitt.ed u> wrote in message news:22******** *************** ***@posting.goo gle.com...
| <...>
| in which the sequence goes *before* the operator. It is interesting to
| notice that somebody recently suggested in the Scheme newsgroup that
|
| (map function sequence)
|
| was a bad idea and that
|
| (map sequence function)
|
| would be better!
|
| <...>

Yes, that's right. Some scientific studies proved that humans think and
express their thoughts in order subject/object-action. So OOP is very
"human" in this aspect, btw.

G-:

[Robin Becker]

In article <17************ **********@twis ter.southeast.r r.com>, Georgy
Pruss <SE************ @hotmail.com> writes

"Michele Simionato" <mi**@pitt.ed u> wrote in message news:2259b0e2.0 311112248.3b
e2****@posting .google.com...
| <...>
| in which the sequence goes *before* the operator. It is interesting to
| notice that somebody recently suggested in the Scheme newsgroup that
|
| (map function sequence)
|
| was a bad idea and that
|
| (map sequence function)
|
| would be better!
|
| <...>

Yes, that's right. Some scientific studies proved that humans think and
express their thoughts in order subject/object-action. So OOP is very
"human" in this aspect, btw.

G-:

on oop in this thread nobody has pointed out that we could have

sequence.sum()
sequence.reduce (operator.add[,init])

or even

sequence.filter (func) etc etc

and similar. That would make these frighteningly incomprehensibl e ;)
concepts seem less like functional programming. Personally I wouldn't
like that to happen.
--
Robin Becker

[Douglas Alan]

pm*****@speakea sy.net (Patrick Maupin) writes:

Douglas Alan wrote:

Your claim is silly. sum() is not *way* simpler than reduce(), and
anyone can be explained reduce() in 10 seconds: "reduce() is just like
sum(), only with reduce() you can specify whatever addition function
you would like." Maybe reduce() can be explained in 10 seconds to someone who has used
sum() a few times, but that has no bearing whatsoever on trying to
explain reduce() to someone if sum() is not available and hasn't
been used by them.
Describing reduce() in 10 seconds is utterly trivial to anyone with an
IQ above 100, whether or not they have ever used sum():

"To add a sequence of numbers together:

reduce(add, seq)

To multiply a sequence of numbers together:

reduce(mul, seq)

To subtract all of the numbers of a sequence (except the first
number) from the first number of the sequence:

reduce(sub, seq)

To divide the first number in a sequence by all the remaining
numbers in the sequence:

reduce(div, seq)

Any two-argument function can be used in place of add, mul, sub, or
div and you'll get the appropriate result. Other interesting
examples are left as an exercise for the reader."
If someone can't understand this quickly, then they shouldn't be
programming!
I'm sorry, but from a pure CS viewpoint, reduce() is way off the
radar screen. Especially for a 101 course.
I'm sorry, but you are incorrect. When I took CS-101, we learned
assembly language, then were assigned to write a text editor in
assembly language, then we learned LISP and were assigned to write
some programs in LISP, and then we learned C, and then we were
assigned to implement LISP in C.

If you can write a !$#@!!%# LISP interpreter in C, you no doubt can
figure out something as mind-achingly simple as reduce()!
You should be assuming that your audience are the smart people
that they are, rather than the idiots you are assuming them to be. Ignorance is not stupidity.
Assuming that your audience cannot learn the simplest of concepts is
assuming that they are stupid, not that they are ignorant.
I sure hope that Python doesn't try to emulate C. It's a terrible,
horrible programming language that held back the world of software
development by at least a decade. I used to hate C. But then, when it borrowed enough good concepts
from Pascal and other languages, and the compilers got smart enough
to warn you (if you cared to see the warnings) about things like
"if (x = y)" I stopped using Modula-2. C held software back 10
years in the same manner as Microsoft did, e.g. by helping to
standardize things to where I can buy a $199 system from WalMart
which would cost over $20,000 if everybody kept writing code like
the pointy-headed ivory tower academics thought it ought to be written.
You score no points for C by saying that it is like Microsoft. That's
a strong damnation in my book. And you really don't know how the
world would have turned out if a different programming language had
been adopted rather than C for all those years. Perhaps computers
would be more expensive today, perhaps not. On the other hand, we
might not have quite so many buffer overflow security exploits.
Perhaps we'd have hardware support for realtime GC, which might be
very nice. On the other hand, perhaps people would have stuck with
assembly language for developing OS's. That wouldn't have been so
pretty, but I'm not sure that that would have made computers more
expensive. Perhaps a variant of Pascal or PL/1 would have taken the
niche that C obtained. Either of those would have been better, though
no great shakes either.

Many of the pointy-headed ivory tower academics, by the way, thought
that code should look something like Python. The reason these
languages are not widely used is because typically they either did not
come with batteries, or there was no lightweight implmentation
provided, or they only ran on special hardware, or all of the above.
The reason for Python's wide acceptance isn't because it is
particularly well-designed compared to other programming languages
that had similar goals of simplicity and minimality (it also isn't
poorly designed compared to any of them -- it is on par with the
better ones) -- the reason for its success is that it was in the right
place at the right time, it had a lightweight implementation, was
well-suited to scripting, and it came with batteries included.

I'd vote this as the statement in this group most likely to start
a religious flamewar since the lisp threads died down.
The only way it could start a religious flamewar is if there are
people who wish to present themselves as fanboys. I have said nothing
extreme -- just what is obvious: There are many nice computer
programming languages -- Python is but one of them. If someone
wishes to disagree with this, then they would have to argue that there
are no other nice programming languages. Now that would be a flame!
I'm not particularly religious, but I _will_ bite on this one: 1) In what way was it at the "right place at the right time?"
Perl was in the right place at the right time because system
administrators had gotten frustrated with doing all their scripts in a
mishmash of shell, awk, sed, and grep, etc. And then web-scripting
kicked Perl into even more wide acceptance. Python was in the right
place in the right time because many such script-writers (like yours
truly) just could not stomach Perl, since it is an ugly monstrocity,
and Python offered such people relief from Perl. If Perl had been a
sane OO language, Python would never have had a chance.
You didn't name names of other languages, but I'll bet that if you
can name 5 which are similar by your criteria, at least two of them
were available when Python first came out.
I'm not sure what you are getting at. There were many nice
programming languages before Python, but not many of them, other than
Perl, were portable and well-suited to scripting.

Oh, yeah, I forgot to mention portability in my list of reasons why
Python caught on. That's an essential one. Sure you could elegantly
script a Lisp Machine with Lisp, and some Xerox computers with
Smalltalk, but they didn't provide versions of these languages
well-suited for scripting other platforms.
2) What part of "lightweigh t implementation, well suited to
scripting" contradicts, or is even merely orthorgonal to
"particular ly well-designed"?
Again, I'm not sure what you are getting at. "Lightweigh t
implementation" and "well-suited to scripting" do not contradict
"well-designed", as Python proves. Lightweightedne ss and capability
at scripting are certainly orthogonal to the property of being
well-designed, however, since there are a plethora of well-designed
languages that are not suited to scripting. They just weren't
designed to address this niche.
3) Do you _really_ think that all the batteries were included when
Python first came out?
It certainly was not a particularly popular language until it came
with pretty hefty batteries. There are many other languages that
would have been equally popular before Python started coming with
batteries.
Do you even think that Python has more batteries _right_ _now_ than
Perl (via CPAN), or that some competing language couldn't or hasn't
already been designed which can coopt other languages' batteries?
Um, the last time I checked Perl was still a lot more popular than
Python, so once again I'm not sure what you are getting at. Regarding
whether or not some future language might also come with batteries and
therefore steal away Python's niche merely due to having more
batteries: Anything is possible, but this will be an uphill battle for
another language because once a language takes a niche, it is very
difficult for the language to be displaced. On the other hand, a new
language can take over a sub-niche by providing more batteries in a
particular area. PHP would be an example of this.
I can accept the premise that, for Python to enjoy the acceptance it
does today, Guido had to be lucky in addition to being an excellent
language designer. But if I were to accept the premise that
Python's popularity is due to sheer luck alone my only logical
course of action would to be to buy Guido a plane ticket to Vegas
and front him $10,000 worth of chips, because he has been extremely
lucky for many years now.

I never claimed *anything* like the assertion that Python's popularity
is due to luck alone!

|>oug

[Alex Martelli]

David Eppstein wrote:
...

Wen seq is any iterable, all you need is izip(seq, islice(seq, 1, None)),
and you'll be creating no new list whatsoever. Still, tradeoffs in
obscurity (and performance for midsized lists) are quite as clear.
If I'm not mistaken, this is buggy when seq is an iterable, and you need

Sorry, I should have said something like "re-iterable" -- an object such
that e.g.:
it1 = iter(seq)
val1 = it1.next()
it2 = iter(seq)
val2 = it2.next()
assert val1 == val2
holds (and keeps holding as you keen next'ing:-). list, tuple, dict, etc.
In particular, when the idiom zip(seq, seq[1:]) works, so should this one
(note in passing that, in said idiom, there is no need to slice the first
seq in the zip call to seq[:-1] -- zip truncates at the end of the
_shorter_ sequence anyway).
to do something like
seq1,seq2 = tee(seq)
izip(seq1,islic e(seq2,1,None))
instead.

Yes, this is totally general. However, even though tee has now (2.4)
been implemented very smartly, this overall approach is still way
"conceptual ly heavy" (IMHO) when compared to, e.g.:

def window_by_2(it) :
it = iter(it)
first = it.next()
for second in it:
yield first, second
fist = second

in any case, I do think that such 'windowing' is a general enough
need that it deserves its own optimized itertool...
Alex

[Christopher A. Craig]

pm*****@speakea sy.net (Patrick Maupin) writes:

Douglas Alan wrote:

Then your children were done a great diservice by receiving a poor
education. (Assuming that is that they wanted to learn Computer
Science, and not Programming in Pascal or Programming in C.)

I'm sorry, but from a pure CS viewpoint, reduce() is way off the
radar screen. Especially for a 101 course. If either of my daughters
wanted to discuss reduce() while taking such a course, I'd want to
see the curriculum to figure out what they _weren't_ being taught.

I'd tend to agree, and I've never found myself on the not teaching
generalities side of this debate before. An intro CS class should be
about intro to Computer Science and not any particular language, but
reduce() is a fairly specific functional programming concept. To say
that it should be taught in an intro class is like saying you should
deal with metaclasses, function pointers, or the stack pointer
immediately to teach good computer science. Algorithms, data
structures, state machines and computational theory are fairly basic
computer science, functional programming can be used to present those,
but is by no means needed.

Also the assumption isn't that Python users are stupid, it's that they
may have little to no CS training. Python wasn't designed for the
exclusive use of CS trained individuals (which is a good thing,
because I use it to teach computers to Boy Scouts).

Back on topic, I very rarely use reduce() in Python and then only for
optimization (which I rarely do). The vast majority of the time I
just use a for loop; it just seems to flow better.

--
Christopher A. Craig <li*********@cc raig.org>
"Tragedy is when I cut my finger. Comedy is when you fall in an open
sewer and die." Mel Brooks.

[eichin]

re cs 101: I'm told (by friends who were undergrads there in the early
90's) that Yale used to have an intro to CS course taught by the late
Alan Perlis. The usual high level concepts you'd expect - but since
it helps a lot to have a language to give examples in, it used APL.

APL has the kind of array manipulation tools that these days you find
in matlab or other specialized tools - I'd say "that other languages
aspire to" but from the looks of it other languages *don't* aspire to
that kind of array handling. But the point here is that "reduce" is
fundamental: x/i5 (where x is multiplication-sign and i is iota) is
a lot like reduce(int.__mu l__, range(1,6)), it's just "readable" if
you're comfortable with the notation (and more general, I can't find
a builtin way to say "coerced multiply" without lambda, in 30 seconds
of poking around.) On the other hand, that readability does assume
you're thinking in terms of throwing arrays around, which can be
an... *odd* way of looking at things, though of course when it fits,
it's very nice.

In other words, I'd expect anyone who had a reasonably rich CS
background to have been exposed to it, either from the historical
perspective, the "languages influence how you think" perspective, or
the mathematical operation perspective.

At the same time, I'll admit to not having used it (I've only been
using python for a year, and will more likely write an
accumulator-style block since it will always be clearer than a lambda
(which is what you generally need for reduce - if you are going to
write a function anyway, it's easier to just write an n-adic instead
of only dyadic function, and skip the need for reduce altogether - and
python has a "bias for functions", the gap between "sum" and "write a
function for the whole thing" is fairly narrow.)

(Hmm, r5rs doesn't have reduce, but mit-scheme does.)

[eichin]

> you're comfortable with the notation (and more general, I can't find

a builtin way to say "coerced multiply" without lambda, in 30 seconds
of poking around.) ...

Sigh, I just realized - everyone talking about operator.mul was being
*literal*, not abstract, there's actually an operator package :-) Oops.
Not that reduce(operator .mul, range(1,6)) is any more readable, I'd
still define Product around it...

[Michael T. Babcock]

On Wed, Nov 12, 2003 at 08:28:29AM +0000, Robin Becker wrote:

sequence.sum()
sequence.reduce (operator.add[,init])
sequence.filter (func) etc etc

That would make these frighteningly incomprehensibl e ;)
concepts seem less like functional programming. Personally I wouldn't
like that to happen.

I'm hoping you were being sarcastic ... but I get the feeling you aren't.

Why, pray-tell, would you want an OO program to do:

results = [ func(x) for x in sequence ]

... instead of ...

results = sequence.map(fu nc) ??

I can understand if you're writing highly LISP-like Python (see IBM's
articles on functional programming with Python for example). However, I
don't see the 'harm' in offering functional methods to lists/arrays.

Looking at this code I wrote today:

matches = [ get_matches(fil e) for file in duplicates ]
todelete = [ get_oldest(file s) for files in matches ]

... would end up being ...

matches = duplicates.map( get_matches)
todelete = matches.map(get _oldest)

... or better ...

todelete = duplicates.map( get_matches).ma p(get_oldest)

... and I somewhat like that as I look at it.
--
Michael T. Babcock
CTO, FibreSpeed Ltd. (Hosting, Security, Consultation, Database, etc)
http://www.fibrespeed.net/~mbabcock/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)
Comment: http://www.fibrespeed.net/~mbabcock/

iQF1AwUBP7JnS2/Ol25qCQydAQIpSA sAkWwusLupCYm59 p2+0Ms6cSfJpdmM CgZB
U6J/0UJgeV6MKdRPVMD E8D3Lbvi5fOldQI HAeRhwbUOJsZYCN P2VHkVEZzuV/2e2
1JDF8FEiIBr7eWN Xwy6lS77A/JZ4223e44rFToS3 dIkh4Zq0UJPWRzv 1rhTsaXp5
F9/qxPzhsAwg7i0Wsa r+7aUTUQtQMGdl1 eYKnKY/nEMnYN5YaKLnrfu XJuC7/vd0
w51tfuvIjZwNjZU tDmx6kFVpBFJOlM 2NcDELA/xHAFfzqJ5KVRTGh av7Mi2TQ7D5
JFeU9dpD7F11gZw/z91z/aVtOMpOC5+3OGku XwFx393Sn42ZziF ct9Bans5SdeyP
HClMFsNdonGpeaG 7nxSBrvI4sNnwJ2/1u4pMni2YxkqhvU eexBPoWhSEGZexu bum
yFg8Bi/CgjlaoiGTNxSyil b8IZ0Jr+w0qHgZk Bf+uVRcgISypH7p 1Q==
=z+Uz
-----END PGP SIGNATURE-----