A critic of Guido's blog on Python's lambda

[Xah Lee]

Python, Lambda, and Guido van Rossum

Xah Lee, 2006-05-05

In this post, i'd like to deconstruct one of Guido's recent blog about
lambda in Python.

In Guido's blog written in 2006-02-10 at
http://www.artima.com/weblogs/viewpo...?thread=147358

is first of all, the title “Language Design Is Not Just Solving
Puzzles�. In the outset, and in between the lines, we are told that
“I'm the supreme intellect, and I created Python�.

This seems impressive, except that the tech geekers due to their
ignorance of sociology as well as lack of analytic abilities of the
mathematician, do not know that creating a language is a act that
requires little qualifications. However, creating a language that is
used by a lot people takes considerable skill, and a big part of that
skill is salesmanship. Guido seems to have done it well and seems to
continue selling it well, where, he can put up a title of belittlement
and get away with it too.

Gaudy title aside, let's look at the content of his say. If you peruse
the 700 words, you'll find that it amounts to that Guido does not like
the suggested lambda fix due to its multi-line nature, and says that he
don't think there could possibly be any proposal he'll like. The
reason? Not much! Zen is bantered about, mathematician's impractical
ways is waved, undefinable qualities are given, human's right brain is
mentioned for support (neuroscience!), Rube Goldberg contrivance
phraseology is thrown, and coolness of Google Inc is reminded for the
tech geekers (in juxtaposition of a big notice that Guido works
there.).

If you are serious, doesn't this writing sounds bigger than its
content? Look at the gorgeous ending: “This is also the reason why
Python will never have continuations, and even why I'm uninterested in
optimizing tail recursion. But that's for another installment.�. This
benevolent geeker is gonna give us another INSTALLMENT!

There is a computer language leader by the name of Larry Wall, who said
that “The three chief virtues of a programmer are: Laziness,
Impatience and Hubris� among quite a lot of other ingenious
outpourings. It seems to me, the more i learn about Python and its
leader, the more similarities i see.

So Guido, i understand that selling oneself is a inherent and necessary
part of being a human animal. But i think the lesser beings should be
educated enough to know that fact. So that when minions follow a
leader, they have a clear understanding of why and what.

----

Regarding the lambda in Python situation... conceivably you are right
that Python lambda is perhaps at best left as it is crippled, or even
eliminated. However, this is what i want: I want Python literatures,
and also in Wikipedia, to cease and desist stating that Python supports
functional programing. (this is not necessarily a bad publicity) And, I
want the Perl literatures to cease and desist saying they support OOP.
But that's for another installment.

----
This post is archived at:
http://xahlee.org/UnixResource_dir/w...bda_guido.html

Â* Â* Xah
Â* Â* xa*@xahlee.org
∑ http://xahlee.org/

[Lasse Rasinen]

Ken Tilton <ke*******@gmail.com> writes:

ps. flaming aside, PyCells really would be amazingly good for Python. And
so Google. (Now your job is on the line. <g>) k

Here's something I wrote this week, mostly as a mental exercise ;-)
The whole code is available at <http://www.iki.fi/~lrasinen/cells.py>,
I'll include a test example below. Feel free to flame away ;-)

(As for background: I like CL better as a language, but I also like Python
a lot. However, I was employed for 3 years as a developer and maintainer
in a Python data mining application, so I'm more fluent in Python than CL.)

The code is mostly based on Kenny's descriptions of Cells in the following
messages:
<0m**************@fe12.lga>
<NU*************@fe11.lga>
<xc************@fe08.lga>

In addition, I have looked at the CL source code briefly, but I'm not sure
if any concepts have survived to the Python version. Since Python's object
model is sufficiently different, the system is based on rules being
defined per-class (however, if you define a rule by hand in the __init__
function, it'll work also. I think; haven't tested).

I can possibly be persuaded to fix bugs in the code and/or to implement
new features ;-)

Features:
- Tracks changes to input cells dynamically (normal attributes are not tracked)
- Callbacks for changes (see caveats)
- Requires Python 2.4 for the decorator syntax (@stuff)
- Should calculate a cell only once per change (haven't tested ;-)

Caveats:
- The input cell callbacks are not called with the class instance
as the first argument, while the rule cell callback are. This
is mostly due to laziness.
- There is no cycle detection. If you write cyclic dependencies, you lose.
- There is very little error checking.

Example follows:

def x_callback(oldval, newval):
print "x changed: %s => %s" % (oldval, newval)

class Test(cellular):
def __init__(self):
self.x = InputCell(10, callback=x_callback)

def y_callback(self, oldval, newval):
print "y changed: %s => %s" %(oldval, newval)

def a_callback(self, oldval, newval):
print "a changed: %s => %s" %(oldval, newval)

def g_callback(self, oldval, newval):
print "g changed: %s => %s" %(oldval, newval)

@rule(callback=y_callback)
def y(self):
return self.x ** 2

@rule(callback=a_callback)
def a(self):
return self.y + self.x

@rule(callback=g_callback)
def g(self):
if self.x % 2 == 0:
return self.y
else:
return self.a
$ python cells.py

y changed: __main__.unbound => 100
a changed: __main__.unbound => 110
g changed: __main__.unbound => 100
=============
x changed: 10 => 4
y changed: 100 => 16
a changed: 110 => 20
g changed: 100 => 16
=============
x changed: 4 => 5
y changed: 16 => 25
a changed: 20 => 30
g changed: 16 => 30
--
Lasse Rasinen
lr******@iki.fi

[Ken Tilton]

Lasse Rasinen wrote:

Ken Tilton <ke*******@gmail.com> writes:

ps. flaming aside, PyCells really would be amazingly good for Python. And
so Google. (Now your job is on the line. <g>) k

Here's something I wrote this week, mostly as a mental exercise ;-)

It's fun, right? But what you have is a complete wreck. :)
The whole code is available at <http://www.iki.fi/~lrasinen/cells.py>,
I'll include a test example below. Feel free to flame away ;-)

(As for background: I like CL better as a language, but I also like Python
a lot. However, I was employed for 3 years as a developer and maintainer
in a Python data mining application, so I'm more fluent in Python than CL.)

The code is mostly based on Kenny's descriptions of Cells in the following
messages:
<0m**************@fe12.lga>
<NU*************@fe11.lga>
<xc************@fe08.lga>

In addition, I have looked at the CL source code briefly, but I'm not sure
if any concepts have survived to the Python version. Since Python's object
model is sufficiently different, the system is based on rules being
defined per-class...
That will be a total disaster for PyCells, if true. But I do not think
it is. You just need a constructor that takes some slot initializers,
and initialize the slots to one of: a normal value; an InputCell itself
initialized with a starting value, if only nil; or a RuledCell itself
initialized with a lambda.

Trust me, you lose a vast amount of power unless different instances of
the same class can have different rules for the same slot.
... (however, if you define a rule by hand in the __init__
function, it'll work also. I think; haven't tested).

I can possibly be persuaded to fix bugs in the code and/or to implement
new features ;-)
PyCells looks like it will be a project for SoC2006, so you may as well
relax. But I understand if you want to keep going, it is great fun. btw,
I have met more than a few people who had done something like Cells
independently, and there are many full-blown similar implementations
around. Mine is just the best. <g> Kidding, i do not really know, there
are so many.

Features:
- Tracks changes to input cells dynamically (normal attributes are not tracked)
Ha! All your rules depend on the input cell itself! How about A depends
on B depends on C? :)
- Callbacks for changes (see caveats)
- Requires Python 2.4 for the decorator syntax (@stuff)
- Should calculate a cell only once per change (haven't tested ;-)
Quite hard to test deliberately, but it happens "in nature". But it will
not happen until you do A->B->C. Once you have /that/ working, make A
the input, then have B and C both use A. But also have B use C, and
jiggle things around until A happens to think it should update B first,
then C. What happens is that B runs and uses C, but C has not been
updated yet. C is inconsistent with A, but is being used to calculate a
new value for B which does see the new value of A. Mismatch! B will get
sorted out in a moment when C gets recalculated and tells B to calculate
a second time, but meanwhile after the first recalculation of B the
on-change callback for that got invoked, missiles were launched, and
Moscow has been destroyed.

Caveats:
- The input cell callbacks are not called with the class instance
as the first argument, while the rule cell callback are. This
is mostly due to laziness.

And unacceptable!

have fun. :)

kenny

ps. In the getattr for any Cell-mediated slot, look to see if "parent"
is non-nil. If so, set up a dependency. k

--
Cells: http://common-lisp.net/project/cells/

"Have you ever been in a relationship?"
Attorney for Mary Winkler, confessed killer of her
minister husband, when asked if the couple had
marital problems.

[Lasse Rasinen]

Ken Tilton <ke*******@gmail.com> writes:

if any concepts have survived to the Python version. Since Python's object
model is sufficiently different, the system is based on rules being
defined per-class...
That will be a total disaster for PyCells, if true. But I do not think it
is. You just need a constructor that takes some slot initializers, and
initialize the slots to one of: a normal value; an InputCell itself
initialized with a starting value, if only nil; or a RuledCell itself
initialized with a lambda.

Hmm, just tried it:

class A(cells.cellular): .... def __init__(self):
.... self.a = cells.InputCell(10)
.... self.b = cells.RuleCell(lambda self: self.a+1, self, None)
a = A()
a.a 10 a.b

11

So it does work out-of-the-box ;-)
PyCells looks like it will be a project for SoC2006, so you may as well
relax.
You really want to start a SoC project on something that takes about two
weeks from an average Python programmer? What does the guy do for the rest
of the summer?

(I think I spent 4-5 hours on this actually sitting on the computer,
sandwiched between remodeling and cleaning and work. The rest of the two
weeks would be making it more robust ;-)
Features:
- Tracks changes to input cells dynamically (normal attributes are not tracked)

Ha! All your rules depend on the input cell itself! How about A depends on
B depends on C? :)

Oops. I'm sorry for the inaccurate terminology. They depend only the cells
they use as inputs (their "children"), and not only on InputCells.

(I use the parent-child terminology because as English is not my native
language, I had trouble remembering which depend* variable was which ;-)
Quite hard to test deliberately, but it happens "in nature". But it will
not happen until you do A->B->C. Once you have /that/ working, make A the
input, then have B and C both use A. But also have B use C, and jiggle
things around until A happens to think it should update B first, then C.
What happens is that B runs and uses C, but C has not been updated yet. C
is inconsistent with A, but is being used to calculate a new value for B
which does see the new value of A. Mismatch! B will get sorted out in a
moment when C gets recalculated and tells B to calculate a second time,
but meanwhile after the first recalculation of B the on-change callback
for that got invoked, missiles were launched, and Moscow has been
destroyed.
If you check the testcase, you'll see there are such dependencies, and all
the callbacks fire just once (and in dependency-related order).

Furthermore, the timestamp mechanism SHOULD take care of those (if the
cell is older than its children, it gets recalculated before it will
provide any data, and thus C will get recalculated before B uses it.
ps. In the getattr for any Cell-mediated slot, look to see if "parent" is
non-nil. If so, set up a dependency. k

Already done, see BaseCell.value() ;-)
--
Lasse Rasinen
lr******@iki.fi

[Ken Tilton]

Lasse Rasinen wrote:

Ken Tilton <ke*******@gmail.com> writes:

if any concepts have survived to the Python version. Since Python's object
model is sufficiently different, the system is based on rules being
defined per-class...
That will be a total disaster for PyCells, if true. But I do not think it
is. You just need a constructor that takes some slot initializers, and
initialize the slots to one of: a normal value; an InputCell itself
initialized with a starting value, if only nil; or a RuledCell itself
initialized with a lambda.

Hmm, just tried it:

class A(cells.cellular):
... def __init__(self):
... self.a = cells.InputCell(10)
... self.b = cells.RuleCell(lambda self: self.a+1, self, None)

a = A()
a.a
10
a.b

11

So it does work out-of-the-box ;-)

So why exactly did you say that the differences in the object model made
it impossible? I was really stunned by that claim. And you sounded so
confident. What went wrong there? It was trivial, right? How did you
miss that?

PyCells looks like it will be a project for SoC2006, so you may as well
relax.

You really want to start a SoC project on something that takes about two
weeks ...

You sound so confident. :)

Do you know the deliverables? I know you do not know Cells. You say you
looked at the code -- it does not show. I can also tell you have not
done much serious programming, or you would know that twelve weeks is
more like twelve minutes than three months.

A new test suite, documentation (a first, there is none now), a full
port of Cells in all their ten years of sophisticated evolution and
variety (no, not your stupid pet trick), and as a demo project an entire
cells-driven GUI, probably a port of my new Celtk (+ Cells Tk) work, all
in a language without macros, without special variables, with a more
modest OO system, and limited first class functions... oh, I think we'll
keep him busy. :)

Now since you are such a genius, maybe you can help with something.
Trust me on this: this is one place where macros would be able to hide a
ton of implementation wiring it does no one any good to look at, and
actually turns into a maintenance nightmare whenever Cells might get
revised.

Is there any experiemntal macro package out there for Python? Maybe a
preprocessor, at least? Or are there ways to actually hack Python to
extend the syntax? My honest guess is that Cells will port readily to
Python but leave everyone very interested in finding some way to hide
implementation boilerplate. Got anything on that?

kenny

[Ben]

Nothing you have described sounds that complicated, and you never come
up with concrete objections to other peoples code (apart that it took
10 years to write in Lisp, so it must be really hard)

Why are you running a SoC project for PyCells if you dislike the
language so much. People who do like Python can implement it if they
need it (which I haven't seen any good examples that they do)

Please don't force a student to create a macro system just to port a
system to Python, as it won't really be python then. Use Pythonic
methodology instead. There are already plenty of ways to hide
complicated functionality, just not necessarily the way you want to do
it.

Cheers,
Ben

[Ken Tilton]

Ben wrote:

Nothing you have described sounds that complicated, and you never come
up with concrete objections to other peoples code (apart that it took
10 years to write in Lisp, so it must be really hard)
Oh, now I have to spend an hour dissecting any code you people toss-off
that does no more than pick the low-hanging fruit? I do not spend enough
time on Usenet already? :)

Why are you running a SoC project for PyCells...
You do not even know what Cells are and have not taken the trouble to
understand, so i will save my breath. Pythonistas will love PyCells, I
promise. Please recall that it is not just me, there is a ton of prior
and current art.
if you dislike the
language so much.
There is a difference between disliking a language and thinking PyCells
might end up persuading folks that macros and/or true lambda might be
worth the trouble to extend the language.

Try to think a little more precisely, OK? Thx.
People who do like Python can implement it if they
need it (which I haven't seen any good examples that they do)

Please don't force a student to create a macro system just to port a
system to Python,
You are getting hysterical, sit down, breathe. I asked a question,
because (unlike you) I can see where this is going. But as you say...
There are already plenty of ways to hide
complicated functionality,

I know. And that is why the mentor is a Pythonista, not me. I made a
simple inquiry as to the options available should Python have trouble
hiding the wiring. just looking ahead a little (as are the student and
mentor). Something wrong with thinking ahead a few moves?

You on the other hand have made up your mind about something you admit
you do not understand, have now ascribed to me a half dozen sentiments I
do not hold, and are feeling absolutely miserable because you think this
is a flamewar.

No, we are just discussing language synatx and how it impacts language
semantics, which has led inadvertently to a few of us starting an SoC
project to put together a Python version of a very successful dataflow
hack I did for Lisp.

I use it every day, and it just plain makes me smile. I wrote more code
than you can imagine Before Cells, and have now used them intensively
and in more ways than you can imagince since. Even if the wiring cannot
be hidden, the productivity win will be trememndous. Note that this
translates ineluctably to "programming will be more fun".

Even you will love them.

:)

kenny

--
Cells: http://common-lisp.net/project/cells/

"Have you ever been in a relationship?"
Attorney for Mary Winkler, confessed killer of her
minister husband, when asked if the couple had
marital problems.

[Ken Tilton]

Ken Tilton wrote:

Ben wrote:

Nothing you have described sounds that complicated, and you never come
up with concrete objections to other peoples code (apart that it took
10 years to write in Lisp, so it must be really hard)

Oh, now I have to spend an hour dissecting any code you people toss-off
that does no more than pick the low-hanging fruit? I do not spend enough
time on Usenet already? :)

I want to clarify something. I did look at the code. It was the same
thing we had with Cells after four-five hours. Yet the author admitted
he had looked at the Cells source, so he should have known he had not
implemented, inter alia, synapses, kid-slotting, ephemerals, optional
laziness, and worst of all he had omitted the data integrity mechanism
encapsulated by with-integrity. In the next exchange we discover he
missed the ability to author Python instances individually while
mistakenly thinking it was impossible.

Exactly how much time am I supposed to spend on someone not willing to
spend enough time to understand Cells? I recognize, tho, a kindred
spirit more interested in writing their own code than reading and
understanding someone else's. :)

You too are more eager to flame me over misperceived slights to The
Sacred Python than in Python having a wicked cool constraints package,
and I am wasting too much time on you. I recognize, tho, a fellow Usenet
dlamewar enthusiast. :)

kenny

--
Cells: http://common-lisp.net/project/cells/

"Have you ever been in a relationship?"
Attorney for Mary Winkler, confessed killer of her
minister husband, when asked if the couple had
marital problems.

[Lasse Rasinen]

[I trimmed some of the newsgroups away; this mostly concerns Python and Lisp]

Ken Tilton <ke*******@gmail.com> writes:

Lasse Rasinen wrote:

Ken Tilton <ke*******@gmail.com> writes:

if any concepts have survived to the Python version. Since Python's object
model is sufficiently different, the system is based on rules being
defined per-class...

That will be a total disaster for PyCells, if true. But I do not think it
is. You just need a constructor that takes some slot initializers, and
initialize the slots to one of: a normal value; an InputCell itself
initialized with a starting value, if only nil; or a RuledCell itself
initialized with a lambda. Hmm, just tried it:

[snip example]

So it does work out-of-the-box ;-)

So why exactly did you say that the differences in the object model made
it impossible? I was really stunned by that claim. And you sounded so
confident. What went wrong there? It was trivial, right? How did you miss
that?

Simple: I didn't think to try that before you asked.

I did not say the differences in the object model made it impossible, I
said the system is based on rules defined per-class. My goal was to
explore the how one would go about defining data flow in Python, which is
why I concentrated on class-level definitions first.

The situation is similar to functions in classes. Normally you'd define
them like this:

class X:
def function(self, ...):
...

However, you can just as well set them, even on per instance basis:
x = X()
x.another_function = lambda self, x: return x+2
I also think(*) that while one would have the option to define
per-instance rules, in a Python implementation one would structure the
code so that common rules would be class-related, and the per-instance
rules would be used less frequently, used only when you absolutely need
them.

(*) Unfounded Gut Feeling(TM); if your project is successful, we can
revisit this prediction in September ;-)

PyCells looks like it will be a project for SoC2006, so you may as well
relax.

You really want to start a SoC project on something that takes about two
weeks ...

You sound so confident. :)

Indeed. Having reread the post I do think the tone was possibly a tad
arrogant. However:
A new test suite, documentation (a first, there is none now), a full port
of Cells in all their ten years of sophisticated evolution and variety
(no, not your stupid pet trick), and as a demo project an entire
cells-driven GUI, probably a port of my new Celtk (+ Cells Tk) work, all
in a language without macros, without special variables, with a more
modest OO system, and limited first class functions... oh, I think we'll
keep him busy. :)
I did not know all this. The list above does sound like a full summer of
work ;)

I assumed that PyCells referred to the core dependency tracking module
which (even now) does not sound like a such huge task, especially when one
has the reference implementation ;-)

As I said above, I was mostly concerned with exploring how the data flow
system would work, so I haven't implemented the various different types of
cells in Cells. So there would obviously be a bit more work if one is
implementing all that, but still, two caffeine and youth powered student
weeks can achieve a lot ;-)

What "your stupid pet trick" would be referring to? The use of the
__getattribute__ method (which is quite close to SLOT-VALUE-USING-CLASS),
or the use of @decorator syntax to reduce typing and redundancy?
Now since you are such a genius, maybe you can help with something. Trust
me on this: this is one place where macros would be able to hide a ton of
implementation wiring it does no one any good to look at, and actually
turns into a maintenance nightmare whenever Cells might get revised.

I'd probably use decorators a lot, since they let you play around with
function objects. If they are suitably chosen and designed, the interface
should stay pretty stable even while the wiring is changed.

(Decorator background:

The regular python Python function definition would be
as follows:

def foo(x,y,z):
...

would be in CL (if CL were Lisp-1 with symbol-function-or-value) more or less:

(setf (symbol-function-or-value 'foo) (lambda (x y z) ...)

The decorated syntax would be:

@magic_decorator
def foo(x,y,z):
...

and the translation:
(setf (symbol-function-or-value 'foo)
(funcall magic-decorator #'(lambda (x y z)
...)))
)
--
Lasse Rasinen
lr******@iki.fi

[Ken Tilton]

Lasse Rasinen wrote:

[I trimmed some of the newsgroups away; this mostly concerns Python and Lisp]

Ken Tilton <ke*******@gmail.com> writes:

Lasse Rasinen wrote:

Ken Tilton <ke*******@gmail.com> writes:
>if any concepts have survived to the Python version. Since Python's object
>model is sufficiently different, the system is based on rules being
>defined per-class...

That will be a total disaster for PyCells, if true. But I do not think it
is. You just need a constructor that takes some slot initializers, and
initialize the slots to one of: a normal value; an InputCell itself
initialized with a starting value, if only nil; or a RuledCell itself
initialized with a lambda.

Hmm, just tried it:

[snip example]

So it does work out-of-the-box ;-)
So why exactly did you say that the differences in the object model made
it impossible? I was really stunned by that claim. And you sounded so
confident. What went wrong there? It was trivial, right? How did you miss
that?

Simple: I didn't think to try that before you asked.

I did not say the differences in the object model made it impossible, I
said the system is based on rules defined per-class.

Oh, please: "Since Python's object model is sufficiently different, the
system is based on rules being defined per-class".
I also think(*) that while one would have the option to define
per-instance rules, in a Python implementation one would structure the
code so that common rules would be class-related,..
What has Python got to do with it? What you describing is the way every
OO system other than a prototype-based system works. It is why OO failed
to deliver on the Grail of object reuse: every time you need different
behavior, you need a new subclass. Literal values only go so far in
making instances authorable. But when an instance can have a rule with
itself as an argument, and when instances exist in a runtime hierarchy
navigable up and down such that rules have effectively global scope,
yowza, now you have authorability, amd now you have object reuse.

and the per-instance
rules would be used less frequently, used only when you absolutely need
them.
You will see, but not until you have the capability. Then you will
discover you absolutely need them all the time. GUIs are dynamic things,
so you cannot just author a widget with a literal value, it has to be a
rule sensitive to the state of other GUI elements and the model itself,
which might be changing underfoot in response to I/O events.

I happened to be discussing this just now over in comp.lang.tcl,
comparing Cells with Actions, a kindred package motivated by this:

"Two large problems exist for developers of such user interfaces. One is
the need to constantly synchronize the controls with the ever-changing
state of the application or data. When no text is selected, for example,
the cut and copy buttons should be disabled.

"Another problem is that as programs evolve over time it can become
tedious and error prone to update the parts of the code that act upon
these controls."

Found here:

http://www.tcl.tk/community/tcl2004/...ley/oakley.pdf

(*) Unfounded Gut Feeling(TM); if your project is successful, we can
revisit this prediction in September ;-)
No need for gut feelings. Anyone working in GUIs knows the problem
(stated by Mr Oakley above) and many partial solutions exist, such as
Tcl/Tk's builtin mechanisms for automatic state management. They called
the company ActiveState for a reason, you know. :)

One person did his homework. vasilsi margioulas sent me cells-gtk, a
two-week marriage of cells and Gtk derived from my Cells-Tk effort. He
was curious if Cells would be useful. he decided, yes. :)

PyCells looks like it will be a project for SoC2006, so you may as well
relax.

You really want to start a SoC project on something that takes about two
weeks ...
You sound so confident. :)

Indeed. Having reread the post I do think the tone was possibly a tad
arrogant. However:

A new test suite, documentation (a first, there is none now), a full port
of Cells in all their ten years of sophisticated evolution and variety
(no, not your stupid pet trick), and as a demo project an entire
cells-driven GUI, probably a port of my new Celtk (+ Cells Tk) work, all
in a language without macros, without special variables, with a more
modest OO system, and limited first class functions... oh, I think we'll
keep him busy. :)

I did not know all this. The list above does sound like a full summer of
work ;)

I assumed that PyCells referred to the core dependency tracking module
which (even now) does not sound like a such huge task, especially when one
has the reference implementation ;-)

I think the real problem here is that you have no idea how fast twelve
weeks can go while programming. The hourly beep on my wristwatch sounds
like a metronome.

As for Cells being so damn easy, well, yeah, that is how we get sucked
into these ten year projects. Do you know how long Knuth thought he
would spend on TeX? Look it up. I have high hopes for you as a
developer, you are blessed with that essential cluelessness as to how
hard things will get before you are done.

If you want to insist on how perfect your code is, please go find
ltktest-cells-inside.lisp in the source you downloaded and read the long
comment detailing the requirements I have identified for "data
integrity". Then (a) tell me how your code fails at integrity, (b) fix
it, and (c) tell me again how easy Cells is. :)

Having the reference implementation is the only thing that makes this
conceivably doable in a summer. What you are missing is something I have
often also gotten wrong: the core, cool functionality always comes easy
in the proof-of-concept stage. We make these ridiculous extrapolations
from that to a shipped product and come in five times over budget.

Not just the extra drudgery (doc, testing) of a finished product, but
also what happens when you scale a package to real-world requirments. I
knew my system was as bad as yours at data integrity for seven years
before I finally hit an application where it mattered (RoboCup, of all
things).

As I said above, I was mostly concerned with exploring how the data flow
system would work, so I haven't implemented the various different types of
cells in Cells. So there would obviously be a bit more work if one is
implementing all that, but still, two caffeine and youth powered student
weeks can achieve a lot ;-)
See above. That is the fun proof-of-concept bit. And go back further to
my point about you really not understanding time in the software
dimension: two weeeks /is/ twelve weeks!

What "your stupid pet trick" would be referring to?
Just picking off the low-hanging fruit. You ducked all the hard issues.
I think in another article in this thread I demonstrated how you
destroyed Moscow.
The use of the
__getattribute__ method (which is quite close to SLOT-VALUE-USING-CLASS),
or the use of @decorator syntax to reduce typing and redundancy?
No, that stuff is fine.

Now since you are such a genius, maybe you can help with something. Trust
me on this: this is one place where macros would be able to hide a ton of
implementation wiring it does no one any good to look at, and actually
turns into a maintenance nightmare whenever Cells might get revised.

I'd probably use decorators a lot, since they let you play around with
function objects. If they are suitably chosen and designed, the interface
should stay pretty stable even while the wiring is changed.

(Decorator background:

The regular python Python function definition would be
as follows:

def foo(x,y,z):
...

would be in CL (if CL were Lisp-1 with symbol-function-or-value) more or less:

(setf (symbol-function-or-value 'foo) (lambda (x y z) ...)

The decorated syntax would be:

@magic_decorator
def foo(x,y,z):
...

and the translation:
(setf (symbol-function-or-value 'foo)
(funcall magic-decorator #'(lambda (x y z)
...)))
)

Wow. I thought Python did not have macros.

This project is looking better all the time. Thx.

kenny

--
Cells: http://common-lisp.net/project/cells/

"Have you ever been in a relationship?"
Attorney for Mary Winkler, confessed killer of her
minister husband, when asked if the couple had
marital problems.

[Rob Warnock]

Ken Tilton <ke*******@gmail.com> wrote:
+---------------
| Having the reference implementation is the only thing that makes this
| conceivably doable in a summer. What you are missing is something I have
| often also gotten wrong: the core, cool functionality always comes easy
| in the proof-of-concept stage. We make these ridiculous extrapolations
| from that to a shipped product and come in five times over budget.
+---------------

Or as Fred Brooks said it in "The Mythical Man-Month" [paraphrased],
if a program takes one unit of effort, a programming *system* takes
three units of effort, and a programming systems *product* takes nine
units of effort.
-Rob

-----
Rob Warnock <rp**@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 [phone]

[Ben]

Ok, I'm sorry. This kind of discussions between two groups of people,
neither of who know the other person's language very well just wind me
up something chronic! It wasn't your post as such, just reading through
most of the thread in one go.

That will teach me to post while cross :). Sorry for any offence.

Anything that makes programming more fun is good, and I hope the Lisp
true way explodes into my head at some point.

Cheers,
Ben

[A. Rogowski]

al*****@yahoo.com (Alex Martelli) writes:

Bill Atkins <NO**********@rpi.edu> wrote:
...

``allow ( as an ordinary single-character identifier'' as for the
unneded feature ``allow unnamed functions with all the flexibility of
named ones''.

Not so infeasible:

(let ((|bizarrely(named()symbol| 3))
(+ |bizarrely(named()symbol| 4))

;; => 7

Read again what I wrote: I very specifically said "ordinary
*single-character* identifier" (as opposed to "one of many characters
inside a multi-character identifier"). Why do you think I said
otherwise, when you just quoted what I had written? (Even just a
_leading_ ( at the start of an identifier may be problematic -- and just
as trivial as having to give names to functions, of course, see below).

bah...

[1]> (setq rtbl (copy-readtable))
#<READTABLE #x203A0F6D>
[2]> (set-syntax-from-char #\{ #\()
T
[3]> (set-syntax-from-char #\( #\a)
T
[4]> {defun ( {a) {1+ a))
(
[5]> {( 1)
2
[6]> {( 4)
5
[7]> {setq *readtable* rtbl)
#<READTABLE #x203A0F6D>
[8]> (1+ 1)
2

With readtable and reader macros you can change the syntax as you
wish.

ajr.

[Ken Tilton]

Ben wrote:

This kind of discussions between two groups of people,
neither of who know the other person's language very well just wind me
up something chronic!
I must say, it is pretty funny how a flamewar turned into a pretty
interesting SoC project.
Anything that makes programming more fun is good, and I hope the Lisp
true way explodes into my head at some point.

Here is an excerpt of an excerpt from famous Gears demo. I notice it
really makes concrete what Lispniks are talking about in re macros and
multi-line lambda (pity about the formatting):

(defmodel gears-demo (window)
((gear-ct :initform (c-in 1) :accessor gear-ct :initarg :gear-ct))
(:default-initargs
:title$ "Rotating Gear Widget Test"
:kids (c? (the-kids
(mk-stack (:packing (c?pack-self))
(mk-row ()
(mk-button-ex (" Add " (incf (gear-ct .tkw))))
(mk-button-ex ("Remove" (when (plusp (gear-ct .tkw))
(decf (gear-ct .tkw)))))
(mk-entry :id :vtime
:md-value (c-in "10")))
(make-instance 'gears
:fm-parent *parent*
:width 400 :height 400
:timer-interval (max 1
(or (parse-integer (fm^v :vtime))
:junk-allowed t)
0)))))))

Don't worry, Lispniks cannot read that either. It is a delarative
construction of a hierarchical GUI. That is such a common task, that I
have rolled up a bunch of GUI-building macrology so that just the stuff
specific to this GUI gets typed in. Since the Gears widget is a custom
widget I have no macrology for that, and the wiring shows in the
expression ":fm-parent *parent*" (which itself leverages Lisp special
variables).

And no, I cannot remember all my macrology. I can certainly read it and
easily modify my GUI, because all the wiring is hidden, but if I have to
build a new GUI I cut and paste from other GUIs.

Let's look at just one form, which I believe destroys Alex's whole case
for naming every lambda:

(mk-button-ex ("Remove" (when (plusp (gear-ct .tkw))
(decf (gear-ct .tkw)))))

"mk-button-ex" (a) makes fun of MS$ naming standards and (b) expands to:

(make-instance 'button
:fm-parent *parent*
:text "remove"
:on-command (c? (lambda (self)
(when (plusp (gear-ct .tkw))
(decf (gear-ct .tkw))))))

The above is what one really needs to write to stick something in my GUI
framework, but who wants to look at all of that when most of it is
boilerplate? I need ":fm-parent *parent*" on every label and widget
because of some internals requirements, I just do not want to look at it
or have to remember to code it all the time (the latter not being a huge
problem because I really am cutting/pasting when I build a new GUI).

Is mk-button-ex some mysterious new language construct that will make
multi-programmer projects collapse in a heap of programmer-specific
constructs inscrutable to anyone else on the team?

(a) mk-button-ex kinda tells you (1) it makes a button and (2) no, this
is not part of Common Lisp, so where is the confusion?

(b) control-alt-. in my IDE shows me:

(defmacro mk-button-ex ((text command) &rest initargs)
`(make-instance 'button
:fm-parent *parent*
:text ,text
:on-command (c? (lambda (self)
(declare (ignorable self))
,command))
,@initargs))

Looks a lot like the expansion, right? That is really important in
making macrology easy. Once one has mastered the syntax (` , @), writing
a macro gets as natural as writing out the code. (In case you are
wondering, in my little example I did not need any other customizations
on the button, so it is hard to make out what the initargs are doign up
there. here is how they would work (also getting a little fancier by
actually disabling the "Remove" button, not just making it do nothing
when pressed, if the gear count is zero):

(mk-button-ex ("Remove" (decf (gear-ct .tkw)))
:fore-color 'red ;; Tcl/Tk will understand
:enabled (c? (plusp (gear-ct .tkw))))

becomes:

(make-instance 'button
:fm-parent *parent*
:text "remove"
:on-command (c? (lambda (self)
(decf (gear-ct .tkw))))
:fore-color 'red
:enabled (c? (plusp (gear-ct .tkw))))

[ps. Do not try that at home, i invented the enabled thing. It really
should be the Tk syntax, which I forget.]

ie, I created mk-button-ex because, jeez, every button I put in a GUI I
/know/ needs its own label and its own command (and the parent thing),
but there are other options, too. They have to be supported if the macro
is to get used all the time (we want that), but I do not want to make
them positional arguments without identifying keywords, because then the
code would be unreadable as well as unwritable (from memory).

Needless to say, there is more macrology in the expansion. One bit of
fun is .tkw. Background: in the GUIs I roll, a widget always knows its
parent. I guess you noticed. Anyway, because of that, a rule can kick
off code to navigate to any other widget and get information, ie, it
pretty much has global scope and that means power. Now one place a rule
will often look is up the hierarchy, say to a containing radio group for
a radio button. So the first thing I wrote was:

(upper self radio-group) -> (container-typed self 'radio-group)

Hmmph. I am /always/ upper-ing off self, I am surprised I have not
written a macro so I can just do (^upper radio-group). Soon. But since I
often look to the containing window in rules, I was looking for
something insanely short, shorter than even parentheses would allow,
like ".tkw":

(define-symbol-macro .tkw (nearest self window))

Nearest is like "upper" except that it is inclusive of the starting
point of the search (and, no, I am not happy with the name <g>).

And so it goes with Lisp macros. All the tedious boilerplate is hiiden,
in ways that cannot be done with functions. Oh, I skipped that point.
Look again at how the command (decf (gear-ct .tkw)) gets spliced into
the lambda form as so much source code:

From:
(mk-button-ex ("Remove" (decf (gear-ct .tkw))))

To:
(make-instance 'button
:fm-parent *parent*
:text "remove"
:on-command (c? (lambda (self)
(decf (gear-ct .tkw))))))

If mk-button-ex were a function, Lisp would try to evaluate

(decf (gear-ct .tkw))

which would be pretty sad because the "self" in there would not even
exist yet (the form is an /input/ to make-instance of what will become
"self" by the time the macro expansion code runs.

Which brings us to the idea of every multi-line lambda needing a name.
Does this:

(lambda (self)
(when (plusp (gear-ct .tkw))
(decf (gear-ct .tkw)))

Not sure what the Python would be, but maybe:

lambda (self):
if nearest(self,'window').gear_ct > 0:
nearest(self,'window').gear_ct = \
nearest(self,'window').gear_ct - 1

Does that need a name?

kenny

--
Cells: http://common-lisp.net/project/cells/

"Have you ever been in a relationship?"
Attorney for Mary Winkler, confessed killer of her
minister husband, when asked if the couple had
marital problems.

[Lasse Rasinen]

Ken Tilton <ke*******@gmail.com> writes:

If you want to insist on how perfect your code is, please go find
ltktest-cells-inside.lisp in the source you downloaded and read the long
comment detailing the requirements I have identified for "data integrity".
Then (a) tell me how your code fails at integrity, (b) fix it, and (c)
tell me again how easy Cells is. :)
Found it and read it; it was most enlightening. I claim my system fulfills
the first three requirements(*), while most likely failing gloriously on
the two last ones.

I'll postpone (b) while I've had a chance to think it over(**), but in the
face of the evidence I'm willing to admit my earlier work estimates (which
should have had a smiley next to them anyway ;-) were in error.

I won't admit to destroying Moscow, though. See (*).

(*) All values depending on the changed cell are marked as invalid before
anything else is done; trying to access an invalid value forces a
recalculation which also marks the cell as valid again, and so on
recursively down to cells that have already converged to a proper value.

Setting input cells in callbacks will f*** that up, though.

(**) Which probably doesn't occur until July or something ;(

[Decorator example]

Wow. I thought Python did not have macros.

Calling decorators macros is an insult to CL macros ;-)
All they do is make it a bit more convinent to apply transformations to
functions, or as the abstract in the original spec[1] says:

The current method for transforming functions and methods (for instance,
declaring them as a class or static method) is awkward and can lead to
code that is difficult to understand. Ideally, these transformations
should be made at the same point in the code where the declaration
itself is made.
This project is looking better all the time. Thx.

In that case I have something else you might like: The "with" Statement[2]

From the abstract:
This PEP adds a new statement "with" to the Python language to make
it possible to factor out standard uses of try/finally statements.

In practice one can use it implement some of the with-macros in a pretty
straightforward manner, especially those that are expanded into
(call-with-* (lambda () ,@body)). I believe the previously advertised
with-integrity macro could also be made to work in a satisfying manner.

To quote from the examples:

2. A template for opening a file that ensures the file is closed
when the block is left:

@contextmanager
def opened(filename, mode="r"):
f = open(filename, mode)
try:
yield f
finally:
f.close()

Used as follows:

with opened("/etc/passwd") as f:
for line in f:
print line.rstrip()

Does that last part look familiar to you?)

The problem with this is that this is Python 2.5 syntax, which currently
appears to be planned for release in late August, a bit late for SoC.
Alpha(*) versions are available, so if you want to take a chance and live
on the bleeding edge, you can probably gain from it.

(*) I don't keep a very close eye on the alpha releases, which is why I
didn't remember this yesterday. I like my programming tools steady
and stable ;-)

[1] http://www.python.org/dev/peps/pep-0318/#abstract
[2] http://www.python.org/dev/peps/pep-0343/
--
Lasse Rasinen
lr******@iki.fi

[Ken Tilton]

Lasse Rasinen wrote:

Ken Tilton <ke*******@gmail.com> writes:

If you want to insist on how perfect your code is, please go find
ltktest-cells-inside.lisp in the source you downloaded and read the long
comment detailing the requirements I have identified for "data integrity".
Then (a) tell me how your code fails at integrity, (b) fix it, and (c)
tell me again how easy Cells is. :)

Found it and read it; it was most enlightening. I claim my system fulfills
the first three requirements(*), while most likely failing gloriously on
the two last ones.

<sigh>

From #1: "recompute all and (for efficiency) only state computed off X
(directly or indirectly through some intermediate datapoint)"

Bzzzt! Another 47hrs, directions from a mentor, and a reference
implementation and you /still/ do not even understand the requirements,
let alone have a working port. The good news is that it is not an
integrity requirement that is being missed, it is the efficiency
requirement I snuck in there. The bad news is, see below.

Want to find the efficiency shortcoming yourself, or should I tell you?
You are entitled to the latter given the rules of the game (simulating a
pythonista student making off with five thousand undeserved dollars). :)

I'll postpone (b) while I've had a chance to think it over(**), but in the
face of the evidence I'm willing to admit my earlier work estimates (which
should have had a smiley next to them anyway ;-) were in error.
Aw, shucks, then I will admit that, before today, I never actually
looked at your code. :)

Well, I followed the URL and glanced at it, but I missed the use of the
timestamp. Speaking of which, Holy Granularity, Batman! You use
Time.time() to determine currency of a computation?!:

"time()
Return the time as a floating point number expressed in seconds
since the epoch, in UTC. Note that even though the time is always
returned as a floating point number, not all systems provide time with a
better precision than 1 second."

One /second/?!!!!! Exactly how slow is Python? I know you guys love that
issue as much as Lispniks. In un-compiled Lisp:

CTK(4): (loop repeat 2 do (print (get-internal-real-time)))
464033837
464033837

And you have no idea how slow PRINT is. btw, I thought Python was
portable. What is with the Time class and "not all systems..."? Check
out the Lisp:

(defun zoom ()
(loop with start = (get-internal-real-time)
while (= start (get-internal-real-time))
count 1 into cities-destroyed
finally (format t "~a cities destroyed in 1/~a of a second"
cities-destroyed internal-time-units-per-second)))

internal-time-units-per-second is (from the standard):

"Constant Value:
A positive integer, the magnitude of which is
implementation-dependent. "

So we vary, too, but my Lisp has to tell me so I can normalize. Anyway,
running that repeatedly I get pretty wild variation. My high score is:

CTK(18): 11637 cities destroyed in 1/1000 of a second

My low was under a thousand! I guess I have to wait until we cross a
millisecond boundary:

(defun zoom ()
(symbol-macrolet ((now (get-internal-real-time)))
(loop with start = (loop for mid = now
while (= mid now)
finally (return now))
while (= start now)
count 1 into cities-destroyed
finally (format t "~a cities destroyed in 1/~a of a second"
cities-destroyed
internal-time-units-per-second))))

Ok, now I am consistently taking out about 11.5k Russian cities. And you
need to fix your system.

Just use a counter -- does Python have bignums? if not, you'll have to
worry about wrapping. (the sound you hear is a project schedule
slipping. <g>)

I won't admit to destroying Moscow, though. See (*).
Sorry, you actually /have/ violated the data integrity requirement. You
confess below to missing this one:

"a corollary: should a client observer SETF a datapoint Y, all the above
must happen with values current with not just X, but also with the value
of Y /prior/ to the change to Y."

Well, how can you claim integrity when some values do not get
recalculated until the world has moved on to state N+2, if you will?
State N+1 had the information that headed off the launch.

Bye bye, Kremlin.

The easiest way to construct such a scenario would be with an ephemeral
cell. As you know... oops. Maybe you do not. Well, spreadsheets are
kinda steady state in orientation. Given a world of other values, this
is what my value should be. But what about events? You have been using
your version of PyCells in real-world applications for a while... oops.
No you have not. Well, when you start trying to handle events from an
event loop, you will discover a need to model events. (Trust your
mentor.) You need a slot that can be assigned normally, propagate
according to the above rules and regulations, and then revert to a null
state but /not/ as a state change -- neither propagating nor
observer-notifying.

They are, like, ephemeral, aka, "fleeting".

Now as I said, I got away with such holes for /years/ in all sorts of
hairy applications of Cells before finally, well, falling down those
holes, but it does happen so the holes are really worth filling in.

And wait till you see what this does to your algorithm. :) I was even
tempted to ban observer writebacks, if you will, but I think someday I
will write an application driven by such things, periodically calling
Tcl_DoOneEvent to keep in touch with the outside world.

Which means we cannot propagate on the stack, as your system would.
<slip...slip...slip>

(*) All values depending on the changed cell are marked as invalid before
anything else is done; trying to access an invalid value forces a
recalculation which also marks the cell as valid again, and so on
recursively down to cells that have already converged to a proper value.

Setting input cells in callbacks will f*** that up, though.

(**) Which probably doesn't occur until July or something ;(

[Decorator example]
Wow. I thought Python did not have macros.

Calling decorators macros is an insult to CL macros ;-)
All they do is make it a bit more convinent to apply transformations to
functions, or as the abstract in the original spec[1] says:

The current method for transforming functions and methods (for instance,
declaring them as a class or static method) is awkward and can lead to
code that is difficult to understand. Ideally, these transformations
should be made at the same point in the code where the declaration
itself is made.

This project is looking better all the time. Thx.

In that case I have something else you might like: The "with" Statement[2]

I had high hopes for that when I saw it, then was not sure. The good
news is that Serious Pythonistas will be doing this, not me. I will just
help them grok the requirements (and hope Ryan is taking notes <g>).

From the abstract:
This PEP adds a new statement "with" to the Python language to make
it possible to factor out standard uses of try/finally statements.

In practice one can use it implement some of the with-macros in a pretty
straightforward manner, especially those that are expanded into
(call-with-* (lambda () ,@body)). I believe the previously advertised
with-integrity macro could also be made to work in a satisfying manner.

To quote from the examples:

2. A template for opening a file that ensures the file is closed
when the block is left:

@contextmanager
def opened(filename, mode="r"):
f = open(filename, mode)
try:
yield f
finally:
f.close()

Used as follows:

with opened("/etc/passwd") as f:
for line in f:
print line.rstrip()

Does that last part look familiar to you?)

The problem with this is that this is Python 2.5 syntax, which currently
appears to be planned for release in late August, a bit late for SoC.
Alpha(*) versions are available, so if you want to take a chance and live
on the bleeding edge, you can probably gain from it.

That is up to the Pythonistas. As a rule, if there is advantage to be
had, I go with the bleeding edge until it proves to me it is unusable.

Thx for the heads up.

Nice job on the code, btw. A lot better than my first efforts.

kenny

--
Cells: http://common-lisp.net/project/cells/

"Have you ever been in a relationship?"
Attorney for Mary Winkler, confessed killer of her
minister husband, when asked if the couple had
marital problems.

[Kay Schluehr]

Ken Tilton wrote:

Is there any experiemntal macro package out there for Python? Maybe a
preprocessor, at least? Or are there ways to actually hack Python to
extend the syntax?

Yes. I've just released EasyExtend that does this kind of job:

http://www.fiber-space.de/EasyExtend/doc/EE.html

It fits quite nice with Python and is conceptually simple, safe and
reasonably fast. Using EasyExtend PyCells could be made an own language
( including Python ) defined in a Python package ( i.e. no C-code and
complex build process is required ). I would be interested in user
experience. I wouldn't consider EE as "experimental" i.e. open for
severe changes. It still lacks some comfort but it also improves
gradually in this respect.

[Michele Simionato]

Kay Schluehr wrote:

http://www.fiber-space.de/EasyExtend/doc/EE.html
Well, I have not read that page yet, but the name "fiber space" reminds
me of old
memories, when I was doing less prosaic things than now. Old times ..
;)

Michele Simionato
It fits quite nice with Python and is conceptually simple, safe and
reasonably fast. Using EasyExtend PyCells could be made an own language
( including Python ) defined in a Python package ( i.e. no C-code and
complex build process is required ). I would be interested in user
experience. I wouldn't consider EE as "experimental" i.e. open for
severe changes. It still lacks some comfort but it also improves
gradually in this respect.

[Kay Schluehr]

Michele Simionato wrote:

Kay Schluehr wrote:

http://www.fiber-space.de/EasyExtend/doc/EE.html

Well, I have not read that page yet, but the name "fiber space" reminds
me of old
memories, when I was doing less prosaic things than now. Old times ..
;)

Michele Simionato

But I guess that time this stuff was taught to you as "fiber bundles",
right? Oh, yes. Old times ;)

Well, besides the cute rhyme on "cyberspace" I had this analogy in mind
and that's why I called the extension languages "fibers". The
association is "fibers over a base space" or a base language as in this
case. The terms are not used strictly of course. I could not even say
what triviality could be in this context. But I still seek for an
acceptable glue mechanism which is beyond the scope of the current
first release :)

Kay