BIG successes of Lisp (was ...)

[mike420]

In the context of LATEX, some Pythonista asked what the big
successes of Lisp were. I think there were at least three *big*
successes.

a. orbitz.com web site uses Lisp for algorithms, etc.
b. Yahoo store was originally written in Lisp.
c. Emacs

The issues with these will probably come up, so I might as well
mention them myself (which will also make this a more balanced
post)

a. AFAIK Orbitz frequently has to be shut down for maintenance
(read "full garbage collection" - I'm just guessing: with
generational garbage collection, you still have to do full
garbage collection once in a while, and on a system like that
it can take a while)

b. AFAIK, Yahoo Store was eventually rewritten in a non-Lisp.
Why? I'd tell you, but then I'd have to kill you :)

c. Emacs has a reputation for being slow and bloated. But then
it's not written in Common Lisp.

Are ViaWeb and Orbitz bigger successes than LATEX? Do they
have more users? It depends. Does viewing a PDF file made
with LATEX make you a user of LATEX? Does visiting Yahoo
store make you a user of ViaWeb?

For the sake of being balanced: there were also some *big*
failures, such as Lisp Machines. They failed because
they could not compete with UNIX (SUN, SGI) in a time when
performance, multi-userism and uptime were of prime importance.
(Older LispM's just leaked memory until they were shut down,
newer versions overcame that problem but others remained)

Another big failure that is often _attributed_ to Lisp is AI,
of course. But I don't think one should blame a language
for AI not happening. Marvin Mins ky, for example,
blames Robotics and Neural Networks for that.

[Lulu of the Lotus-Eaters]

Pascal Costanza <co******@web.d e> wrote previously:
|Wouldn't it be better if everyone could contribute to the evolution of a
|language, and then let the community decide what the best approaches are?

I can think of almost nothing worse than this!

Well, "design-by-committee" results like XSLT are even worse. But the
"benevolent dictator" model is VASTLY better than the "let 1000 flowers
bloom" approach.

Incidentally, I have never seen--and expect never to see--some new
mysterious domain where Python is too limited because the designers did
not forsee the problem area. Nor similarly with other very high level
languages. It NEVER happens that you just cannot solve a problem
because of the lack of some novel syntax to do so... that's what
libraries are for.

Yours, Lulu...

--
Keeping medicines from the bloodstreams of the sick; food from the bellies
of the hungry; books from the hands of the uneducated; technology from the
underdeveloped; and putting advocates of freedom in prisons. Intellectual
property is to the 21st century what the slave trade was to the 16th.

[David Steuber]

Ville Vainio <vi************ ********@spamtu t.fi> writes:

I only use Emacs Lisp. I have Python for the problem domains that CL
and Scheme would cover. I guess I might start hacking CL if/when
XEmacs switched their Lisp implementation, which I believe is in the
roadmaps...

If even XEmacs is ported over to Common Lisp in my lifetime I will be
surprised. If it happens to GNU Emacs, surprise will not be enough
and I will have to resort to astonishment.

I'm not thinking of the core code. I'm thinking of all those *.el
files floating around the universe.

--
One Emacs to rule them all. One Emacs to find them,
One Emacs to take commands and to the keystrokes bind them,

All other programming languages wish they were Lisp.

[Björn Lindberg]

Alex Martelli <al***@aleax.it > writes:

The approach for Lisp is to write a domain-specific language for the
problem at hand, and then to write the program in that domain-specific

Right. People who fancy themselves as language designers will surely
be happier that way. People who prefer to use languages used by better
language designers than themselves, such as me, will be happier with
Python.

Any programmer writing a program is by necessity also a language
designer. The act of writing a program consists of designing a
language. Did you read Steele's "Growing a Language", which someone
posted a pointer to earlier in the thread? It is essentially about
Java, but is very enlightening on the "programmin g as language design"
issue.

There is nothing magical or special about syntactic abstraction. It is
just an abstraction mechanism like the rest of them.
Björn

[Alex Martelli]

Pascal Costanza wrote:
...

In the case of Python, couldn't you rightfully regard it as driven by a
one-man commitee? ;-)
Ah, what a wonderfully meaningful view that is.

Specifically: when you want to ALTER SYNTAX...

If it were only about making small alterations to the syntax, I wouldn't

I didn't say SMALL. Small or large, it's about alteration to the
syntax. Other lispers have posted (on several of this unending
multitude of threads, many but not all of which I've killfiled)
stating outright that there is no semantic you can only implement
with macros: that macros are ONLY to "make things pretty" for
given semantics. If you disagree with them, I suggest pistols at
ten paces, but it's up to you lispers of course -- as long as
you guys with your huge collective experience of macros stop saying
a million completely contradictory things about them and chastising
me because (due, you all keep claiming, to my lack of experience)
I don't agree with all of them, I'll be glad to debate this again.

Till then, this is yet another thread that get killfiled.

lead this discussion. I mean what I say, I am talking about adding
full-fledged language constructs.
Right: exactly what I _DON'T_ want to happen to my favourite language.

you have to step outside of the language, and for example use program
generators . Python doesn't give you a choice here.

...you can't "just alter Python's own syntax with your own changes":
you have plenty of choices (implement your chosen semantics within
Python's syntax, use any of N available parsers to parse your favourite
syntax -- user EITHER "program generators" OR interpreters at any
level on the parsed forms -- etc, etc). You just don't have the one
choice of *altering Python's own syntax wholly within Python itself*.

Exactly. Lisp-style Macros make these things a breeze. The other
alternatives you suggest are considerably more complicated. As I have

What's "considerab ly more complicated" in, say,
my_frobazzer = frobaz_compiler ('''
oh my pretty, my beauteous,
my own all my own special unique frambunctious *LANGUAGE*!!!
''')
and later on call my_frobazzed(bi m. bum, bam) at need? The complexity
of the frobaz_compiler factory callable depends exclusively on the
complexity of the language you want it to parse and compile, and you
of course have total choice ("Python doesn't give you a choice" MY
LEFT FOOT) on how to apportion the work between the factory and the
__call__ method of the resulting my_frobazzer.

You just have to respect Python's syntax and fundamental semantics
(e.g., strict order when your Python code calls a Python callable --
you can ALWAYS, TOTALLY count on this, NO surprises possible EVER),
and you still have a BAZILLION of choices, even AFTER making the
decision, which SHOULD be momentous rather than "a breeze" (!!!),
to CHOOSE to structure your whole application around your own unique
language or sub-language.

If Python makes it just as appealing to structure applications in
other ways, reserving "language design" for those occasions which
really call for it, so much the better. As I've repeatedly posted,
I've SUFFERED the consequences of computer scientists "breezily"
designing embedded Lisp subdialects for "domain-specific languages"
for domains they DIDN'T know 1/10th as well as the poor non-CS'ers
forced to suffer under that mishmash (hardware design the domain,
hardware design engineers the poor non-CS'ers) -- while at other
competing firms language design was treated with the respect it
deserves, producing not-embedded-in-anything, not-necessarily-
distorted-extended-subsets-of-anything hardware design languages
that let people actually DESIGN HARDWARE when that was their job.
said before, Lisp makes the simple things a little bit harder (-> syntax
for beginners), so that the hard things become considerably simpler (->
adding sophisticated abstractions in order to considerably reduce the
amount of code for complex tasks).
The vast majority of applications has absolutely no need to tinker
with the language's syntax and fundamental semantics. When you need
a language with different syntax, this is best treated as a serious
task and devoted all the respect it deserves, NOT treated as "a
breeze". PARTICULARLY for domain-specific languages, the language's
designers NEED access to domain-specific competence, which typically
they won't have enough of for any domain that doesn't happen to be
what they've spent many years of their life actually DOING (just
STUDYING doesn't cut it); the people WITH the domain-specific
competence will in turn not typically be experienced programmers --
they've spent their lives getting experience in the specific domain
instead. Making things harder for "beginners" (which may well be
the domain-specific experts) is then the wrong choice: those
non-programmers fully deserve, if a domain-specific language is
indeed warranted by the task's complexity, a language that does NOT
just happens to embed bits and pieces of another, harder-for-
beginners language, just because it was "a breeze" to design it thus.

I'm talking about substantial applications meant to get into heavy
production use. For experimentation , play, research in computing
itself, whatever, go ahead, not my own interest. For application
programming, IF a domain-specific language is needed, I want it to
be designed FOR THE USERS' NEEDS AND CONVENIENCE (==productivity !),
NOT because it's easy to hack together with macros. And that IF is
a big one; more often than not, no weird new syntax and semantics
are warranted. If it's too easy to tweak syntax and semantics, they
will be tweaked whether that's warranted or not; such frequent and
lightly undertaken SYNTAX CHANGES are NOT what I want in the language
I will use, in collaboration with many others, to build applications.

Some resistance to change is useful. Without attrition, walking
would be VERY hard. Without a constitution that's FAR harder to
change than ordinary laws, and "balance of powers" so that the
legislature, the executive and the courts hamper each other, a
democracy might be dangerously unstable (the "Federalist Papers"
make that case particularly well). Without an unchanging core,
such as syntax and fundamental semantics, a language may be less
useful for application development by middling-large groups (the
language might be wonderful for play and free-spirited experiments
AND at the same time less wonderful for such usage).

Describing a situation of having 99 available strategies rather than
100 as "doesn't give you a choice" can be charitably described as
"ridiculous ", it seems to me.

I know about these choices, I have tried them all. I have written

So, you KNOW "doesn't give you a choice" is ridiculous.
compilers, interpreters, with and without parser generators, and have
co-designed and used transformation frameworks. These things are
considerably harder to build, and you need to spend a considerable
amount of time on the technology itself which distracts you from solving
You are (by far) exaggerating the effort. Check out such tools as
spark or SimpleParse and you'll see that parsing a custom language,
e.g. into an AST, is reasonably trivial. Once you have your AST,
you're basically in the same position as you start out with
S-exprs, so compiling and/or interpreting them is the same work.

What you don't do this way is MERGING LANGUAGE AND META-LANGUAGE --
sprinkling bits and pieces of your own custom language in the middle
of your implementation language and vice versa -- not "casually": if
and when you decide to embed languages into each other you must
deliberately take all decisions about arranging that.

As a consequence, the language you design is NOT influenced by the
language[s] you choose to use for the implementation thereof -- no
special push to make the supposedly application-domain-specific
language conform to uses and conventions typical of the implementation
language, no fragments of the implementation language suddenly perking
up in the middle of the application-domain-specific language.

So, for example, the BLM Language being developed at AB Strakt
(www.strakt.com) is INDEPENDENT from Python (though the designers
have of course been influenced by the parts of Python's syntax
they most liked, e.g., significant indentation); in the first
implementation, the embedded language for expressions and actions
is also Python, but the syntax already provides for embedding
multiple procedural/imperative languages even inside a single
BLM module. The interface between the declarative parts and the
embedded imperative ones is defined in terms of functions and
objects, so that, if and when the application-domain-specific
experts demand it (and pay for it:-) other languages may just as
well be adapter (sufficiently non-dynamic ones will necessarily
end up with lots of boilerplate code such as
attrib := blm.services->get_attribute( current_name);
and the like, but then, users of non-dynamic languages do appear
to like such boilerplate:-).

Sure, it could perfectly have been done in lisp (or any other
sufficiently high-level language just as well), but one would
have had to resist the temptation to orient the language being
designed towards "ease of implementation with lisp macros and
ease of integration with snippets of lisp", making it a
totally separate language -- and I've seen, and suffered on
my own skin, what typical CS'ers do about resisting such
temptation, particularly when convinced (as most, though not
all, of you guys seem to be) that lisp is obviously the best
language for ANY task, so why EVER could one POSSIBLY want a
totally different one for [insert any application domain]...?!
[Well, I've seen what they did about it 20+ years ago -- no
doubt human nature HAS changed drastically since, I'm sure].
the problem at hand. With Lisp-style macros, these things are much
simpler to handle once you have gotten it.
But will the simple way be the RIGHT one? Or will you end up
designing application-domain-specific languages needlessly, in
applications that would be much better structured in other ways,
AND when an application-domain-specific language IS called for,
end up "along the continuum" designing it ``by inertia'' as just
some "extended subset" of lisp, instead...?

But _in practice, people don't_, except for play and experiments
of various kinds. You can find a bazillion extremely varied
extensions to Python of all imaginable kinds, *BUT* they're ALL
*add-ons*, NEVER alteration of syntax or fundamental semantics

You still have a wrong idea of how macros work. Macros are of course
also only add-ons. You don't alter unrelated syntax or unrelated
portions of source code, and you don't change fundamental semantics.

I have HAD to use (several mutually incompatible varieties of)
"applicatio n-domain-specific languages" built by lispers with
their beloved macros (in the early '80s, so it wasn't common
lisp yet, but rather a congeries of different lisp dialects,
including scheme). I know how macros work: they get hacked by
computer scientists that don't really grok the application domain,
then the resulting "extended subset of lisp" du jour is foisted
on people who DO know the application domain, CLAIMING falsely
to be appropriate for the specific application domain -- why,
they're (some kind of) lisp, OF COURSE they're automatically ideal
for ANY application right? And I'm not sure what the "unrelated"
syntax or portions of sources were, if any -- we had to describe
system-level, interface-level, circuit-level &c views of a chip
being developed in one or more of those (expl.del) "applicatio n
awful languages" and NO of _course_ you can't use COND here
because [gibbledygook] -- that's in AAL 1 as designed in Lab 1,
while in AAL 2 designed in Lab 2 OBVIOUSLY you must use COND here,
I thought you were HIRED because you had used lisp (I had, scheme
actually, but that was the favoured lisp of lab _3_ so mentioning
it in labs 1 or 2 made you an instant leper obviously).

No doubt "the solution" you proposed would have been to learn a
few of those GD macro systems and write the N-plus-1-th divergent
incompatible subdialect. Yeah right. I found a better one --
jump ship to IBM Research, get a raise, get to use a GOOD (for
the times, mind you -- still over 20 years ago) hardware design
language, have the fun of my life designing HW for a while until
I had to go back to SW -- and reconciling myself to the latter when
I found out it didn't HAVE to mean endlessly designing languages
for domains you didn't truly grasp with congeries of pieces of an
existing language and half-baked concoctions of your own -- you
COULD actually design good systems and algorithms, just like you
could with hardware, using SOLID, UNCHANGING (inside a project's
timeframe:-), _almost_-decently-designed-for-the-general-domain
languages [PLURAL!!!! *YES*!!!]. Admittedly, "almost". I tried
my hand at designing specialized languages (one of those sins
that just about everybody practices in youth, I guess) and found
out that, even when I grasped the application domain perfectly
well, that still didn't make my languages usable by other experts
of the domain (maybe because I kept falling onto prefix s-exproid
syntax in those years -- and just about everybody else hated it).
Instead of spreading misconceptions you should really start to get to
know the things better that you criticize.

Been there, done that, *NEVER, NEVER AGAIN*. Don't keep repeating
this claim, analogous to one that somebody who's lived and suffered
under a dictatorship shouldn't be spreading misconceptions but try
being a dictator to "know the things better that you criticize". I
don't want to foist my half-baked macro-based lisp dialects on
others any more than I'm going to suffer others foisting their own
on me, thankyouverymuc h. If and when I should find myself working
on a solo project for which the need to tweak syntax within an
s-expr world appears, I may try some kind of lisp (hopefully, by
then, Graham's new baby -- if he's as good as the reverence every
lisper devotes him seems to indicate, then his claim that he can
make a lisp that's better than both scheme and CL while having the
strenths of both -- and then some -- should be credible).

But, until then -- bye. And now, to killfile this thread too....
Alex

[Bengt Richter]

On Mon, 20 Oct 2003 21:25:55 +0200, Pascal Costanza <co******@web.d e> wrote:

Alex Martelli wrote:

[...]

...you can't "just alter Python's own syntax with your own changes":
you have plenty of choices (implement your chosen semantics within
Python's syntax, use any of N available parsers to parse your favourite
syntax -- user EITHER "program generators" OR interpreters at any
level on the parsed forms -- etc, etc). You just don't have the one
choice of *altering Python's own syntax wholly within Python itself*.

Exactly. Lisp-style Macros make these things a breeze. The other
alternatives you suggest are considerably more complicated. As I have
said before, Lisp makes the simple things a little bit harder (-> syntax
for beginners), so that the hard things become considerably simpler (->
adding sophisticated abstractions in order to considerably reduce the
amount of code for complex tasks).

Describing a situation of having 99 available strategies rather than
100 as "doesn't give you a choice" can be charitably described as
"ridiculous ", it seems to me.

I know about these choices, I have tried them all. I have written
compilers, interpreters, with and without parser generators, and have
co-designed and used transformation frameworks. These things are
considerably harder to build, and you need to spend a considerable
amount of time on the technology itself which distracts you from solving
the problem at hand. With Lisp-style macros, these things are much
simpler to handle once you have gotten it.

But _in practice, people don't_, except for play and experiments
of various kinds. You can find a bazillion extremely varied
extensions to Python of all imaginable kinds, *BUT* they're ALL
*add-ons*, NEVER alteration of syntax or fundamental semantics

You still have a wrong idea of how macros work. Macros are of course
also only add-ons. You don't alter unrelated syntax or unrelated
portions of source code, and you don't change fundamental semantics.

Instead of spreading misconceptions you should really start to get to
know the things better that you criticize.

Maybe we can put this thing to bed by a Python foreign-language-module-encapsulating class,
and a functionally equivalent lisp a macro for lispers, so we could write something like (sketch)

module_from_lis p = FLModule('lisp' ,'lisp_file_pat h_or_remote_spe c',
'interface spec string', 'make info string')

where the last args would have sensible defaults, but all would be expected to be passed
to a translation process selected by the language name (the first arg) unless a cached module
could be accessed without calling for any translation.

For C or C++ this could generate a DLL extension module and import that, assuming the
relevant tools are available. Otherwise an informative exception could be raised.

For other languages, a DLL that serves as an access proxy for whatever. Letting the source
be specified by a remote spec like an URL with some authentication requirement might open
some interesting uses (language name 'python' would be allowed ;-) (maybe even 'python1.5.2' ;-)

I know there are already pieces for integrating/accessing C, but the idea here is to
unify the concept into one standard general form expressed by a standard class which could
always have a presence in builtins.

Just an alpha idea at this point ;-)

Hm, think I'll switch the title, in case people aren't reading the old subject thread ;-)

Regards,
Bengt Richter

[Scott McIntire]

"Lulu of the Lotus-Eaters" <me***@gnosis.c x> wrote in message
news:ma******** *************** **************@ python.org...

Pascal Costanza <co******@web.d e> wrote previously:
|Wouldn't it be better if everyone could contribute to the evolution of a
|language, and then let the community decide what the best approaches are?

I can think of almost nothing worse than this!

Well, "design-by-committee" results like XSLT are even worse. But the
"benevolent dictator" model is VASTLY better than the "let 1000 flowers
bloom" approach.

Incidentally, I have never seen--and expect never to see--some new
mysterious domain where Python is too limited because the designers did
not forsee the problem area. Nor similarly with other very high level
languages. It NEVER happens that you just cannot solve a problem
because of the lack of some novel syntax to do so... that's what
libraries are for.

Yours, Lulu...

Can't one make the same argument for Visual Basic. Use libraries as the
method to "build up the language" to a domain. Keep the language as simple
as possible and just use libraries - yeah, that's the ticket.

-R. Scott McIntire
--
Keeping medicines from the bloodstreams of the sick; food from the bellies
of the hungry; books from the hands of the uneducated; technology from the
underdeveloped; and putting advocates of freedom in prisons. Intellectual
property is to the 21st century what the slave trade was to the 16th.

[Paul Rubin]

David Steuber <da***********@ verizon.net> writes:

If even XEmacs is ported over to Common Lisp in my lifetime I will be
surprised. If it happens to GNU Emacs, surprise will not be enough
and I will have to resort to astonishment.

I'm not thinking of the core code. I'm thinking of all those *.el
files floating around the universe.

Plans have been to convert Emacs to use a Scheme dialect with hacks to
support old .el files.

[Jock Cooper]

Alex Martelli <al***@aleax.it > writes:

Pascal Costanza wrote:
snip
What's "considerab ly more complicated" in, say,
my_frobazzer = frobaz_compiler ('''
oh my pretty, my beauteous,
my own all my own special unique frambunctious *LANGUAGE*!!!
''')
and later on call my_frobazzed(bi m. bum, bam) at need? The complexity
of the frobaz_compiler factory callable depends exclusively on the
complexity of the language you want it to parse and compile, and you
of course have total choice ("Python doesn't give you a choice" MY
LEFT FOOT) on how to apportion the work between the factory and the
__call__ method of the resulting my_frobazzer.

Just imagine if your embedded language also used white space to denote
program structure.. cut and paste would get a little more tricky...

[David Mertz]

|"Lulu of the Lotus-Eaters" <me***@gnosis.c x> wrote in message
|> It NEVER happens that you just cannot solve a problem
|> because of the lack of some novel syntax to do so... that's what
|> libraries are for.

"Scott McIntire" <mc************ ******@comcast. net> wrote previously:
|Can't one make the same argument for Visual Basic. Use libraries as the
|method to "build up the language" to a domain.

Sure, in a sense. But the problem(s) with VB are not that it cannot be
extended to some novel domain syntax because it lacks macros.

VB is a bad language, but it is JUST AS BAD in its "core" areas as it is
when extended to new problems. There is a very strong disanalogy with
Python here. Python does a good job of expressing all the basic
*programming* constructs that go into ANY programming domain: loops,
branches, collections, objects and inheritence, HOFs, declared
constraints, etc. I have no expectation of EVER seeing some new problem
area that isn't built out of these same application parts.

Once in a great while, something really new comes along in programming
constructs themselves. OOP was such a thing, HOFs were, multimethods
are at a smaller level, constraint declarations (i.e. logic programming)
were; maybe coroutines are. Python does all of these things well--see
some of my articles for explanations of some of them where the basic
syntax needs some straightforward (pure-Python) extension modules.

The ONLY example I can think of where a basic construct is difficult to
shoehorn into Python might be AOP. I remain agnostic about the staying
power of this one, but it might prove important. There are some modules
out there, and metaclasses let you do it (in a less than elegant way).
On the other hand, Lisp--macros and all--is not any better suited to AOP
than is Python. I know you can work up something that nominally
qualifies as AOP using obscenely brittle macros... but it's not
something I would want to do.

Yours, David...

--
Keeping medicines from the bloodstreams of the sick; food from the bellies
of the hungry; books from the hands of the uneducated; technology from the
underdeveloped; and putting advocates of freedom in prisons. Intellectual
property is to the 21st century what the slave trade was to the 16th.

[Donn Cave]

In article <OiZkb.838841$u u5.148203@sccrn sc04>,
"Scott McIntire" <mc************ ******@comcast. net> wrote:

"Lulu of the Lotus-Eaters" <me***@gnosis.c x> wrote in message
news:ma******** *************** **************@ python.org... ....

Incidentally, I have never seen--and expect never to see--some new
mysterious domain where Python is too limited because the designers did
not forsee the problem area. Nor similarly with other very high level
languages. It NEVER happens that you just cannot solve a problem
because of the lack of some novel syntax to do so... that's what
libraries are for.

Can't one make the same argument for Visual Basic. Use libraries as the
method to "build up the language" to a domain. Keep the language as simple
as possible and just use libraries - yeah, that's the ticket.

Well - yes! I mean, I honestly don't know a thing about Visual
Basic, but I find no irony in this. Relatively humble language
is widely used thanks to rich library support - QED.

Now I'm sure Python programmers will mostly say that its core
language features are superior to Basic's and a move to that
language would be a big step backwards, and of course the same
will eventually apply to Python when a radically better language
arrives. But that's thanks to basic language design, not
incremental tinkering. As the man said, incremental tinkering
doesn't open the door to new application areas, it's just something
to keep the language designer busy so he doesn't get bored and
design a new language that competes with the old one.

Donn Cave, do**@u.washingt on.edu