BIG successes of Lisp (was ...)

[mike420]

In the context of LATEX, some Pythonista asked what the big
successes of Lisp were. I think there were at least three *big*
successes.

a. orbitz.com web site uses Lisp for algorithms, etc.
b. Yahoo store was originally written in Lisp.
c. Emacs

The issues with these will probably come up, so I might as well
mention them myself (which will also make this a more balanced
post)

a. AFAIK Orbitz frequently has to be shut down for maintenance
(read "full garbage collection" - I'm just guessing: with
generational garbage collection, you still have to do full
garbage collection once in a while, and on a system like that
it can take a while)

b. AFAIK, Yahoo Store was eventually rewritten in a non-Lisp.
Why? I'd tell you, but then I'd have to kill you :)

c. Emacs has a reputation for being slow and bloated. But then
it's not written in Common Lisp.

Are ViaWeb and Orbitz bigger successes than LATEX? Do they
have more users? It depends. Does viewing a PDF file made
with LATEX make you a user of LATEX? Does visiting Yahoo
store make you a user of ViaWeb?

For the sake of being balanced: there were also some *big*
failures, such as Lisp Machines. They failed because
they could not compete with UNIX (SUN, SGI) in a time when
performance, multi-userism and uptime were of prime importance.
(Older LispM's just leaked memory until they were shut down,
newer versions overcame that problem but others remained)

Another big failure that is often _attributed_ to Lisp is AI,
of course. But I don't think one should blame a language
for AI not happening. Marvin Mins ky, for example,
blames Robotics and Neural Networks for that.

[David Mertz]

Bruce Hoult <br***@hoult.or g> wrote previously:
|~bruce$ ls -l `which emacs`
|-rwxr-xr-x 1 root wheel 4596224 Sep 24 04:29 /usr/bin/emacs
|~bruce$ ls -l /Applications/Microsoft\ Office\ X/Microsoft\ Word
|-rwxr-xr-x 1 bruce admin 10568066 Sep 26 2002
|/Applications/Microsoft Office X/Microsoft Word

Not even Windows users use MS-Word to edit program code; this is a
completely irrelevant comparison.

For more realistic ones (from my OS/2 machine that I'm sitting at, and
the editors I actually use):

D:\editors % ls boxer\b2.exe jEdit4.2pre1\je dit.jar fte\fte.exe
3-24-95 7:00a 317127 0 b2.exe
5-06-03 1:36a 2797098 219 jedit.jar
1-04-00 9:19p 585235 0 fte.exe

On a Linux box where my website is hosted (and emacs isn't installed
even if I wanted to use it):

~$ ls -l /usr/bin/joe /usr/bin/vim
-r-xr-xr-x 1 root root 166160 Feb 28 2001 /usr/bin/joe*
-rwxr-xr-x 1 root root 1172464 Dec 5 2001 /usr/bin/vim*

On my Macs I use jEdit too, so the size is basically the same (although
I have a bit more up-to-date version there). On my FreeBSD box maybe
I'll use kEdit or kDevel, or jed or vim; but I'm not sure of file sizes
from here.

IOW: Emacs is BLOATED.

Yours, David...

--
Keeping medicines from the bloodstreams of the sick; food from the bellies
of the hungry; books from the hands of the uneducated; technology from the
underdeveloped; and putting advocates of freedom in prisons. Intellectual
property is to the 21st century what the slave trade was to the 16th.

[Thomas Lindgren]

me***@gnosis.cx (David Mertz) writes:

[examples snipped]

IOW: Emacs is BLOATED.

I have 512 MB of memory on my $300 PC. Mozilla at the time of posting
eats 36M, emacs 15 M. I reckon that BLOAT will be approaching a real
problem when I run, like, twenty or thirty emacses or so, at once.

In short, a non-issue for the ordinary computer user.

Best,
Thomas
--
Thomas Lindgren
"It's becoming popular? It must be in decline." -- Isaiah Berlin

[Paul Rubin]

me***@gnosis.cx (David Mertz) writes:

|-rwxr-xr-x 1 root wheel 4596224 Sep 24 04:29 /usr/bin/emacs
|~bruce$ ls -l /Applications/Microsoft\ Office\ X/Microsoft\ Word
|-rwxr-xr-x 1 bruce admin 10568066 Sep 26 2002
|/Applications/Microsoft Office X/Microsoft Word

Not even Windows users use MS-Word to edit program code; this is a
completely irrelevant comparison.
OK, compare it to Visual Studio (not counting the compilers) instead.
IOW: Emacs is BLOATED.

Or whatever else you're using is underpowered...

[Kaz Kylheku]

Alex Martelli <al***@aleax.it > wrote in message news:<rD******* *************** *@news2.tin.it> ...

Pascal Costanza wrote:

Exactly. Lisp-style Macros make these things a breeze. The other
alternatives you suggest are considerably more complicated. As I have
What's "considerab ly more complicated" in, say,
my_frobazzer = frobaz_compiler ('''
oh my pretty, my beauteous,
my own all my own special unique frambunctious *LANGUAGE*!!!
''')
and later on call my_frobazzed(bi m. bum, bam) at need?

The problem is that

program = compiler(charac ter-string)

is too much of a closed design. A more flexible design resembles:

abstract-syntax-tree = reader(characte r-string)

target-syntax-tree = translator(abst ract-syntax-tree)

program = compiler(target-syntax-tree)

The input to a compiler should not be a character string, but
structured data.
The complexity
of the frobaz_compiler factory callable depends exclusively on the
complexity of the language you want it to parse and compile, and you
In Lisp, the complexity of the compiler is constant; it's a language
builtin. The complexity of the reader depends on the complexity of the
lexical properties of the language, and the complexity of the
translator depends on the semantic complexity of the language.
The vast majority of applications has absolutely no need to tinker
with the language's syntax and fundamental semantics.
This is a common fallacy: namely that the difficulty of doing
something, and the consequent infrequency of doing it, constitute
evidence for a lack of need of doing it. In fact this is nothing more
than rationalization : ``I know I have inadequate tools, but I really
don't need them''. It's not unlike ``I can't reach those grapes, but I
know they are sour anyway''.

In Lisp, tinkering with the syntax and semantics is done even in
trivial programs.

By the way, your use of ``fundamental'' suggests a misunderstandin g:
namely that some kind of destructive manipulation of the language is
going on to change the foundations, so that existing programs are no
longer understood or change in meaning. This is not so: rather, users
extend the language to understand new constructs. The fundamental
syntax and semantics stay what they are; they are the stable target
language for the new constructs.
When you need
a language with different syntax, this is best treated as a serious
task and devoted all the respect it deserves, NOT treated as "a
breeze".
This is a common viewpoint in computer science. Let me state it like
this: ``language design is a Hard Problem that requires you to whip
out lexical analyzers, parser constructors, complex data structures
for symbol table management, intermediate code generation, target code
generation, instruction selection, optimization, etc.'' But when you
have this:

abstract-syntax-tree = reader(characte r-string)

target-syntax-tree = translator(abst ract-syntax-tree)

program = compiler(target-syntax-tree)

you can do most of the language design work in the second step:
translation of syntax trees into other trees. This is where macro
programming is done, and there is a large amount of clever
infrastructure in Lisp which makes it easy to work at this level. Lisp
contains a domain language for language construction.
PARTICULARLY for domain-specific languages, the language's
designers NEED access to domain-specific competence, which typically
they won't have enough of for any domain that doesn't happen to be
what they've spent many years of their life actually DOING (just

Bingo! This is where Lisp comes in; it gives the domain experts the
power to express what they want, without requiring them to become
compiler construction experts. This is why Lisp is used by some
artificial intelligence researchers, biologists, linguists, musicians,
etc.

[Raffael Cavallaro]

me***@gnosis.cx (David Mertz) wrote in message news:<ma******* *************** *************** @python.org>...

IOW: Emacs is BLOATED.

In an era of 250 GB hard drives, and GB RAM modules, who cares that
emacs is about 5MB in size, and uses 4 (real) to 10 (virtual) MB of
RAM.

Moore's law is lisp's friend. ;^)

[Andrew Dalke]

Kaz Kylheku:

The problem is that

program = compiler(charac ter-string)

is too much of a closed design. A more flexible design resembles:

abstract-syntax-tree = reader(characte r-string)

target-syntax-tree = translator(abst ract-syntax-tree)

program = compiler(target-syntax-tree)
But you didn't see the implementation of Alex's 'compiler' function.
It looks like

def compiler(charac ter_string):
return compile_tree(tr anslator(reader (character_stri ng)))

and those intermediates are part of the publically usable
API.
The input to a compiler should not be a character string, but
structured data.
I think you're arguing naming preferences here. That's fine;
Alex's point remains unchanged. In a dynamic language like
Python, parsing a domain specific language can be done at
run-time, parsed, converted to a Python parse tree, and
compiled to Python byte codes, just like what you want and
in the form you want it.
In Lisp, the complexity of the compiler is constant; it's a language
builtin. The complexity of the reader depends on the complexity of the
lexical properties of the language, and the complexity of the
translator depends on the semantic complexity of the language.
Replace 'Lisp' with 'Python' and the result is still true. Ditto
for 'C'. So I'm afraid I don't understand your point.

The vast majority of applications has absolutely no need to tinker
with the language's syntax and fundamental semantics.

This is a common fallacy: namely that the difficulty of doing
something, and the consequent infrequency of doing it, constitute
evidence for a lack of need of doing it. In fact this is nothing more
than rationalization : ``I know I have inadequate tools, but I really
don't need them''. It's not unlike ``I can't reach those grapes, but I
know they are sour anyway''.

That's not Alex's argument. Python has the ability to do exactly
what you're saying (domain language -> AST -> Python code or AST ->
compiler). It's rarely needed (I've used it twice now in my six years
or so of Python), so why should a language cater to make that
easy at the expense of making frequent things harder?
In Lisp, tinkering with the syntax and semantics is done even in
trivial programs.
And that's a good thing? That means that everyone looking at
new Lisp code needs to understand the modifications to the syntax
and semantics. That may be appropriate for an insular organization,
but otherwise it makes it harder for others to understand any code.
By the way, your use of ``fundamental'' suggests a misunderstandin g:
namely that some kind of destructive manipulation of the language is
going on to change the foundations, so that existing programs are no
longer understood or change in meaning. This is not so: rather, users
extend the language to understand new constructs. The fundamental
syntax and semantics stay what they are; they are the stable target
language for the new constructs.
Alex and others have responded to this argument many times. The
summary is that 1) in practice those who extend the language are most
often not the domain experts so the result doesn't correctly capture
the domain, 2) because it's easy to do, different groups end up with
different, incompatible domain-specific modifications, 3) rarely
is that approach better than using OOP, HOF and other approaches,
where better is defined as more flexible, easier to read, more
succinct, etc.
This is a common viewpoint in computer science. Let me state it like
this: ``language design is a Hard Problem that requires you to whip
out lexical analyzers, parser constructors, complex data structures
for symbol table management, intermediate code generation, target code
generation, instruction selection, optimization, etc.''
Actually, language design doesn't require any of those. They
are needed to implement a language.

Let me add that implementing a language *was* a Hard Problem,
but effectively solved in the 1970s. The solutions are now well
known and there are a huge number of tools to simplify all
the steps in that process, books on the subject, and people with
experience in doing it.

There are still hard problems, but they are hard engineering
problems, not hard scientific ones where the theory is not
well understood.
But when you have this:

abstract-syntax-tree = reader(characte r-string)

target-syntax-tree = translator(abst ract-syntax-tree)

program = compiler(target-syntax-tree)

you can do most of the language design work in the second step:
translation of syntax trees into other trees. This is where macro
programming is done, and there is a large amount of clever
infrastructure in Lisp which makes it easy to work at this level. Lisp
contains a domain language for language construction.

Python has all that, including the ability to turn a string into
a Python AST, manipulate that tree, and compile that tree.

It's not particularly clever; there's no real need for that. In
general, the preference is to be clear and understandable over
being clever. (The New Jersey approach, perhaps?)

(Though the compiler module is pretty clumsy to use.)

PARTICULARLY for domain-specific languages, the language's
designers NEED access to domain-specific competence, which typically
they won't have enough of for any domain that doesn't happen to be
what they've spent many years of their life actually DOING (just

Bingo! This is where Lisp comes in; it gives the domain experts the
power to express what they want, without requiring them to become
compiler construction experts. This is why Lisp is used by some
artificial intelligence researchers, biologists, linguists, musicians,
etc.

Speaking as a representative from the biology community, the
Lisp programmers are a minority and far behind C, Fortran, Perl,
and still behind Python, Tcl, and even Ruby.

*If* the domain expert is also an expert Lisp program then
what you say is true. It's been my experience that most domain
experts are not programmers -- most domains aren't programming.
Even in what I do, computational life sciences, most chemists
and biologists can do but a smattering of programming. That's
why they hire people like me. And I've found that objects and
functions are good enough to solve the problems in that domain;
from a CS point of view, it's usually pretty trivial.

Andrew
da***@dalkescie ntific.com

[Ville Vainio]

"Andrew Dalke" <ad****@mindspr ing.com> writes:

It's not particularly clever; there's no real need for that. In
general, the preference is to be clear and understandable over
being clever. (The New Jersey approach, perhaps?)

Some people (academics) are paid for being clever. Others (engineers)
are paid for creating systems that work (in the wide meaning of the
word), in a timeframe that the company/client can afford.

In the for-fun area, by analogy, some people get the kick from
creating systems that work (be it a Linux distribution or a network
programming framework), and some from creating an uber-3133t hacks in
order to impress their friends.

Macros provide billions of different ways to be "clever", so obviously
Lisp gives greater opportunity of billable hours for people who can
bill for clever stuff. I'm studying Grahams "On Lisp" as bad-time
reading ATM, and can also sympathize w/ people who use Lisp just for
the kicks.

Lisp might have a good future ahead of it if it was only competing
againt C++, Java and others. Unfortunately for Lisp, other dynamic
languages exist at the moment, and they yield greater
productivity. Most bosses are more impressed with getting stuff done
fast than getting it done slowly, using gimmicks that would have given
you an A+ if it was a CS research project.

--
Ville Vainio http://www.students.tut.fi/~vainio24

[Ville Vainio]

Ville Vainio <vi************ ********@spamtu t.fi> writes:

bill for clever stuff. I'm studying Grahams "On Lisp" as bad-time

^^^

Typo, s/bad/bed, obviously :).

--
Ville Vainio http://www.students.tut.fi/~vainio24

[Espen Vestre]

me***@gnosis.cx (David Mertz) writes:

Not even Windows users use MS-Word to edit program code; this is a
completely irrelevant comparison.

I'm not sure you're right. Using Words little brother Wordpad is just
as weird, and we just had proof of that.
--
(espen)

[Pascal Costanza]

Andrew Dalke wrote:

That's not Alex's argument. Python has the ability to do exactly
what you're saying (domain language -> AST -> Python code or AST ->
compiler). It's rarely needed (I've used it twice now in my six years
or so of Python), so why should a language cater to make that
easy at the expense of making frequent things harder?
Maybe you have only rarely used it because it is hard, and therefore
just think that you rarely need it. At least, this is my assessment of
what I have thought to be true before I switched to Lisp.

In Lisp, tinkering with the syntax and semantics is done even in
trivial programs.

And that's a good thing? That means that everyone looking at
new Lisp code needs to understand the modifications to the syntax
and semantics. That may be appropriate for an insular organization,
but otherwise it makes it harder for others to understand any code.

That's true for any language. In any language you build new data
structures, classes, methods/functions/procedures, and everyone looking
at new code in any language needs to understand these new definitions.
There is no difference here _whatsoever_.

Modifying syntax and creating new language abstractions only _sounds_
scary, but these things are like any other activity during programming
that take care, as soon as the language you use supports them well.

By the way, your use of ``fundamental'' suggests a misunderstandin g:
namely that some kind of destructive manipulation of the language is
going on to change the foundations, so that existing programs are no
longer understood or change in meaning. This is not so: rather, users
extend the language to understand new constructs. The fundamental
syntax and semantics stay what they are; they are the stable target
language for the new constructs.

Alex and others have responded to this argument many times. The
summary is that 1) in practice those who extend the language are most
often not the domain experts so the result doesn't correctly capture
the domain,

Are these non-experts any better off with just data structures and
functions?
2) because it's easy to do, different groups end up with
different, incompatible domain-specific modifications,
Do they also come up with different APIs? How is divergence of APIs
solved in practice? Can't you use the same solutions for macro libraries?
3) rarely
is that approach better than using OOP, HOF and other approaches,
where better is defined as more flexible, easier to read, more
succinct, etc.

Are you guessing, or is this based on actual experience?

BTW, macros are definitely more flexible and more succinct. The only
claim that I recall being made by Pythonistas is that macros make code
harder to read. The Python argument is that uniformity eases
readability; the Lisp argument is that a better match to the problem
domain eases readability. I think that...

(with-open-file (f "...")
...
(read f)
...)

....is much clearer than...

try:
f=open('...')
...
f.read()
...
finally:
f.close()

Why do I need to say "finally" here? Why should I even care about
calling close? What does this have to do with the problem I am trying to
solve? Do you really think it does not distract from the problem when
you first encounter that code and try to see the forest from the trees?

BTW, this is one of the typical uses for macros: When designing APIs,
you usually want to make sure that certain protocols are followed. For
the typical uses of your library you can provide high-level macros that
hide the details of your protocol.

Here is an arbitrary example that I have just picked from the
documentation for a Common Lisp library I have never used before:

(with-transaction
(insert-record :into [emp]
:attributes '(x y z)
:values '(a b c))
(update-records [emp]
:attributes [dept]
:values 50
:where [= [dept] 40])
(delete-records :from [emp]
:where [> [salary] 300000]))

(see
http://www.lispworks.com/reference/l...m#pgfId-889797
)

What do you think the with-transaction macro does? Do you need any more
information than that to understand the code?

BTW, note that the third line of this example is badly indented. Does
this make reading the code more difficult?

Here is why I think that Python is successful: it's because it favors
dynamic approaches over static approaches (wrt type system, and so on).
I think this is why languages like Ruby, Perl and PHP are also
successful. Languages like Java, C and C++ are very static, and I am
convinced that static approaches create more problems than they solve.

It's clear that Python is a very successful language, but I think this
fact is sometimes attributed to the wrong reasons. I don't think its
success is based on prettier syntax or uniformity. Neither give you an
objectively measurable advantage.
Pascal

--
Pascal Costanza University of Bonn
mailto:co****** @web.de Institute of Computer Science III
http://www.pascalcostanza.de Römerstr. 164, D-53117 Bonn (Germany)