BIG successes of Lisp (was ...)

[mike420]

In the context of LATEX, some Pythonista asked what the big
successes of Lisp were. I think there were at least three *big*
successes.

a. orbitz.com web site uses Lisp for algorithms, etc.
b. Yahoo store was originally written in Lisp.
c. Emacs

The issues with these will probably come up, so I might as well
mention them myself (which will also make this a more balanced
post)

a. AFAIK Orbitz frequently has to be shut down for maintenance
(read "full garbage collection" - I'm just guessing: with
generational garbage collection, you still have to do full
garbage collection once in a while, and on a system like that
it can take a while)

b. AFAIK, Yahoo Store was eventually rewritten in a non-Lisp.
Why? I'd tell you, but then I'd have to kill you :)

c. Emacs has a reputation for being slow and bloated. But then
it's not written in Common Lisp.

Are ViaWeb and Orbitz bigger successes than LATEX? Do they
have more users? It depends. Does viewing a PDF file made
with LATEX make you a user of LATEX? Does visiting Yahoo
store make you a user of ViaWeb?

For the sake of being balanced: there were also some *big*
failures, such as Lisp Machines. They failed because
they could not compete with UNIX (SUN, SGI) in a time when
performance, multi-userism and uptime were of prime importance.
(Older LispM's just leaked memory until they were shut down,
newer versions overcame that problem but others remained)

Another big failure that is often _attributed_ to Lisp is AI,
of course. But I don't think one should blame a language
for AI not happening. Marvin Mins ky, for example,
blames Robotics and Neural Networks for that.

[Pascal Costanza]

Ville Vainio wrote:

Lisp might have a good future ahead of it if it was only competing
againt C++, Java and others. Unfortunately for Lisp, other dynamic
languages exist at the moment, and they yield greater
productivity.
This is true for the things that are currently en vogue.
Most bosses are more impressed with getting stuff done
fast than getting it done slowly, using gimmicks that would have given
you an A+ if it was a CS research project.

I have implemented an AOP extension for Lisp that took about a weekend
to implement. The implementation is one page of Lisp code and is rather
efficient (wrt usability and performance) because of macros.

I have heard rumors that the development of an AOP extension for Python
would take considerably longer.
Pascal

--
Pascal Costanza University of Bonn
mailto:co****** @web.de Institute of Computer Science III
http://www.pascalcostanza.de Römerstr. 164, D-53117 Bonn (Germany)

[prunesquallor]

Ville Vainio <vi************ ********@spamtu t.fi> writes:

"Andrew Dalke" <ad****@mindspr ing.com> writes:

It's not particularly clever; there's no real need for that. In
general, the preference is to be clear and understandable over
being clever. (The New Jersey approach, perhaps?)
Some people (academics) are paid for being clever. Others
(engineers) are paid for creating systems that work (in the wide
meaning of the word), in a timeframe that the company/client can
afford.

And we all know that nothing made by academics actually works.
Conversely there is no need to be clever if you are an engineer.
Macros provide billions of different ways to be "clever", so obviously
Lisp gives greater opportunity of billable hours for people who can
bill for clever stuff. I'm studying Grahams "On Lisp" as bad-time
reading ATM, and can also sympathize w/ people who use Lisp just for
the kicks.
Shhh! Don't alert the PHB's to how we pad our hours!
Lisp might have a good future ahead of it if it was only competing
againt C++, Java and others. Unfortunately for Lisp, other dynamic
languages exist at the moment, and they yield greater productivity.
For instance, a productivity study done by Erann Gat showed ... no
wait. Where was that productivity study that showed how far behind
Lisp was?
Most bosses are more impressed with getting stuff done fast than
getting it done slowly, using gimmicks that would have given you an
A+ if it was a CS research project.

Which is *precisely* the reason that bosses have adopted C++ over C.

[Björn Lindberg]

Alex Martelli <al***@aleax.it > writes:

Pascal Costanza wrote:
...

In the case of Python, couldn't you rightfully regard it as driven by a
one-man commitee? ;-)
Ah, what a wonderfully meaningful view that is.

Specifically: when you want to ALTER SYNTAX...

If it were only about making small alterations to the syntax, I wouldn't

I didn't say SMALL. Small or large, it's about alteration to the
syntax. Other lispers have posted (on several of this unending
multitude of threads, many but not all of which I've killfiled)
stating outright that there is no semantic you can only implement
with macros: that macros are ONLY to "make things pretty" for
given semantics. If you disagree with them, I suggest pistols at
ten paces, but it's up to you lispers of course -- as long as
you guys with your huge collective experience of macros stop saying
a million completely contradictory things about them and chastising
me because (due, you all keep claiming, to my lack of experience)
I don't agree with all of them, I'll be glad to debate this again.

Why is this so surprising? Maybe different lispers use macros for
different things, or see different advantages to them? What all have
in common though, is that they all consider macros a valuable and
important part of a programming language. What I have seen in this
thread are your (Alex) lengthy posts where you reiterate the same
uniformed views of macros over and over again. You seem to have made
your mind up already, even though you don not seem to have fully
understood Common Lisp macros yet. Am I wrong?
Till then, this is yet another thread that get killfiled.
Why do you bother to post if you are not even going to read the
responses?

<snip>
But, until then -- bye. And now, to killfile this thread too....

What is the point in initiating a subthread by an almost 300-line,
very opinionated post just to immediately killfile it?
Björn

[Brian Kelley]

Pascal Costanza wrote:

BTW, macros are definitely more flexible and more succinct. The only
claim that I recall being made by Pythonistas is that macros make code
harder to read. The Python argument is that uniformity eases
readability; the Lisp argument is that a better match to the problem
domain eases readability. I think that...

(with-open-file (f "...")
...
(read f)
...)

...is much clearer than...

try:
f=open('...')
...
f.read()
...
finally:
f.close()

Why do I need to say "finally" here? Why should I even care about
calling close? What does this have to do with the problem I am trying to
solve? Do you really think it does not distract from the problem when
you first encounter that code and try to see the forest from the trees? Your two examples do completely different things and the second is
written rather poorly as f might might not exist in the finally block.
It certainly won't if the file doesn't exist.

A better comparison would (*might*, I haven't used lisp in a while) be:

(with-open-file (f filename :direction :output :if-exists :supersede)
(format f "Here are a couple~%of test data lines~%")) => NIL

if os.path.exists( filename):
f = open(filename, 'w')
print >> f, "Here are a couple of test data lines"

I think that the latter is easier to read and I don't have to worry
about f.close(), but that is just me. I don't know what with-open-file
does if filename actually can't be opened but you haven't specified what
to do in this case with your example so I won't dig deeper.
BTW, this is one of the typical uses for macros: When designing APIs,
you usually want to make sure that certain protocols are followed. For
the typical uses of your library you can provide high-level macros that
hide the details of your protocol. I tend to use a model in this case. For example if I want to always
retrieve a writable stream even if a file isn't openable I just supply a
model function or method.

model.safeOpen( filename)
"""filename -> return a writable file if the file can be opened or a
StringIO buffer object otherwise"""

In this case what happens is explicit as it would be with a macro. Note
that I could have overloaded the built-in "open" function but I don't
really feel comfortable doing that. So far, I haven't been supplied an
urgent need to use macros.
Here is an arbitrary example that I have just picked from the
documentation for a Common Lisp library I have never used before:

(with-transaction
(insert-record :into [emp]
:attributes '(x y z)
:values '(a b c))
(update-records [emp]
:attributes [dept]
:values 50
:where [= [dept] 40])
(delete-records :from [emp]
:where [> [salary] 300000]))

(see
http://www.lispworks.com/reference/l...m#pgfId-889797
)

What do you think the with-transaction macro does? Do you need any more
information than that to understand the code?
Yep. Where's the database? I have to look at the specification you
provided to realize that it is provided by *default-database*. I also
am not aware from this macro whether the transaction is actually
committed or not and whether it rolls back if anything goes wrong. I
can *assume* this at my own peril but I, of course, had to look at the
documentation to be sure. Now this might be lisp-centric but I, being a
lowly scientist, couldn't use this macro without that piece of
knowledge. Of course, now that I know what the macro does I am free to
use it in the future. And yet, if I actually want to *deal* with errors
that occur in the macro besides just rolling back the transaction I
still need to catch the exceptions ( or write another macro :) )

The macro's that you have supplied seem to deal with creating a standard
API for dealing with specific exceptions.

Again, I could create an explicit database model

model.transacti on(commands)
"""(command s) Execute a list of sql commands in a transaction.
The transaction is rolled back if any of the commands fail
and the correspoding failed exception is raised"""

BTW, note that the third line of this example is badly indented. Does
this make reading the code more difficult? This is a red herring. Someone had to format this code to bake it
suitably readable. I could add a red-herring of my own reformatting
this into an undreadable blob but that would be rather childish on my
part and completely irrelevant.
Here is why I think that Python is successful: it's because it favors
dynamic approaches over static approaches (wrt type system, and so on).
I think this is why languages like Ruby, Perl and PHP are also
successful. Languages like Java, C and C++ are very static, and I am
convinced that static approaches create more problems than they solve. We are in agreement here.
It's clear that Python is a very successful language, but I think this
fact is sometimes attributed to the wrong reasons. I don't think its
success is based on prettier syntax or uniformity. Neither give you an
objectively measurable advantage. It all depends on your perspective. I think that I have limited brain
power for remembering certain operations. A case in point, I was
re-writing some documentation yesterday for the embedded metakit
database. Python uses a slice notation for list operations:

list[lo:hi] -> returns a list from index lo to index hi-1

The database had a function call select:

view.select(lo, hi) -> returns a list from index lo to index hi

While it seems minor, this caused me major grief in usage and I wish it
had been uniform with the way python selects ranges. Now I have two
things to remember. I can objectively measure the difference in this
case. The difference is two hours of debugging because of a lack of
uniformity. Now, I brought this on myself by not reading the
documentation closely enough and missing the word "(inclusive )" so I
can't gripe to much. I will just say that the documentation now clearly
shows this lack of uniformity from the standard pythonic way. Of course
we could talk about the "should indexed arrays start with 0 or 1?" but I
respect that there are different desired levels of uniformity. Mine is
probably a little higher than yours :)

Note, that I have had similar experiences in lisp where macros that I
expected to work a certain way, as they were based on common CLHS
macros, didn't. For example, you wouldn't expect (with-open-file ...)
to behave fundamentally different from (with-open-stream ...) and would
probably be annoyed if they did.

In case anyone made it this far, I'm not dissing lisp or trying to
promote python. Both languages are remarkably similar. Macros are one
of the constructs that make lisp lisp and indentation are one of the
things that make python python. Macros could be extremely useful in
python and perhaps to someone who uses them regularly, their ommision is
a huge wart. Having used macros in the past, all I can say is that for
*MY* programming style, I can't say that I miss them that much and have
given a couple of examples of why not.
Pascal

Brian Kelley
Whitehead institute for Biomedical Research

[Jock Cooper]

Pascal Costanza <co******@web.d e> writes:

Andrew Dalke wrote:

(with-open-file (f "...")
...
(read f)
...)

...is much clearer than...

try:
f=open('...')
...
f.read()
...
finally:
f.close()

Why do I need to say "finally" here? Why should I even care about
calling close? What does this have to do with the problem I am trying
to solve? Do you really think it does not distract from the problem
when you first encounter that code and try to see the forest from the
trees?
snip

Can you implement with-open-file as a function? If you could how would
it compare to the macro version? It would look something like:

(defun with-open-file (the-func &rest open-args)
(let ((stream (apply #'open open-args)))
(unwind-protect
(funcall the-func stream)
(close stream))))

(defun my-func (stream)
... operate on stream...
)

(defun do-stuff ()
(with-open-file #'my-func "somefile" :direction :input))

One of the important differences is that MY-FUNC is lexically isolated
from the environment where WITH-OPEN-FILE appears. The macro version
does not suffer this; and it is often convenient for the code block
in the WITH-OPEN-FILE to access that environment.

[Kaz Kylheku]

Ville Vainio <vi************ ********@spamtu t.fi> wrote in message news:<du******* ******@mozart.c c.tut.fi>...

Some people (academics) are paid for being clever. Others (engineers)
are paid for creating systems that work (in the wide meaning of the
word), in a timeframe that the company/client can afford.
[ snip ]
--
Ville Vainio http://www.students.tut.fi/~vainio24

^^^^^^^^

Tee hee! :)

[Kaz Kylheku]

"Andrew Dalke" <ad****@mindspr ing.com> wrote in message news:<V5******* ********@newsre ad4.news.pas.ea rthlink.net>...

Kaz Kylheku:

The input to a compiler should not be a character string, but
structured data.

I think you're arguing naming preferences here. That's fine;
Alex's point remains unchanged. In a dynamic language like
Python, parsing a domain specific language can be done at
run-time, parsed, converted to a Python parse tree, and
compiled to Python byte codes, just like what you want and
in the form you want it.

Ah, but in Lisp, this is commonly done at *compile* time. Moreover,
two or more domain-specific languages can be mixed together, nested in
the same lexical scope, even if they were developed in complete
isolation by different programmers. Everything is translated and
compiled together. Some expression in macro language B can appear in
an utterance of macro language A. Lexical references across these
nestings are transparent:

(language-a
... establish some local variable foo ...
(language-b
... reference to local variable foo set up in language-a!
))

The Lisp parse tree is actually just normal Lisp code. There is no
special target language for the compiler; it understands normal Lisp,
and that Lisp is very conveniently manipulated by the large library of
list processing gadgets. No special representation or API is required.

Do people write any significant amount of code in the Python parse
tree syntax? Can you use that syntax in a Python source file and have
it processed together with normal code?

What is Python's equivalent to the backquote syntax, if I want to put
some variant pieces into a parse tree template?

[Matthew Danish]

On Wed, Oct 22, 2003 at 12:30:01PM -0400, Brian Kelley wrote:

Your two examples do completely different things and the second is
written rather poorly as f might might not exist in the finally block.
It certainly won't if the file doesn't exist.

A better comparison would (*might*, I haven't used lisp in a while) be:

(with-open-file (f filename :direction :output :if-exists :supersede)
(format f "Here are a couple~%of test data lines~%")) => NIL

if os.path.exists( filename):
f = open(filename, 'w')
print >> f, "Here are a couple of test data lines"

How are these in any way equivalent? Pascal posted his example with
try...finally and f.close () for a specific reason. In your Python
example, the file is not closed until presumably the GC collects the
descriptor and runs some finalizer (if that is even the case). This is
much different from the Lisp example which guarantees the closing of the
file when the body of the WITH-OPEN-FILE is exited. In addition:

``When control leaves the body, either normally or abnormally (such as
by use of throw), the file is automatically closed. If a new output file
is being written, and control leaves abnormally, the file is aborted and
the file system is left, so far as possible, as if the file had never
been opened.''

http://www.lispworks.com/reference/H...with-open-file

So in fact, you pointed out a bug in Pascal's Python example, and one
that is easy to make. All this error-prone code is abstracted away by
WITH-OPEN-FILE in Lisp.

--
; Matthew Danish <md*****@andrew .cmu.edu>
; OpenPGP public key: C24B6010 on keyring.debian. org
; Signed or encrypted mail welcome.
; "There is no dark side of the moon really; matter of fact, it's all dark."

[Matthew Danish]

On Wed, Oct 22, 2003 at 09:52:42AM -0700, Jock Cooper wrote:

One of the important differences is that MY-FUNC is lexically isolated
from the environment where WITH-OPEN-FILE appears. The macro version
does not suffer this; and it is often convenient for the code block
in the WITH-OPEN-FILE to access that environment.

(call-with-open-file
#'(lambda (stream)
...)
"somefile"
:direction :input)

WITH-OPEN-FILE happens to be one of those macros which doesn't require
compile-time computation, but rather provides a convenient interface to
the same functionality as above.

--
; Matthew Danish <md*****@andrew .cmu.edu>
; OpenPGP public key: C24B6010 on keyring.debian. org
; Signed or encrypted mail welcome.
; "There is no dark side of the moon really; matter of fact, it's all dark."

[Andrew Dalke]

Kaz Kylheku:

Ah, but in Lisp, this is commonly done at *compile* time.
Compile vs. runtime is an implementation issue. Doesn't
change expressive power, only performance. Type inferencing
suggests that there are other ways to get speed-ups from
dynamic languages.
Moreover,
two or more domain-specific languages can be mixed together, nested in
the same lexical scope, even if they were developed in complete
isolation by different programmers.
We have decidedly different definitions of what a "domain-specific
language" means. To you it means the semantics expressed as
an s-exp. To me it means the syntax is also domain specific. Eg,
Python is a domain specific language where the domain is
"languages where people complain about scope defined by
whitespace." ;)

Yes, one can support Python in Lisp as a reader macro -- but
it isn't done because Lispers would just write the Python out
as an S-exp. But then it wouldn't be Python, because the domain
language *includes*domai n*syntax*.

In other words, writing the domain language as an S-exp
is a short cut to make it easier on the programmer, and not
on the domain specialist. Unless the domain is programming.
And you know, very few of the examples of writing a domain
specific language in Lisp have been for tasks other than
programming.
Do people write any significant amount of code in the
Python parse tree syntax?
No. First, it isn't handled as a syntax, it's handled as
as operations on a tree data structure. Second -- and
this point has been made several times -- that style of
programming isn't often needed, so there of course isn't
a "significan t amount."
Can you use that syntax in a Python source file and have
it processed together with normal code?
Did you look at my example doing just that? I built
an AST for Python and converted it into a normal function.
What is Python's equivalent to the backquote syntax, if I
want to put some variant pieces into a parse tree template?

There isn't. But then there isn't need. The question isn't
"how do I do this construct that I expect in Lisp?" it's "how
do I solve this problem?" There are other ways to solve
that problem than creating a "parse tree template" and to
date there have been few cases where the alternatives were
significantly worse -- even in the case of translating a domain
language into local syntax, which is a Lisp specialty, it's only
about twice as long for Python as for Lisp and definitely
not "impossible " like you claimed. Python is definitely worse
for doing research in new programming styles, but then
again that's a small part of what most programmers need,
and an even smaller part of what most non-professional
programmers need. (Eg, my science work, from a computer
science viewpoint, is dead boring.)

There's very little evidence that Lisp is significantly better
than Python (or vice versa) for solving most problems.
It's close enough that it's a judgement call to decide which
is more appropriate.

But that's a Pythonic answer, which acknowledges that
various languages are better for a given domain and that it's
relatively easy to use C/Java bindings, shared memory, sockets,
etc to make them work together, and not a Lispish answer,
which insists that Lisps are the best and only languages
people should consider. (Broad brush, I know.)

Andrew
da***@dalkescie ntific.com