BIG successes of Lisp (was ...)

[mike420]

In the context of LATEX, some Pythonista asked what the big
successes of Lisp were. I think there were at least three *big*
successes.

a. orbitz.com web site uses Lisp for algorithms, etc.
b. Yahoo store was originally written in Lisp.
c. Emacs

The issues with these will probably come up, so I might as well
mention them myself (which will also make this a more balanced
post)

a. AFAIK Orbitz frequently has to be shut down for maintenance
(read "full garbage collection" - I'm just guessing: with
generational garbage collection, you still have to do full
garbage collection once in a while, and on a system like that
it can take a while)

b. AFAIK, Yahoo Store was eventually rewritten in a non-Lisp.
Why? I'd tell you, but then I'd have to kill you :)

c. Emacs has a reputation for being slow and bloated. But then
it's not written in Common Lisp.

Are ViaWeb and Orbitz bigger successes than LATEX? Do they
have more users? It depends. Does viewing a PDF file made
with LATEX make you a user of LATEX? Does visiting Yahoo
store make you a user of ViaWeb?

For the sake of being balanced: there were also some *big*
failures, such as Lisp Machines. They failed because
they could not compete with UNIX (SUN, SGI) in a time when
performance, multi-userism and uptime were of prime importance.
(Older LispM's just leaked memory until they were shut down,
newer versions overcame that problem but others remained)

Another big failure that is often _attributed_ to Lisp is AI,
of course. But I don't think one should blame a language
for AI not happening. Marvin Mins ky, for example,
blames Robotics and Neural Networks for that.

[Marcin 'Qrczak' Kowalczyk]

On Fri, 24 Oct 2003 20:00:23 +0000, Rainer Deyke wrote:

I DON'T want to manually close files. I DON'T want to deal with the
limitations of with-open-file. And, here's the important bit, I DON'T WANT
TO COMBINE OR CHOOSE BETWEEN THESE TWO METHODS, BOTH OF WHICH ARE FLAWED.

What I want to open a file and have it close automatically when I am done
with it. I can do that in C++. Why can't I do it in Python?

In C++ you must choose between a local variable for the stream (which is
equivalent to with-open-file) or dynamically allocated object (which is
equivalent to open & close, where close is spelled delete).

You can't say that you want the C++ way and you don't want explicit open &
close and with-open-file, because they are equivalent! They differ only in
syntax details, but the style of usage and limitations are the same.

The only Python's problem is that there is no nice way to pass an
anonymous function to with-open-file, and try:finally: doesn't look so
nice either. I'm not defending Python's lack of large anonymous functions
but the interface of open, close & with-open-file in general, irrespective
of the programming language.

In particular you shouldn't wish for GC to find unused files because it's
impossible without sacrificing other GC properties. No languages does that
except misused CPython (you shouldn't rely on that).

--
__("< Marcin Kowalczyk
\__/ qr****@knm.org. pl
^^ http://qrnik.knm.org.pl/~qrczak/

[Matthew Danish]

On Fri, Oct 24, 2003 at 08:00:23PM +0000, Rainer Deyke wrote:

I DON'T want to manually close files. I DON'T want to deal with the
limitations of with-open-file. And, here's the important bit, I DON'T WANT
TO COMBINE OR CHOOSE BETWEEN THESE TWO METHODS, BOTH OF WHICH ARE FLAWED.
I already discussed this in another post.
What I want to open a file and have it close automatically when I am done
with it. I can do that in C++. Why can't I do it in Python?

One major difference between C++ and Lisp/Python is that C++ lacks
closures. This means that local variables do not have to be considered
for allocation on the heap. So saying

{
ofstream f;
f.open ("output");
...
// I presume ofstream dtor closes file? I'm a bit rusty
}

is okay in C++ to obtain a similar effect as WITH-OPEN-FILE, though
I am not sure how well it does when it comes to abnormal conditions.

In Common Lisp, and presumably now Python, lexical scope combined with
higher-order functions implies that variables may be captured in a
closure and have indefinite extent. In CL, also, the language does not
define memory management; leaving open the possibility for a better
solution than GC. CL's answer to the handling of dynamic-extent
resources is WITH-OPEN-FILE, and also the other operators from which
WITH-OPEN-FILE is defined (such as UNWIND-PROTECT).

One final question: do you expect C++ to clean up dynamically allocated
ofstream objects, automatically?

--
; Matthew Danish <md*****@andrew .cmu.edu>
; OpenPGP public key: C24B6010 on keyring.debian. org
; Signed or encrypted mail welcome.
; "There is no dark side of the moon really; matter of fact, it's all dark."

[Joe Marshall]

"Rainer Deyke" <ra*****@eldwoo d.com> writes:

Peter Seibel wrote:

But--and this is the bit I think you may have missed--if that's *not*
what you want you don't use WITH-OPEN-FILE. Lisp also provides OPEN
and CLOSE, that act as you'd expect and allow you to explicitly
control the lifetime of the file stream.

Which leads us back to having to manually close files.

I DON'T want to manually close files. I DON'T want to deal with the
limitations of with-open-file. And, here's the important bit, I DON'T WANT
TO COMBINE OR CHOOSE BETWEEN THESE TWO METHODS, BOTH OF WHICH ARE FLAWED.

Let me see if I get this:

1. You don't want to tell the computer when you are done with the
file.

2. But the computer shouldn't wait until it can prove you are done
to close the file.

3. And you definitely are opposed to the idea of letting the
computer figure it out in general, but giving it hints when
it is important that it be prompt.

Do you want to tell the computer the file name, or should it read your
mind on that as well?

[Rainer Deyke]

Marcin 'Qrczak' Kowalczyk wrote:

In C++ you must choose between a local variable for the stream (which
is equivalent to with-open-file) or dynamically allocated object
(which is equivalent to open & close, where close is spelled delete).
I can also choose to have the stream reference counted (through
boost::shared_p tr or similar), and I can make it a member variable of an
object (tying the lifetime of the stream to the lifetime of the owning
object).

Now, I *could* write my own reference counting stream wrapper in Python,
together with a with_reference_ to_stream HOF. I could, but it would be
expensive, not only in terms of performnace, but also in terms of syntax and
mental overhead.
You can't say that you want the C++ way and you don't want explicit
open & close and with-open-file, because they are equivalent! They
differ only in syntax details, but the style of usage and limitations
are the same.

The syntax is actually very significant here. Python's lack of anonymous
code blocks hurts, but even without this, the C++ syntax is more lightweight
and therefore more managable. Compare the levels of indentation:

void f()
{
std::fstream a("a"), b("b");
do_something(a, b);
std::fstream c;
do_something_el se(a, b, c);
}

def f():
with_open_file( a = open("a")):
with_open_file( b = open("b")):
do_something(a, b)
with_open_file( c = open("c")):
do_something_el se(a, b, c)
--
Rainer Deyke - ra*****@eldwood .com - http://eldwood.com

[Rainer Deyke]

Joe Marshall wrote:

2. But the computer shouldn't wait until it can prove you are done
to close the file.

Wrong. The computer can prove that I am done with the file the moment my
last reference to the file is gone. I demand nothing more (or less).
--
Rainer Deyke - ra*****@eldwood .com - http://eldwood.com

[Donn Cave]

In article <rxfmb.19034$Tr 4.39346@attbi_s 03>,
"Rainer Deyke" <ra*****@eldwoo d.com> wrote:

Peter Seibel wrote:

But--and this is the bit I think you may have missed--if that's *not*
what you want you don't use WITH-OPEN-FILE. Lisp also provides OPEN
and CLOSE, that act as you'd expect and allow you to explicitly
control the lifetime of the file stream.

Which leads us back to having to manually close files.

I DON'T want to manually close files. I DON'T want to deal with the
limitations of with-open-file. And, here's the important bit, I DON'T WANT
TO COMBINE OR CHOOSE BETWEEN THESE TWO METHODS, BOTH OF WHICH ARE FLAWED.

What I want to open a file and have it close automatically when I am done
with it. I can do that in C++. Why can't I do it in Python?

None of the above makes a whole lot of sense to me, and judging
by the use of upper case I'm inclined to believe that discussing
this with comp.lang.lisp participants has caused you to become
disturbed. I am accordingly posting this only to comp.lang.pytho n.

You can have files close automatically in Python, but automatically
isn't by itself a rather vacuous term, and `when I am done with it'
doesn't help a bit. In C Python, when a file object is no longer
referenced by any part of the program, it closes. If it's local
to a function, including bound only to a function parameter or some
such thing, that will occur when control returns from the function.

Unless the file becomes involved in a circular reference, in which
case the close will be deferred until the references are discovered
and broken by the garbage collector. Unless the garbage collector
is unable to do so because of some property of the members, such as
a __del__ method.

No doubt there is a great deal of C++ arcana that I have missed out
on, but the basic finalization issues are the same as far as I know.
C++ programmers aren't subject to the above exceptions only because
they don't get any of the functionality to which these are exceptions!
Function local objects are always deleted on return from the function
regardless - and any other part of the program that retains a pointer
to such an object will become unsound. Objects allocated on the heap
have to be deleted explicitly, after which other parts of the program
that reference them will become unsound. If you do a fraction of the
explicit management that C++ requires, you can get reliable finalization.

I am not familiar with with-open-file, but I imagine if you decide to
write your software in Lisp, you will probably want to give it a try.

There is also nothing wrong with closing a file explicitly. There are
reasons for it that have nothing to do with the language, and then there
is a school of thought (actually the party line for Python) that says
you should explicitly close files in any case because the memory
management rules are different for Java Python and could in theory
change in a later release of C Python. Apparently Java's finalization
is not immediate.

Donn Cave, do**@u.washingt on.edu

[Rainer Deyke]

Donn Cave wrote:

You can have files close automatically in Python, but automatically
isn't by itself a rather vacuous term, and `when I am done with it'
doesn't help a bit. In C Python, when a file object is no longer
referenced by any part of the program, it closes. If it's local
to a function, including bound only to a function parameter or some
such thing, that will occur when control returns from the function.

I understand both the specification and the implementation of finalization
in Python, as well as the reasoning behind them. My point is, I would
prefer it if Python guaranteed immediate finalization of unreferenced
objects. Maybe I'll write a PEP about if someday.
--
Rainer Deyke - ra*****@eldwood .com - http://eldwood.com

[Kaz Kylheku]

"Andrew Dalke" <ad****@mindspr ing.com> wrote in message news:<64******* **********@news read4.news.pas. earthlink.net>. ..

Kaz Kylheku:

Moreover,
two or more domain-specific languages can be mixed together, nested in
the same lexical scope, even if they were developed in complete
isolation by different programmers.
We have decidedly different definitions of what a "domain-specific
language" means. To you it means the semantics expressed as
an s-exp. To me it means the syntax is also domain specific. Eg,
Python is a domain specific language where the domain is
"languages where people complain about scope defined by
whitespace." ;)

There are two levels of syntax: the read syntax, and a deeper abstract
syntax. People who understand only the first one to be syntax are
scared of syntactic manipulation, because they are used to being
burned by syntax: stupid precedence rules, quirky punctuation,
semicolon diseases, strange whitespace handling, and so on.

A symbolic expression is a printed form which codes for a data
structure. That data structure continues to exhibit syntax, long after
the parentheses, whitespace and other lexical elements are processed
and gone.
Yes, one can support Python in Lisp as a reader macro -- but
it isn't done because Lispers would just write the Python out
as an S-exp.
They would do that in cases when they have to invoke so many escape
hatches to embed Lisp code in the Python that the read syntax no
longer provides any benefit.
But then it wouldn't be Python, because the domain
language *includes*domai n*syntax*.

In other words, writing the domain language as an S-exp
is a short cut to make it easier on the programmer, and not
on the domain specialist.
Writing the language in terms of a data structure (which can be
converted back and forth to a printed symbolic expression) is not a
shortcut; it's proper layering at work. The read syntax can be
independently developed; the symbolic expressions merely give you a
generic one for free! Read syntax is just lexical sugar. It can be
quite important. Heck symbolic expressions have enough of it; it would
be a pain to type (QUOTE X) all the time instead of 'X, or
#.(make-array '(4) :element-type 'bit :initial-contents '(1 1 0 1))
instead of #*1101!

Can you use that syntax in a Python source file and have
it processed together with normal code?

Did you look at my example doing just that? I built
an AST for Python and converted it into a normal function.

I'm looking for a Python example which adds a new kind of statement to
the language, and then later in the source file uses that statement to
write a part of the program, such that it's all processed in the same
pass. The statement can be nested in existing ones, and can have
existing syntax embedded in it. Oh yeah, and it should work on at
least two Python implementations .

What is Python's equivalent to the backquote syntax, if I
want to put some variant pieces into a parse tree template?

There isn't. But then there isn't need. The question isn't

When you start talking about need, that quickly becomes a losing
proposition. Because needs are actually wants in disguise. Do human
beings need electricity, running water or food prepared by heating?
"how do I do this construct that I expect in Lisp?" it's "how
do I solve this problem?"
The answer is: if I can get Lisp, I use that. Otherwise I work around
that. If you are in the middle of civilization, and the question is
``what's for dinner'', the answers are to go to the supermarket or a
restaurant. If you are lost in the woods, then you have to reason
differently.
There are other ways to solve
that problem than creating a "parse tree template" and to
date there have been few cases where the alternatives were
significantly worse -- even in the case of translating a domain
What cases? Where? Can you be more specific? Can ``significantly
worse'' be put into some kind of numbers?
language into local syntax, which is a Lisp specialty, it's only
about twice as long for Python as for Lisp and definitely
not "impossible " like you claimed.

Okay, I'm expecting that example I asked for to be only twice as long
as the Lisp version.

[prunesquallor]

"Rainer Deyke" <ra*****@eldwoo d.com> writes:

Joe Marshall wrote:

2. But the computer shouldn't wait until it can prove you are done
to close the file.

Wrong. The computer can prove that I am done with the file the moment my
last reference to the file is gone. I demand nothing more (or less).

If you don't care about performance, this is trivial.

[Bjorn Pettersen]

tf*@famine.OCF. Berkeley.EDU (Thomas F. Burdick) wrote in
news:xc******** *****@famine.OC F.Berkeley.EDU:

Bjorn Pettersen <bj************ *@comcast.net> writes:

Raymond Wiker <Ra***********@ fast.no> wrote:

> "Rainer Deyke" <ra*****@eldwoo d.com> writes:
>
> > Personally I'd prefer guaranteed immediate destructors over
> > with-open-file. More flexibility, less syntax, and it matches
> > what the CPython implementation already does.
>
> Right... all along until CPython introduces a more
> elaborate
> gc scheme.
... which is highly unlikely to happen without preservation of
reference counting semantics for files... google if you're _really_
interested :-/

I hope you put assertion in your code so that it won't run under
Jython, because that's the kind of insidious bug that would be
*aweful* to find the hard way.

I'm assuming the "import win32con" etc., will do :-) Seriously, this
would be a minor issue compared with everything else I want to connect
to (e.g. 950KLocs of legacy c++ -- somehow I don't think I'd get the go-
ahead to rewrite that in Java <smile>).
(I really don't understand why you wouldn't just want at least real
closures, so you can just use call_with_open_ file, and not have to
worry about what GC you're using when *opening* *files*)

I'm not the one worrying here <wink>. I know exactly what Python does
and it is no cognitive burden. How would you rewrite these without
obfuscating the real work with book-keeping tasks, or leaving the file
open longer than necessary?

timestamp = datetime.now(). strftime('%Y%m% d%H%M%S\n')
file('log', 'a').write(time stamp)

file('output', 'w').write(file ('template').re ad() % locals())

nwords = sum([len(line.split( )) for line in file('input')])

for line in file('input'):
print line[:79]

-- bjorn