Python syntax in Lisp and Scheme

[mike420]

I think everyone who used Python will agree that its syntax is
the best thing going for it. It is very readable and easy
for everyone to learn. But, Python does not a have very good
macro capabilities, unfortunately. I'd like to know if it may
be possible to add a powerful macro system to Python, while
keeping its amazing syntax, and if it could be possible to
add Pythonistic syntax to Lisp or Scheme, while keeping all
of the functionality and convenience. If the answer is yes,
would many Python programmers switch to Lisp or Scheme if
they were offered identation-based syntax?

[Alex Martelli]

Erann Gat wrote:
...

But if you focus on examples like this you really miss the point. Imagine
that you wanted to be able to write this in Python:

def vector_fill(v, x):
for i from 0 to len(v)-1:
v[i] = x

You can't do it because Python doesn't support "for i from ... to ...",
only "for i in ...". What's more, you can't as a user change the language
so that it does support "for i from ... to ...". (That's why the xrange
hack was invented.)
Almost right, except that xrange is a hack. Since in Python you cannot
change the language to suit your whims, you USE the language (designed
by a pretty good language designer) -- by coding an iterator that is
suitable to put where the ... are in "for i in ...".
In Lisp you can. If Lisp didn't already have LOOP or DOTIMES as part of
the standard you could add them yourself, and the way you do it is by
writing a macro.
Good summary: if you fancy yourself as a language designer, go for Lisp;
if you prefer to use a language designed by somebody else, without you
_or any of the dozens of people working with you on the same project_
being able to CHANGE the language, go for Python.
That's what macros are mainly good for, adding features to the langauge in
ways that are absolutely impossible in any other language. S-expression
syntax is the feature that enables users to so this quickly and easily.
Doesn't Dylan do a pretty good job of giving essentially the same
semantics (including macros) without S-expression syntax? That was
my impression, but I've never used Dylan in production.
For example, imagine you want to be able to traverse a binary tree and do
an operation on all of its leaves. In Lisp you can write a macro that
lets you write:

(doleaves (leaf tree) ...)

You can't do that in Python (or any other langauge).
Well, in Ruby, or Smalltalk, you would pass your preferred code block
to the call to the doleaves iterator, giving something like:

doleaves(tree) do |leaf|
...
end

while in Python, where iterators are "the other way around" (they
get relevant items out rather than taking a code block in), it would be:

for leaf in doleaves(tree):
...

In either case, it may not be "that" (you are not ALTERING the syntax
of the language, just USING it for the same purpose), but it's sure close.
(In Dylan, I do believe you could ``do that'' -- except the surface
syntax would not be Lisp-ish, of course).

Here's another example of what you can do with macros in Lisp:

(with-collector collect
(do-file-lines (l some-file-name)
(if (some-property l) (collect l))))

This returns a list of all the lines in a file that have some property.
DO-FILE-LINES and WITH-COLLECTOR are macros, and they can't be implemented
any other way because they take variable names and code as arguments.

If you consider than giving e.g. the variable name as an argument to
do-file-lines is the crucial issue here, then it's probably quite true
that this fundamental (?) feature "cannot be implemented any other way";
in Ruby, e.g., the variable name would not be an argument to dofilelines,
it would be a parameter at the start of the block receiving & using it:

dofilelines(som efilename) do |l|
collect l if someproperty? l
end

However, it appears to me that the focus on where variable names are
to be determined may be somewhat misplaced. The key distinction does
seem to be: if you're happy using a language as it was designed (e.g.,
in this example, respecting the language designer's concept that the
names for the control variables of a block must appear within | vertical
bars | at the start of the block -- or, in Python, the reversed concept
that they must appear between the 'for' and 'in' in the "for ... in ...:
statement), macros are not relevant; if you do want to design and use
your own language (including, for example, placing variable names in
new and interesting places) then macros can let you do that, while
other constructs would be insufficiently powerful.

If you dream of there being "preferably only one obvious way to do it",
as Pythonistas do (try "import this" at an interactive Python prompt),
macros are therefore a minus; if you revel in the possibilities of there
being many ways to do it, even ones the language designer had never even
considered (or considered and rejected in disgust:-), macros then become
a huge plus.

Therefore, I entirely agree that people who pine for macros should
use them in a language that accomodates them quite well, is designed
for them, cherishes and nurtures and exhalts them, like some language
of the Lisp family (be it Common, ISO, Scheme, ...), or perhaps Dylan
(which often _feels_ as if "of the Lisp family" even though it does
not use S-expressions), rather than trying to shoehorn them willy-nilly
into a language to whose overall philosophy they are SO utterly foreign,
like Python (Ruby, and even more Perl, may be a different matter;
google for "Lingua Latina Perligata" to see what Perl is already able
to do in terms of within-the-language language design and syntax
alteration, even without anything officially deemed to be 'macros'...
it IS, after all, a language CENTERED on "more than one way to do it").
Alex

[Alan Crowe]

MetalOne wrote

If a set of macros could be written to improve LISP
syntax, then I think that might be an amazing thing. An
interesting question to me is why hasn't this already been
done.

I think the issue is the grandeur of the Lisp vision. More
ambitious projects require larger code bases. Ambition is
hard to quantify. Nevertheless one must face the issue of
scaling. Does code size go as the cube of ambition, or is it
the fourth power of ambition? Or something else entirely.

Lisp aspires to change the exponent, not the constant
factor. The constant factor is very important. That is why
CL has FILL :-) but shrinking the constant factor has been
done (and with C++ undone).

Macros can be used to abbreviate code. One can spot that one
is typing similar code over and over again. One says
"whoops, I'm typing macro expansions". Do you use macros to
tune the syntax, so that you type N/2 characters instead of
N characters, or do you capture the whole concept in macro
and eliminate the repetition altogether?

The point is that there is nowhere very interesting to go
with syntax tuning. It is the bolder goal of changing the
exponent, and thus seriously enlarging the realm of the
possible, that excites.

Alan Crowe

[Frode Vatvedt Fjeld]

Alex Martelli <al***@aleax.it > writes:

Good summary: if you fancy yourself as a language designer, go for
Lisp; if you prefer to use a language designed by somebody else,
without you _or any of the dozens of people working with you on the
same project_ being able to CHANGE the language, go for Python.

I believe it is very unfortunate to view lisp macros as something that
is used to "change the language". Macros allow syntactic abstraction
the same way functions allow functional abstraction, and is almost as
important a part of the programmer's toolchest. While macros _can_ be
used to change the language in the sense of writing your own
general-purpose iteration construct or conditional operator, I believe
this is an abuse of macros, precisely because of the implications this
has for the readability of the code and for the language's user
community.

--
Frode Vatvedt Fjeld

[David Rush]

On 03 Oct 2003 14:44:36 +0300, Toni Nikkanen <to**@tuug.fi > wrote:

It's be interesting to know where people got the idea of learning
Scheme/LISP from (apart from compulsory university courses)?

Emacs. I've noticed over the years that people don't really get Emacs
religion until they've started hacking elisp. I know that the frustration
of having almost-but-not-quite the behavior I wanted on top of having all
that source code was a powerful incentive for me to learn Lisp. Of course
my apreciation of Emacs only increased as I went...

The thing that sealed it for me was re-programming SCWM's behavior so that
I could use X w/no mouse &cet. That got me hooked on Scheme (I had been
hacking SML at roughly the same time while looking for the foundations of
OOP), which was really just about perfect semantically.

david rush
--
(\x.(x x) \x.(x x)) -> (s i i (s i i))
-- aki helin (on comp.lang.schem e)

[Grzegorz Chrupala]

bo**@oz.net (Bengt Richter) wrote in message news:<bl******* ***@216.39.172. 122>...

Do you like this better?

>>> def foo(n):

... box = [n]
... def foo(i): box[0]+=i; return box[0]
... return foo
...

It's still a hack that shows an area where Python has unnecessary
limitations, isn't it?
As Paul Graham says (<URL:http://www.paulgraham. com/icad.html>):
Python users might legitimately ask why they can't just write

def foo(n):
return lambda i: return n += i

or even

def foo(n):
lambda i: n += i

Cheers,
-- Grzegorz

[prunesquallor]

Frode Vatvedt Fjeld <fr****@cs.uit. no> writes:

Alex Martelli <al***@aleax.it > writes:

Good summary: if you fancy yourself as a language designer, go for
Lisp; if you prefer to use a language designed by somebody else,
without you _or any of the dozens of people working with you on the
same project_ being able to CHANGE the language, go for Python.

I believe it is very unfortunate to view lisp macros as something that
is used to "change the language". Macros allow syntactic abstraction
the same way functions allow functional abstraction, and is almost as
important a part of the programmer's toolchest. While macros _can_ be
used to change the language in the sense of writing your own
general-purpose iteration construct or conditional operator, I believe
this is an abuse of macros, precisely because of the implications this
has for the readability of the code and for the language's user
community.

But syntactic abstractions *are* a change to the language, it just
sounds fancier.

I agree that injudicious use of macros can destroy the readability of
code, but judicious use can greatly increase the readability. So
while it is probably a bad idea to write COND1 that assumes
alternating test and consequence forms, it is also a bad idea to
replicate boilerplate code because you are eschewing macros.

[Alex Martelli]

Grzegorz Chrupala wrote:
...

>>> def foo(n): ... box = [n]
... def foo(i): box[0]+=i; return box[0]
... return foo
...

It's still a hack that shows an area where Python has unnecessary
limitations, isn't it?

Debatable, and debated. See the "Rebinding names in enclosing
scopes" section of http://www.python.org/peps/pep-0227.html .

Essentially, Guido prefers classes (and instances thereof) to
closures as a way to bundle state and behavior; thus he most
emphatically does not want to add _any_ complication at all,
when the only benefit would be to have "more than one obvious
way to do it".

Guido's generally adamant stance for simplicity has been the
key determinant in the evolution of Python. Guido is also on
record as promising that the major focus in the next release
of Python where he can introduce backwards incompatibiliti es
(i.e. the next major-number-incrementing release, 3.0, perhaps,
say, 3 years from now) will be the _elimination_ of many of
the "more than one way to do it"s that have accumulated along
the years mostly for reasons of keeping backwards compatibility
(e.g., lambda, map, reduce, and filter, which Guido mildly
regrets ever having accepted into the language).

As Paul Graham says (<URL:http://www.paulgraham. com/icad.html>):

Python users might legitimately ask why they can't just write

def foo(n):
return lambda i: return n += i
The rule Python currently use to determine whether a variable
is local is maximally simple: if the name gets bound (assigned
to) in local scope, it's a local variable. Making this rule
*any* more complicated (e.g. to allow assignments to names in
enclosing scopes) would just allow "more than one way to do
it" (making closures a viable alternative to classes in more
cases) and therefore it just won't happen. Python is about
offering one, and preferably only one, obvious way to do it,
for any value of "it". And another key principle of the Zen
of Python is "simple is better than complex".

Anybody who doesn't value simplicity and uniformity is quite
unlikely to be comfortable with Python -- and this should
amply answer the question about the motivations for reason
number 1 why the above foo is unacceptable in Python (the
lambda's body can't rebind name n in an enclosing scope).

Python draws a firm distinction between expressions and
statements. Again, the deep motivation behind this key
distinction can be found in several points in the Zen of
Python, such as "flat is better than nested" (doing away
with the expression/statement separation allows and indeed
encourages deep nesting) and "sparse is better than dense"
(that 'doing away' would encourage expression/statements
with a very high density of operations being performed).

This firm distinction should easily explain other reasons
why the above foo is unacceptable in Python: n+=i is a
statement (not an expression) and therefore it cannot be
held by a 'return' keyword; 'return' is a statement and
therefore cannot be in the body of a 'lambda' keyword.
or even

def foo(n):
lambda i: n += i

And this touches on yet another point of the Zen of Python:
explicit is better than implicit. Having a function
implicitly return the last expression it computes would
violate this point (and is in fact somewhat error-prone,
in my experience, in the several languages that adopt
this rule).

Somebody who is unhappy with this drive for explicitness,
simplicity, uniformity, and so on, cannot be happy with
Python. If he wants a very similar language from most
points of view, BUT with very different philosophies, he
might well be quite happy with Ruby. Ruby does away with
any expression/statement distinction; it makes the 'return'
optional, as a method returns the last thing it computes;
it revels in "more than one way to do it", clever and cool
hacks, not perhaps to the extent of Perl, but close enough.

In Ruby, the spaces of methods and data are separate (i.e.,
most everything is "an object" -- but, differently from
Python, methods are not objects in Ruby), and I do not
think, therefore, that you can write a method that builds
and returns another method, and bind the latter to a name --
but you can return an object with a .call method, a la:

def outer(a) proc do |b| a+=b end end

x = outer(23)
puts x.call(100) # emits 123
puts x.call(100) # emits 223

[i.e., I can't think of any way you could just use x(100)
at the end of such a snippet in Ruby -- perhaps somebody
more expert of Ruby than I am can confirm or correct...?]
but apart from this it seems closer to what the above
quotes appear to be probing for. In particular, it lets
you be MUCH, MUCH denser, if that is your purpose in life,
easily squeezing that outer function into a (short) line.
Python is NOT about making code very dense, indeed, as
above mentioned, it sees _sparseness_ as a plus; a typical
Pythonista would cringe at the density of that 'outer'
and by contrast REVEL at the "sparsity" and "explicitne ss"
(due to the many names involved:-) of, e.g.:

def make_accumulato r(initial_value ):
accumulator = Bunch(value=ini tial_value)
def accumulate(adde nd):
accumulator.val ue += addend
return accumulator.val ue
return accumulate

accumulate = make_accumulato r(23)
print accumulate(100) # emits 123
print accumulate(100) # emits 223
(using the popular Bunch class commonly defined as:
class Bunch(object):
def __init__(self, **kwds):
self.__dict__.u pdate(kwds)
). There is, of course, a cultural gulf between this
verbose 6-liner [using an auxiliary class strictly for
reasons of better readability...!] and the terse Ruby
1-liner above, and no doubt most practitioners of both
languages would in practice choose intermediate levels,
such as un-densifying the Ruby function into:
def outer(a)
proc do |b|
a+b
end
end

or shortening/densifying the Python one into:

def make_accumulato r(a):
value = [a]
def accumulate(b):
value[0] += b
return value[0]
return accumulate

but I think the "purer" (more extreme) versions are
interesting "tipization s" for the languages, anyway.
Alex

[Alex Martelli]

Frode Vatvedt Fjeld wrote:

Alex Martelli <al***@aleax.it > writes:

Good summary: if you fancy yourself as a language designer, go for
Lisp; if you prefer to use a language designed by somebody else,
without you _or any of the dozens of people working with you on the
same project_ being able to CHANGE the language, go for Python.
I believe it is very unfortunate to view lisp macros as something that
is used to "change the language". Macros allow syntactic abstraction

Maybe "enhance" can sound more positive? An enhancement, of course,
IS a change -- and if one were to perform any change, he'd surely be
convinced it WAS going to be an enhancement. (Whether it really
turned out to be one is another issue).
the same way functions allow functional abstraction, and is almost as
important a part of the programmer's toolchest. While macros _can_ be
used to change the language in the sense of writing your own
general-purpose iteration construct or conditional operator, I believe
this is an abuse of macros, precisely because of the implications this
has for the readability of the code and for the language's user
community.

Sure, but aren't these the examples that are being presented? Isn't
"with-collector" a general purpose iteration construct, etc? Maybe
only _special_ purpose ones should be built with macros (if you are
right that _general_ purpose ones should not be), but the subtleness
of the distinction leaves me wondering about the practice.
Alex

[MetalOne]

Thanks for everybody's responses. I found them quite informative.

[Jens Axel Søgaard]

Alex Martelli wrote:

Essentially, Guido prefers classes (and instances thereof) to
closures as a way to bundle state and behavior; thus he most
emphatically does not want to add _any_ complication at all,
when the only benefit would be to have "more than one obvious
way to do it".

Guido's generally adamant stance for simplicity has been the
key determinant in the evolution of Python.

The following is taken from "All Things Pythonic - News from Python UK"
written by Guido van Rossum April 17,
<2003:http://www.artima.com/weblogs/viewpost.jsp?th read=4550>

During Simon's elaboration of an example (a type-safe printf function)
I realized the problem with functional programming: there was a simple
programming problem where a list had to be transformed into a
different list. The code to do this was a complex two-level lambda
expression if I remember it well, and despite Simon's lively
explanation (he was literally hopping around the stage making
intricate hand gestures to show how it worked) I failed to "get" it. I
finally had to accept that it did the transformation without
understanding how it did it, and this is where I had my epiphany about
loops as a higher level of abstraction than recursion - I'm sure that
the same problem would be easily solved by a simple loop in Python,
and would leave no-one in the dark about what it did.

Hmm.

--
Jens Axel Søgaard