473,574 Members | 2,357 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

macro FAQ

Here's a proposed Q&A for the FAQ based on a couple recent
threads. Appropriate comments appreciated

X.Y: Why doesn't Python have macros like in Lisp or Scheme?

Before answering that, a clarification on what 'macro' means.
A Lisp macro is a way of modifying code when that code is first
defined. It can rearrange the structure of the code, and add and
remove parts of it. Unlike C's #define macro language, Lisp
macros understand the structure of the expression and the
context (or "closure") in which it is found.

Here's a simple example of what a macro might do for Python.
A complaint about Python is that constructors for simple data
types require typing the parameter names three times, as in

class Country:
def __init__(self, name, capitol):
self.name = name
self.capitol = capitol

Suppose you could create a new way to create functions, which
looked at the argument list and automatically set the corresponding
member. This would let you replace the above with something
more like

class Country:
def __init__(self, name, capitol) [arg_assign]:
pass

where the 'arg_assign' would use the function definition
and code to make a new function used in place of the
existing one and with the 'self.name = name' and
'self.capitol = capitol' code automatically added to
the front of the code.

This is a very powerful technique and especially nice for
high-performance code where the macro can optimize
code before it is actually evaluated. Macros work well in
Lisp or Scheme, where the code is the data, all written
pretty much as a parse tree.

The first problem with implementing a macro system in Python
is coming up with a readable syntax; that is, how it's "spelled."
The example shown above, using the [] after the function
definition, was taken from a proposed syntax for adding
function modifiers like 'staticmethod' and 'classmethod' to
Python.

The second problem is that Python's code blocks do not
store the original parse tree, so there's nothing to manipulate.
If you use 'dir()' or the inspect module you'll see some fields
like the list of argument names and the default value to use
for the fields. Nowhere will you find the original parse tree.
(The C implementation of Python does give access to the
Python byte code, but this is not part of the language
specification and Java implementation doesn't support it.)

These problems can be resolved, and Dylan shows that
an infix-based/Algol-like language can support macros
similar to those used in Lisp.

The deeper question is, should Python include macros?

People with a Lisp background might have a problem
understanding this viewpoint. Macros add flexibility, and
more flexibilty lets programmers be more productive; or
so the argument goes.

Those against including macros in Python point out that
Python has a different focus. It has a broad target audience.
Beginners can pick it up pretty easily yet advanced
developers also enjoy working in the language. New features,
like new builtin functions, additional core types, expanding
the standard libraries, list comprensions, metaclasses and
deriving from object, and macros, all make it harder to
learn how to use Python well and shifts the balance more
towards advanced programmers.

More importantly, the detractors -- including those
with plenty of experience using macros in Lisp -- argue that
macros cause dialects to form. Macros can modify other code to
make it fit the problem better, while functions only use other
code but make no modifications. This makes them very
powerful but means that understanding a section of code
requires also knowing about any macros which might use the
code. In an extreme case which wouldn't be used in real
projects, every * could be replaced with a +.

(As an aside, some proponents argue that macros and
functions are essentially the same. Alex Martelli made
an interesting observation about one difference: it's often
worthwhile to turn a set of code into a function even if it
is only called from one place, because it helps simplify
the code into smaller chucks which are easier to understand.
Macros, on the other hand, should almost never be used
unless they are used many times.)

With only one or a small group of people working together
on a project there is little problem. Macros help in developing
idioms specific to the problem and group. When groups
share code, they also share idioms, and anyone who has had
to deal with UK vs. US English knows the effect idioms have
in mututal understanding.

Those against macros say their benefits do not outweigh
the likelihood that the Python community will be broken up
into distinct subsets, where people from one group find it
hard to use code from another.

The proponents for macros say this has not been a problem
in Lisp. Macros are misused, but so are functions and classes.
The same community feedback which encourages people to
use proper coding styles also encourages people to use macros
correctly. Those against reassert that not only can it be a
problem but it's happened to them personally.

At this point the discussion usually breaks down, with the
Lispers pulling out the old canard about Python (and every
other language) being a half-complete, poor reimplementatio n
of Lisp, while Pythonistas trot out the expression "Lisp is like
a ball of mud. You can add any amount of mud to it and it
still looks like a ball of mud."

Those interested in actual language improvements then ask for
concrete examples of how macros might be used for Python.
So far, the examples given are ones which could equally well
be done in Python using metaclasses, properties, and other
existing mechanisms. For example, the arg_assign example
given above could, I think, be done with a modified__new__ .

The problem at this point seems to be that the two languages
have different enough object models and accepted best
practices that it's hard for an intuition based on Lisp to
know immediately how to apply that intuition to Python.

If you want to encourage Python to have macros, you must
therefore propose a syntax for it and give solid examples of
what the result would look like when applied to real Python
code. Simply stating that macros are needed is not good
enough nor are examples showing how macros are useful
for Lisp code. (Besides, believe it or not but some of us
don't know enough Lisp to follow your examples ;)

In the meanwhile, if you want to experiment with manipulating
code trees, you can use decompyle to turn byte code back
into Python source and the compiler module to turn source into
an AST and thence into byte code.

Andrew Dalke
da***@dalkescie ntific.com
Jul 18 '05 #1
25 3223
"Andrew Dalke" <ad****@mindspr ing.com> wrote in message news:<Ds******* *********@newsr ead4.news.pas.e arthlink.net>.. .
Here's a proposed Q&A for the FAQ based on a couple recent
threads. Appropriate comments appreciated

X.Y: Why doesn't Python have macros like in Lisp or Scheme?


This is all very interesting and I do agree that there shouuld be a
FAQ about macros, nevertheless I don't see your message as suitable
for a FAQ. It is ways too verbose. I mean, if one asks for a macro,
s/he already knows what a macro is. The FAQ should be much
shorter. What you wrote is more useful and interesting for
Pythonistas than from Lispers.
I think your document should be made available on the Web somewhere,
but not in the FAQ. Of course the FAQ could and should give a pointer
to it. Maybe lispers would be helped from a page describing how
some of the things made in macros in Lisp can be made in Python.
This page should be referred in the FAQ too. It is clear, however,
that many things involving macros cannot just be done in Python
(I mean, unless implementing a custom macro interpreter in Python,
which actually I wrote few months ago as a proof of concept (google
on c.l.py for simionato macros, first hit) and which I DO NOT use ;).

Michele Simionato, Ph. D.
Mi************* *@libero.it
http://www.phyast.pitt.edu/~micheles
--- Currently looking for a job ---
Jul 18 '05 #2
"Andrew Dalke" <ad****@mindspr ing.com> wrote in message news:<Ds******* *********@newsr ead4.news.pas.e arthlink.net>.. .
Here's a proposed Q&A for the FAQ based on a couple recent
threads. Appropriate comments appreciated

X.Y: Why doesn't Python have macros like in Lisp or Scheme?


This is all very interesting and I do agree that there shouuld be a
FAQ about macros, nevertheless I don't see your message as suitable
for a FAQ. It is ways too verbose. I mean, if one asks for a macro,
s/he already knows what a macro is. The FAQ should be much
shorter. What you wrote is more useful and interesting for
Pythonistas than from Lispers.
I think your document should be made available on the Web somewhere,
but not in the FAQ. Of course the FAQ could and should give a pointer
to it. Maybe lispers would be helped from a page describing how
some of the things made in macros in Lisp can be made in Python.
This page should be referred in the FAQ too. It is clear, however,
that many things involving macros cannot just be done in Python
(I mean, unless implementing a custom macro interpreter in Python,
which actually I wrote few months ago as a proof of concept (google
on c.l.py for simionato macros, first hit) and which I DO NOT use ;).

Michele Simionato, Ph. D.
Mi************* *@libero.it
http://www.phyast.pitt.edu/~micheles
--- Currently looking for a job ---
Jul 18 '05 #3
mi**@pitt.edu (Michele Simionato) writes:
"Andrew Dalke" <ad****@mindspr ing.com> wrote in message news:<Ds******* *********@newsr ead4.news.pas.e arthlink.net>.. .
Here's a proposed Q&A for the FAQ based on a couple recent
threads. Appropriate comments appreciated

X.Y: Why doesn't Python have macros like in Lisp or Scheme?
[...] I think your document should be made available on the Web somewhere,
but not in the FAQ. Of course the FAQ could and should give a pointer
to it.

[...]

+1, FWIW
John
Jul 18 '05 #4
mi**@pitt.edu (Michele Simionato) writes:
"Andrew Dalke" <ad****@mindspr ing.com> wrote in message news:<Ds******* *********@newsr ead4.news.pas.e arthlink.net>.. .
Here's a proposed Q&A for the FAQ based on a couple recent
threads. Appropriate comments appreciated

X.Y: Why doesn't Python have macros like in Lisp or Scheme?
[...] I think your document should be made available on the Web somewhere,
but not in the FAQ. Of course the FAQ could and should give a pointer
to it.

[...]

+1, FWIW
John
Jul 18 '05 #5
"Andrew Dalke" <ad****@mindspr ing.com> writes:
Here's a proposed Q&A for the FAQ based on a couple recent
threads. Appropriate comments appreciated [...] Those against including macros in Python point out that
Python has a different focus. It has a broad target audience.
Beginners can pick it up pretty easily yet advanced
developers also enjoy working in the language. New features,
like new builtin functions, additional core types, expanding
the standard libraries, list comprensions, metaclasses and
deriving from object, and macros, all make it harder to
learn how to use Python well and shifts the balance more
towards advanced programmers.

More importantly, the detractors -- including those
with plenty of experience using macros in Lisp -- argue that
macros cause dialects to form. Macros can modify other code to
make it fit the problem better, while functions only use other
code but make no modifications. This makes them very
powerful but means that understanding a section of code
requires also knowing about any macros which might use the
code. In an extreme case which wouldn't be used in real
projects, every * could be replaced with a +.

[...]

About the first paragraph here: I don't recall anyone arguing that,
but maybe I forget. Anyway, my problem with it is that it reinforces
the "Python is a scripting language for amateurs, not suitable for
large scale / systems programming" attitude. I think Python
demonstrates that there isn't much tension in reality between the
'advanced programmers' and the beginners -- both seem to want a simple
language. And as you say, the argument about dialects is more
important anyway.

Other than that, excellent stuff.
John
Jul 18 '05 #6
"Andrew Dalke" <ad****@mindspr ing.com> writes:
Here's a proposed Q&A for the FAQ based on a couple recent
threads. Appropriate comments appreciated [...] Those against including macros in Python point out that
Python has a different focus. It has a broad target audience.
Beginners can pick it up pretty easily yet advanced
developers also enjoy working in the language. New features,
like new builtin functions, additional core types, expanding
the standard libraries, list comprensions, metaclasses and
deriving from object, and macros, all make it harder to
learn how to use Python well and shifts the balance more
towards advanced programmers.

More importantly, the detractors -- including those
with plenty of experience using macros in Lisp -- argue that
macros cause dialects to form. Macros can modify other code to
make it fit the problem better, while functions only use other
code but make no modifications. This makes them very
powerful but means that understanding a section of code
requires also knowing about any macros which might use the
code. In an extreme case which wouldn't be used in real
projects, every * could be replaced with a +.

[...]

About the first paragraph here: I don't recall anyone arguing that,
but maybe I forget. Anyway, my problem with it is that it reinforces
the "Python is a scripting language for amateurs, not suitable for
large scale / systems programming" attitude. I think Python
demonstrates that there isn't much tension in reality between the
'advanced programmers' and the beginners -- both seem to want a simple
language. And as you say, the argument about dialects is more
important anyway.

Other than that, excellent stuff.
John
Jul 18 '05 #7
Jacek Generowicz <ja************ **@cern.ch> wrote in message news:<ty******* ******@lxplus01 4.cern.ch>...
Macros can modify other code to make it fit the problem better,


What do you mean by "other" code? Macros modify the code that is
passed to them as an argument, transforming it before the compiler
gets to see it. I get the impression that you believe that macros can
somehow modify code from other parts of the program. They can't.

Ultimately, macros just save you a lot of typing of source code. (And
thereby save you a lot of bugs.) If you can't type it as soure code,
then a macro can't do it.
while functions only use other code but make no modifications.


This only strengthens the above suspicion, but I'm not really sure
what you mean, here.
This makes them very powerful but means that understanding a section
of code requires also knowing about any macros which might use the
code.


What do you mean by "macros using code" ?

The macros are part of the code, just like functions are. To
understand the code, you must understand what the macros and functions
do.
In an extreme case which wouldn't be used in real projects, every *
could be replaced with a +.


This almost completely convinces me that you are very confused about
what macros can achieve.

Either, you are suggesting that one might write a macro which replaces
every * operator with a + operator, and then pass _the entire source
code_ of a project to it (possible, but patently absurd); or you are
suggesting that it is possible to write a macro "over here" which,
somehow, surreptitiously modifies existing source code "over there".

a) Macros cannot do the latter.

b) You can achieve something very similar in Python, by re-binding
attributes of __builtins__.

Think of the point of macros in another way. The point is _not_ to
take existing source code, and to change it. The point is to take
something that is not (necessarily) valid source code, and turn it
into valid source code, before the compiler gets to see it. Why?
Because this allows you to create a shorthand notation, and use macros
to expand it to real code. Think in terms of encoding design patterns.


I think Jacek has made an extremely good point here, and I want to
follow up on it to determine if everyone has a clear understanding of
what macros are and are not. Following I have outlined a series of
questions an answers to attempt to illuminate the issue. Please
suggest any clarifications or distinctions that need to be made. Also
note that the examples presented are purposefully trivial and could in
many instances be accomplished other ways, however I am presenting
these examples in order to provide a meaningful context to proceed
from.

Question: What are Macros?

Answer: Macros are an advanced templating system that allow dynamic
evaluation of code during run time. Essentially Macros allow you to
pass in some variables that are then expanded into valid Python syntax
which is then executed.

Question: Can't I accomplish this same functionality via string
parsing and eval?

Answer: Eval will accept any valid Python expression and execute it.
This is however regarded by many as a security hole. Macros allow
this type of expressiveness while limiting many of the security risks.
With a Macro you can more specifically control what types of actions
are permissible.

Question: I have some existing modules, if I import a macro from
someone else, could the macro make my existing code perform in
unexpected ways?

Answer: No. A macro doesn't modify existing code, in order to invoke
a macro it must be called just like a function call. So a Macro
couldn't alter the functionality of existing code any more than a
function could. The only way to accomplish such a feat would be to
manually open each file and parse through them and re-evaluate them.
This type of functionality is not what macros are about any more than
a function call would be.

Question: How would Macros work in practice?

Answer: Here is a trivial example macro to start with:

def HelloFunc(messa ge):
print "%s\n" % (message)

defmacro uselessmacro(me ssage):
macroexpansion = "HelloFunc( %s)" % (message)

defmacro callfunc(funcna me, message):
macroexpansion = """
calllog = open("/var/log/functions.log", "a")
calllog.write(" %(funcname)s was called.\n")
calllog.flush()
calllog.close()
%(funcname)s(%( message)s)
""" % locals()

if __name__ == "__main__":
uselessmacro("H ello World")
callfunc("Hello Func", "Hello World")

# Note, Jacek's method of using indentation is an interesting
alternative to calling them as a
# function.

# Second note, this textarea box does not have a compiler, so please
forgive any syntax errors :)

uselessmacro simply passes it's arguments through to HelloFunc
undisturbed
callfunc logs all calls to a function before evaluating the function

You could clearly accomplish this functionality using other means,
however I have used such a trivial example in order to illustrate
several points:

1) Macros once they are defined would most likely be invoked the same
way any other function call would be or represented as a block of
code.

2) Macros taken in isolation would not allow any strange changes to
the language to be made (eg changing all *'s to +'s, or changing the
behavior of existing classes - metaclasses, builtins and magic methods
are required for this)

3) Understanding a macro's behavior is typically easy for someone
familiar with Python's syntax and constructs. In otherwords a Macro
allows you to encapsulate commonly used idioms into a cleaner syntax,
but would not allow you to alter the behavior of the language itself.

My examples are barely touching the surface of what is possible with
macros, but I wanted to start with some simple examples in order to
give everyone some solid examples of what Macros are, and what Macros
are not.
Jul 18 '05 #8

"Jacek Generowicz" <ja************ **@cern.ch> wrote in message
news:ty******** *****@lxplus014 .cern.ch...
This has turned out rather long, but I hope that it demystifies these scary exotic macros for at least one or two people on the list.


Definitely. The sort of concrete example I asked for. Thank you.

Terry
Jul 18 '05 #9
Jacek Generowicz <ja************ **@cern.ch> wrote in message news:<ty******* ******@lxplus01 4.cern.ch>...
==== Example 1: a debugging aid =============== =============== ==

One litte macro I have for debugging purposes (let's call it "show"),
allows me to print out the source code of a number of expressions,
along with their values. At this point, note that Python's horribly
irregular syntax <0.5 wink> already starts causing problems: should
the macro look like a block of code, or should it look like a function
call; in Lisp both look identical.

The "block" version of the show macro invocation:

show:
sin(x)
a
a + b + c

The "function" version of the show macro invocation:

show(sin(x), a, a + b + c)

In both cases, the action of the macro should be to replace itself
with its expansion, _before_ the compiler gets to see the source code.
The expansion should look like this:

print "sin(x) =>", sin(x)
print "a =>", a
print "a + b + c =>", a + b + c

Note the key points:

1) The macro receives some data, and transforms it into valid Python
source code

2) This happens before compile time

3) Nothing "outside" the macro call gets affected by the macro's
action.

Can this be implemented using metaclasses, or any other existing
Python mechanism ?
Yes.

http://groups.google.it/groups?hl=it....lang.python.*

At the end it uses metaclasses+exe c+a simple argument parser based on
tokenize. EXPERIMENTAL code, which I would not trust except as proof
of concept.
=== Example 2: Alexander Schmolck's updating classes ===============


I think this also can be done with metaclasses (somebody already
posted a recipe in some other thread, Diederich?)

Michele Simionato, Ph. D.
Mi************* *@libero.it
http://www.phyast.pitt.edu/~micheles
--- Currently looking for a job ---
Jul 18 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

699
33569
by: mike420 | last post by:
I think everyone who used Python will agree that its syntax is the best thing going for it. It is very readable and easy for everyone to learn. But, Python does not a have very good macro capabilities, unfortunately. I'd like to know if it may be possible to add a powerful macro system to Python, while keeping its amazing syntax, and if it...
2
3473
by: Pete | last post by:
In Access 95/97 I used to be able to create pull down menus (File,Edit ...) from a macro. It seems there used to be some wizard for that. However in Access 2000 it seems you have to build your menus by customizing a toolbar. With this method you have to create a separate macro for every single menu and sub menu. The old method would allow...
7
23534
by: Newbie_sw2003 | last post by:
Where should I use them? I am giving you my understandings. Please correct me if I am wrong: MACRO: e.g.:#define ref-name 99 The code is substituted by the MACRO ref-name. So no overhead. Execution is faster. Where will it be stotred?(Is it in bss/stack/?) FUNCTION:
3
8464
by: Alexander Ulyanov | last post by:
Hi all. Is it possible to pass the whole blocks of code (possibly including " and ,) as macro parameters? I want to do something like: MACRO(FOO, "Foo", "return "Foobar";", "foo(); bar();")
8
10853
by: lasek | last post by:
Hi...in some posts i've read...something about using macro rather then function...but difference ??. Best regards....
12
6503
by: Laurent Deniau | last post by:
I was playing a bit with the preprocessor of gcc (4.1.1). The following macros expand to: #define A(...) __VA_ARGS__ #define B(x,...) __VA_ARGS__ A() -nothing, *no warning* A(x) -x B() -nothing, *warning ISO C99 requires rest arguments to be used*
6
6425
by: Takeadoe | last post by:
Dear NG, Can someone assist me with writing the little code that is needed to run an update table query each time the database is opened? From what I've been able to glean from this group, the Autoexec Macro looks like the way to go. Could someone please assist? Thank you very much! Mike
5
3476
by: Bill | last post by:
This database has no forms. I am viewing an Access table in datasheet view. I'd like to execute a macro to execute a function (using "runcode"). In the function, I'll reading data from the record the cursor was on in the datasheet at the time I executed the macro. So, the questions are: 1) In the macro, how to I get my hands on the record...
0
2035
by: =?Utf-8?B?TGV0emRvXzF0?= | last post by:
I'd like to create a Macro that will sort some raw data, apprx 20k lines, remove some lines based upon a condition in a certain column. Then copy this data into a new spreadsheet and sort the data again and delete the unwanted data and repeat few more times in new sheets. End product will be apprximately 7 or 8 sheets - 1 for Active...
0
7797
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
7715
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
8041
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
1
7806
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
6440
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
0
5296
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3739
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
1
2233
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1333
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.