473,320 Members | 2,177 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Recompile AST?

Is it possible to recompile the AST generated by compiler.parse, back
into code or an executable code object? My aim here is to allow a
script to manipulate Python code as elements within a list. However, it
doesn't look like the module has any native methods designed to do
this.

Chris

Nov 10 '05 #1
6 1433
[ch*******@gmail.com]
Is it possible to recompile the AST generated by compiler.parse, back
into code or an executable code object? My aim here is to allow
a script to manipulate Python code as elements within a list. However,
it doesn't look like the module has any native methods designed to do
this.


Pynits does something like this for pretty-printing Python statements,
from within Vim. I did not touch this project for a while, yet I still
intend to revisit it in a few weeks or months (for an actual need). You
may peek at:

http://fp-etc.progiciels-bpi.ca/pynits.html

--
François Pinard http://pinard.progiciels-bpi.ca
Nov 10 '05 #2
On 10 Nov 2005 13:21:56 -0800, ch*******@gmail.com wrote:
Is it possible to recompile the AST generated by compiler.parse, back
into code or an executable code object? My aim here is to allow a
script to manipulate Python code as elements within a list. However, it
doesn't look like the module has any native methods designed to do
this.

It is possible, but if you want code from a particular python source
string that you are giving to compiler.parse, why not use compile?
Or do you want to modify the AST and then compile? This can be done too,
but modifying the AST can be tricky.

Based on the starting point providided by
http://groups.google.com/group/comp....2351b221063a8c

I've done a few experiments, e.g.,
http://groups.google.com/group/comp....3f473ae5c23751

I've also posted sporadic musings about the possibilities for AST-transforming
custom import functions to do optimizations and macros and special forms etc.,
but no one seemed much interested (or maybe they quietly went off to do something
of their own ;-)

I've seen some references to new AST stuff in the latest version, but I haven't
got the new version. I really must clean up my disk for space ;-/

Regards,
Bengt Richter
Nov 10 '05 #3
Bengt Richter wrote:
I've also posted sporadic musings about the possibilities for AST-transforming
custom import functions to do optimizations and macros and special forms etc.,
but no one seemed much interested (or maybe they quietly went off to do
something of their own ;-)


For an example of AST transformations, take a look at the
libxml2macro.py script in the libxml2dom distribution [1]: it contains
code which transforms ASTs based on simple rules (changing method calls
to function calls) as well as the magic sequence of instructions to
generate .pyc files from ASTs. Originally, the analysis distribution
[2] also compiled modified ASTs to Python bytecode, but such activities
didn't really fit in with the goals of that particular project.

Paul

[1] http://www.python.org/pypi/libxml2dom
[2] http://www.python.org/pypi/analysis

Nov 11 '05 #4
ch*******@gmail.com wrote:
Is it possible to recompile the AST generated by compiler.parse, back
into code or an executable code object?


Into a bytecode object:
from compiler.pycodegen import ModuleCodeGenerator
from compiler.misc import set_filename
from compiler import parse

tree = parse('foo = 42')
set_filename('<foo>', tree)
code = ModuleCodeGenerator(tree).getCode()
exec code
foo

42

Into Python source code: <http://ecritters.biz/asttosource.py>.
Nov 11 '05 #5
Leif K-Brooks wrote:
ch*******@gmail.com wrote:
Is it possible to recompile the AST generated by compiler.parse, back
into code or an executable code object?

Into a bytecode object:
>>> from compiler.pycodegen import ModuleCodeGenerator
>>> from compiler.misc import set_filename
>>> from compiler import parse
>>>
>>> tree = parse('foo = 42')
>>> set_filename('<foo>', tree)
>>> code = ModuleCodeGenerator(tree).getCode()
>>> exec code
>>> foo

42

Into Python source code: <http://ecritters.biz/asttosource.py>.


Thanks, that's exactly what I was looking for. I had almost figured that
out, but I had overlooked the need for set_filename.

Chris
Nov 11 '05 #6
On 10 Nov 2005 16:07:56 -0800, "Paul Boddie" <pa**@boddie.org.uk> wrote:
Bengt Richter wrote:
I've also posted sporadic musings about the possibilities for AST-transforming
custom import functions to do optimizations and macros and special forms etc.,
but no one seemed much interested (or maybe they quietly went off to do
something of their own ;-)


For an example of AST transformations, take a look at the
libxml2macro.py script in the libxml2dom distribution [1]: it contains
code which transforms ASTs based on simple rules (changing method calls
to function calls) as well as the magic sequence of instructions to
generate .pyc files from ASTs. Originally, the analysis distribution
[2] also compiled modified ASTs to Python bytecode, but such activities
didn't really fit in with the goals of that particular project.

Paul

[1] http://www.python.org/pypi/libxml2dom
[2] http://www.python.org/pypi/analysis

Thanks for the links. I downloaded the tgzs but it will be a while
before I get into anything ;-)

Regards,
Bengt Richter
Nov 11 '05 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: ggg | last post by:
In regards to mod_python, I keep getting this warning when I do apachectl configtest: Loaded DSO libexec/mod_python.so uses plain Apache 1.3 API, this module might crash under EAPI! (please...
3
by: P | last post by:
I would like to call my stored proc and invoke the WITH RECOMPILE option. I can't figure out what to set in ADO to make that happen. Anyone know how to make this work? BTW, I _don't_ want to...
4
by: J?r?my | last post by:
Hello: The installation details: W2K SP4, SQL Server 2000 Ent with 1GB RAM. It is a Bi-P3. When I run the Profiler to trace Stored Procedure performance, I get a bunch of SP:CacheMiss for...
7
by: Harold | last post by:
Hi, I have a question in SQL Server 2K, I use SQL Profile to trace, and find Stored Procedure was auto recompiled, like this row in the trace: SP:Recompile 15 1680 76 2004-02-27 16:01:11.610 ...
1
by: JCBrinegar | last post by:
Here is the question: We have a .NET service that makes calls to (using interop) COM+ components compiled under VB6. If we make changes to a VB6 COM Component, without breaking compatability,...
4
by: GG | last post by:
We currently run .net version 1.1 with no service packs. Do we need to recompile all the applications with sp1 and release them on the machines that will get sp1 or just install the sp1 on the...
3
by: Dai Hao | last post by:
Hi all, I have a solution that includes several projects, and one of them depends on all others. When I compile it in Visual Studio, it always recompile all the other projects even though I...
3
by: vgv888 | last post by:
How can I recompile ActiveX dll from VisualBasic6 code with Visual Studio.NET? Can I mix VB6 sources with C#.NET?
1
by: sarojanand | last post by:
Hi, I have created a project (abc) in my wwwroot folder on my system. From visual studio, i am able to run and view this page. Now I have deployed on wwwroot folder and for this I have edited...
2
by: Vols | last post by:
I have a class that many libraries depend on. If I modify the class with 'adding an argument with default value to an existing member function', is it necessary to recompile all library? How about...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.