I have some significantly extended syntax for Python that I need to
create a reference implementation for. My new syntax includes new
keywords, statements and objects that are sort of like classes but not
really. The implementation is all possible using standard Python, but
the implementation isn't the point of what I'm doing. Speed and having
an extra step to run a program are not issues that I need to be
concerned with.
I'd like to create a preprocessor if possible, because it would
probably be easier than implementing the changes in the interpreter. I
could just drop in standard Python code that provides the functionality
when I encounter a part of my extended syntax. Modifying the
interpreter, on the other hand, sounds like it would be pretty nasty,
even though I have experience in interpreter hacking already.
So my question is: what's the easiest way to implement a preprocessor
system in Python? I understand I could use the tokenize module, but
that would still require a lot of manual parsing of the Python syntax.
Is it possible to use any of the parser module facilities to accomplish
this without them choking on the unknown syntax? Or, alternatively,
would modifying the interpreter ultimately be easier? 3 1508
Hi! as******@gmail. com wrote: I have some significantly extended syntax for Python that I need to create a reference implementation for. My new syntax includes new keywords, statements and objects that are sort of like classes but not really. The implementation is all possible using standard Python, but the implementation isn't the point of what I'm doing. Speed and having an extra step to run a program are not issues that I need to be concerned with. I'd like to create a preprocessor if possible, because it would probably be easier than implementing the changes in the interpreter. I could just drop in standard Python code that provides the functionality when I encounter a part of my extended syntax. Modifying the interpreter, on the other hand, sounds like it would be pretty nasty, even though I have experience in interpreter hacking already. So my question is: what's the easiest way to implement a preprocessor system in Python? I understand I could use the tokenize module, but that would still require a lot of manual parsing of the Python syntax. Is it possible to use any of the parser module facilities to accomplish this without them choking on the unknown syntax? Or, alternatively, would modifying the interpreter ultimately be easier?
I cannot really say much about how easy it would be to just write a
preprocessor. However, I think what you are trying to do could be done
reasonably easy with the PyPy project: http://codespeak.net/pypy
PyPy is an implementation of a Python interpreter written in Python.
(Disclaimer: I am a PyPy developer). It has a quite flexible
parser/bytecode compiler that could probably be tweaked to support your
new syntax (especially if the new constructs can be mapped to standard
python). Feel free to ask question on the pypy developer mailing list: http://codespeak.net/mailman/listinfo/pypy-dev
Cheers,
Carl Friedrich Bolz
Carl Friedrich Bolz wrote: I cannot really say much about how easy it would be to just write a preprocessor. However, I think what you are trying to do could be done reasonably easy with the PyPy project:
http://codespeak.net/pypy
PyPy is an implementation of a Python interpreter written in Python. (Disclaimer: I am a PyPy developer). It has a quite flexible parser/bytecode compiler that could probably be tweaked to support your new syntax (especially if the new constructs can be mapped to standard python).
It looks like the lack of thread support means I can't just use PyPy by
itself, unfortunately. But the tokeniser, lexer, parser and AST builder
could do what I need with modification, then I could walk the generated
AST and produce standard Python code from that. How easy would it be to
separate these parts out from the rest of PyPy? as******@gmail. com wrote: Carl Friedrich Bolz wrote:
I cannot really say much about how easy it would be to just write a preprocesso r. However, I think what you are trying to do could be done reasonably easy with the PyPy project:
http://codespeak.net/pypy
PyPy is an implementation of a Python interpreter written in Python. (Disclaimer : I am a PyPy developer). It has a quite flexible parser/bytecode compiler that could probably be tweaked to support your new syntax (especially if the new constructs can be mapped to standard python).
It looks like the lack of thread support means I can't just use PyPy by itself, unfortunately.
There is thread-support (using a GIL), but it is indeed not perfect yet.
This will definitively be a topic in the next months, though.
But the tokeniser, lexer, parser and AST builder could do what I need with modification, then I could walk the generated AST and produce standard Python code from that. How easy would it be to separate these parts out from the rest of PyPy?
Should be reasonably easy, although I am no expert in that area of PyPy.
Especially since the output of the parser/compiler is regular python
2.4 bytecode.
Cheers,
Carl Friedrich Bolz This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Ken in Melbourne Australia |
last post by:
If I use the curly bracket syntax (referred to as the
complex syntax) within a string, how do I get to call a
function within it?
The php manual says that the first (or previous)
character for the curly bracket has to be a dollar sign '$'.
This is fine for variables, arrays and some objects but
doesn't allow me to call a function such as addslashes() or
trim() before I return the string in the variable.
|
by: Edvard Majakari |
last post by:
Hi all ya unit-testing experts there :)
Code I'm working on has to parse large and complex files and detect
equally complex and large amount of errors before the contents of the file
is fed to module interpreting it. First I created a unit-test class named
TestLoad which loaded, say, 40 files of which about 10 are correct and
other 30 files contained over 20 different types of errors. Different
methods on the TestLoad class were coded so...
|
by: Andrew |
last post by:
Hi,
I am thinking about ways for efficient techniques for isolation testing.
Here is the problem as I see it:
Complex OO systems are designed with massive number of dependencies between
modules and classes. In most cases these dependencies end-up in external
systems such as databases and OS. Overhead in designing systems with "could
be stubbed" in mind is great, so in many cases dropped.
I would appreciate if anyone interested in this...
|
by: Charax |
last post by:
I maintain an academic web site accessed regularly by schools without the
latest equipment. This means that Version 4 IE and NS browsers are frequent
visitors.
When I validate CSS and HTML, which versions and DOCTYPE should I test
against?
Thanks you,
Chris Hopkins
|
by: Jonathan Allen |
last post by:
We have found that our method of testing does not match traditional
text-book methodologies. I decided to write a short white paper on it so
that I could get your opinions. Does anyone else use this method? If so,
what the heck is it called? Do you think it would work in other projects, or
did we just luck out?
Jonathan Allen
*****************
| |
by: newbiecpp |
last post by:
I am looking for a C++ unit testing program. I googled and found there are
quite a few there, such as CppTest, C++Test, and DejaGnu. Can someone give
me a recommendation among these programs? I want to the software is easy to
use and has a nice interface. Please excuse me if it is out of the topic.
Thanks in advance!
|
by: Diffident |
last post by:
Hello All,
Can anyone please suggest me a good unit testing tool. I have seen NUnit but
not sure on how I can use it to test my methods which involve session
variables, viewstate variables, textbox values.
I understand that NUnit is more suitable for OO methods which take set of
parameters and return an output parameter. How about tools for testing
methods which use state variables and form values?
|
by: Andrew Wan |
last post by:
I have been developing web applications with ASP & Javascript for a long
time. I have been using Visual Studio 2003.NET. While VS2003 is okay for
intellisense of ASP & Javascript, it's still not that great.
One of the cons of ASP & Javascript is that they're both interpreted, which
means one has twice the amount of work to do interms of syntax checking &
semantic/runtime checking.
Another bad thing is that ASP & Javascript doesn't have...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
| |
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |