By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,939 Members | 1,602 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,939 IT Pros & Developers. It's quick & easy.

Using Python to generate code?

P: n/a
Dear all:

I need your advice on this matter. I am working on a program which
takes some pieces of System-C code in and generate some other System-C
code. (System-C code is just C++ with some special libraries, hence
you can consider it as C++).

Right now, the generator program is written in C++. And I feel not so
comfortable with C++. I feel C++ is an overkill. Because, I need to
generate some code, hence in the program there are a lot of something
like this:

printf(out, "for (%s = 1, %s < %s, %s < %s )", varName, varName,
varName1, varname, varName2);

It is just too messy if I have more than 20 lines like this.

So my question is:
1. Can Python help to solve this issue?
2. Does Python has a parser? It seems to me that there is no-standard
Parser for Python right now. If it is the case, can I interface Python
with existing cup and lex?

Many thanks!
Tuan Anh
P/S: Also because it is NO big code generator, it only needs to
generate some simple codes. Hence, using some abstract-syntax-tree
library is also too an overkill.
Jul 18 '05 #1
Share this Question
Share on Google+
16 Replies


P: n/a
On Tue, 07 Sep 2004 23:51:38 -0700, Tran Tuan Anh wrote:
printf(out, "for (%s = 1, %s < %s, %s < %s )", varName, varName,
varName1, varname, varName2);

It is just too messy if I have more than 20 lines like this.
It looks like you want a templater. I don't know what such programs exist
for C++, if any, or how good they are.

You may get a lot of milage out of just adding some utility functions, to
turn that to something like

forFromOne(out, var, max, min);

Changing languages is pretty drastic if you have working, trusted code.
(If those words don't describe your code, then that assessment may change.)
So my question is:
1. Can Python help to solve this issue?
Maybe a little, but if the above code snippet is representative, the
Python equivalent of that isn't much shorter. Python would start to shine
if you want to start doing smarter things with the data, but without
knowing about the data it is hard to be specific. Basically, the moment
you aren't just assembling strings, Python will *quite likely* have an
advantage.

(I find it very likely that a Pythonic outlook on the problem would allow
significant simplification at acceptable effort levels, but I can't
*promise* that; if all you really, honestly need is printf, which I think
would be rare but certainly conceivable, then you may get no advantage.)
2. Does Python has a parser? It seems to me that there is no-standard
Parser for Python right now. If it is the case, can I interface Python
with existing cup and lex?


Why are you asking about a parser when you are talking about *outputting*
code? Is there something left out of the problem description?

There are a lot of parsers for Python, covering a wide range of needs.

Jul 18 '05 #2

P: n/a
Tran Tuan Anh <anhtt <at> hotmail.com> writes:
1. Can Python help to solve this issue?
I have used Python for quite simple code generation/templating - not in C++, but
that makes no difference. There are even special and not that complex tools for
code generation, e.g. Cog (http://www.nedbatchelder.com/code/cog/index.html).
2. Does Python has a parser? It seems to me that there is no-standard


There are all kinds of parsers, but I've never used any of them. Google gives
tons of links when searching for "python parser", so a more specific search
would be wise.

Yours,

Andrei

Jul 18 '05 #3

P: n/a
I use Python to generate quite a bit of HTML and
JavaScript code and it works very well. If you
can use Python's classes to abstract individual
elements and give each of them a __str__ method
you can build very complex code generator classes
that know how to render their own output.

Larry Bates
Syscon, Inc.

"Tran Tuan Anh" <an***@hotmail.com> wrote in message
news:ed**************************@posting.google.c om...
Dear all:

I need your advice on this matter. I am working on a program which
takes some pieces of System-C code in and generate some other System-C
code. (System-C code is just C++ with some special libraries, hence
you can consider it as C++).

Right now, the generator program is written in C++. And I feel not so
comfortable with C++. I feel C++ is an overkill. Because, I need to
generate some code, hence in the program there are a lot of something
like this:

printf(out, "for (%s = 1, %s < %s, %s < %s )", varName, varName,
varName1, varname, varName2);

It is just too messy if I have more than 20 lines like this.

So my question is:
1. Can Python help to solve this issue?
2. Does Python has a parser? It seems to me that there is no-standard
Parser for Python right now. If it is the case, can I interface Python
with existing cup and lex?

Many thanks!
Tuan Anh
P/S: Also because it is NO big code generator, it only needs to
generate some simple codes. Hence, using some abstract-syntax-tree
library is also too an overkill.

Jul 18 '05 #4

P: n/a
On 7 Sep 2004 23:51:38 -0700, Tran Tuan Anh <an***@hotmail.com> wrote:
Right now, the generator program is written in C++. And I feel not so
comfortable with C++. I feel C++ is an overkill. Because, I need to
generate some code, hence in the program there are a lot of something
like this:

printf(out, "for (%s = 1, %s < %s, %s < %s )", varName, varName,
varName1, varname, varName2);

It is just too messy if I have more than 20 lines like this.
Agreed, reached that same conclusion.
Therefore, I switched to something like

output="""for ($VAR = 1, $VAR < $START, $VAR < $END)"""

output=string.replace(output,"$VAR",varName)
output=string.replace(output,"$START",varName1)
output=string.replace(output,"$END",varName2)

Obviously, this can be enhanced.
(use other conventions than $identifier, and performing the substitution
in a loop).
BTW: I am not sure string.replace works OK as shown here.

In a sense, I am using the string replacement to substitute variables
names thus making my own text templating system.
1. Can Python help to solve this issue?
yep.
2. Does Python has a parser? It seems to me that there is no-standard
yes, several.
I have good experiences with spark. you have to be careful what you
enter, and it is not fast, but it can handle just about any
grammar you can think of.
P/S: Also because it is NO big code generator, it only needs to
generate some simple codes. Hence, using some abstract-syntax-tree
library is also too an overkill.


Anything small, simple, and beautiful grows ......
:-)

Albert
--
Unlike popular belief, the .doc format is not an open publically available format.
Jul 18 '05 #5

P: n/a
Jeremy Bowers <je**@jerf.org> wrote:
...
2. Does Python has a parser? It seems to me that there is no-standard
Parser for Python right now. If it is the case, can I interface Python
with existing cup and lex?
Why are you asking about a parser when you are talking about *outputting*
code? Is there something left out of the problem description?


the original poster started his original post with:
I need your advice on this matter. I am working on a program which
takes some pieces of System-C code in and generate some other System-C


That "takes some pieces of ... code" sounded to me like an indication
that a parser is needed -- nothing left out of the problem description,
IMHO.
Alex
Jul 18 '05 #6

P: n/a
Albert Hofkamp <ha*@se-126.se.wtb.tue.nl> wrote:
On 7 Sep 2004 23:51:38 -0700, Tran Tuan Anh <an***@hotmail.com> wrote:
Right now, the generator program is written in C++. And I feel not so
comfortable with C++. I feel C++ is an overkill. Because, I need to
generate some code, hence in the program there are a lot of something
like this:

printf(out, "for (%s = 1, %s < %s, %s < %s )", varName, varName,
varName1, varname, varName2);

It is just too messy if I have more than 20 lines like this.
Agreed, reached that same conclusion.
Therefore, I switched to something like

output="""for ($VAR = 1, $VAR < $START, $VAR < $END)"""


Uh, OT, but, doesn't this work better with semicolons than commas...?

output=string.replace(output,"$VAR",varName)
output=string.replace(output,"$START",varName1)
output=string.replace(output,"$END",varName2)

Obviously, this can be enhanced.
(use other conventions than $identifier, and performing the substitution
in a loop).
BTW: I am not sure string.replace works OK as shown here.


Yep, though output.replace(...) would be neater. But the best
replacement is already in the Python 2.4 standard library: it uses
exactly the $identifier convention, and takes a dictionary of mapping of
identifier to string...:

In [4]: tpl=string.Template('for ($VAR = 1, $VAR < $START, $VAR <
$END)')

In [5]: tpl % dict(VAR='foo', START='bar', END='baz')
Out[5]: u'for (foo = 1, foo < bar, foo < baz)'

In [6]: tpl % dict(VAR='fee', START='fie', END='fofum')
Out[6]: u'for (fee = 1, fee < fie, fee < fofum)'

Alex
Jul 18 '05 #7

P: n/a
Alex Martelli wrote:
output=string.replace(output,"$VAR",varName)
output=string.replace(output,"$START",varName1 )
output=string.replace(output,"$END",varName2)

Obviously, this can be enhanced.
(use other conventions than $identifier, and performing the substitution
in a loop).
BTW: I am not sure string.replace works OK as shown here.

Yep, though output.replace(...) would be neater. But the best
replacement is already in the Python 2.4 standard library: it uses
exactly the $identifier convention, and takes a dictionary of mapping of
identifier to string...:

In [4]: tpl=string.Template('for ($VAR = 1, $VAR < $START, $VAR <
$END)')

In [5]: tpl % dict(VAR='foo', START='bar', END='baz')
Out[5]: u'for (foo = 1, foo < bar, foo < baz)'

In [6]: tpl % dict(VAR='fee', START='fie', END='fofum')
Out[6]: u'for (fee = 1, fee < fie, fee < fofum)'


If you are not contrained to using $ identifiers:

d={'var':'foo', 'start':1, 'end': 100}
'for (%(var)s = %(start)i; %(var)s < %(end)i; %(var)s++)' % d

-> 'for (foo = 1; foo < 100; foo++)'

Regards
--
////
(@ @)
----------------------------oOO----(_)----OOo--------------------------
<> Ojo por ojo y el mundo acabara ciego
/\ Alexis Roda - Universitat Rovira i Virgili - Reus, Tarragona (Spain)
-----------------------------------------------------------------------

Jul 18 '05 #8

P: n/a
Alexis Roda <al*********@urv.es> wrote:
...
Yep, though output.replace(...) would be neater. But the best
replacement is already in the Python 2.4 standard library: it uses
exactly the $identifier convention, and takes a dictionary of mapping of
identifier to string...: ... In [6]: tpl % dict(VAR='fee', START='fie', END='fofum')
Out[6]: u'for (fee = 1, fee < fie, fee < fofum)'


If you are not contrained to using $ identifiers:

d={'var':'foo', 'start':1, 'end': 100}
'for (%(var)s = %(start)i; %(var)s < %(end)i; %(var)s++)' % d


yeah, but the template string for this long-standing Python feature is
hard to read and hard to write, which is why 2.4 finally grew a simpler
approach -- the string.Template class.
Alex

Jul 18 '05 #9

P: n/a
Alex Martelli wrote:
Alexis Roda <al*********@urv.es> wrote:
...
Yep, though output.replace(...) would be neater. But the best
replacement is already in the Python 2.4 standard library: it uses
exactly the $identifier convention, and takes a dictionary of mapping of
identifier to string...:
...
In [6]: tpl % dict(VAR='fee', START='fie', END='fofum')
Out[6]: u'for (fee = 1, fee < fie, fee < fofum)'


If you are not contrained to using $ identifiers:

d={'var':'foo', 'start':1, 'end': 100}
'for (%(var)s = %(start)i; %(var)s < %(end)i; %(var)s++)' % d

yeah, but the template string for this long-standing Python feature is
hard to read and hard to write, which is why 2.4 finally grew a simpler
approach -- the string.Template class.
Alex


If you come from Perl its not so hard :-). Anyway, I agree, templates
are nicer, but not everyone is running python 2.4 ...
Alex
--
////
(@ @)
----------------------------oOO----(_)----OOo--------------------------
<> Ojo por ojo y el mundo acabara ciego
/\ Alexis Roda - Universitat Rovira i Virgili - Reus, Tarragona (Spain)
-----------------------------------------------------------------------

Jul 18 '05 #10

P: n/a
Alex Martelli <aleaxit <at> yahoo.com> writes:
yeah, but the template string for this long-standing Python feature is
hard to read and hard to write, which is why 2.4 finally grew a simpler
approach -- the string.Template class.


Just to make the comparison as fair as possible:

2.4 Template solution
tpl = string.Template('for ($VAR = 1, $VAR < $START, $VAR < $END)')
tpl % dict(VAR='fee', START='fie', END='fofum')

<2.4 format string solution
tpl = 'for (%(VAR)s = 1, %(VAR)s < %(START)s, %(VAR)s < %(END)s)'
tpl % dict(VAR='fee', START='fie', END='fofum')

So, yes, the Template solution is cleaner, but not drastically so. It saves
you 3 characters per variable -- '(', ')' and 's'. Of course the more
variables you have to write, the bigger the deal it is, so I suspect the OP
would much prefer the Template solution.

Steve

Jul 18 '05 #11

P: n/a
On Wed, 8 Sep 2004 12:59:57 -0500, Greg Lindstrom
<gr************@novasyshealth.com> wrote:
How can I get the name (or ip) of the machine where my python script is
running? I'd like to add it to my log entry.

import socket

print socket.gethostname()
print socket.getfqdn()

this_host = socket_gethostname()
print socket.gethostbyname(this_host)

# the following will give several IP addresses if you are
#multi-homed

print socket.gethostbyname_ex(this_host)

Jul 18 '05 #12

P: n/a
Steven Bethard wrote:
So, yes, the Template solution is cleaner, but not drastically so.**It
saves you 3 characters per variable -- '(', ')' and 's'.**Of*course*the
more variables you have to write, the bigger the deal it is


True, and calling string.Template() is an extra 17 characters, so there
would have to be 6 variables in your string before you got a typing
payoff. :-) Of course, some people might prefer the look of the Template.

Jeffrey
Jul 18 '05 #13

P: n/a
Many thanks for your comments and advice!

Another concern is that how I can interface Python with Cup/Lex.
Writing parser is a no-joke and tedious to me, hence I would like to
re-use
the existing Cup/Lex scanner and parser.
Could anyone confirm that it is possible to do so? (I mean naturally
:)

In general, the program I am dealing with needs to manipulate strings
quite a lot. Eventhough we can certainly do with C++, it does not
appeal naturally to me.
Jul 18 '05 #14

P: n/a
- I'm familar with Java, C, C++, and Pascal, some experiences with
SML. Could you elaborate about some Python's string-manipulation
features?

- So now imagine if I have to convince my supervisor, an
theoretic-non-progammer, about switching to Python for this code
generator program, what kind of advantages I can get from doing so?

Thanks!
Tuan Anh
Jul 18 '05 #15

P: n/a
On 9 Sep 2004 07:09:10 -0700, Tran Tuan Anh <an***@hotmail.com> wrote:
- I'm familar with Java, C, C++, and Pascal, some experiences with
SML. Could you elaborate about some Python's string-manipulation
features?
I can never beat the online documentation:

http://www.python.org/doc, click 'Module index', click 'strings'.

I don't know where the string template thingies are described, but it
has to be there somewhere.
- So now imagine if I have to convince my supervisor, an
theoretic-non-progammer, about switching to Python for this code
generator program, what kind of advantages I can get from doing so?


Same:

http://www.python.org/ look in the 'Documentation links', in particular
the 'introductions to python'. There is a section of links to convince
those that control what we do (they think :-) ).
[ python documentation is awesome, very complete and very good. Browse
the website for answers to just about any question.
]
Albert
--
Unlike popular belief, the .doc format is not an open publically available format.
Jul 18 '05 #16

P: n/a
On Wed, 8 Sep 2004 17:47:16 +0200, Alex Martelli <al*****@yahoo.com> wrote:
Jeremy Bowers <je**@jerf.org> wrote:
...
> 2. Does Python has a parser? It seems to me that there is no-standard
> Parser for Python right now. If it is the case, can I interface Python
> with existing cup and lex?


Why are you asking about a parser when you are talking about *outputting*
code? Is there something left out of the problem description?


the original poster started his original post with:
I need your advice on this matter. I am working on a program which
takes some pieces of System-C code in and generate some other System-C


That "takes some pieces of ... code" sounded to me like an indication
that a parser is needed -- nothing left out of the problem description,
IMHO.


For the original poster:

I just discovered that there is a Topic guide about scanners/parsers at
python.org !!
It has quite a list of module links.
Albert
--
Unlike popular belief, the .doc format is not an open publically available format.
Jul 18 '05 #17

This discussion thread is closed

Replies have been disabled for this discussion.