467,076 Members | 885 Online
Bytes | Developer Community
Ask Question

Home New Posts Topics Members FAQ

Post your question to a community of 467,076 developers. It's quick & easy.

Parsing text file with #include and #define directives

I'm parsing a text file for a proprietary product that has the following
2 directives:

#include <somefile>
#define <name<value>

Defined constants are referenced via <#name#syntax.

I'm looking for a single text stream that results from processing a file
containing these directives. Even better would be an iterator(?) type
object that tracked file names and line numbers as it returns individual
lines.

Is there a Python parsing library to handle this type of task or am I
better off writing my own?

The effort to write one from scratch doesn't seem too difficult (minus
recursive file and constant loops), but I wanted to avoid re-inventing
the wheel if this type of component already exists.

Thank you,

Malcolm
Jun 27 '08 #1
  • viewed: 2263
Share:
2 Replies
py****@bdurham.com writes:
I'm parsing a text file for a proprietary product that has the following
2 directives:

#include <somefile>
#define <name<value>

Defined constants are referenced via <#name#syntax.

I'm looking for a single text stream that results from processing a file
containing these directives. Even better would be an iterator(?) type
object that tracked file names and line numbers as it returns individual
lines.

Is there a Python parsing library to handle this type of task or am I
better off writing my own?

The effort to write one from scratch doesn't seem too difficult (minus
recursive file and constant loops), but I wanted to avoid re-inventing
the wheel if this type of component already exists.

Thank you,

Malcolm
I think it's straightforward enough to be dealt with simply. Here is
a solution that doesn't handle errors but should work with well-formed
input and handles recursive expansions.

expand(filename) returns an iterator over expanded lines in the file,
inserting lines of included files.

import re

def expand(filename):
defines = {}
def define_repl(matchobj):
return defines[matchobj.group(1)]
define_regexp = re.compile('#(.+?)#')
for line in open(filename):
if line.startswith('#include '):
recfilename = line.strip().split(None, 1)[1]
for recline in expand(recfilename):
yield recline
elif line.startswith('#define '):
_, name, value = line.strip().split(None, 2)
defines[name] = value
else:
yield define_regexp.sub(define_repl, line)

It would be easy to modify it to keep track of line numbers and file
names.

HTH

--
Arnaud
Jun 27 '08 #2
Arnaud,

Wow!!! That's beautiful. Thank you very much!

Malcolm

<snip>

I think it's straightforward enough to be dealt with simply. Here is
a solution that doesn't handle errors but should work with well-formed
input and handles recursive expansions.

expand(filename) returns an iterator over expanded lines in the file,
inserting lines of included files.

import re

def expand(filename):
defines = {}
def define_repl(matchobj):
return defines[matchobj.group(1)]
define_regexp = re.compile('#(.+?)#')
for line in open(filename):
if line.startswith('#include '):
recfilename = line.strip().split(None, 1)[1]
for recline in expand(recfilename):
yield recline
elif line.startswith('#define '):
_, name, value = line.strip().split(None, 2)
defines[name] = value
else:
yield define_regexp.sub(define_repl, line)

It would be easy to modify it to keep track of line numbers and file
names.

</snip>
Jun 27 '08 #3

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

4 posts views Thread by Gert Van den Eynde | last post: by
4 posts views Thread by ralphNOSPAM@primemail.com | last post: by
4 posts views Thread by Stuk | last post: by
8 posts views Thread by Claude Yih | last post: by
5 posts views Thread by Martin Walke | last post: by
11 posts views Thread by andreyvul | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.