By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
446,369 Members | 1,253 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 446,369 IT Pros & Developers. It's quick & easy.

Parsing text file with #include and #define directives

P: n/a
I'm parsing a text file for a proprietary product that has the following
2 directives:

#include <somefile>
#define <name<value>

Defined constants are referenced via <#name#syntax.

I'm looking for a single text stream that results from processing a file
containing these directives. Even better would be an iterator(?) type
object that tracked file names and line numbers as it returns individual
lines.

Is there a Python parsing library to handle this type of task or am I
better off writing my own?

The effort to write one from scratch doesn't seem too difficult (minus
recursive file and constant loops), but I wanted to avoid re-inventing
the wheel if this type of component already exists.

Thank you,

Malcolm
Jun 27 '08 #1
Share this Question
Share on Google+
2 Replies


P: n/a
py****@bdurham.com writes:
I'm parsing a text file for a proprietary product that has the following
2 directives:

#include <somefile>
#define <name<value>

Defined constants are referenced via <#name#syntax.

I'm looking for a single text stream that results from processing a file
containing these directives. Even better would be an iterator(?) type
object that tracked file names and line numbers as it returns individual
lines.

Is there a Python parsing library to handle this type of task or am I
better off writing my own?

The effort to write one from scratch doesn't seem too difficult (minus
recursive file and constant loops), but I wanted to avoid re-inventing
the wheel if this type of component already exists.

Thank you,

Malcolm
I think it's straightforward enough to be dealt with simply. Here is
a solution that doesn't handle errors but should work with well-formed
input and handles recursive expansions.

expand(filename) returns an iterator over expanded lines in the file,
inserting lines of included files.

import re

def expand(filename):
defines = {}
def define_repl(matchobj):
return defines[matchobj.group(1)]
define_regexp = re.compile('#(.+?)#')
for line in open(filename):
if line.startswith('#include '):
recfilename = line.strip().split(None, 1)[1]
for recline in expand(recfilename):
yield recline
elif line.startswith('#define '):
_, name, value = line.strip().split(None, 2)
defines[name] = value
else:
yield define_regexp.sub(define_repl, line)

It would be easy to modify it to keep track of line numbers and file
names.

HTH

--
Arnaud
Jun 27 '08 #2

P: n/a
Arnaud,

Wow!!! That's beautiful. Thank you very much!

Malcolm

<snip>

I think it's straightforward enough to be dealt with simply. Here is
a solution that doesn't handle errors but should work with well-formed
input and handles recursive expansions.

expand(filename) returns an iterator over expanded lines in the file,
inserting lines of included files.

import re

def expand(filename):
defines = {}
def define_repl(matchobj):
return defines[matchobj.group(1)]
define_regexp = re.compile('#(.+?)#')
for line in open(filename):
if line.startswith('#include '):
recfilename = line.strip().split(None, 1)[1]
for recline in expand(recfilename):
yield recline
elif line.startswith('#define '):
_, name, value = line.strip().split(None, 2)
defines[name] = value
else:
yield define_regexp.sub(define_repl, line)

It would be easy to modify it to keep track of line numbers and file
names.

</snip>
Jun 27 '08 #3

This discussion thread is closed

Replies have been disabled for this discussion.