Parsing environment variables in ConfigParser files

[Matthew Barnes]

I'm considering submitting a patch for Python 2.4 to allow environment
variable expansion in ConfigParser files. The use cases for this
should be obvious. I'd like to be able to specify something like the
following in a configuration file:

[section_name]
data_file=${HOME}/mydata.dat

....(where HOME=/home/matt) and have ConfigParser automatically expand
it to:

[section_name]
data_file=/home/matt/mydata.dat

The change is pretty straight-forward, but I'm interested in feedback
on whether this is a good idea, what the syntax for environment
variable references should look like (currently I'm thinking
${varname}), or whether there are any hidden complexities or
backward-compatibility concerns.

Matthew Barnes

[Andrew Dalke]

Matthew Barnes:

I'm considering submitting a patch for Python 2.4 to allow environment
variable expansion in ConfigParser files. [...] I'd like to be
able to specify something like the following in a configuration file:

[section_name]
data_file=${HOME}/mydata.dat

Personally, I'm against it. The less magic the better. Eg,
what if I have a configuration file which sets the default
configurations, so that I want the actual string
${HOME}/mydata.dat
in the file. What's the escape rule for that.

Whereas as it is now I can get the string value then use
os.path.expandvars (or os.path.expanduser if I want ~home
expansion). It's an extra couple lines of code, granted,
but it's explicit instead of implicit and less likely to introduce
subtle errors.

Andrew
da***@dalkescientific.com

[Skip Montanaro]

Matthew> I'm considering submitting a patch for Python 2.4 to allow
Matthew> environment variable expansion in ConfigParser files.

data_file=${HOME}/mydata.dat -> data_file=/home/matt/mydata.dat

Matthew> The change is pretty straight-forward, but I'm interested in
Matthew> feedback on whether this is a good idea, what the syntax for
Matthew> environment variable references should look like (currently I'm
Matthew> thinking ${varname}), or whether there are any hidden
Matthew> complexities or backward-compatibility concerns.

There is already a %(foo)s expansion capability in ConfigParser, though the
source of key/value pairs is different. I'd check to see if it can be bent
(twisted?) to your needs by subclassing ConfigParser and merging a suitably
munged os.environ with the user's config file. Using your example, the
DEFAULT info would be synthesized or augmented to look like (logically
speaking, this might simply be modified to parse os.environ before reading
the config file):

[DEFAULT]
# stuff imported from os.environ
HOME=/home/matt
USER=matt
...
# user's other DEFAULT settings
...

and a simple regular expression replace would be performed on the actual
config file roughly like so:

configf = file("config.ini")
configdata = re.sub(r'${([^}]+)}', r'%(\1)s', configf.read())
configf = StringIO.StringIO(configdata)
configf.seek(0)

Skip

[Peter Otten]

Matthew Barnes wrote:

I'm considering submitting a patch for Python 2.4 to allow environment
variable expansion in ConfigParser files. The use cases for this
should be obvious. I'd like to be able to specify something like the
following in a configuration file:

[section_name]
data_file=${HOME}/mydata.dat

...(where HOME=/home/matt) and have ConfigParser automatically expand
it to:

[section_name]
data_file=/home/matt/mydata.dat

The change is pretty straight-forward, but I'm interested in feedback
on whether this is a good idea, what the syntax for environment
variable references should look like (currently I'm thinking
${varname}), or whether there are any hidden complexities or
backward-compatibility concerns.

Matthew Barnes

I think this can be easily achieved by subclassing the ConfigParser:

from ConfigParser import ConfigParser
class ExpandingParser(ConfigParser): .... def getexpanded(self, section, option):
.... import os
.... return self._get(section, os.path.expandvars, option)
.... p = ExpandingParser()
p.read("expand.ini")
p.get("paths", "mydir") '$HOME/of/the/brave' p.getexpanded("paths", "mydir") '/home/peter/of/the/brave'

If you will bother with the library implementation at all, I would prefer
this level of explicitness, i. e. a dedicated getexpanded() method in
ConfigParser, or alternatively a subclass that overrides _get() to always
expand in the getXXX() methods. With both approaches you can stay safely
away from backwards compatibility problems.

Peter

[Jarek Zgoda]

Matthew Barnes <ma*****@barnes.net> pisze:

[section_name]
data_file=${HOME}/mydata.dat

Curly braces are so... php'ish... No, I don't like this idea.

--
Jarek Zgoda
Unregistered Linux User # -1
http://www.zgoda.biz/ JID:zg***@chrome.pl http://zgoda.jogger.pl/

[Bogdan Marinescu]

Good idea, but since you want to do this, why stop here? I would really
like to have the possibility to expand some previous defined variables in
the file:

[sect1]
var1 = ...
..............
expansion=${var1}

Of course this should merely complete the environment variable
replacement feature.

----- Original Message -----
From: "Matthew Barnes" <ma*****@barnes.net>
Newsgroups: comp.lang.python
To: <py*********@python.org>
Sent: Friday, December 19, 2003 7:36 PM
Subject: Parsing environment variables in ConfigParser files

I'm considering submitting a patch for Python 2.4 to allow environment
variable expansion in ConfigParser files. The use cases for this
should be obvious. I'd like to be able to specify something like the
following in a configuration file:

[section_name]
data_file=${HOME}/mydata.dat

...(where HOME=/home/matt) and have ConfigParser automatically expand
it to:

[section_name]
data_file=/home/matt/mydata.dat

The change is pretty straight-forward, but I'm interested in feedback
on whether this is a good idea, what the syntax for environment
variable references should look like (currently I'm thinking
${varname}), or whether there are any hidden complexities or
backward-compatibility concerns.

Matthew Barnes
--
http://mail.python.org/mailman/listinfo/python-list

[Skip Montanaro]

Bogdan> Good idea, but since you want to do this, why stop here? I would
Bogdan> really like to have the possibility to expand some previous
Bogdan> defined variables in the file:

...

ConfigParser already does this.

Skip