473,320 Members | 1,817 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

ConfigParser shootout, preliminary entry

A few weeks ago, the suggestion was made on Python-Dev that it might be time
to consider replacing the ConfigParser module and that we should hold a
"shootout" (ie ask for implementations and see what we get).

Since then I've been playing around with this... not the parsing part (which
so far I have completely ignored) but the programmer interface. There needs
to be a well-thought-out data model for the information stored, and the user
interface needs to be very easy to use, yet not so "magical" that it becomes
difficult to understand.

I have put together what I think is probably my best proposal. It is based
on a superset of ini config files and java .property files. There is a
convenient access mechanism ("config.my_app.some_value") as well as more
general approaches ("config.values['my_app.serviceByPort.80']"). I have
tried to consider issues like unicode (I permit fairly lenient mixing of
unicode and str), and unit testing ("... call config.clear_all() in the
tearDown() method of any unittests that use the config module..."). I have
even considered carefully what to leave OUT (converting to non-string data
types, interpolating values, things like that).

I think that I am now at the point where I could really use some input from
others. So I'd like to invite people to review my design and send me your
suggestions. I'm not expecting this as a *useful* module yet (it doesn't
yet parse files!), but it seemed like a good stage at which to ask for
feedback. I'm attaching two files, config.py and configTest.py, and they
are also available from these urls:

http://www.mcherm.com/publish/2004-10-17/config.py
http://www.mcherm.com/publish/2004-10-17/configTest.py

Thanks in advance for reviewing this.

-- Michael Chermside

Jul 18 '05 #1
14 1449
Michael Chermside wrote:
A few weeks ago, the suggestion was made on Python-Dev that it might be time
to consider replacing the ConfigParser module and that we should hold a
"shootout" (ie ask for implementations and see what we get).

Since then I've been playing around with this... not the parsing part (which
so far I have completely ignored) but the programmer interface. There needs
to be a well-thought-out data model for the information stored, and the user
interface needs to be very easy to use, yet not so "magical" that it becomes
difficult to understand.

I have put together what I think is probably my best proposal. It is based
on a superset of ini config files and java .property files. There is a
convenient access mechanism ("config.my_app.some_value") as well as more
general approaches ("config.values['my_app.serviceByPort.80']"). I have
tried to consider issues like unicode (I permit fairly lenient mixing of
unicode and str), and unit testing ("... call config.clear_all() in the
tearDown() method of any unittests that use the config module..."). I have
even considered carefully what to leave OUT (converting to non-string data
types, interpolating values, things like that).

I think that I am now at the point where I could really use some input from
others. So I'd like to invite people to review my design and send me your
suggestions. I'm not expecting this as a *useful* module yet (it doesn't
yet parse files!), but it seemed like a good stage at which to ask for
feedback. I'm attaching two files, config.py and configTest.py, and they
are also available from these urls:

http://www.mcherm.com/publish/2004-10-17/config.py
http://www.mcherm.com/publish/2004-10-17/configTest.py

Thanks in advance for reviewing this.

-- Michael Chermside


I doubt it conforms to your thinking on the matter, but some time ago I
wrote and released such a creature of my own invention:

https://www.tundraware.com/Software/tconfpy/

--
----------------------------------------------------------------------------
Tim Daneliuk tu****@tundraware.com
PGP Key: http://www.tundraware.com/PGP/
Jul 18 '05 #2
From the docs:
"The config module can read config files in Microsoft's ini file format,
java's properties file format, or its own python config format -- these
can even be mixed."
To me this does not sound appealing. People might just end
up being confused of what the actual file format is.
All these formats are so simple that supporting them all
only makes the usage more complicated.
"as long as the components of the path are valid Python identifiers,
there is a more convenient attribute syntax available:"


This means that some features can only be used if the parameter
names are valid python identifiers, right? When I put it that way,
it is a bit less attractive.

Istvan.
Jul 18 '05 #3
Config file reading is an area where python is 'well served' with
various options.
For sheer simplicty of use you can't beat my ConfigObj. It reads the
file and presents the values as a dictionary (keyed by keyword of
course). It supports writing hte file as well. The trouble with with
using attribute names is that you will have problems with keywords
that are reserved - like 'print' and 'pass'.

Of course the obligatory URL
http://www.voidspace.org.uk/atlantibots/configobj.html

Regards,

Fuzzyman

http://www.voidpace.org.uk/atlantibots/pythonutils.html

Michael Chermside <mc****@mcherm.com> wrote in message news:<ma**************************************@pyt hon.org>...
A few weeks ago, the suggestion was made on Python-Dev that it might be time
to consider replacing the ConfigParser module and that we should hold a
"shootout" (ie ask for implementations and see what we get).

Since then I've been playing around with this... not the parsing part (which
so far I have completely ignored) but the programmer interface. There needs
to be a well-thought-out data model for the information stored, and the user
interface needs to be very easy to use, yet not so "magical" that it becomes
difficult to understand.

I have put together what I think is probably my best proposal. It is based
on a superset of ini config files and java .property files. There is a
convenient access mechanism ("config.my_app.some_value") as well as more
general approaches ("config.values['my_app.serviceByPort.80']"). I have
tried to consider issues like unicode (I permit fairly lenient mixing of
unicode and str), and unit testing ("... call config.clear_all() in the
tearDown() method of any unittests that use the config module..."). I have
even considered carefully what to leave OUT (converting to non-string data
types, interpolating values, things like that).

I think that I am now at the point where I could really use some input from
others. So I'd like to invite people to review my design and send me your
suggestions. I'm not expecting this as a *useful* module yet (it doesn't
yet parse files!), but it seemed like a good stage at which to ask for
feedback. I'm attaching two files, config.py and configTest.py, and they
are also available from these urls:

http://www.mcherm.com/publish/2004-10-17/config.py
http://www.mcherm.com/publish/2004-10-17/configTest.py

Thanks in advance for reviewing this.

-- Michael Chermside

Jul 18 '05 #4
Istvan Albert <ia*****@mailblocks.com> wrote in message news:<-J********************@giganews.com>...
From the docs:
> "The config module can read config files in Microsoft's ini file format,
> java's properties file format, or its own python config format -- these
> can even be mixed."


To me this does not sound appealing. People might just end
up being confused of what the actual file format is.
All these formats are so simple that supporting them all
only makes the usage more complicated.

I don't think this is a problem. ini file format is simple enough - if
people just want a simple config file format the following is
straightforward enough :

[section name]
keyword = value

However the fact that it can support alternative formats is a good
thing. The important thing then becomes how good is the documentation.
If you document the simple use case first with alternatives afterwards
it should be easy enough.

> "as long as the components of the path are valid Python identifiers,
> there is a more convenient attribute syntax available:"


This means that some features can only be used if the parameter
names are valid python identifiers, right? When I put it that way,
it is a bit less attractive.

Istvan.


This is a valid point though. Dictionary syntax is good ;-)

Regards,

Fuzzy

http://www.voidspace.org.uk/atlantib...thonutils.html
Jul 18 '05 #5
Michael Foord wrote:
Config file reading is an area where python is 'well served' with
various options.
For sheer simplicty of use you can't beat my ConfigObj. It reads the
file and presents the values as a dictionary (keyed by keyword of
course). It supports writing hte file as well. The trouble with with
using attribute names is that you will have problems with keywords
that are reserved - like 'print' and 'pass'.


well served yes but they are all basically the same. For camram, I
needed a different configuration file service and I put a wrapper around
the stock with a configuration file utility.

Configuration data is composite of three data sources, global, shadow,
and user. In global are the system defaults, shadow contains local
system overrides, and user contains user specific overrides. the global
configuration file contains the patterns describing the locations of the
shadow and user configuration files.

I was experimenting with some other features based on how people might
want to use and anti-spam system, I added the ability for the user to
delegate or merge certain aspects of their profile of another user.

I will admit it's a bit of a dogs breakfast because I've made so many
passes and tried out a bunch of different things over the past year.
One of these days I'll go back over it and simplify things.

---eric

Jul 18 '05 #6

MC> A few weeks ago, the suggestion was made on Python-Dev that it might
MC> be time to consider replacing the ConfigParser module and that we
MC> should hold a "shootout" (ie ask for implementations and see what we
MC> get).

...

FM> Config file reading is an area where python is 'well served' with
FM> various options.

FM> For sheer simplicty of use you can't beat my ConfigObj.

Before we get too much stuff posted (which will just get lost), might I
suggest a ConfigurationParserShootout page for the wiki?

Thx,

Skip
Jul 18 '05 #7
Mike,

Ditto on "I got one too". I wrote a configuration module that is more
powerful than any I've seen to date, yet it has a simple interface and
simple configuration file syntax. In particular it handles arbitrary
levels of hierarchical organizations of settings quite nicely. Its
drawback (and the reason it would probably never make it into the
standard python distribution) is that it has a security hole. The
configuration file format is Python (the configuration module just
executes it rather than parses it).

That said, you may want to look at it for ideas. I'd be happy to
participate in discussions.

Good Luck,
Dan Gass
Jul 18 '05 #8
Oops, forgot the link:

https://sourceforge.net/projects/config-py/
Jul 18 '05 #9
Istvan comments:
> "as long as the components of the path are valid Python identifiers,
> there is a more convenient attribute syntax available:"


This means that some features can only be used if the parameter
names are valid python identifiers, right? When I put it that way,
it is a bit less attractive.


I'm afraid that I can't take credit for the idea of using attribute
syntax for accessing config values... it was Guido's suggestion in
the first place. But I'm not interested in an "appeal to authority"
argument here -- I think I can provide a good argument for why the
attribute syntax is a good idea (so long as there is a different
syntax available also for use with possible non-identifiers).

Code that accesses configuration values usually uses a constant for
the "key". In other words, you often look up "config.get('my_app.maxRows')",
but rarely something like "config.get('my_app.%s' % some_var)".
Furthermore, the keys themselves are rarely determined by external
forces... the programmer is usually free to select whatever name she
likes. Because of this, it is (almost always) quite easy to ensure that
only valid identifiers are used. With little downside, the convenience
to the programmer of typing "config.my_app.maxRows" rather than
"config.values['my_app.maxRows']" or "config.get('my_app.maxRows') is
worth considering.

Of course, there are still a few issues. One is explicitness... it's
unwise to have too much "magic". On the other hand, the more common
something becomes the more it becomes an idiom in its own right, and
having a somewhat "magical" syntax is more acceptable. For instance,
something like "dup_list = a_list[:]" looks like some kind of peculiar
smiley the first time any user sees it, but once you're used to it
you instantly recognize it as the Python idiom for making a copy.
(Although the new idiom: "dup_list = list(a_list)" is probably better.)
Features like logging and configuration which are ubiquitous are good
canidates for "convenience syntax".

And the other issue is that SOMETIMES one DOES use arbitrary strings
within config files. It isn't always a good idea to tell users to enter
something like "my_app.<name-of-the-server>.numConnections", but when
you DO something like this the server could be named "def", or "44832"
or even something truly dangerous like u"xp\u01a9" or "". So there
MUST be some OTHER means of accessing values in addition to the
attribute syntax so that arbitrary values can be passed.

Anyway... I certainly think that the matter is less than clear-cut, but
in my opinion, this is one of those areas where it's worth having a
little well-bounded "magic" to make the code that much more readable.

-- Michael Chermside
Jul 18 '05 #10
mc****@mcherm.com (Michael Chermside) wrote in message news:<1c**************************@posting.google. com>...
[snip..]

Anyway... I certainly think that the matter is less than clear-cut, but
in my opinion, this is one of those areas where it's worth having a
little well-bounded "magic" to make the code that much more readable.

-- Michael Chermside


Of course the good thing about python is that doing both is easy.
Using __setattr__ and sublassing dict will provide both methods with
minimal fuss.

I subclass caselessDict to get a case insensitive dictionary access -
but have been thinking about adding attributes - with a little magic
to avoid overriding any existing methods/attributes. (Just do a
dir(config) on an empty config object and that is the list of reserved
names...)

Regards,

Fuzzy

http://www.voidspace.org.uk/atlantib...thonutils.html
Jul 18 '05 #11
Michael Chermside <mc****@mcherm.com> wrote:
http://www.mcherm.com/publish/2004-10-17/config.py


One thing that's really obviously missing from this as an interface,
possibly because you're seeing it as a parsing issue, is how does
config know what file (or files) the data is in? That is, given:

config.my_app.some_value

how is it going to map between "my_app" and a file name? I ask
through bitter experience of having to work around wx's "wxConfig
knows best" to supply alternative and shared configurations.

--
\S -- si***@chiark.greenend.org.uk -- http://www.chaos.org.uk/~sion/
___ | "Frankly I have no feelings towards penguins one way or the other"
\X/ | -- Arthur C. Clarke
her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump
Jul 18 '05 #12
On Tue, 19 Oct 2004 20:27:53 -0500, Paramjit Oberoi <pa***@cs.wisc.edu> wrote:
Since then I've been playing around with this... not the parsing part (which
so far I have completely ignored) but the programmer interface. There needs
to be a well-thought-out data model for the information stored, and the user
interface needs to be very easy to use, yet not so "magical" that it becomes
difficult to understand.

From the me-too department:


For me personally, the problems with ConfigParser were inconvenient value
access/update and lack of order-preservation by the INI parser/writer. I
have developed my own solution which:

* Is convenient: both attribute access as well as container syntax can be
used to access as well as update values:

from cfgparse.iniparser import ini_namespace
conf = ini_namespace()
conf.user.name = 'Oberoi'
print conf['user']['name']
'Oberoi'

* Preserves order/indentation/spacing when the INI file is updated and
when the data is accessed.

* I have given some thought to the data model and interface question, but
probably not as much as you. The data model simply is that of arbitrarily
nested namespaces, each containing values (although each specific
implementation of the model can have it's own restrictions - for example,
INI files don't allow top-level values). The abstract model/interface
allos conversion from one config format to another.

* I have some code written using the ConfigParser API which I didn't want
to have to update, so an API-compatible interface is also available which
implements the old interface, as well as allowing access to the new one.

The code can be found here:

http://www.cs.wisc.edu/~param/software/cfgparse/

-param
--
http://mail.python.org/mailman/listinfo/python-list

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: ca********@gmail.com
mail: ca********@yahoo.com
Jul 18 '05 #13
So, I've been working on this a little yesterday and today and I've got
something that I like so far. I just thought about a "feature" and I'd
like to get some opinions on it. Should the ConfigParser object be
able to read and store configs from multiple config files or should
each config file require a new ConfigParser object?

If the ConfigParser object could read and store multiple files, I see
referencing it like so

# Load the config files
cpObj.load(file0)
cpObj.load(file1)

# get values from them
ret0 = cpObj.file0.property0
ret1 = cpObj.file1.property0

where file0 and file1 can be whatever you would like to name them. I
ask this because I like to put a lot of information in config files
making my code as configurable as possible. Sometimes, managing the
config file can be as messy as managing the code itself. So, breaking
the config files up into several smaller files may be handy.
Any thoughts on this?

Jul 18 '05 #14
I've given this some thought yesterday and today and would like to pose
another question. Should the configparser be able to read and store
properties from multiple config files or should each config file
require a separate configparser?

Jul 18 '05 #15

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Greg Krohn | last post by:
I'm trying to subclass ConfigParser so I can use a custom __read method (the custom format doesn't conform to RFC 822) when needed. Needless to say, it's not working as expected. In the...
1
by: Martin Maney | last post by:
Summary: with batteries like these, get a longer extension cord? <transcript from interactive session> Python 2.2.1 (#1, Sep 7 2002, 14:34:30) on linux2 Type "help", "copyright",...
2
by: Roy H. Berger | last post by:
If I want to subclass ConfigParser and changed the optionxform method to not return things in lower case wouldn't I just need the following code in my subclasss module? from ConfigParser import...
2
by: rzed | last post by:
I am working with PythonCard in one of my apps. For its purposes, it uses an .ini file that is passed to ConfigParser. For my app, I also need configuration information, but for various reasons,...
3
by: Sergey | last post by:
Is there an alternative to standard module ConfigParser, which can use delimitier symbol other than ":" and "=", preferaby just space? I need to parse such configs: 2:5020/758 xxxx 2:5020/794...
2
by: Greg Buchholz | last post by:
I was browsing the C++ programs on "The Computer Language Shootout" http://shootout.alioth.debian.org/ and for the heck of it, I thought I'd try my hand at seeing if I couldn't shorten the entry...
10
by: Terry Carroll | last post by:
It looks like ConfigParser will accept a list to be writing to the *.ini file; but when reading it back in, it treats it as a string. Example: ############################### import...
1
by: pipehappy | last post by:
Hello everyone: I came across the module ConfigParser and can use it correctly. import ConfigParser fp = open('test.cfg','w+') config = ConfigParser.ConfigParser() config.readfp(fp)...
4
by: Danil Dotsenko | last post by:
Wrote a little "user-friedly" wrapper for ConfigParser for a KDE's SuperKaramba widget. (http://www.kde-look.org/content/show.php?content=32185) I was using 2.4.x python docs as reference and...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.