473,699 Members | 2,377 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

config files in python

Hi,
In my application, I have some configurable information which is used
by different processes. currently I have stored configration in a
conf.py file as name=value pairs, and I am importing conf.py file to
use this variable. it works well

import conf
print conf.SomeVariab le

but if I need to change some configuration parameteres, it would need
me to restart processes.

I want to store this data in some conf file (txt) and would like to
use it same way as I am using these variables as defined in py
files.

one solution I can think of is writing data as a dictionary into conf
file. and then by reading data, apply eval on that data. and update
local dict? but this is not a good solution....

any pointers?

Sandip
Jun 27 '08 #1
10 3249
sandipm wrote:
In my application, I have some configurable information which is used
by different processes. currently I have stored configration in a
conf.py file as name=value pairs, and I am importing conf.py file to
use this variable. it works well

import conf
print conf.SomeVariab le

but if I need to change some configuration parameteres, it would need
me to restart processes.
reload(conf)

http://docs.python.org/lib/built-in-funcs.html#l2h-61

But you should carefully consider what updating the configuration means
in your application.

Ciao, Michael.
Jun 27 '08 #2
On Mon, 05 May 2008 00:35:51 -0700, sandipm wrote:
Hi,
In my application, I have some configurable information which is used
by different processes. currently I have stored configration in a
conf.py file as name=value pairs, and I am importing conf.py file to
use this variable. it works well

import conf
print conf.SomeVariab le

but if I need to change some configuration parameteres, it would need
me to restart processes.

I want to store this data in some conf file (txt) and would like to
use it same way as I am using these variables as defined in py
files.

one solution I can think of is writing data as a dictionary into conf
file. and then by reading data, apply eval on that data. and update
local dict? but this is not a good solution....

any pointers?

Sandip
The 'simple but relatively dangerous way', already suggested, is to
reload() the module.

A safer way - but requiring more work - could be to build something around
the Configparser module in the standard library ...

Ciao
-----
FB
Jun 27 '08 #3
On May 5, 10:22 am, Francesco Bochicchio <bock...@virgil io.itwrote:
On Mon, 05 May 2008 00:35:51 -0700, sandipm wrote:
Hi,
In my application, I have some configurable information which is used
by different processes. currently I have stored configration in a
conf.py file as name=value pairs, and I am importing conf.py file to
use this variable. it works well
import conf
print conf.SomeVariab le
but if I need to change some configuration parameteres, it would need
me to restart processes.
I want to store this data in some conf file (txt) and would like to
use it same way as I am using these variables as defined in py
files.
one solution I can think of is writing data as a dictionary into conf
file. and then by reading data, apply eval on that data. and update
local dict? but this is not a good solution....
any pointers?
Sandip

The 'simple but relatively dangerous way', already suggested, is to
reload() the module.

A safer way - but requiring more work - could be to build something around
the Configparser module in the standard library ...

Ciao
-----
FB
How is it relatively dangerous? I mean, aside from any `dangers'
associated with importing a user defined module in the first place.

Matt
Jun 27 '08 #4
On May 4, 11:35 pm, sandipm <sandip.m...@gm ail.comwrote:
Hi,
In my application, I have some configurable information which is used
by different processes. currently I have stored configration in a
conf.py file as name=value pairs, and I am importing conf.py file to
use this variable. it works well

import conf
print conf.SomeVariab le

but if I need to change some configuration parameteres, it would need
me to restart processes.

I want to store this data in some conf file (txt) and would like to
use it same way as I am using these variables as defined in py
files.

one solution I can think of is writing data as a dictionary into conf
file. and then by reading data, apply eval on that data. and update
local dict? but this is not a good solution....

any pointers?

Sandip
I would load the configuration file using `imp.load_sourc e'. This
allows you to load the config file by filename, and gets away from the
issue of accidentally importing a file somewhere else in pythons
search path. Also, calling imp.load_source will reload the module when
called a second time.

http://docs.python.org/lib/module-imp.html

[conf.py]
a = 1
b = 2
class c:
a = "hello"
b = "world"
[/end conf.py]
>>conf = imp.load_source ("conf", "./conf.py")
conf.a
1
>>conf.b
2
>>conf.c.a
'hello'
>>conf.c.b
'world'

There are so many ways potential solutions to your problem that,
without any more details, it is hard to suggest anything.

Here are some potential solutions:

ConfigParser - module for handling ini files
xml - several built-in modules for handling XML files
sqlite3 - a `lite' SQL database built-in in python 2.5 + (can be used
for config data)
windows registry - _winreg module
pickle - serialize python objects
marshal - similar to pickle, only works for simple objects

Those are just the built-in solutions. If you wanna look at 3rd party
solutions, prepare for overload. The number of alternative INI parsers
alone is staggering.

Also, there are many ways to organize your data and use a solution
similar to what you are already using.

I guess what I'm trying to say is... don't roll your own, it would be
a waste of time, this problem has been solved 100s of times. That is,
unless you want to do it for fun.

Matt
Jun 27 '08 #5
On May 5, 6:57 pm, Matimus <mccre...@gmail .comwrote:
On May 5, 10:22 am, Francesco Bochicchio <bock...@virgil io.itwrote:
On Mon, 05 May 2008 00:35:51 -0700, sandipm wrote:
Hi,
In my application, I have some configurable information which is used
by different processes. currently I have stored configration in a
conf.py file as name=value pairs, and I am importing conf.py file to
use this variable. it works well
import conf
print conf.SomeVariab le
but if I need to change some configuration parameteres, it would need
me to restart processes.
I want to store this data in some conf file (txt) and would like to
use it same way as I am using these variables as defined in py
files.
one solution I can think of is writing data as a dictionary into conf
file. and then by reading data, apply eval on that data. and update
local dict? but this is not a good solution....
any pointers?
Sandip
The 'simple but relatively dangerous way', already suggested, is to
reload() the module.
A safer way - but requiring more work - could be to build something around
the Configparser module in the standard library ...
Ciao
-----
FB

How is it relatively dangerous? I mean, aside from any `dangers'
associated with importing a user defined module in the first place.
Read the caveats in reload()'s description. Briefly:
- It does not reload names imported with "from ... import ...".
- Any existing instances of classes defined in the module still refer
to the original class, not the reloaded one (assuming it is still
present).
- The module's dictionary (containing the module's global variables)
is retained and updated in place, so names that are deleted in the
module are still available.
- It is not recursive.
- And more...

George
Jun 27 '08 #6
Thanks for various useful suggestions.
actually right now I am using conf files only in psp handler of
mod_python/apache
but I have other processes which might use same config files.

One way is I can put conf related data directly in database and
database handling module can directly pickup values from db.
but the problem with that approach is that it is difficult in managing
changes from version control system.as I need to explicitly
create DB scripts to put it in version control and other devs require
them run those scripts.

but if I put those conf params in files it would be easy for me to
change params and useful for other developers
to integrate those changes in their dev environments without doing any
explicit ops.

here I would like to have python file which read conf from text file
and load those params in current process space.
so only importing that python file should read up the conf file and
load the current process with configurable parameters.
I thought this would be good way to do it, rather than getting
involved in slightly complicted reload mechanisms of python modules?

Regards,
Sandip

Jun 27 '08 #7
On Tue, May 6, 2008 at 2:00 AM, sandipm <sa*********@gm ail.comwrote:
>
here I would like to have python file which read conf from text file
and load those params in current process space.
so only importing that python file should read up the conf file and
load the current process with configurable parameters.
I thought this would be good way to do it, rather than getting
involved in slightly complicted reload mechanisms of python modules?
I think you should work your way into a proper configuration system.

for ones you can use the buildin module
http://docs.python.org/lib/module-ConfigParser.html

or migrate to the more powerful configObj
http://www.voidspace.org.uk/python/configobj.html
>
with configobj, you will have an instance in memory that you can
modify on runtime with a gui or something and when you hit save it
will rewrite the new config file.
>
Regards,
Sandip

--
http://mail.python.org/mailman/listinfo/python-list
Jun 27 '08 #8
On 2008-05-06 01:16, Matimus wrote:
On May 4, 11:35 pm, sandipm <sandip.m...@gm ail.comwrote:
>Hi,
In my application, I have some configurable information which is used
by different processes. currently I have stored configration in a
conf.py file as name=value pairs, and I am importing conf.py file to
use this variable. it works well

import conf
print conf.SomeVariab le

but if I need to change some configuration parameteres, it would need
me to restart processes.

I want to store this data in some conf file (txt) and would like to
use it same way as I am using these variables as defined in py
files.

one solution I can think of is writing data as a dictionary into conf
file. and then by reading data, apply eval on that data. and update
local dict? but this is not a good solution....

any pointers?

Sandip

I would load the configuration file using `imp.load_sourc e'. This
allows you to load the config file by filename, and gets away from the
issue of accidentally importing a file somewhere else in pythons
search path. Also, calling imp.load_source will reload the module when
called a second time.

http://docs.python.org/lib/module-imp.html
Why not just use execfile() ?

http://www.python.org/doc/2.2.3/lib/built-in-funcs.html
[conf.py]
a = 1
b = 2
class c:
a = "hello"
b = "world"
[/end conf.py]
>>>conf = imp.load_source ("conf", "./conf.py")
conf.a
1
>>>conf.b
2
>>>conf.c.a
'hello'
>>>conf.c.b
'world'

There are so many ways potential solutions to your problem that,
without any more details, it is hard to suggest anything.

Here are some potential solutions:

ConfigParser - module for handling ini files
xml - several built-in modules for handling XML files
sqlite3 - a `lite' SQL database built-in in python 2.5 + (can be used
for config data)
windows registry - _winreg module
pickle - serialize python objects
marshal - similar to pickle, only works for simple objects

Those are just the built-in solutions. If you wanna look at 3rd party
solutions, prepare for overload. The number of alternative INI parsers
alone is staggering.

Also, there are many ways to organize your data and use a solution
similar to what you are already using.

I guess what I'm trying to say is... don't roll your own, it would be
a waste of time, this problem has been solved 100s of times. That is,
unless you want to do it for fun.

Matt
--
http://mail.python.org/mailman/listinfo/python-list
--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, May 06 2008)
>>Python/Zope Consulting and Support ... http://www.egenix.com/
mxODBC.Zope.D atabase.Adapter ... http://zope.egenix.com/
mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
_______________ _______________ _______________ _______________ ____________

:::: Try mxODBC.Zope.DA for Windows,Linux,S olaris,MacOSX for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
Jun 27 '08 #9
On Tue, May 6, 2008 at 4:33 AM, M.-A. Lemburg <ma*@egenix.com wrote:
>
On 2008-05-06 01:16, Matimus wrote:
On May 4, 11:35 pm, sandipm <sandip.m...@gm ail.comwrote:
Hi,
In my application, I have some configurable information which is used
by different processes. currently I have stored configration in a
conf.py file as name=value pairs, and I am importing conf.py file to
use this variable. it works well
>
import conf
print conf.SomeVariab le
>
but if I need to change some configuration parameteres, it would need
me to restart processes.
>
I want to store this data in some conf file (txt) and would like to
use it same way as I am using these variables as defined in py
files.
>
one solution I can think of is writing data as a dictionary into conf
file. and then by reading data, apply eval on that data. and update
local dict? but this is not a good solution....
>
any pointers?
>
Sandip
>
I would load the configuration file using `imp.load_sourc e'. This
allows you to load the config file by filename, and gets away from the
issue of accidentally importing a file somewhere else in pythons
search path. Also, calling imp.load_source will reload the module when
called a second time.

>
http://docs.python.org/lib/module-imp.html

Why not just use execfile() ?

http://www.python.org/doc/2.2.3/lib/built-in-funcs.html
that is very bad for this case, from what he is suggesting this is a
server install so you are basically giving a vector of remote code
execution (same with the first approach) but then execfile has the
issue that it goes into your current namespace possibly creating a
namespace crash which is even worst because an attacker can shallow
say your auth module with something that will just return.
>

[conf.py]
a = 1
b = 2
class c:
a = "hello"
b = "world"
[/end conf.py]

>

conf = imp.load_source ("conf", "./conf.py")
conf.a
>

>
1
>

conf.b
>

>
2
>

conf.c.a
>

>
'hello'
>

conf.c.b
>

>
'world'

There are so many ways potential solutions to your problem that,
without any more details, it is hard to suggest anything.

Here are some potential solutions:

ConfigParser - module for handling ini files
xml - several built-in modules for handling XML files
sqlite3 - a `lite' SQL database built-in in python 2.5 + (can be used
for config data)
windows registry - _winreg module
pickle - serialize python objects
marshal - similar to pickle, only works for simple objects

Those are just the built-in solutions. If you wanna look at 3rd party
solutions, prepare for overload. The number of alternative INI parsers
alone is staggering.

Also, there are many ways to organize your data and use a solution
similar to what you are already using.

I guess what I'm trying to say is... don't roll your own, it would be
a waste of time, this problem has been solved 100s of times. That is,
unless you want to do it for fun.

Matt
--
http://mail.python.org/mailman/listinfo/python-list

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, May 06 2008)
>>Python/Zope Consulting and Support ... http://www.egenix.com/
>>mxODBC.Zope.D atabase.Adapter ... http://zope.egenix.com/
>>mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
_______________ _______________ _______________ _______________ ____________

:::: Try mxODBC.Zope.DA for Windows,Linux,S olaris,MacOSX for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
--
http://mail.python.org/mailman/listinfo/python-list
Jun 27 '08 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

13
3017
by: Maxim Khesin | last post by:
I want to have a config file with my python proggie, satisfying the following requirements: 1) support key->(value, default) 2) simple and intuitive to read and edit 3) easyly readable into a python datastructure (a dictionary?) 4) not requiring any heavy libraries needed (I am distributing my proggie as a py2exe executable and do not want to bloat the size) can you guys suggest some format for this? thanks, max
3
3681
by: Peter Maas | last post by:
Hi, currently I'm trying to create a pgsql backend for the roundup issue tracker using the mysql backend as a template (is somebody aware of such a thing? I couldn't find one). The author has written a config.py as many authors do. From the programmer's perspective this is a satis- factory solution: he can use his favourite language and put complex structures into the configuration. From the sysadmin's perspective this is not so...
0
1044
by: Dan Gass | last post by:
config-py version 1.1 has been released and is available at SourceForge. Highlights of the changes are: 1) License changed from GPL to MIT (less restrictive) 2) Settings available as attributes of configuration instances 3) Allow user exact control of when environment keys are utilized 4) Exception class was added 5) Other (backwards compatible) interface and documentation improvements
4
3824
by: Fuzzyman | last post by:
There have been a couple of config file 'systems' announced recently, that focus on building more powerful and complex configuration files. ConfigObj is a module to enable you to much more *simply* access config files. This is version 3, which is a big overhaul. It extends ConfigObj to reading config files with sections and various other simplifications. I find ConfigObj extremely easy to use and use it for reading config files and data...
5
1642
by: Tom Willis | last post by:
How are the expert pythoneers dealing with config files? Is there anything similair to .net's config files or java's .properties? A quick search on google didn't return anything that looked useful, and I almost would expect to see some module that would be for dealing with config information. I can think of at least one way to do it, but I'm sure there are shortcomings I can't see yet, and I'd rather use something someone
13
1531
by: gaudetteje | last post by:
Hi All, I've been trying to come up with an elegant solution to this problem, but can't seem to think of anything better than my solution below. I have a Python program that needs to be converted into an executable. The problem is that I have a "config" template file that I've been using to modify initialization constants such as paths, global variables, structures, etc. Obviously, once I convert my code into an executable I can no...
5
2631
by: qqcq6s59 | last post by:
Hi all I am a newbie and I just saw a ongoing thread on Fileprocessing which talks abt config parser. I have writen many pyhton program to parse many kind of text files by using string module and regex. But after reading that config parser thread I feel stunned. Can somebody tell me some intro info how to parse huge data (most of them are input data to application softwares like nastran, abaqus etc)
20
2616
by: tomerfiliba | last post by:
hey i've been seeing lots of config-file-readers for python. be it ConfigObj (http://www.voidspace.org.uk/python/configobj.html) or the like. seems like a trend to me. i came to this conclusion a long time ago: YOU DON'T NEED CONFIG FILES FOR PYTHON. why re-invent stuff and parse text by yourself, why the interpreter can do it for you? and anyway, i find this a very ugly format:...
21
213
by: Nick Craig-Wood | last post by:
Lance Gamet <lance@gamet.comwrote: I've found http://docs.python.org/lib/module-ConfigParser.html To be easy to use and built in. It makes human readable / editable ..ini - like files. As for where to store it, I use os.path.expanduser("~") to find the
0
9184
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9042
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8929
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8891
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7759
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6538
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5878
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4380
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
3
2013
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.