473,695 Members | 1,998 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Passing options around your program

I have another question where I am not so much looking for a
solution but rather hoping to get some feedback on *which*
solutions people here consider good Pythonic ways to approach a
issue.

The situation is this: I am writing fairly large console scripts
in Python. They have quite a few command-line options, which lead
to configuration variables that are needed all over the program
(e.g. the "--verbose" option alone is used by just about every
function and method).

I am using optparse to handle the actual option parsing, so I
have a main function that contains:

opts,args = parser.parse_ar gs()

My question in a nutshell is: what is the best way to make the
parsed options in 'opts' easily available to the rest of my
program?

- Having 'opts' trickle down by passing it as an explicit
parameter is the most flexible option. It allows me to easily
separate the options from the command-line parsing -- if I want
to call my script interactively or from a GUI instead, I can just
craft my own 'opts' object and pass that.

But on the other hand, I feel that the proliferation of 'opts'
parameters gets ugly, and does not do wonders for the readability
of the program.

- So especially for shorter scripts, a solution I've used is to
define a global config class, and transfer the options to it:

class Config:
pass

With in main():

for opt in vars(opts):
setattr(Config, opt, getattr(opts, opt))

or:

Config.cmd_opts = opts

Now every method that needs it can just call Config.verbose or
Config.cmd_opts .verbose or whatever.

A variant I have also used occasionally is to derive Config from
a Martelli Borg parent class with shared-state instances, so that
every function or class that needs access to the config
parameters can just instantiate its own 'local' Config() object.
I find this works nicely if the program is spread out over more
than one file.

- A third option I have seen mentioned (possibly even here on
c.l.p.), but have not actually tried yet, is that if you decide
to have something 'global' anyway, why not just use a module
instead of a class to store the options, so that every file can
do "import config" and then say "config.verbose " etc.
So, does anybody have any particular positive or negative
opinions on any of these approaches, and/or are there perhaps
possibilities I have overlooked?

Many thanks in advance for your feedback.

--
Leo Breebaart <le*@lspace.org >
May 7 '06 #1
7 1539
When I have done this, I've used the last option -- creating a "config"
module which can be imported wherever the configuration info is needed.
(This module also handles any config files and/or Windows registry
keys needed.) Using a Borg is a nice idea... but that ConfigBorg is
probably going to be in its own module, which will need to be imported
anyhow... so why not let sys.__import__( ) do your singleton-enforcement
for you?

Of course, I hesitate to claim anything that I do as being necessarily
related to "best practices", but what would Usenet be without swarms of
half-informed opinions? ;)

Jeff Shannon

May 7 '06 #2
Leo Breebaart wrote:
I have another question where I am not so much looking for a
solution but rather hoping to get some feedback on *which*
solutions people here consider good Pythonic ways to approach a
issue.

The situation is this: I am writing fairly large console scripts
in Python. They have quite a few command-line options, which lead
to configuration variables that are needed all over the program
(e.g. the "--verbose" option alone is used by just about every
function and method).


<SNIP>

One question I have is about the "--verbose" option.

If you're doing something that is equivalent to logging to <file |
console>, rather than continuing to pass the '--verbose' flag around,
why not just use the built-in logging facility to manage the extra
output?

By having the different modules call logging( <level>, "<message"> )
throughout your programs, you only need to set the initial level from
where you're checking options (whether from command line or
configuration file) rather than having to pass the '--verbose' option
around.

May 7 '06 #3
"alisonken1 " <al********@gma il.com> writes:
Leo Breebaart wrote:
I am writing fairly large console scripts in Python. They
have quite a few command-line options, which lead to
configuration variables that are needed all over the program
(e.g. the "--verbose" option alone is used by just about
every function and method).


<SNIP>

One question I have is about the "--verbose" option.

If you're doing something that is equivalent to logging to
<file | console>, rather than continuing to pass the
'--verbose' flag around, why not just use the built-in logging
facility to manage the extra output?


Good point. It's not a real conscious decision, and I have one or
two other projects lying around where I do in fact use logging. I
think the main reason why I am not using it by default is
because, when all is said and done, it still comes easier to me
to resort to guarded print statements then to set up and use the
logging machinery.

This may well be a false economy in the long run, but it is
nevertheless how I perceive it in day-to-day programming.

(Related to this is that while I found e.g. the optparse
documentation very clear and relevant to my needs, the logging
documentation and examples, while readable, just never seem to
relate to what I actually need. This is not a complaint -- just
a subjective observation.)

But you are right. Especially for the larger projects I really
should bite the bullet and start using the logging module.

--
Leo Breebaart <le*@lspace.org >
May 8 '06 #4
Leo Breebaart enlightened us with:
I think the main reason why I am not using it by default is because,
when all is said and done, it still comes easier to me to resort to
guarded print statements then to set up and use the logging
machinery.


The logging "machinery" isn't that huge nor is it difficult to set up:

=============== =============== =============== =====
import logging

logging.basicCo nfig()
log = logging.getLogg er('your.module .name')
=============== =============== =============== =====

and that's all there is to it.

Sybren
--
The problem with the world is stupidity. Not saying there should be a
capital punishment for stupidity, but why don't we just take the
safety labels off of everything and let the problem solve itself?
Frank Zappa
May 8 '06 #5
Sybren Stuvel <sy*******@YOUR thirdtower.com. imagination> writes:
Leo Breebaart enlightened us with:

I think the main reason why I am not using [logging] by
default is because, when all is said and done, it still comes
easier to me to resort to guarded print statements then to
set up and use the logging machinery.


The logging "machinery" isn't that huge nor is it difficult to set up:

=============== =============== =============== =====
import logging

logging.basicCo nfig()
log = logging.getLogg er('your.module .name')
=============== =============== =============== =====

and that's all there is to it.


I did say: "it comes easier *to me*" -- I am aware that my own
tastes, which may not be shared by others, or be entirely
logical, feature into this.

Having said that, I do think it's also not *quite* as trivial as
you claim. In order to duplicate what I already have ("if config.verbose:
print msg"), I probably need to log at the DEBUG level, and I
need to change the format so that that output looks like the
'print' output (no timestamping cruft etc.)

Okay, you say, that's still easy. It's just:

logging.basicCo nfig(level=logg ing.DEBUG,
format='%(messa ge)s')

logging.debug(' A debug message')

And that's true, but I did have to go and look this up, and I
made a couple of false starts first. I can only repeat that the
workings of the logging framework are rarely intuitive to me.

Also, assume that I have set it up as above. Now I want certain
other print statements to go to sys.stderr alone. If I understand
the docs correctly (possibly I don't), the way to do this is to
start attaching explicit StreamHandlers and whatnot. Whereas with
print, I can say "print >>sys.stderr, msg".

I know that it won't take much for a script to become so complex
that the extra stuff logging buys you vastly overpowers the
convenience of the print statements. I have just not needed that
often enough for it to become second nature to use in smaller
scripts as well.

--
Leo Breebaart <le*@lspace.org >
May 8 '06 #6
Leo Breebaart enlightened us with:
Okay, you say, that's still easy. It's just:

logging.basicCo nfig(level=logg ing.DEBUG,
format='%(messa ge)s')
I always use a separate logger, as per my example. That would then
just require an additional line:

log.setLeveL(lo gging.DEBUG)
And that's true, but I did have to go and look this up, and I made a
couple of false starts first. I can only repeat that the workings of
the logging framework are rarely intuitive to me.
If you're just writing software for yourself, that's fine. If you want
to share your software, things that are intuitive to you might not be
to others. In such a case (and usually software is going to be shared
at some point or another) it's generally a good idea to spend a few
minutes learning how to use such a system, because then other people
will see your code using things they are already familiar with.

Reusing other people's code instead of writing your own logging, also
means that other people will fix bugs for you.
I know that it won't take much for a script to become so complex
that the extra stuff logging buys you vastly overpowers the
convenience of the print statements. I have just not needed that
often enough for it to become second nature to use in smaller
scripts as well.


I don't use the logging module in small scripts either, but as soon as
it gets over 100 lines, I do. It's quite easy to get it going, once
you get the hang of it.

Sybren
--
The problem with the world is stupidity. Not saying there should be a
capital punishment for stupidity, but why don't we just take the
safety labels off of everything and let the problem solve itself?
Frank Zappa
May 8 '06 #7

Leo Breebaart wrote:

<SNIP>
Also, assume that I have set it up as above. Now I want certain
other print statements to go to sys.stderr alone. If I understand
the docs correctly (possibly I don't), the way to do this is to
start attaching explicit StreamHandlers and whatnot. Whereas with
print, I can say "print >>sys.stderr, msg".

<SNIP>

Something else to consider, every time your script calls "print
sys.stderr, msg", it is making another file open request.


I've run across situations where this format will actually cause a
system to return "too many open files" error.

As was pointed out earlier, it's much easier to learn how to use the
logging facility and create a default stdout logger as well as a
secondary stderr logger that only maintains one file handle than to try
to find out where you're getting some errors that are not script errors
but system limit errors.

The other nice aspect of using the logging facility is the availability
of changing the logger to save output to a file or stdout/stderr than
to try and go through a more-than-one-file program.

It's easy to learn how to redirect stdout/stderr within a script, but
it's usually more flexible to use the logging facility that's already
been included. Before the logging module was included in the library, I
usually ended up writing my own logging module to do the same thing.
Especially since the logger allows you to specify levels.

For example, I typically have my logging facility setup as follows:

10 - Very basic logging - typically main routine changes
20 - Log imports
30 - Log class instantiation
40 - Log module calls
50 - Log function calls

The other aspect of using the logging facility, you can also define
your own in-between log levels:

51 - Entering function
52 - Exiting function
53 - Function sub-routines
60 - Everyhing under the sun

As part of the configParser options:
-v : Increment logging level by 1 level for every -v option on command
line
-loglevel=<level > : Specify log level
-logfile=<filena me> : File to save stdout messages
-errfile=<filena me> : File to save stderr messages

Once you get used to the logging module, it's hard to go back to using
file redirects and the <sometimes> system limits troubleshooting in
larger programs.

May 8 '06 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
2166
by: Chris | last post by:
I'm trying to come up with a not-so-ugly manner of passing many command-line options between modules. I have a Steering.py file, which contains my main() and handles the getopts getting of several command-line options. Since having 5-6 additional parameters for every class instantiation or function call is obviously an ugly way to handle this, what would you folks recommend as a superior option? Maybe passing a dict of options to each...
6
2295
by: Garma | last post by:
According to what I have learnt so far, instantiating global objects should be the last resort. Is there any reasons why so? Sometimes some objects or their pointers have to be shared among several other objects, I'd like to know how this is handled commonly. Could someone elaborate on this? Another situation is some objects could be shared among several threads. How to handle it commonly?
58
10138
by: jr | last post by:
Sorry for this very dumb question, but I've clearly got a long way to go! Can someone please help me pass an array into a function. Here's a starting point. void TheMainFunc() { // Body of code... TCHAR myArray; DoStuff(myArray);
3
4754
by: Simon Harvey | last post by:
Hi, In my application I get lots of different sorts of information from databases. As such, a lot of information is stored in DataSets and DataTable objects. Up until now, I have been passing around chunks of data in DataTables/DataSets, simply because that was the format that they were in when the data was taken from the database. Now, I know this maybe a pretty silly question with a standard "it depends" answer, but I'm going to...
3
1567
by: Lee | last post by:
Hi All How can I pass options from one webpage into another webpage. When the user clicks on the hyperlink I want them to be go to the next page but I need to pass in a number that the next page needs to reference. Thanks Lee
28
3121
by: Skeets | last post by:
i'm passing session and hidden variables between pages. not to mention post values. i'm a little concerned that someone with sufficient knowledge could spoof these vlaues and manipulate the program. is this a valid concern? i'm thinking i can check the submitting page setting up something around the following the following code... $base_name = basename($_SERVER);
5
3016
by: Chris Hieronymus | last post by:
Hi, I have a bunch of x-y data contained in an array. I would like to plot the data using an external program (psxy in GMT). The plotting program takes x-y couples as standard input. How do I get the data into the system call? I used to do things in csh and awk, i.e., something like
5
1907
by: Markus Ernst | last post by:
Hello A class that composes the output of shop-related data gets some info from the main shop class. Now I wonder whether it is faster to store the info in the output class or get it from the main class whenever it is needed: class shop_main { var $prices = null; function &get_prices() {
1
1654
by: Harold Fellermann | last post by:
Dear list, I looked through the list but could not find any solutions for my current problem. Within my program, I am importing a module via __import__(module_name,globals(),locals()) and I want to pass comand line options to this module. I would prefer not to save them in a config module or a Config class but rather use the globals and locals dictionaries
0
8630
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8568
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9115
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
1
8845
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8825
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
6491
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
4340
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
2
2272
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
1976
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.