Choosing log file destination in logging configuration file

[guybenron]

Hey,

I have a sort of petty-neurotic question, I'm kinda pedantic and like
to keep my log directories clean, and this thing is bothering me to
the point of actually posting a question (couldn't find any post about
this..).

My application has two types of components: daemons and regular ones
that are run every on crontab. Because the log structure is very
similar, I want both to use the same logging configuration file.

For clarity, I want all of the files to have the date as part of the
filename.
The TimedRotatingFileHandler has it built in (once it rotates), and I
found a dirty hack to make it work for the FileHandler (see shameful
code below). Basically I'm all set - have the daemons use the rotating
handler, since they're, um, daemonic and on all the time, and have the
regular components use the regular handler, using the dirty hack to
insert the date into the filename.

So far so good. In the relevant applications, the code looks something
like this:
logging.config.fileConfig('log.ini')
logger = logging.getLogger('log.regular') <orlogger =
logging.getLogger('log.daemonic')
... and start logging.

The thorn in my side is that after the fileConfig call, BOTH handlers
are instantiated, meaning both types of files are created for every
component, even if I don't need it: component.log (for the rotating
handler) and compenent.date.log (for the regular file handler).

...So finally, here's my question:
Apart from splitting the logging configuration into two separate
files, is there any way to NOT create the file until you actually use
it?

Here's the logging configuration file for reference.. Thanks !

[loggers]
keys=root,regular,daemonic

[handlers]
keys=fileHandler,consoleHandler,timedRotatingFileH andler

[formatters]
keys=simpleFormatter

[logger_root]
level=DEBUG
handlers=consoleHandler

[logger_regular]
level=INFO
handlers=fileHandler,consoleHandler
propagate=0
qualname=log.regular

[logger_daemonic]
level=INFO
handlers=timedRotatingFileHandler,consoleHandler
propagate=0
qualname=log.daemonic

[handler_timedRotatingFileHandler]
class=handlers.TimedRotatingFileHandler
level=INFO
formatter=simpleFormatter
args=('mylog.log', 'midnight')

[handler_fileHandler]
class=FileHandler
level=INFO
formatter=simpleFormatter
; sorry about the grossness
args=('mylog.' + str(time.localtime().tm_year) + '_' +
str(time.localtime().tm_mon).zfill(2) + '_' +
str(time.localtime().tm_mday).zfill(2) + '.log', 'a')

[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout,)

[formatter_simpleFormatter]
format=%(asctime)s - %(filename)s - %(name)s - %(levelname)s - %
(message)s

[Miki]

Hello,

So far so good. In the relevant applications, the code looks something
like this:
logging.config.fileConfig('log.ini')
logger = logging.getLogger('log.regular') <orlogger =
logging.getLogger('log.daemonic')
.. and start logging.

The thorn in my side is that after the fileConfig call, BOTH handlers
are instantiated, meaning both types of files are created for every
component, even if I don't need it: component.log (for the rotating
handler) and compenent.date.log (for the regular file handler).

..So finally, here's my question:
Apart from splitting the logging configuration into two separate
files, is there any way to NOT create the file until you actually use
it?

You can generate the .ini file on the fly and then load it:

>>log_ini = gen_log_ini(type) # type is either "deamon" or "regular"
atexit.register(lambda: remove(log_ini))
logging.config.fileConfig(log_ini)

HTH,
--
Miki <mi*********@gmail.com>
http://pythonwise.blogspot.com

[guybenron]

On Apr 22, 12:57 pm, Miki <[e-mail]:

Hello,

So far so good. In the relevant applications, the code looks something
like this:
logging.config.fileConfig('log.ini')
logger = logging.getLogger('log.regular') <orlogger =
logging.getLogger('log.daemonic')
.. and start logging.

The thorn in my side is that after the fileConfig call, BOTH handlers
are instantiated, meaning both types of files are created for every
component, even if I don't need it: component.log (for the rotating
handler) and compenent.date.log (for the regular file handler).

..So finally, here's my question:
Apart from splitting the logging configuration into two separate
files, is there any way to NOT create the file until you actually use
it?

You can generate the .ini file on the fly and then load it:

>log_ini = gen_log_ini(type) # type is either "deamon" or "regular"
atexit.register(lambda: remove(log_ini))
logging.config.fileConfig(log_ini)

HTH,
--
Miki <[e-mail]>http://pythonwise.blogspot.com

I think it misses the point of having a file to config..
It looks as if there's no solution. A peek at the logging module shows
this:
klass = cp.get(sectname, "class")
...
klass = eval(klass, vars(logging))
args = cp.get(sectname, "args")
args = eval(args, vars(logging))
h = apply(klass, args)

Bah. I'll have to split them into two configuration files (or subclass
and have the respective file and rotating handlers lazy evaluate until
the actual log call is made).
Thanks though.

[Vinay Sajip]

On Apr 22, 9:48 pm, [e-mail] wrote:

On Apr 22, 12:57 pm, Miki <[e-mail]:

Hello,

So far so good. In the relevant applications, the code looks something
like this:
>logging.config.fileConfig('log.ini')
logger =logging.getLogger('log.regular') <orlogger =
>logging.getLogger('log.daemonic')
.. and startlogging.

The thorn in my side is that after the fileConfig call, BOTH handlers
are instantiated, meaning both types of files are created for every
component, even if I don't need it: component.log (for the rotating
handler) and compenent.date.log (for the regular file handler).

..So finally, here's my question:
Apart from splitting theloggingconfiguration into two separate
files, is there any way to NOT create the file until you actually use
it?

You can generate the .ini file on the fly and then load it:

>>log_ini = gen_log_ini(type) # type is either "deamon" or "regular"
>>atexit.register(lambda: remove(log_ini))
>>>logging.config.fileConfig(log_ini)

HTH,
--
Miki <[e-mail]>http://pythonwise.blogspot.com

I think it misses the point of having a file to config..
It looks as if there's no solution. A peek at theloggingmodule shows
this:
klass = cp.get(sectname, "class")
...
klass = eval(klass, vars(logging))
args = cp.get(sectname, "args")
args = eval(args, vars(logging))
h = apply(klass, args)

Bah. I'll have to split them into two configuration files (or subclass
and have the respective file and rotating handlers lazy evaluate until
the actual log call is made).
Thanks though.

Not sure if it's any help - the SVN version contains a new optional
"delay" parameter for FileHandler and subclasses (including the
rotating handlers) which delays opening the file until there's a need
to, i.e. when an emit() call occurs.

Regards,

Vinay Sajip