473,382 Members | 1,703 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,382 software developers and data experts.

Problem with logging module

I have a logger class that uses the Python logging module. When I
call it within a program using the unittest module, I get one line in
the log file for the first test, two identical ones for the second,
etc. I'm using local variables, which should go out of context with
each test. Setting the 'propagate' property to False doesn't have any
affect.

import logging, os, signal, sys
from datetime import datetime

class Logger:
def __init__(self, dir, name):
self.dir = dir
self.name = name

self.logger = logging.getLogger(name)
self.logger.propagate = False
handler = logging.FileHandler(dir + '/' + name + '.log')
self.logger.addHandler(handler)
self.logger.setLevel(logging.INFO)

def log(self, text):
msg = datetime.utcnow().isoformat(' ') + ' ' + sys.argv[0] + ' -- '
+ text
self.logger.info(msg)

import os, sys, unittest
import log

class Test(unittest.TestCase):

def test_1(self):
homedir = sys.path[0]
logger = log.Logger(homedir, "test")
logger.log("Test 1")

def test_2(self):
homedir = sys.path[0]
logger = log.Logger(homedir, "test")
logger.log("Test 2")

def test_3(self):
homedir = sys.path[0]
logger = log.Logger(homedir, "test")
logger.log("Test 3")

def test_4(self):
homedir = sys.path[0]
logger = log.Logger(homedir, "test")
logger.log("Test 4")

def test_5(self):
homedir = sys.path[0]
logger = log.Logger(homedir, "test")
logger.log("Test 5")

if __name__ == '__main__':
unittest.main()

And the output file looks like:

2004-10-13 15:54:08.321112 test.py -- Test 1
2004-10-13 15:54:08.326594 test.py -- Test 2
2004-10-13 15:54:08.326594 test.py -- Test 2
2004-10-13 15:54:08.329742 test.py -- Test 3
2004-10-13 15:54:08.329742 test.py -- Test 3
2004-10-13 15:54:08.329742 test.py -- Test 3
2004-10-13 15:54:08.333411 test.py -- Test 4
2004-10-13 15:54:08.333411 test.py -- Test 4
2004-10-13 15:54:08.333411 test.py -- Test 4
2004-10-13 15:54:08.333411 test.py -- Test 4
2004-10-13 15:54:08.336997 test.py -- Test 5
2004-10-13 15:54:08.336997 test.py -- Test 5
2004-10-13 15:54:08.336997 test.py -- Test 5
2004-10-13 15:54:08.336997 test.py -- Test 5
2004-10-13 15:54:08.336997 test.py -- Test 5
Jul 18 '05 #1
8 1968
> self.logger = logging.getLogger(name)
self.logger.propagate = False
handler = logging.FileHandler(dir + '/' + name + '.log')
self.logger.addHandler(handler)

^^^^
Thats your problem - only set the handler once for the runtime of your
program.

--
Regards,

Diez B. Roggisch
Jul 18 '05 #2
Steve Erickson wrote:
I have a logger class that uses the Python logging module. When I
call it within a program using the unittest module, I get one line in
the log file for the first test, two identical ones for the second,
etc. I'm using local variables, which should go out of context with
each test. Setting the 'propagate' property to False doesn't have any
affect.

import logging, os, signal, sys
from datetime import datetime

class Logger:
def __init__(self, dir, name):
self.dir = dir
self.name = name

self.logger = logging.getLogger(name)
self.logger.propagate = False
handler = logging.FileHandler(dir + '/' + name + '.log')
self.logger.addHandler(handler)
self.logger.setLevel(logging.INFO)

def log(self, text):
msg = datetime.utcnow().isoformat(' ') + ' ' + sys.argv[0] + ' -- '
+ text
self.logger.info(msg)


As I understand it, logging uses internal (global/"static") data
structures to keep track of all instantiated loggers. Thus, each time
you create an instance of your logger class, it gets the (pre-existing)
logging.Logger instance named "test" and adds a *new* FileHandler to
it. Thus, you end up with an ever-growing list of (identical) handlers
on the same logging.Logger instance.

You can fix this either by using a global log.Logger object, or by
having a separate initialize() method for it that's called only once and
which adds the handler and sets the logging level.

By the way, you can also set a formatter on the handler which will
automatically include the time (and probably sys.argv[0] as well). With
that done, you could do away with your class. Just initialize the
logger once, and then in each test case, you can do

logging.getLogger("test").info(text)

The logging module will keep track of all of the handler and formatting
details for you.

Jeff Shannon
Technician/Programmer
Credit International

Jul 18 '05 #3
"Diez B. Roggisch" <de*********@web.de> wrote in message news:<ck*************@news.t-online.com>...
self.logger = logging.getLogger(name)
self.logger.propagate = False
handler = logging.FileHandler(dir + '/' + name + '.log')
self.logger.addHandler(handler)

^^^^
Thats your problem - only set the handler once for the runtime of your
program.

Thanks. Coming from C++, I thought that the local logger instance in
each test method would go away when I went out of the context of the
method. Sounds like the logging class in Python maintains artifacts
that affect instantiations in other methods. Or maybe I'm confused
about how Python handles local versus class variables.
Jul 18 '05 #4
Steve Erickson wrote:
Thanks. Coming from C++, I thought that the local logger instance in
each test method would go away when I went out of the context of the
method. Sounds like the logging class in Python maintains artifacts
that affect instantiations in other methods. Or maybe I'm confused
about how Python handles local versus class variables.


Nothing to do with python - its the way logging is designed, and it makes
sense: if at one place you attach the logger named foo.bar to a handler,
and at another place obtain a reference to foo.bar to log some stuff, you
don't want to reiterate the whole handler-attaching stuff.

So you simply abused the logging module - just set up the logger once, and
then obtain a reference to it using logging.getLogger whenever you need it
- that totally rids you of your logger class, that so far doesn't do much
anyway. Like this:

------
logger = logging.getLogger(name)
logger.propagate = False
handler = logging.FileHandler(dir + '/' + name + '.log')
logger.addHandler(handler)

class Test:

def test_me(self):
logger = logging.getLogger(name)
logger.debug("test_me")

------
--
Regards,

Diez B. Roggisch
Jul 18 '05 #5
Thanks a bunch for your patience and explanations. It would be nice
if there was a method of checking the logger or handler for an
instance of it:

logger = logging.getLogger(name)
if not logger.handler(dir + '/' + name + '.log'):
logger.propagate = False
handler = logging.FileHandler(dir + '/' + name + '.log')
logger.addHandler(handler)

"Diez B. Roggisch" <de*********@web.de> wrote in message news:<ck*************@news.t-online.com>...
Steve Erickson wrote:
Thanks. Coming from C++, I thought that the local logger instance in
each test method would go away when I went out of the context of the
method. Sounds like the logging class in Python maintains artifacts
that affect instantiations in other methods. Or maybe I'm confused
about how Python handles local versus class variables.


Nothing to do with python - its the way logging is designed, and it makes
sense: if at one place you attach the logger named foo.bar to a handler,
and at another place obtain a reference to foo.bar to log some stuff, you
don't want to reiterate the whole handler-attaching stuff.

So you simply abused the logging module - just set up the logger once, and
then obtain a reference to it using logging.getLogger whenever you need it
- that totally rids you of your logger class, that so far doesn't do much
anyway. Like this:

------
logger = logging.getLogger(name)
logger.propagate = False
handler = logging.FileHandler(dir + '/' + name + '.log')
logger.addHandler(handler)

class Test:

def test_me(self):
logger = logging.getLogger(name)
logger.debug("test_me")

------

Jul 18 '05 #6
Steve Erickson wrote:
Thanks a bunch for your patience and explanations. It would be nice
if there was a method of checking the logger or handler for an
instance of it:

logger = logging.getLogger(name)
if not logger.handler(dir + '/' + name + '.log'):
logger.propagate = False
handler = logging.FileHandler(dir + '/' + name + '.log')
logger.addHandler(handler)


There is the (unfortunately undocumented) property "handlers" on a Logger
insnstance that you can use:

if not fname in [h.stream.name for h in logger.handlers]:
...

--
Regards,

Diez B. Roggisch
Jul 18 '05 #7
[Steve]
if not logger.handler(dir + '/' + name + '.log'):
[Diez] There is the (unfortunately undocumented) property "handlers" on a Logger
insnstance that you can use:


The handlers property is not documented because it is not normally
meant to be used by clients of the logging package. Since handlers can
include third party handlers, the answer to 'what is an existing
handler?' is not in general known to the logging package. This is why
something like the functionality suggested by Steve is not provided in
the box.

Regards,
Vinay Sajip
Jul 18 '05 #8
"Diez B. Roggisch" <de*********@web.de> wrote in message news:<ck*************@news.t-online.com>...
Steve Erickson wrote:
Thanks a bunch for your patience and explanations. It would be nice
if there was a method of checking the logger or handler for an
instance of it:

logger = logging.getLogger(name)
if not logger.handler(dir + '/' + name + '.log'):
logger.propagate = False
handler = logging.FileHandler(dir + '/' + name + '.log')
logger.addHandler(handler)


There is the (unfortunately undocumented) property "handlers" on a Logger
insnstance that you can use:

if not fname in [h.stream.name for h in logger.handlers]:
...


Excellent!! Thanks.
Jul 18 '05 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
by: Ville Vainio | last post by:
Just posting this for the sake of google: Like everyone else, I figured it's time to start using the 'logging' module. I typically want to dump "info" level (and up) log information to...
2
by: rh0dium | last post by:
Hi all, So I have a slice of code which calls other python code. I have started to take a real liking to the logging module, but I want to extend this into the called python code. I have no...
1
by: Maksim Kasimov | last post by:
hello in my modules, I'm using logging module, doing thus (there is a few modules): in module1.py: hdl = logging.StreamHandler() fmt =...
23
by: Rotem | last post by:
Hi, while working on something in my current project I have made several improvements to the logging package in Python, two of them are worth mentioning: 1. addition of a logging record field...
0
by: robert | last post by:
As more and more python packages are starting to use the bloomy (Java-ish) 'logging' module in a mood of responsibility and as I am not overly happy with the current "thickener" style of usage, I...
5
by: Ritesh Raj Sarraf | last post by:
import os, sys, logging logger = logging.getLogger("my_app") conerr = logging.StreamHandler(sys.stderr) conerr.setLevel(logging.DEBUG) conerr_formatter = logging.Formatter('%(levelname)s...
12
by: Tekkaman | last post by:
I'm getting a strange behaviour from the "pathname" and "lineno" formatter mapping keys. Instead of my file and my line number I get: /usr/lib/python2.4/logging/__init__.py as the file, and...
0
by: Karlo Lozovina | last post by:
I've just upgraded to Python 2.5, SQLAlchemy 0.3.3, and py2exe 0.6.5 (the py2.5 version, yes). Simple: --- import sqlalchemy print 'Test' ---
3
by: Lowell Alleman | last post by:
Here is the situation: I wrote my own log handler class (derived from logging.Handler) and I want to be able to use it from a logging config file, that is, a config file loaded with the...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.