473,727 Members | 1,989 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Problem with logging module

I have a logger class that uses the Python logging module. When I
call it within a program using the unittest module, I get one line in
the log file for the first test, two identical ones for the second,
etc. I'm using local variables, which should go out of context with
each test. Setting the 'propagate' property to False doesn't have any
affect.

import logging, os, signal, sys
from datetime import datetime

class Logger:
def __init__(self, dir, name):
self.dir = dir
self.name = name

self.logger = logging.getLogg er(name)
self.logger.pro pagate = False
handler = logging.FileHan dler(dir + '/' + name + '.log')
self.logger.add Handler(handler )
self.logger.set Level(logging.I NFO)

def log(self, text):
msg = datetime.utcnow ().isoformat(' ') + ' ' + sys.argv[0] + ' -- '
+ text
self.logger.inf o(msg)

import os, sys, unittest
import log

class Test(unittest.T estCase):

def test_1(self):
homedir = sys.path[0]
logger = log.Logger(home dir, "test")
logger.log("Tes t 1")

def test_2(self):
homedir = sys.path[0]
logger = log.Logger(home dir, "test")
logger.log("Tes t 2")

def test_3(self):
homedir = sys.path[0]
logger = log.Logger(home dir, "test")
logger.log("Tes t 3")

def test_4(self):
homedir = sys.path[0]
logger = log.Logger(home dir, "test")
logger.log("Tes t 4")

def test_5(self):
homedir = sys.path[0]
logger = log.Logger(home dir, "test")
logger.log("Tes t 5")

if __name__ == '__main__':
unittest.main()

And the output file looks like:

2004-10-13 15:54:08.321112 test.py -- Test 1
2004-10-13 15:54:08.326594 test.py -- Test 2
2004-10-13 15:54:08.326594 test.py -- Test 2
2004-10-13 15:54:08.329742 test.py -- Test 3
2004-10-13 15:54:08.329742 test.py -- Test 3
2004-10-13 15:54:08.329742 test.py -- Test 3
2004-10-13 15:54:08.333411 test.py -- Test 4
2004-10-13 15:54:08.333411 test.py -- Test 4
2004-10-13 15:54:08.333411 test.py -- Test 4
2004-10-13 15:54:08.333411 test.py -- Test 4
2004-10-13 15:54:08.336997 test.py -- Test 5
2004-10-13 15:54:08.336997 test.py -- Test 5
2004-10-13 15:54:08.336997 test.py -- Test 5
2004-10-13 15:54:08.336997 test.py -- Test 5
2004-10-13 15:54:08.336997 test.py -- Test 5
Jul 18 '05 #1
8 1997
> self.logger = logging.getLogg er(name)
self.logger.pro pagate = False
handler = logging.FileHan dler(dir + '/' + name + '.log')
self.logger.add Handler(handler )

^^^^
Thats your problem - only set the handler once for the runtime of your
program.

--
Regards,

Diez B. Roggisch
Jul 18 '05 #2
Steve Erickson wrote:
I have a logger class that uses the Python logging module. When I
call it within a program using the unittest module, I get one line in
the log file for the first test, two identical ones for the second,
etc. I'm using local variables, which should go out of context with
each test. Setting the 'propagate' property to False doesn't have any
affect.

import logging, os, signal, sys
from datetime import datetime

class Logger:
def __init__(self, dir, name):
self.dir = dir
self.name = name

self.logger = logging.getLogg er(name)
self.logger.pro pagate = False
handler = logging.FileHan dler(dir + '/' + name + '.log')
self.logger.add Handler(handler )
self.logger.set Level(logging.I NFO)

def log(self, text):
msg = datetime.utcnow ().isoformat(' ') + ' ' + sys.argv[0] + ' -- '
+ text
self.logger.inf o(msg)


As I understand it, logging uses internal (global/"static") data
structures to keep track of all instantiated loggers. Thus, each time
you create an instance of your logger class, it gets the (pre-existing)
logging.Logger instance named "test" and adds a *new* FileHandler to
it. Thus, you end up with an ever-growing list of (identical) handlers
on the same logging.Logger instance.

You can fix this either by using a global log.Logger object, or by
having a separate initialize() method for it that's called only once and
which adds the handler and sets the logging level.

By the way, you can also set a formatter on the handler which will
automatically include the time (and probably sys.argv[0] as well). With
that done, you could do away with your class. Just initialize the
logger once, and then in each test case, you can do

logging.getLogg er("test").info (text)

The logging module will keep track of all of the handler and formatting
details for you.

Jeff Shannon
Technician/Programmer
Credit International

Jul 18 '05 #3
"Diez B. Roggisch" <de*********@we b.de> wrote in message news:<ck******* ******@news.t-online.com>...
self.logger = logging.getLogg er(name)
self.logger.pro pagate = False
handler = logging.FileHan dler(dir + '/' + name + '.log')
self.logger.add Handler(handler )

^^^^
Thats your problem - only set the handler once for the runtime of your
program.

Thanks. Coming from C++, I thought that the local logger instance in
each test method would go away when I went out of the context of the
method. Sounds like the logging class in Python maintains artifacts
that affect instantiations in other methods. Or maybe I'm confused
about how Python handles local versus class variables.
Jul 18 '05 #4
Steve Erickson wrote:
Thanks. Coming from C++, I thought that the local logger instance in
each test method would go away when I went out of the context of the
method. Sounds like the logging class in Python maintains artifacts
that affect instantiations in other methods. Or maybe I'm confused
about how Python handles local versus class variables.


Nothing to do with python - its the way logging is designed, and it makes
sense: if at one place you attach the logger named foo.bar to a handler,
and at another place obtain a reference to foo.bar to log some stuff, you
don't want to reiterate the whole handler-attaching stuff.

So you simply abused the logging module - just set up the logger once, and
then obtain a reference to it using logging.getLogg er whenever you need it
- that totally rids you of your logger class, that so far doesn't do much
anyway. Like this:

------
logger = logging.getLogg er(name)
logger.propagat e = False
handler = logging.FileHan dler(dir + '/' + name + '.log')
logger.addHandl er(handler)

class Test:

def test_me(self):
logger = logging.getLogg er(name)
logger.debug("t est_me")

------
--
Regards,

Diez B. Roggisch
Jul 18 '05 #5
Thanks a bunch for your patience and explanations. It would be nice
if there was a method of checking the logger or handler for an
instance of it:

logger = logging.getLogg er(name)
if not logger.handler( dir + '/' + name + '.log'):
logger.propagat e = False
handler = logging.FileHan dler(dir + '/' + name + '.log')
logger.addHandl er(handler)

"Diez B. Roggisch" <de*********@we b.de> wrote in message news:<ck******* ******@news.t-online.com>...
Steve Erickson wrote:
Thanks. Coming from C++, I thought that the local logger instance in
each test method would go away when I went out of the context of the
method. Sounds like the logging class in Python maintains artifacts
that affect instantiations in other methods. Or maybe I'm confused
about how Python handles local versus class variables.


Nothing to do with python - its the way logging is designed, and it makes
sense: if at one place you attach the logger named foo.bar to a handler,
and at another place obtain a reference to foo.bar to log some stuff, you
don't want to reiterate the whole handler-attaching stuff.

So you simply abused the logging module - just set up the logger once, and
then obtain a reference to it using logging.getLogg er whenever you need it
- that totally rids you of your logger class, that so far doesn't do much
anyway. Like this:

------
logger = logging.getLogg er(name)
logger.propagat e = False
handler = logging.FileHan dler(dir + '/' + name + '.log')
logger.addHandl er(handler)

class Test:

def test_me(self):
logger = logging.getLogg er(name)
logger.debug("t est_me")

------

Jul 18 '05 #6
Steve Erickson wrote:
Thanks a bunch for your patience and explanations. It would be nice
if there was a method of checking the logger or handler for an
instance of it:

logger = logging.getLogg er(name)
if not logger.handler( dir + '/' + name + '.log'):
logger.propagat e = False
handler = logging.FileHan dler(dir + '/' + name + '.log')
logger.addHandl er(handler)


There is the (unfortunately undocumented) property "handlers" on a Logger
insnstance that you can use:

if not fname in [h.stream.name for h in logger.handlers]:
...

--
Regards,

Diez B. Roggisch
Jul 18 '05 #7
[Steve]
if not logger.handler( dir + '/' + name + '.log'):
[Diez] There is the (unfortunately undocumented) property "handlers" on a Logger
insnstance that you can use:


The handlers property is not documented because it is not normally
meant to be used by clients of the logging package. Since handlers can
include third party handlers, the answer to 'what is an existing
handler?' is not in general known to the logging package. This is why
something like the functionality suggested by Steve is not provided in
the box.

Regards,
Vinay Sajip
Jul 18 '05 #8
"Diez B. Roggisch" <de*********@we b.de> wrote in message news:<ck******* ******@news.t-online.com>...
Steve Erickson wrote:
Thanks a bunch for your patience and explanations. It would be nice
if there was a method of checking the logger or handler for an
instance of it:

logger = logging.getLogg er(name)
if not logger.handler( dir + '/' + name + '.log'):
logger.propagat e = False
handler = logging.FileHan dler(dir + '/' + name + '.log')
logger.addHandl er(handler)


There is the (unfortunately undocumented) property "handlers" on a Logger
insnstance that you can use:

if not fname in [h.stream.name for h in logger.handlers]:
...


Excellent!! Thanks.
Jul 18 '05 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
10880
by: Ville Vainio | last post by:
Just posting this for the sake of google: Like everyone else, I figured it's time to start using the 'logging' module. I typically want to dump "info" level (and up) log information to screen, and "debug" level (and up) to a log file for in-depth analysis. This is for scripts, so date/time/severity information is not wanted. I assumed such a simple use case would have an example in the docs (py 2.4), but no luck.
2
1742
by: rh0dium | last post by:
Hi all, So I have a slice of code which calls other python code. I have started to take a real liking to the logging module, but I want to extend this into the called python code. I have no idea how to pass the handle from the calling code into the modules.. So basically here is what I do.. -- Main Program --
1
1611
by: Maksim Kasimov | last post by:
hello in my modules, I'm using logging module, doing thus (there is a few modules): in module1.py: hdl = logging.StreamHandler() fmt = logging.Formatter("%(name)s:\t%(levelname)s:\t%(asctime)s:\t%(message)s") hdl.setFormatter(fmt) log = logging.getLogger('module1')
23
2218
by: Rotem | last post by:
Hi, while working on something in my current project I have made several improvements to the logging package in Python, two of them are worth mentioning: 1. addition of a logging record field %(function)s, which results in the name of the entity which logged the record. My version even deduces the class name in the case which the logger is a bound method, and assuming the name of the "self" variable is indeed "self".
0
1862
by: robert | last post by:
As more and more python packages are starting to use the bloomy (Java-ish) 'logging' module in a mood of responsibility and as I am not overly happy with the current "thickener" style of usage, I want to put this comment and a alternative most simple default framework for discussion. Maybe there are more Python users which like to see that imported (managed) logging issue more down-to-earth and flexible ? ... Vinay Sajip wrote: >...
5
6081
by: Ritesh Raj Sarraf | last post by:
import os, sys, logging logger = logging.getLogger("my_app") conerr = logging.StreamHandler(sys.stderr) conerr.setLevel(logging.DEBUG) conerr_formatter = logging.Formatter('%(levelname)s %(message)s') conerr.setFormatter(conerr_formatter) console = logging.StreamHandler(sys.stdout)
12
1973
by: Tekkaman | last post by:
I'm getting a strange behaviour from the "pathname" and "lineno" formatter mapping keys. Instead of my file and my line number I get: /usr/lib/python2.4/logging/__init__.py as the file, and 1072 as the line number. I set up my config as follows: logBaseConf = { 'level' : logging.DEBUG,
0
1365
by: Karlo Lozovina | last post by:
I've just upgraded to Python 2.5, SQLAlchemy 0.3.3, and py2exe 0.6.5 (the py2.5 version, yes). Simple: --- import sqlalchemy print 'Test' ---
3
6423
by: Lowell Alleman | last post by:
Here is the situation: I wrote my own log handler class (derived from logging.Handler) and I want to be able to use it from a logging config file, that is, a config file loaded with the logging.config.fileConfig() function. Let say my logging class is called "MyLogHandler" and it's in a module called "mylogmodule", I want to be able to make an entry something like this in my logging config file:
0
8890
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8752
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9259
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
9120
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
8101
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6702
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6011
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4785
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3228
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.