473,545 Members | 1,739 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

selective logger disable/enable

Suppose I have 3 modules that belong to a project, 'A', 'A.a' and 'B',
and each module has its own logger, created with:

module1logger = logging.getLogg er('project.A')

and

module2logger = logging.getLogg er('project.A.a ')

and

module3logger = logging.getLogg er('project.B')
And I want to selectively enable/disable these, per module (for
example, I only want logging from A and A.a, or just from B, etc). It
seems like logging.Filter is supposed to let me do this, but I can't
see how to apply it globally. Is there really no other way that to add
a addFilter(filte r) call to each of these loggers individually?

logging.basicCo nfig gets inherited by all loggers, but it doesn't seem
capable of giving a Filter to all loggers. Is there any way to do this?

Jan 19 '07 #1
10 8067
Gary Jefferson wrote:
Suppose I have 3 modules that belong to a project, 'A', 'A.a' and 'B',
and each module has its own logger, created with:

module1logger = logging.getLogg er('project.A')

and

module2logger = logging.getLogg er('project.A.a ')

and

module3logger = logging.getLogg er('project.B')
And I want to selectively enable/disable these, per module (for
example, I only want logging from A and A.a, or just from B, etc). It
seems like logging.Filter is supposed to let me do this, but I can't
see how to apply it globally. Is there really no other way that to add
a addFilter(filte r) call to each of these loggers individually?
The documentation for Logger - see

http://docs.python.org/lib/node406.html

- shows that there are addFilter() and removeFilter() methods on the
Logger class which you can use to add or remove filters from individual
Logger instances. From the above page (entitled "14.5.1 Logger
Objects"):

addFilter(filt)
Adds the specified filter filt to this logger.

removeFilter(fi lt)
Removes the specified filter filt from this logger.

The parent section of Section 14.5.1, which is Section 14.5, was the
first search result when I just searched Google for "python logging".

Best regards,

Vinay Sajip

Jan 19 '07 #2

Vinay Sajip wrote:
>
The documentation for Logger - see

http://docs.python.org/lib/node406.html

- shows that there are addFilter() and removeFilter() methods on the
Logger class which you can use to add or remove filters from individual
Logger instances. From the above page (entitled "14.5.1 Logger
Objects"):

addFilter(filt)
Adds the specified filter filt to this logger.

removeFilter(fi lt)
Removes the specified filter filt from this logger.

The parent section of Section 14.5.1, which is Section 14.5, was the
first search result when I just searched Google for "python logging".

Thanks for the reply Vinay. I had been reading those docs prior to
posting, but addFilter/removeFilter only work on individual logger
instances and handlers. I want to globally enable/disable multiple
logger instances, based on their namespaces (see the example I
provided). I cannot find a way to do this. I can certainly call
addFilter on every instance of every logger throughout the source code,
but this is more than inconvenient -- it breaks when new modules are
added with their own loggers in their own namespaces (until fixed by
manually adding addFilter() to those instances).

e.g., suppose you have a source tree with a several components that do
network access, and several components that do other things like saving
state to disk, accessing the filesystem, etc. And suppose you find a
bug that you think is in the networking code. It would be nice to be
able to GLOBALLY enable just the loggers that belong in the networking
namespace, and disable all others (similar to how Filters are supposed
to work for individual loggers). And then to be able to do this with
any component, by providing, for example, a command line switch or
environment variable. Otherwise, the poor programmer is forced to go
and edit every module in the source tree to selectively turn on/off
their respecitve loggers. Or am I missing something really obvious
about how this is done with the logging module?

thanks,
Gary

Jan 21 '07 #3
Gary Jefferson wrote:
Thanks for the reply Vinay. I had been reading those docs prior to
posting, but addFilter/removeFilter only work on individual logger
instances and handlers. I want to globally enable/disable multiple
logger instances, based on their namespaces (see the example I
provided). I cannot find a way to do this. I can certainly call
addFilter on every instance of every logger throughout the source code,
but this is more than inconvenient -- it breaks when new modules are
added with their own loggers in their own namespaces (until fixed by
manually adding addFilter() to those instances).

e.g., suppose you have a source tree with a several components that do
network access, and several components that do other things like saving
state to disk, accessing the filesystem, etc. And suppose you find a
bug that you think is in the networking code. It would be nice to be
able to GLOBALLY enable just the loggers that belong in the networking
namespace, and disable all others (similar to how Filters are supposed
to work for individual loggers). And then to be able to do this with
any component, by providing, for example, a command line switch or
environment variable. Otherwise, the poor programmer is forced to go
and edit every module in the source tree to selectively turn on/off
their respecitve loggers. Or am I missing something really obvious
about how this is done with the logging module?
I don't know enough about your target environment and application to
necessarily give you the best advice, but I'll make some general
comments which I hope are useful. You seem to be thinking that loggers
are binary - i.e. you turn them on or off. But they can be controlled
more finely than that; you should be able to get the effect that you
want by using the different logging levels available judiciously, as
well as using the fact that loggers inhabit a named hierarchy. Note
that loggers, by default, inherit the level of the first ancestor
logger which has an explicitly set level (by ancestor, I mean in the
name hierarchy). The root logger's default level is WARNING, so by
default all loggers will work at this level. [N.B. You can also set
levels for individual handlers, e.g. to ensure that only CRITICAL
conditions are emailed to a specified email address using SMTPHandler,
or that ERROR conditions and above are written to file but not to
console.]

So, for your networking scenario, let's suppose you do the following:
Have all network loggers live in the hierarchy namespace below
"network" (e.g. "network", "network.tc p", "network.ht tp" etc.). By
default, all of these will only log events of severity WARNING and
above. Also, suppose you log events in your application code, which are
sometimes but not always of interest, at level DEBUG or level INFO.
Then, these events will never show up in the logging output, since they
are below WARNING in severity. Subsequently, if you want to turn on
logging verbosity for the network code only, you can arrange, via a
command-line switch or environment variable or configuration file, to
do

logging.getLogg er("network").s etLevel(logging .DEBUG)

whereupon you will start seeing events from the networking code at
severity DEBUG and INFO. This will affect all loggers in the "network"
hierarchy whose levels you have not explicitly set (so that they will
get the effective level of the first ancestor which has a level
explicitly set - the logger named "network").

If all you are interested in is turning on verbosity based on different
event severities (levels), you should not need to use or set Filters.
Filters are for use only when levels don't meet your use case
requirements.

Best regards,

Vinay Sajip

Jan 21 '07 #4

Vinay Sajip wrote:
>
I don't know enough about your target environment and application to
necessarily give you the best advice, but I'll make some general
comments which I hope are useful. You seem to be thinking that loggers
are binary - i.e. you turn them on or off. But they can be controlled
more finely than that; you should be able to get the effect that you
want by using the different logging levels available judiciously, as
well as using the fact that loggers inhabit a named hierarchy. Note
that loggers, by default, inherit the level of the first ancestor
logger which has an explicitly set level (by ancestor, I mean in the
name hierarchy). The root logger's default level is WARNING, so by
default all loggers will work at this level. [N.B. You can also set
levels for individual handlers, e.g. to ensure that only CRITICAL
conditions are emailed to a specified email address using SMTPHandler,
or that ERROR conditions and above are written to file but not to
console.]

So, for your networking scenario, let's suppose you do the following:
Have all network loggers live in the hierarchy namespace below
"network" (e.g. "network", "network.tc p", "network.ht tp" etc.). By
default, all of these will only log events of severity WARNING and
above. Also, suppose you log events in your application code, which are
sometimes but not always of interest, at level DEBUG or level INFO.
Then, these events will never show up in the logging output, since they
are below WARNING in severity. Subsequently, if you want to turn on
logging verbosity for the network code only, you can arrange, via a
command-line switch or environment variable or configuration file, to
do

logging.getLogg er("network").s etLevel(logging .DEBUG)

whereupon you will start seeing events from the networking code at
severity DEBUG and INFO. This will affect all loggers in the "network"
hierarchy whose levels you have not explicitly set (so that they will
get the effective level of the first ancestor which has a level
explicitly set - the logger named "network").

If all you are interested in is turning on verbosity based on different
event severities (levels), you should not need to use or set Filters.
Filters are for use only when levels don't meet your use case
requirements.

Best regards,

Vinay Sajip

Vinay, okay, I think what you described will work out for me -- thank
you very much for the explanation.

I am still a bit confused about Filters, though. It seems they are a
bit of an anomoly in the hierarchical view of loggers that the API
supports elsewhere, i.e., filters don't seem to inherit... Or am I
missing something again? Here's a quick example:

import logging

log1 = logging.getLogg er("top")
log2 = logging.getLogg er("top.network ")
log3 = logging.getLogg er("top.network .tcp")
log4 = logging.getLogg er("top.network .http")
log5 = logging.getLogg er("top.config" )
log6 = logging.getLogg er("top.config. file")

logging.basicCo nfig(level=logg ing.DEBUG,
format='%(ascti me)s %(levelname)s %(message)s')

filter = logging.Filter( "top.networ k")
log1.addFilter( filter) # only affects log1, do this for each of log2-7
too?

log1.debug("I'm top")
log2.debug("I'm top.network")
log3.debug("I'm top.network.tcp ")
log4.debug("I'm top.network.htt p")
log5.debug("I'm top.config")
log6.debug("I'm top.config.file ")
This is only for the binary case (and I think if I ignore the binary
case and filters altogether as you suggested), but it really would be
nice to be able to squelch /all/ output from loggers that belong to
certain parts of the namespace by using a filter as above (which I
can't get to work).

Perhaps if I set up a basicConfig with a loglevel of nothing, I can get
this to approximate squelching of everything but that which I
explicitly setLevel (which does inherit properly).

In other words, the addFilter/removeFilter part of the API seems rather
impotent if it can't be inherited in the logging namespaces. In fact,
I can't really figure out a use case where I could possibly want to use
it without it inheriting. Obviously I'm missing something. I'm sure
I've consumed more attention that I deserve already in this thread,
but, do you have any pointers which can enlighten me as to how to
effectively use addFilter/removeFilter?

many thanks,
Gary

Jan 22 '07 #5
Gary Jefferson wrote:
I am still a bit confused about Filters, though. It seems they are a
bit of an anomoly in the hierarchical view of loggers that the API
supports elsewhere, i.e., filters don't seem to inherit... Or am I
missing something again? Here's a quick example:

import logging

log1 = logging.getLogg er("top")
log2 = logging.getLogg er("top.network ")
log3 = logging.getLogg er("top.network .tcp")
log4 = logging.getLogg er("top.network .http")
log5 = logging.getLogg er("top.config" )
log6 = logging.getLogg er("top.config. file")

logging.basicCo nfig(level=logg ing.DEBUG,
format='%(ascti me)s %(levelname)s %(message)s')

filter = logging.Filter( "top.networ k")
log1.addFilter( filter) # only affects log1, do this for each of log2-7
too?

log1.debug("I'm top")
log2.debug("I'm top.network")
log3.debug("I'm top.network.tcp ")
log4.debug("I'm top.network.htt p")
log5.debug("I'm top.config")
log6.debug("I'm top.config.file ")
This is only for the binary case (and I think if I ignore the binary
case and filters altogether as you suggested), but it really would be
nice to be able to squelch /all/ output from loggers that belong to
certain parts of the namespace by using a filter as above (which I
can't get to work).

Perhaps if I set up a basicConfig with a loglevel of nothing, I can get
this to approximate squelching of everything but that which I
explicitly setLevel (which does inherit properly).

In other words, the addFilter/removeFilter part of the API seems rather
impotent if it can't be inherited in the logging namespaces. In fact,
I can't really figure out a use case where I could possibly want to use
it without it inheriting. Obviously I'm missing something. I'm sure
I've consumed more attention that I deserve already in this thread,
but, do you have any pointers which can enlighten me as to how to
effectively use addFilter/removeFilter?
I don't really think there's a problem with the logging API - it went
through a fair amount of peer review on python-dev before making it
into the Python distribution. You need to use what's there rather than
shoehorn it into how you think it ought to be. Filters are for more
esoteric requirements - you can see some examples of filters in the old
(out of date) standalone distribution at
http://www.red-dove.com/python_logging.html (download and examine some
of the test scripts). Loggers aren't binary - levels are there to be
used. Most people get by with judicious use of levels, using Filters on
loggers only for unusual cases. Since Filters are only meant to be used
in unusual situations, there is no need to think about inheriting them.

Filters can be set on Handlers so that even if Loggers log the events,
they don't go to any output if filtered out at the handlers for that
output.

BTW I would also advise reading PEP-282 to understand more about the
logging approach.

Best regards,

Vinay Sajip

Jan 22 '07 #6

Vinay Sajip wrote:
>
BTW I would also advise reading PEP-282 to understand more about the
logging approach.

You've been most helpful, Vinay. The PEP section on Filters states
that I can do what I've been trying to do with filters, but doesn't
provide enough information to do it (or, at least, I'm too thick to
guess at how it would work by looking at the API and PEP and trying a
dozen different ways). Luckily, the examples you point to from your
original package do provide enough info; log_test15.py held the key.

I still feel like it would be more intuitive if filters were inherited
down the hierarchy instead of having to go through the extra steps of
getting at the root handler first, but I'm sure there are good reasons
for not doing this.

One more question, is there any way to get the list of all named
loggers (from the Manager, perhaps)? Or... maybe I don't need this,
either, as MatchFilter (log_test18.py) seems to do what I was thinking
I need the list of logger names for... most excellent.

Thanks,
Gary

BTW, the python logging module is one of the best readily available
loggers I've come across in any language.

Jan 23 '07 #7
Gary Jefferson wrote:
Suppose I have 3 modules that belong to a project, 'A', 'A.a' and 'B',
and each module has its own logger, created with:

module1logger = logging.getLogg er('project.A')

and

module2logger = logging.getLogg er('project.A.a ')

and

module3logger = logging.getLogg er('project.B')
And I want to selectively enable/disable these, per module (for
example, I only want logging from A and A.a, or just from B, etc). It
seems like logging.Filter is supposed to let me do this, but I can't
see how to apply it globally. Is there really no other way that to add
a addFilter(filte r) call to each of these loggers individually?

logging.basicCo nfig gets inherited by all loggers, but it doesn't seem
capable of giving a Filter to all loggers. Is there any way to do this?
An alternative approach might be to set the 'propagate' flag:

import logging

def warn_all(logger s, message):
for lgr in loggers:
lgr.warn(messag e)

logging.basicCo nfig()
root = logging.getLogg er()
root.manager.em ittedNoHandlerW arning = True
loggers = [logging.getLogg er(name) for name in """
alpha
alpha.1
alpha.2
alpha.2.1
alpha.2.2
alpha.2.2.1
beta
beta.1
beta.2
""".split()]

warn_all(logger s, "all on")

print "---"
logging.getLogg er("alpha").pro pagate = False
warn_all(logger s, "alpha off")
logging.getLogg er("alpha").pro pagate = True

print "---"
logging.getLogg er("alpha.2").p ropagate = False
warn_all(logger s, "alpha.2 off")

Peter
Jan 23 '07 #8
Gary Jefferson wrote:
Vinay Sajip wrote:

BTW I would also advise reading PEP-282 to understand more about the
logging approach.


You've been most helpful, Vinay. The PEP section on Filters states
that I can do what I've been trying to do with filters, but doesn't
provide enough information to do it (or, at least, I'm too thick to
guess at how it would work by looking at the API and PEP and trying a
dozen different ways). Luckily, the examples you point to from your
original package do provide enough info; log_test15.py held the key.

I still feel like it would be more intuitive if filters were inherited
down the hierarchy instead of having to go through the extra steps of
getting at the root handler first, but I'm sure there are good reasons
for not doing this.

One more question, is there any way to get the list of all named
loggers (from the Manager, perhaps)? Or... maybe I don't need this,
either, as MatchFilter (log_test18.py) seems to do what I was thinking
I need the list of logger names for... most excellent.

Thanks,
Gary

BTW, the python logging module is one of the best readily available
loggers I've come across in any language.
Thanks. Glad the tests/examples (log_testxx.py) helped. When I get a
chance, I will try to work some of them into the docs...

Best regards,
Vinay Sajip

Jan 23 '07 #9
So maybe I don't have all this figured out quite as well as I thought.
What I really want to do is set an environment variable, MYDEBUG, which
contains a list of wildcarded logger names, such as "a.*.c a.d" (which
becomes ['a.*.c', 'a.d'], and then selectively crank the loglevel up to
DEBUG for those that match.

In order to do that, I think I need something kind of like filters, but
for logger name... I'm not seeing how to do this, even after playing
with variations of test15, 18, and 21.

What I do have working at the moment is passing a list of non-wildcard
logger names, i.e., doing the wildcard expansion manually such as
"['a.b.c', 'a.c.c', 'a.f.c', 'a.d']. Is there anyway to automate this
dynamically?

BTW, I do understand that 'a.d' is essentially equivalent to 'a.d.*',
but I'm dealing with a hierarchy that doesn't always tidy up like that.
For example, I have top.network.ser ver.http, top.network.ser ver.smtp,
top.network.cli ent.http, and top.network.cli ent.smtp. Obviously, if I
only want server or client DEBUG msgs, this is easy. And sometimes
that's exactly what I want. Other times, I want only smtp DEBUG msgs,
and the hierarchy won't help me get that (unless I break it for just
getting client or server msgs), etc. So I would really like to figure
out how to do 'a.*.c'.

Any ideas?

Thanks again,
Gary
On Jan 23, 3:01 am, "Vinay Sajip" <vinay_sa...@ya hoo.co.ukwrote:
Glad the tests/examples (log_testxx.py) helped. When I get a
chance, I will try to work some of them into the docs...

Best regards,

Vinay Sajip
Jan 25 '07 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
15941
by: Alphonse Giambrone | last post by:
I am trying to enable/disable a requiredfieldvalidator on the client side and am generating an error. I had found some documentation on validation which states that I should be able to enable/disable validators on the client side. http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnaspp/html/aspplusvalid.asp According to it...
0
3357
by: Robert Ladd | last post by:
Hi, I'm trying to disable the asp.net calendar control from a javascript function, but it doesn't disable the doPostBack. To simplify the situation, assume a page with 4 controls. A dropdownlist, a calendar control, a textbox and an update button. What I'm doing is initially setting the button to disabled, the other 3 controls are...
0
2392
by: talal | last post by:
How to check whether javascript is enable or disable in client browser. Remeber i can check wheter browser client support javascript with BrowserCapability class. But what if the client has disable from security setting in a browser. Tell me is there any class like browserCapability in asp.net to check wheter javascript is enable or disable....
4
10733
by: mabond | last post by:
Does anyone know some code that will allow me to enable and/or disable a task which already exists in the task scheduer. I have a task which runs at 15 min intervals and from time to time I want to disable (and later enable) that task. I'd like to do so programatically as part of another application instead of opening the task scheduler and...
2
2524
by: PamelaDV | last post by:
I am wondering if there is a way to disable the "X" used to close the Access application window? I know how to disable it for individual forms, but I would like to disable it for the application in general and force users to use a button on the switchboard to close (because I run important code on the close of the database from that button)....
4
14149
by: Phoe6 | last post by:
Hi all, I am trying to disable the NIC card (and other cards) enabled in my machine to test diagnostics on that card. I am trying to disable it programmatic using python. I checked python wmi and i could not find ways to disable/enable, (listing is however, possible). Where should I look for to enable/disable devices in python. Thanks,
3
11624
by: Pietro | last post by:
Hi all, First of all I'd like to thank you very very much ,as finally after many years of searching,I could find a code to disable/enable the shift key,but actually i cannot use the code as I'm very new to VBA,i tried to follow the instructions reported in the code,but i got no result,i still can use the shift key,can you explain in details...
8
17213
by: sebouh181 | last post by:
I am writing to registry using javascript. var wsh = new ActiveXObject("WScript.Shell"); var key = "HKLM\\Software\\Neos\\2.0\\LightMode\\my-script"; wsh.RegWrite (key, 1999, "REG_SZ"); when i run this javascript code in my HTML page the following warning message appears An ActiveX control might be unsafe to interact with other parts of the...
2
4356
by: Naushad | last post by:
Hi all, I am using the countinous form. I want to Enable/Disable the some fields for perticular records as per the following condition when open the form. I have written this code in "On Current Event". I go on the perticular record its enable/dosable the following field for all records. I have tried this code in "On Open Even" but there is...
0
7453
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
7802
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
1
7410
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
7742
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
0
5958
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5318
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
4940
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
1
1865
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1007
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.