473,406 Members | 2,259 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,406 software developers and data experts.

__builtins__.loglog - logging more pythonic, decent & scalable ?

As more and more python packages are starting to use the bloomy
(Java-ish) 'logging' module in a mood of responsibility and as I am not
overly happy with the current "thickener" style of usage, I want to put
this comment and a alternative most simple default framework for
discussion.
Maybe there are more Python users which like to see that imported
(managed) logging issue more down-to-earth and flexible ? ...

Vinay Sajip wrote:
robert wrote:
some packages like paramiko use the logging. I get this messages:
"No handlers could be found for logger xxx" on stderr

Why is un-initialized logging allowed to chatter at all?
You could invoke logging.basicConfig with a level of CRITICAL. This
will generally filter out logging messages.


Meanwhile i looked on logging and its doc:

The doc of for example basicConfig (promising to be quite simple/silent):

"basicConfig( )

Does basic configuration for the logging system by creating a
StreamHandler with a default Formatter and adding it to the root logger.
The functions debug(), info(), warning(), error() and critical() will
call basicConfig() automatically if no handlers are defined for the root
logger. "

=> So far I don't get any idea what will happen after doing this. What
is a StreamHandler? a "default Formatter"? CRITICAL? what is this
default magic behavior? even if silent behavior is installed log(),
debug() consume still a lot of execution time => I have to read the
logging code anyway => for my needs i rendered dead the Logger() in
app-init / dead the line in logging which chatters within my standard
set of Python patches.

I looked around a little more and think, that logging it is really a
_FRAMEWORK_ - non-pythonic . a declarative monster.

Meaning: too big the interface and too big its basic footprint. without
a decent scaling. too loud. it takes lots of time until one understands
the basics. even the simplest setup example has not less than 8 lines -
strange "filehandlers" and "formatters" are going on - and it is a
framework which does a lot of obscure things, its class collection reads
more like an magic application only understandable by its immediate
programmers - rather than a stdlib part.
Its almost a Java-ish ego class bunch.

Some wide spread packages like paramiko use it already - inside stdlib
only cookielib so far. Yet, just for providing the option they are
costly pre-importing almost the whole framework functionality (in
memory, frozen apps etc.) and have considerable costs during execution
(regarding my step-in debugging sessions: seems to do very very late
dispatching).
Python is still not an OS on its own but a programming language/library
which should fit into OS'es.
( though they meanwhile put the CJK codecs etc. into the python core !?
:-) )

Yes, one could use such functionality - but one does only in rarest cases.
I did and do logging a lot, but always had quite simple paths. I'd
usually still not go though that framework in almost all cases, because
it would require many hours until I really know whats going magically in
all kinds of cases.

99% of my app logging needs were and are fulfilled by a simple direct
module function as simple as:

def doLog(text,level,**kwattrs):
# in case write to file or whatever ...

and so I simply know really what will happen. Collecting all logs is
also so easy with Python, that it could be _enabled_ by a much much
simpler and pythonic one-function-framework: a empty log-hook function
in the python __builtins__:

================================================== =======

def loglog(text, channel='default', level=0, **kwattrs):
pass

================================================== =======

yet there could go any complex hierarchical channel names through that
like channel="mypackage.mymodule.subtheme" or even multi-hierarchical
channel classes/objects similar to exception-class-catching in addition
to string-catching. Example:

================================================== =======
class MyLogChannelAA(MyLogChannelA, MyLogAspectXY)

loglog(text, MyLogChannelAA, __debug__, lcls=locals() )
================================================== =======

If that simple would be the recommended standard logging interface in
Python, I think, really a significant fraction of Python programmers
would first understand and than use ( call and consume ) it - and
(decent) modules would not blow up the machine unconditionally just for
a mere specialized option.

For massive usage apps could shortcut THEMSELVES with bloomy names and
level-constants like logdebug() logerror() ....
(Recommendations at best - no more framework.)

Still this would be flexible enough and a super-hierarchical obscure
logger and handler collection as exposed by "logging" or any other
alternative could still be hooked in, while no module which does logging
has to PRE-IMPORT SUCH A FAT SPECIAL LOGGING FRAMEWORK.
Un-initialized logging chatters because in development environments,
it's useful to be able to spot misconfigured loggers. For production
use, set logging.raiseExceptions to 0 and logging should then be quiet.


Think a potentially "misconfigured logger" is not a reason to chatter.

If there is an error in setup you'd 99.99% get an exception. If you
really totally forget to setup - you simply have forgotten as you can
forget many things. If you really want log's you'll have 1000 chances as
developer to see that there are no logs - and add the setup.

My opinion: _By default_ logging should really

* be totally silent
* not import all the logging code and handler classes etc. (but only a
minimal stub or do ** (see below))
* consume only minimum execution time: <if-not-initialized: return>

Consider a typical complex program: Hundreds and thousands of things
have to be be set up and configured which are much more important than a
debug logger. Yet not every imported library and every class yells
because it was not used or not basicConfig'ed :-). The logging is not
the center of the world. In 99% only the developer of a module
uses/wants the log output.

I see there were already other complaints about this chattering - and
other people also wondered how to make it silent.

--

In cookielib recently the logging initialization has been **virtualized
=> its not imported / necessary (in frozen apps) until the cookielib
debugging was not really requested.
Think thats a better/acceptable style of use and such guidline should be
put at the top of the logging docs to complete/alternate the above 3 issues.

That all maybe could be healed most easily by the proposed
__builtin__.loglog in Python : not unconditional wiring to such a
specific and fat package.
-robert
Jun 3 '06 #1
0 1822

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

15
by: Ville Vainio | last post by:
Pythonic Nirvana - towards a true Object Oriented Environment ============================================================= IPython (by Francois Pinard) recently (next release - changes are...
8
by: qwweeeit | last post by:
Hi all, I wonder if it is possible to change (temporarily) a built-in function for logging purposes. Let me explain: I want to log all the 'open' operations, recording the file to be opened, the...
23
by: Rotem | last post by:
Hi, while working on something in my current project I have made several improvements to the logging package in Python, two of them are worth mentioning: 1. addition of a logging record field...
3
by: nicholas.petrella | last post by:
I am currently trying to use the python logging system as a core enterprise level logging solution for our development and production environments. The rotating file handler seems to be what I...
3
by: loquehumaine | last post by:
Hi there, I'm a newby in python (I know a little in programmation) and I have a lot of questions on builtins but my first one is about modules... I have seen that if I type help() at a prompt,...
1
by: hyperboreean | last post by:
Hi, I am writing the application server for a three-tier architecture and sending the client's response in xml. My question is: is there a way to build the xml dom in a more scalable way and faster...
4
by: Aaron \Castironpi\ Brady | last post by:
Sometimes questions come up on here about unpickling safely and executing foreign code. I was thinking a minimum install that didn't even have access to modules like 'os' could be safe. ...
4
by: alag20 | last post by:
Hi Guys, Sorry for duplicate posting as this Question refers to both c# and perl cgi script on the net, so please help. Here is the original post...
7
by: nissanbi | last post by:
Hi, I need to find a data structure that supports insertion, deletion and finding the median of the elements stored at the data structure. All the operation should run in O(loglogn) amortized...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.