473,703 Members | 3,002 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

__builtins__.lo glog - logging more pythonic, decent & scalable ?

As more and more python packages are starting to use the bloomy
(Java-ish) 'logging' module in a mood of responsibility and as I am not
overly happy with the current "thickener" style of usage, I want to put
this comment and a alternative most simple default framework for
discussion.
Maybe there are more Python users which like to see that imported
(managed) logging issue more down-to-earth and flexible ? ...

Vinay Sajip wrote:
robert wrote:
some packages like paramiko use the logging. I get this messages:
"No handlers could be found for logger xxx" on stderr

Why is un-initialized logging allowed to chatter at all?
You could invoke logging.basicCo nfig with a level of CRITICAL. This
will generally filter out logging messages.


Meanwhile i looked on logging and its doc:

The doc of for example basicConfig (promising to be quite simple/silent):

"basicConfi g( )

Does basic configuration for the logging system by creating a
StreamHandler with a default Formatter and adding it to the root logger.
The functions debug(), info(), warning(), error() and critical() will
call basicConfig() automatically if no handlers are defined for the root
logger. "

=> So far I don't get any idea what will happen after doing this. What
is a StreamHandler? a "default Formatter"? CRITICAL? what is this
default magic behavior? even if silent behavior is installed log(),
debug() consume still a lot of execution time => I have to read the
logging code anyway => for my needs i rendered dead the Logger() in
app-init / dead the line in logging which chatters within my standard
set of Python patches.

I looked around a little more and think, that logging it is really a
_FRAMEWORK_ - non-pythonic . a declarative monster.

Meaning: too big the interface and too big its basic footprint. without
a decent scaling. too loud. it takes lots of time until one understands
the basics. even the simplest setup example has not less than 8 lines -
strange "filehandle rs" and "formatters " are going on - and it is a
framework which does a lot of obscure things, its class collection reads
more like an magic application only understandable by its immediate
programmers - rather than a stdlib part.
Its almost a Java-ish ego class bunch.

Some wide spread packages like paramiko use it already - inside stdlib
only cookielib so far. Yet, just for providing the option they are
costly pre-importing almost the whole framework functionality (in
memory, frozen apps etc.) and have considerable costs during execution
(regarding my step-in debugging sessions: seems to do very very late
dispatching).
Python is still not an OS on its own but a programming language/library
which should fit into OS'es.
( though they meanwhile put the CJK codecs etc. into the python core !?
:-) )

Yes, one could use such functionality - but one does only in rarest cases.
I did and do logging a lot, but always had quite simple paths. I'd
usually still not go though that framework in almost all cases, because
it would require many hours until I really know whats going magically in
all kinds of cases.

99% of my app logging needs were and are fulfilled by a simple direct
module function as simple as:

def doLog(text,leve l,**kwattrs):
# in case write to file or whatever ...

and so I simply know really what will happen. Collecting all logs is
also so easy with Python, that it could be _enabled_ by a much much
simpler and pythonic one-function-framework: a empty log-hook function
in the python __builtins__:

=============== =============== =============== ============

def loglog(text, channel='defaul t', level=0, **kwattrs):
pass

=============== =============== =============== ============

yet there could go any complex hierarchical channel names through that
like channel="mypack age.mymodule.su btheme" or even multi-hierarchical
channel classes/objects similar to exception-class-catching in addition
to string-catching. Example:

=============== =============== =============== ============
class MyLogChannelAA( MyLogChannelA, MyLogAspectXY)

loglog(text, MyLogChannelAA, __debug__, lcls=locals() )
=============== =============== =============== ============

If that simple would be the recommended standard logging interface in
Python, I think, really a significant fraction of Python programmers
would first understand and than use ( call and consume ) it - and
(decent) modules would not blow up the machine unconditionally just for
a mere specialized option.

For massive usage apps could shortcut THEMSELVES with bloomy names and
level-constants like logdebug() logerror() ....
(Recommendation s at best - no more framework.)

Still this would be flexible enough and a super-hierarchical obscure
logger and handler collection as exposed by "logging" or any other
alternative could still be hooked in, while no module which does logging
has to PRE-IMPORT SUCH A FAT SPECIAL LOGGING FRAMEWORK.
Un-initialized logging chatters because in development environments,
it's useful to be able to spot misconfigured loggers. For production
use, set logging.raiseEx ceptions to 0 and logging should then be quiet.


Think a potentially "misconfigu red logger" is not a reason to chatter.

If there is an error in setup you'd 99.99% get an exception. If you
really totally forget to setup - you simply have forgotten as you can
forget many things. If you really want log's you'll have 1000 chances as
developer to see that there are no logs - and add the setup.

My opinion: _By default_ logging should really

* be totally silent
* not import all the logging code and handler classes etc. (but only a
minimal stub or do ** (see below))
* consume only minimum execution time: <if-not-initialized: return>

Consider a typical complex program: Hundreds and thousands of things
have to be be set up and configured which are much more important than a
debug logger. Yet not every imported library and every class yells
because it was not used or not basicConfig'ed :-). The logging is not
the center of the world. In 99% only the developer of a module
uses/wants the log output.

I see there were already other complaints about this chattering - and
other people also wondered how to make it silent.

--

In cookielib recently the logging initialization has been **virtualized
=> its not imported / necessary (in frozen apps) until the cookielib
debugging was not really requested.
Think thats a better/acceptable style of use and such guidline should be
put at the top of the logging docs to complete/alternate the above 3 issues.

That all maybe could be healed most easily by the proposed
__builtin__.log log in Python : not unconditional wiring to such a
specific and fat package.
-robert
Jun 3 '06 #1
0 1861

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

15
2282
by: Ville Vainio | last post by:
Pythonic Nirvana - towards a true Object Oriented Environment ============================================================= IPython (by Francois Pinard) recently (next release - changes are still in CVS) got the basic abilities of a system shell (like bash). Actually, using it as a shell is rather comfortable. This all made me think... Why do we write simple scripts to do simple things? Why do we serialize data to flat text files in...
8
1543
by: qwweeeit | last post by:
Hi all, I wonder if it is possible to change (temporarily) a built-in function for logging purposes. Let me explain: I want to log all the 'open' operations, recording the file to be opened, the "mode" (r/w/a...) and (possibly) the module which made the call. After that the 'open' can carry on following the standard built-in path. Possibly I don't want to have to modify the source of the application
23
2215
by: Rotem | last post by:
Hi, while working on something in my current project I have made several improvements to the logging package in Python, two of them are worth mentioning: 1. addition of a logging record field %(function)s, which results in the name of the entity which logged the record. My version even deduces the class name in the case which the logger is a bound method, and assuming the name of the "self" variable is indeed "self".
3
5400
by: nicholas.petrella | last post by:
I am currently trying to use the python logging system as a core enterprise level logging solution for our development and production environments. The rotating file handler seems to be what I am looking for as I want the ability to have control over the number and size of log files that are written out for each of our tools. I have noticed a few problems with this handler and wanted to post here to get your impressions and possibly...
3
4169
by: loquehumaine | last post by:
Hi there, I'm a newby in python (I know a little in programmation) and I have a lot of questions on builtins but my first one is about modules... I have seen that if I type help() at a prompt, and then 'modules', I'll be given a list of all modules available, thanks to this group.. But I have seen the differences between them and the one in dir(__builtins__). Why are some modules in __builtins__ and others don't ? (UserDict for example)
1
1100
by: hyperboreean | last post by:
Hi, I am writing the application server for a three-tier architecture and sending the client's response in xml. My question is: is there a way to build the xml dom in a more scalable way and faster way than just creating every textNode and element for it? I have tons of data to transmit and it drives me crazy having to build that dom manually. I am not sure if this is a stupid question as I don't know other alternatives ... maybe just...
4
1333
by: Aaron \Castironpi\ Brady | last post by:
Sometimes questions come up on here about unpickling safely and executing foreign code. I was thinking a minimum install that didn't even have access to modules like 'os' could be safe. (Potentially.) I have time to entertain this a little, though all the devs are busy. I can bring it up again in a few months if it's a better time. I browsed for info on 'rexec'. Two c-l-py threads:...
4
3077
by: alag20 | last post by:
Hi Guys, Sorry for duplicate posting as this Question refers to both c# and perl cgi script on the net, so please help. Here is the original post http://bytes.com/topic/c-sharp/answers/870412-internet-logging-using-c-perl-cgi-script#post3493381 which contains the perl cgi script and c# code i am using. Please help - any help will be highly appreciated.
7
1912
by: nissanbi | last post by:
Hi, I need to find a data structure that supports insertion, deletion and finding the median of the elements stored at the data structure. All the operation should run in O(loglogn) amortized time. Any ideas? I tried using two heaps so I can find the median in O(1), but I don't thing it's good enough...
0
8743
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8659
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9239
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9102
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8951
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7844
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6585
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
4417
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4674
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.