Hello,
I'm trying to understand the behavior of the Python 2.3 logging module (MS
Windows 2k) with regard to RotatingFileHandler. The following script
illustrates a puzzling problem. What is wrong with this script?
Thanks,
-- jv
BEGIN FILE _________________________________________
'''
This script terminates as follows:
Traceback (most recent call last):
File "D:\$PROJECTS\experimental\Py Logging\t_xb.py", line 63, in ?
shutdown()
File "C:\Python23\lib\logging\__init__.py", line 1195, in shutdown
h.flush()
File "C:\Python23\lib\logging\__init__.py", line 661, in flush
self.stream.flush()
ValueError: I/O operation on closed file
What is wrong with it?
'''
from logging import getLogger, Formatter, shutdown, DEBUG, INFO, WARNING,
ERROR, CRITICAL
from logging.handlers import RotatingFileHandler
def logger_for(component):
'''
RETURNS a logger for the specified component.
SIDE-EFFECTS
(re)assigns the logger handler
'''
global handler
logger = getLogger(component)
if handler:
handler.flush()
handler.close()
logger.removeHandler(handler)
# In normal, operational mode, the following parameters:
filename = '%s.log'%component
mode = 'a'
maxBytes = 100
backupCount = 5
# would be user-configurable "on the fly" hence the reason for this
function.
handler = RotatingFileHandler(filename, mode, maxBytes, backupCount)
handler.setLevel(DEBUG)
logger.addHandler(handler)
return logger
handler = None
for i in range(20):
log = logger_for('supplier')
log.error('testing Python logging module')
shutdown()
END FILE _________________________________________ 5 11208
j vickroy wrote: I'm trying to understand the behavior of the Python 2.3 logging module (MS Windows 2k) with regard to RotatingFileHandler. The following script illustrates a puzzling problem. What is wrong with this script?
if handler: handler.flush() handler.close() logger.removeHandler(handler)
The handler is stored in the logging._handlers dictionary in order to close
it when shutdown() is called. But you already did close it manually.
I think you have three options to fix your script.
(1) Don't call shutdown() at all and manually close the last handler
instead.
(2) Change the above to
if handler:
logger.removeHandler(handler)
handler.flush()
handler.close()
del logging._handlers[handler]
so that shutdown() cannot touch closed handlers, or
(3) don't close handlers manually
if handler:
logger.removeHandler(handler)
so that shutdown() gets a still open handler as expected. I would go with
the latter, as it does not rely on implementation details.
(all untested, use at your own risk)
Peter
Here is the result of not closing a handler manually -- Peter's suggestion
(3)
Traceback (most recent call last):
File
"C:\Python23\lib\site-packages\Pythonwin\pywin\framework\scriptutils.py" ,
line 310, in RunScript
exec codeObject in __main__.__dict__
File "E:\$PROJECTS\experimental\Py Logging\t_xc.py", line 61, in ?
log.error('testing Python logging module')
File "C:\Python23\lib\logging\__init__.py", line 923, in error
apply(self._log, (ERROR, msg, args), kwargs)
File "C:\Python23\lib\logging\__init__.py", line 994, in _log
self.handle(record)
File "C:\Python23\lib\logging\__init__.py", line 1004, in handle
self.callHandlers(record)
File "C:\Python23\lib\logging\__init__.py", line 1037, in callHandlers
hdlr.handle(record)
File "C:\Python23\lib\logging\__init__.py", line 592, in handle
self.emit(record)
File "C:\Python23\lib\logging\handlers.py", line 105, in emit
self.doRollover()
File "C:\Python23\lib\logging\handlers.py", line 90, in doRollover
os.rename(self.baseFilename, dfn)
OSError: [Errno 13] Permission denied
and here is the script that generated the above behavior:
begin -------------------------------------------------
from logging import getLogger, Formatter, shutdown, DEBUG, INFO, WARNING,
ERROR, CRITICAL
from logging.handlers import RotatingFileHandler
def logger_for(component):
'''
RETURNS a logger for the specified component.
SIDE-EFFECTS
(re)assigns the logger handler
'''
global handler
logger = getLogger(component)
if handler:
## handler.flush()
## handler.close()
logger.removeHandler(handler)
# In normal, operational mode, the following parameters:
filename = '%s.log'%component
mode = 'a'
maxBytes = 100
backupCount = 5
# would be user-configurable "on the fly" hence the reason for this
function.
handler = RotatingFileHandler(filename, mode, maxBytes, backupCount)
handler.setLevel(DEBUG)
logger.addHandler(handler)
return logger
handler = None
for i in range(20):
log = logger_for('supplier')
log.error('testing Python logging module')
shutdown()
end -----------------------------------------
As a follow-up question, why is a handle object not removed from
logging._handlers when its (i.e., handle) close() procedure is applied?
That behavior appears to be responsible for the logging.shutdown() failure.
"j vickroy" <ji*********@noaa.gov> wrote in message
news:bp**********@boulder.noaa.gov... Hello,
I'm trying to understand the behavior of the Python 2.3 logging module (MS Windows 2k) with regard to RotatingFileHandler. The following script illustrates a puzzling problem. What is wrong with this script?
Thanks,
-- jv
BEGIN FILE _________________________________________ ''' This script terminates as follows:
Traceback (most recent call last): File "D:\$PROJECTS\experimental\Py Logging\t_xb.py", line 63, in ? shutdown() File "C:\Python23\lib\logging\__init__.py", line 1195, in shutdown h.flush() File "C:\Python23\lib\logging\__init__.py", line 661, in flush self.stream.flush() ValueError: I/O operation on closed file
What is wrong with it? ''' from logging import getLogger, Formatter, shutdown, DEBUG, INFO,
WARNING, ERROR, CRITICAL from logging.handlers import RotatingFileHandler def logger_for(component): ''' RETURNS a logger for the specified component.
SIDE-EFFECTS (re)assigns the logger handler '''
global handler
logger = getLogger(component)
if handler: handler.flush() handler.close() logger.removeHandler(handler)
# In normal, operational mode, the following parameters: filename = '%s.log'%component mode = 'a' maxBytes = 100 backupCount = 5 # would be user-configurable "on the fly" hence the reason for this function.
handler = RotatingFileHandler(filename, mode, maxBytes, backupCount)
handler.setLevel(DEBUG)
logger.addHandler(handler)
return logger handler = None
for i in range(20): log = logger_for('supplier') log.error('testing Python logging module')
shutdown() END FILE _________________________________________
j vickroy wrote: Here is the result of not closing a handler manually -- Peter's suggestion (3)
Traceback (most recent call last): File "C:\Python23\lib\site-packages\Pythonwin\pywin\framework\scriptutils.py" , line 310, in RunScript exec codeObject in __main__.__dict__ File "E:\$PROJECTS\experimental\Py Logging\t_xc.py", line 61, in ? log.error('testing Python logging module') File "C:\Python23\lib\logging\__init__.py", line 923, in error apply(self._log, (ERROR, msg, args), kwargs) File "C:\Python23\lib\logging\__init__.py", line 994, in _log self.handle(record) File "C:\Python23\lib\logging\__init__.py", line 1004, in handle self.callHandlers(record) File "C:\Python23\lib\logging\__init__.py", line 1037, in callHandlers hdlr.handle(record) File "C:\Python23\lib\logging\__init__.py", line 592, in handle self.emit(record) File "C:\Python23\lib\logging\handlers.py", line 105, in emit self.doRollover() File "C:\Python23\lib\logging\handlers.py", line 90, in doRollover os.rename(self.baseFilename, dfn) OSError: [Errno 13] Permission denied
Works here. Consider switching to an OS where you can rename an open file...
Well, you read my discalimer :-)
Peter
j vickroy wrote: As a follow-up question, why is a handle object not removed from logging._handlers when its (i.e., handle) close() procedure is applied?
That behavior appears to be responsible for the logging.shutdown() failure.
Seems that the author did not consider the use case of consecutively using
different handlers operating on the same file set - I've not yet made up my
mind, if you are misusing the logging system or if that's a bug.
Anyway, as of 2.3.2 the package has still __status__ = "beta", so patches
might be welcome.
Peter This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: max |
last post by:
I have an application which calls a module runx, in runx I import
logging and get a logger add handlers. I want to be able to rename
the file created by fileHandler.
several problems, I can't...
|
by: Neil Benn |
last post by:
Hello,
I'm running a test and having issues with logging, if I call
logging.shutdown() and then want to start the logging going again then I
get a problem as if I call shutdown, I can't get the...
|
by: flupke |
last post by:
Hi,
i have a class and a class attribute log which is a logger object. In
the __del__() function i want to log a message but it fails even if i
use self.__class__.log.
The error i get is...
|
by: Oliver Eichler |
last post by:
Hi,
I experience several exceptions from python's logging system when using the
rollover feature on Windows.
Traceback (most recent call last):
File "c:\Python24\lib\logging\handlers.py",...
|
by: usenet |
last post by:
I'm having some problems getting the logging module to work with the
threading module. I've narrowed the problem down to the following
code:
import logging, threading
update_log =...
|
by: flupke |
last post by:
Hi,
i'm getting errors with the log module concerning RotatingFileHandler.
I'm using Python 2.4.3 on Windows XP SP2. This used to work in previous
python versions but since i upgraded to 2.4.3...
|
by: s99999999s2003 |
last post by:
hi
i have defined a function
def logger(logfile,msg):
import logging
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s %(levelname)-8s
%(message)s',
datefmt='%a, %d %b %Y...
|
by: nicholas.petrella |
last post by:
I am currently trying to use the python logging system as a core
enterprise level logging solution for our development and production
environments.
The rotating file handler seems to be what I...
|
by: scriptlearner |
last post by:
OS: Solaris 9
Python Version: 2.4.4
I need to log certain data in a worker thread; however, I am getting
an error now when I use two worker threads.
I think the problem comes from the line...
|
by: DolphinDB |
last post by:
Tired of spending countless mintues downsampling your data? Look no further!
In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
|
by: ryjfgjl |
last post by:
ExcelToDatabase: batch import excel into database automatically...
|
by: isladogs |
last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM).
In this month's session, we are pleased to welcome back...
|
by: ArrayDB |
last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
|
by: PapaRatzi |
last post by:
Hello,
I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
|
by: Defcon1945 |
last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
|
by: af34tf |
last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
|
by: Faith0G |
last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome former...
| |