473,324 Members | 1,678 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,324 software developers and data experts.

logging.shutdown() ValueError: I/O operation on closed file

Hello,

I'm trying to understand the behavior of the Python 2.3 logging module (MS
Windows 2k) with regard to RotatingFileHandler. The following script
illustrates a puzzling problem. What is wrong with this script?

Thanks,

-- jv

BEGIN FILE _________________________________________
'''
This script terminates as follows:

Traceback (most recent call last):
File "D:\$PROJECTS\experimental\Py Logging\t_xb.py", line 63, in ?
shutdown()
File "C:\Python23\lib\logging\__init__.py", line 1195, in shutdown
h.flush()
File "C:\Python23\lib\logging\__init__.py", line 661, in flush
self.stream.flush()
ValueError: I/O operation on closed file

What is wrong with it?
'''

from logging import getLogger, Formatter, shutdown, DEBUG, INFO, WARNING,
ERROR, CRITICAL
from logging.handlers import RotatingFileHandler

def logger_for(component):
'''
RETURNS a logger for the specified component.

SIDE-EFFECTS
(re)assigns the logger handler
'''

global handler

logger = getLogger(component)

if handler:
handler.flush()
handler.close()
logger.removeHandler(handler)

# In normal, operational mode, the following parameters:
filename = '%s.log'%component
mode = 'a'
maxBytes = 100
backupCount = 5
# would be user-configurable "on the fly" hence the reason for this
function.

handler = RotatingFileHandler(filename, mode, maxBytes, backupCount)

handler.setLevel(DEBUG)

logger.addHandler(handler)

return logger

handler = None

for i in range(20):
log = logger_for('supplier')
log.error('testing Python logging module')

shutdown()
END FILE _________________________________________
Jul 18 '05 #1
5 11208
j vickroy wrote:
I'm trying to understand the behavior of the Python 2.3 logging module (MS
Windows 2k) with regard to RotatingFileHandler. The following script
illustrates a puzzling problem. What is wrong with this script? if handler:
handler.flush()
handler.close()
logger.removeHandler(handler)


The handler is stored in the logging._handlers dictionary in order to close
it when shutdown() is called. But you already did close it manually.
I think you have three options to fix your script.

(1) Don't call shutdown() at all and manually close the last handler
instead.

(2) Change the above to

if handler:
logger.removeHandler(handler)
handler.flush()
handler.close()
del logging._handlers[handler]

so that shutdown() cannot touch closed handlers, or

(3) don't close handlers manually

if handler:
logger.removeHandler(handler)

so that shutdown() gets a still open handler as expected. I would go with
the latter, as it does not rely on implementation details.

(all untested, use at your own risk)

Peter

Jul 18 '05 #2

Here is the result of not closing a handler manually -- Peter's suggestion
(3)

Traceback (most recent call last):
File
"C:\Python23\lib\site-packages\Pythonwin\pywin\framework\scriptutils.py" ,
line 310, in RunScript
exec codeObject in __main__.__dict__
File "E:\$PROJECTS\experimental\Py Logging\t_xc.py", line 61, in ?
log.error('testing Python logging module')
File "C:\Python23\lib\logging\__init__.py", line 923, in error
apply(self._log, (ERROR, msg, args), kwargs)
File "C:\Python23\lib\logging\__init__.py", line 994, in _log
self.handle(record)
File "C:\Python23\lib\logging\__init__.py", line 1004, in handle
self.callHandlers(record)
File "C:\Python23\lib\logging\__init__.py", line 1037, in callHandlers
hdlr.handle(record)
File "C:\Python23\lib\logging\__init__.py", line 592, in handle
self.emit(record)
File "C:\Python23\lib\logging\handlers.py", line 105, in emit
self.doRollover()
File "C:\Python23\lib\logging\handlers.py", line 90, in doRollover
os.rename(self.baseFilename, dfn)
OSError: [Errno 13] Permission denied

and here is the script that generated the above behavior:

begin -------------------------------------------------
from logging import getLogger, Formatter, shutdown, DEBUG, INFO, WARNING,
ERROR, CRITICAL
from logging.handlers import RotatingFileHandler

def logger_for(component):
'''
RETURNS a logger for the specified component.

SIDE-EFFECTS
(re)assigns the logger handler
'''

global handler

logger = getLogger(component)

if handler:
## handler.flush()
## handler.close()
logger.removeHandler(handler)

# In normal, operational mode, the following parameters:
filename = '%s.log'%component
mode = 'a'
maxBytes = 100
backupCount = 5
# would be user-configurable "on the fly" hence the reason for this
function.

handler = RotatingFileHandler(filename, mode, maxBytes, backupCount)

handler.setLevel(DEBUG)

logger.addHandler(handler)

return logger

handler = None

for i in range(20):
log = logger_for('supplier')
log.error('testing Python logging module')

shutdown()
end -----------------------------------------

Jul 18 '05 #3
As a follow-up question, why is a handle object not removed from
logging._handlers when its (i.e., handle) close() procedure is applied?

That behavior appears to be responsible for the logging.shutdown() failure.
"j vickroy" <ji*********@noaa.gov> wrote in message
news:bp**********@boulder.noaa.gov...
Hello,

I'm trying to understand the behavior of the Python 2.3 logging module (MS
Windows 2k) with regard to RotatingFileHandler. The following script
illustrates a puzzling problem. What is wrong with this script?

Thanks,

-- jv

BEGIN FILE _________________________________________
'''
This script terminates as follows:

Traceback (most recent call last):
File "D:\$PROJECTS\experimental\Py Logging\t_xb.py", line 63, in ?
shutdown()
File "C:\Python23\lib\logging\__init__.py", line 1195, in shutdown
h.flush()
File "C:\Python23\lib\logging\__init__.py", line 661, in flush
self.stream.flush()
ValueError: I/O operation on closed file

What is wrong with it?
'''

from logging import getLogger, Formatter, shutdown, DEBUG, INFO, WARNING, ERROR, CRITICAL
from logging.handlers import RotatingFileHandler

def logger_for(component):
'''
RETURNS a logger for the specified component.

SIDE-EFFECTS
(re)assigns the logger handler
'''

global handler

logger = getLogger(component)

if handler:
handler.flush()
handler.close()
logger.removeHandler(handler)

# In normal, operational mode, the following parameters:
filename = '%s.log'%component
mode = 'a'
maxBytes = 100
backupCount = 5
# would be user-configurable "on the fly" hence the reason for this
function.

handler = RotatingFileHandler(filename, mode, maxBytes, backupCount)

handler.setLevel(DEBUG)

logger.addHandler(handler)

return logger

handler = None

for i in range(20):
log = logger_for('supplier')
log.error('testing Python logging module')

shutdown()
END FILE _________________________________________

Jul 18 '05 #4
j vickroy wrote:

Here is the result of not closing a handler manually -- Peter's suggestion
(3)

Traceback (most recent call last):
File
"C:\Python23\lib\site-packages\Pythonwin\pywin\framework\scriptutils.py" ,
line 310, in RunScript
exec codeObject in __main__.__dict__
File "E:\$PROJECTS\experimental\Py Logging\t_xc.py", line 61, in ?
log.error('testing Python logging module')
File "C:\Python23\lib\logging\__init__.py", line 923, in error
apply(self._log, (ERROR, msg, args), kwargs)
File "C:\Python23\lib\logging\__init__.py", line 994, in _log
self.handle(record)
File "C:\Python23\lib\logging\__init__.py", line 1004, in handle
self.callHandlers(record)
File "C:\Python23\lib\logging\__init__.py", line 1037, in callHandlers
hdlr.handle(record)
File "C:\Python23\lib\logging\__init__.py", line 592, in handle
self.emit(record)
File "C:\Python23\lib\logging\handlers.py", line 105, in emit
self.doRollover()
File "C:\Python23\lib\logging\handlers.py", line 90, in doRollover
os.rename(self.baseFilename, dfn)
OSError: [Errno 13] Permission denied


Works here. Consider switching to an OS where you can rename an open file...
Well, you read my discalimer :-)

Peter
Jul 18 '05 #5
j vickroy wrote:
As a follow-up question, why is a handle object not removed from
logging._handlers when its (i.e., handle) close() procedure is applied?

That behavior appears to be responsible for the logging.shutdown()
failure.


Seems that the author did not consider the use case of consecutively using
different handlers operating on the same file set - I've not yet made up my
mind, if you are misusing the logging system or if that's a bug.

Anyway, as of 2.3.2 the package has still __status__ = "beta", so patches
might be welcome.

Peter
Jul 18 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: max | last post by:
I have an application which calls a module runx, in runx I import logging and get a logger add handlers. I want to be able to rename the file created by fileHandler. several problems, I can't...
0
by: Neil Benn | last post by:
Hello, I'm running a test and having issues with logging, if I call logging.shutdown() and then want to start the logging going again then I get a problem as if I call shutdown, I can't get the...
2
by: flupke | last post by:
Hi, i have a class and a class attribute log which is a logger object. In the __del__() function i want to log a message but it fails even if i use self.__class__.log. The error i get is...
1
by: Oliver Eichler | last post by:
Hi, I experience several exceptions from python's logging system when using the rollover feature on Windows. Traceback (most recent call last): File "c:\Python24\lib\logging\handlers.py",...
1
by: usenet | last post by:
I'm having some problems getting the logging module to work with the threading module. I've narrowed the problem down to the following code: import logging, threading update_log =...
7
by: flupke | last post by:
Hi, i'm getting errors with the log module concerning RotatingFileHandler. I'm using Python 2.4.3 on Windows XP SP2. This used to work in previous python versions but since i upgraded to 2.4.3...
1
by: s99999999s2003 | last post by:
hi i have defined a function def logger(logfile,msg): import logging logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %(levelname)-8s %(message)s', datefmt='%a, %d %b %Y...
3
by: nicholas.petrella | last post by:
I am currently trying to use the python logging system as a core enterprise level logging solution for our development and production environments. The rotating file handler seems to be what I...
2
by: scriptlearner | last post by:
OS: Solaris 9 Python Version: 2.4.4 I need to log certain data in a worker thread; however, I am getting an error now when I use two worker threads. I think the problem comes from the line...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.