473,857 Members | 1,773 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

RotatingFileHan dler bugs/errors and a general logging question.

I am currently trying to use the python logging system as a core
enterprise level logging solution for our development and production
environments.

The rotating file handler seems to be what I am looking for as I want
the ability to have control over the number and size of log files that
are written out for each of our tools. I have noticed a few problems
with this handler and wanted to post here to get your impressions and
possibly some ideas about whether these issues can be resolved.
The first issue is with multiple copies of the same tool trying to log
to the same location. This should not be an issue as the libraries are
supposed to be thread safe and therefore also should be safe for
multiple instances of a tool. I have run into two problems with
this...
1.
When a log file is rolled over, occasionally we see the following
traceback in the other instance or instances of the tool:

Traceback (most recent call last):
File "/usr/local/lib/python2.4/logging/handlers.py", line 62, in
emit
if self.shouldRoll over(record):
File "/usr/local/lib/python2.4/logging/handlers.py", line 132, in
shouldRollover
self.stream.see k(0, 2) #due to non-posix-compliant Windows
feature
ValueError: I/O operation on closed file

As best I can tell this seems to be caused by instance A closing the
log file and rolling it over and instance B is still trying to use
it's file handle to that log file. Except that A has replaced the file
during rollover. It seems that a likely solution would be to handle
the exception and reopen the log file. It seems that the newer
WatchedFileHand ler (http://www.trentm.com/python/dailyhtml/lib/
node414.html) provides the functionality that is needed, but I think
it would be helpful to have the functionality included with the
RotaingFileHand ler to prevent these errors.

2.
I am seeing that at times when two instances of a tool are logging,
the log will be rotated twice. It seems that ass app.log approaches
the size limeit (10 MB in my case), the rollover is triggered in both
instances of the application causing a small log file to be created.
>ls -l
-rw-rw-rw- 1 petrella user 10485641 May 8 16:23 app.log
-rw-rw-rw- 1 petrella user 2758383 May 8 16:22 app.log.1 <----
Small log
-rw-rw-rw- 1 petrella user 10485903 May 8 16:22 app.log.2
-rw-rw-rw- 1 petrella user 2436167 May 8 16:21 app.log.3

It seems that the rollover should also be protected so that the log
file is not rolled twice.


I also wanted to ask for anyone's thoughts on maybe a better way to
implement python logging to meet our needs.

The infrastructure in which I am work needs the ability to have log
files written to from multiple instances of the same script and
potentially from hundreds or more different machines.

I know that the documentation suggests using a network logging server
but I wanted to know if anyone had any other solutions to allow us to
build off of the current python logging packages.

Thanks in advance for any of your responses.

-Nick

May 8 '07 #1
3 5419
On May 9, 12:52 am, nicholas.petre. ..@gmail.com wrote:
The infrastructure in which I am work needs the ability to have log
files written to from multiple instances of the same script and
potentially from hundreds or more different machines.

I know that the documentation suggests using a networkloggings erver
but I wanted to know if anyone had any other solutions to allow us to
build off of the current pythonloggingpa ckages.
Dennis is right - the logging system is threadsafe but not safe
against multiple processes (separate Python instances) writing to the
same file. It certainly sounds like you need a scalable solution - and
having each script send the events to a network logging server seems a
good way of handling the scalability requirement. The logger name used
can include the script instance and machine name, e.g. by starting
with hostname.script name.scriptpid. .. The socket server which receives
the events can demultiplex them based on this information and write
them to a central repository in any arrangement you care to implement
(e.g. into one file or several).

Given that the example in the docs is a (basic) working example, is
there any particular reason why you don't want to follow the suggested
approach?

Regards,

Vinay Sajip

May 9 '07 #2
On May 9, 12:37 am, Vinay Sajip <vinay_sa...@ya hoo.co.ukwrote:
On May 9, 12:52 am, nicholas.petre. ..@gmail.com wrote:The infrastructure in which I am work needs the ability to have log
files written to from multiple instances of the same script and
potentially from hundreds or more different machines.
I know that the documentation suggests using a networkloggings erver
but I wanted to know if anyone had any other solutions to allow us to
build off of the current pythonloggingpa ckages.

Dennis is right - the logging system is threadsafe but not safe
against multiple processes (separate Python instances) writing to the
samefile. It certainly sounds like you need a scalable solution - and
having each script send the events to a network logging server seems a
good way of handling the scalability requirement. The logger name used
can include the script instance and machine name, e.g. by starting
with hostname.script name.scriptpid. .. The socket server which receives
the events can demultiplex them based on this information and write
them to a central repository in any arrangement you care to implement
(e.g. into onefileor several).

Given that the example in the docs is a (basic) working example, is
there any particular reason why you don't want to follow the suggested
approach?

Regards,

Vinay Sajip

Our biggest concerns with the network solution is having a single
point of failure and the need for scalability. We could have
potentially thousands of machines doing logging across multiple
geographic sites. Our fear with the network solution is overwhelming
the server or group of servers as well as having a single point of
failure for the logging interface. In addition using a server or
servers would require added support for the logigng server machines.
Our NFS infrastructure is very well supported and can handle the load
generated by these machines already (A load which would be many times
more than what the logging would generate) which is why we would like
to log directly to file system without going through a separate
server. Also adding in a logging server introduces one more level
where we could potentially have failure. We would like to keep the
infrastructure for our logging as simple as possible as we rely on log
files to give us critical information when troubleshooting issues.

It sounds like my only option may be using a server in order to handle
the logging from different hosts. That or possibly having individual
log files for each host.

Thanks for your input. It is much appreciated.

-Nick

May 10 '07 #3
On May 10, 6:37 pm, nicholas.petre. ..@gmail.com wrote:
On May 9, 12:37 am, Vinay Sajip <vinay_sa...@ya hoo.co.ukwrote:

Our biggest concerns with the network solution is having a single
point of failure and the need for scalability. We could have
potentially thousands of machines doingloggingacr oss multiple
geographic sites. Our fear with the network solution is overwhelming
the server or group of servers as well as having a single point of
failure for thelogginginter face. In addition using a server or
servers would require added support for the logigng server machines.
Our NFS infrastructure is very well supported and can handle the load
generated by these machines already (A load which would be many times
more than what theloggingwould generate) which is why we would like
to log directly to file system without going through a separate
server. Also adding in aloggingserver introduces one more level
where we could potentially have failure. We would like to keep the
infrastructure for ourloggingas simple as possible as we rely on log
files to give us critical information when troubleshooting issues.

It sounds like my only option may be using a server in order to handle
theloggingfrom different hosts. That or possibly having individual
log files for each host.
There are other options - you don't necessarily need a separate
logging server. For example, you could

(a) Have a single network receiver process per host which writes to
disk and avoids the problem of contention for the file. Although this
process could be a point of failure, it's a pretty simple piece of
software and it should be possible to manage the risks.
(b) If you wanted to centralize log information, you could move the
log files from each host onto a central NFS disk using standard tools
such as e.g. rsync, and then manipulate them for reporting purposes
however you want.

Regards,

Vinay Sajip

May 10 '07 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
2460
by: Neal D. Becker | last post by:
Why isn't my formatter used when I use DatagramHandler? Here's the server: #!/usr/bin/env python import socket import logging import pickle sock = socket.socket (socket.AF_INET, socket.SOCK_DGRAM)
1
1207
by: Yosh | last post by:
I am looking for a mail UI component for .Net. I know how to send emails using the System.Web namespace but I am interested in a component with a UI built in? Does anyone know of a component or have any suggestions? Thanks Yosh
3
2070
by: Lauren Quantrell | last post by:
A general design question: Assuming I can figure out a way to link some local tables in an .MDB file to my Access2000 .ADP database (any help on this is appreciated as well), I'm wondering which of the following methods will yield faster performance over a slow WAN network: Method A: Creating tables on my SQL Server to store temporary records that are linked to records in permanent SQL Server tables. Method B: Linking tables in a...
1
1209
by: rutledj | last post by:
Is there anyway to force logging to the file (writing to the log file) immediately as each log command is executed? Currently, it appears to write the log when the program ends. Thanks, Rut
2
1438
by: ZorpiedoMan | last post by:
I'm new to the world of sockets, and this question is not VB specific: If multiple clients access the same server on the same port, and the server is set up to do some async communication, does the server's response back to all the clients on that port, or just to the one who sent the request? In other words: Client One - Request Data From Server (It takes a few seconds for the server to get the answer)
2
1302
by: Ed L. | last post by:
In testing slony with pg 8.0 (from cvs), I see a number of these "CONTEXT" log statements showing up for what appear to be successfully executed saveplans. I see the CONTEXT statements are controlled by log_error_verbosity in postgresql.conf. I want these CONTEXT log statements for *errors*, but not for non-error statements. Is that possible? Ed
2
1184
by: roy anderson | last post by:
Hey all, I'm not new to web development, but totally clueless regarding web security/logins and such. I have a website which requires users to login, checks their info against a SQL Server backend, then saves their login as a session variable. As they browse through the various pages of the site, each page checks to see if the Session variable exists/is not null, etc. If the check fails, the page redirects them to the login form. ...
2
1361
by: Angus | last post by:
I want to setup a macro to log informational strings if eg INFOLOGGING is defined. So I created a macro which outputs informational strings to a log file. I want to have a #define which switches this option on or off. My problem is that I need to do a load of string processing to build up the informational string. Then the informational string is passed
3
1630
by: =?Utf-8?B?Ymxi?= | last post by:
I am posting to the general discussion group - but I cannot find my postings... or replies...
0
9923
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, weíll explore What is ONU, What Is Router, ONU & Routerís main usage, and What is the difference between ONU and Router. Letís take a closer look ! Part I. Meaning of...
0
9768
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
11083
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10711
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10808
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
7104
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5971
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4592
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
3215
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.