473,385 Members | 1,492 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

home > topics > python > questions > line duplication using logging to file

Join Bytes to post your question to a community of 473,385 software developers and data experts.

line duplication using logging to file

seb
Hi, I am writing to a file some basic information using the logging
module. It is working but in the log file some line are printed
several time. I had put some print debugging messages in the logging
function (so they appear on the consile) and they are called once only.
Obviously there is some understantding of the logging module that I am
missing.

My simple logging program (see below) is called by several processes.
In this way I can collect the information from various sources (and not
use the network enabled logging module)

I am using python 2.4 on WinXP SP2.

Do you have any idea ? Thanks in advance.

Seb.

*****************************************
The very simple "logging program ":
*****************************************
import logging, logging.handlers
import time

def write_log(level, message):
# Utilisation de l'API pour le Handler global
print "time.asctime()",time.asctime(),"received level=",level,"message
=",message
nom_logger="main_log_file"
logger=logging.getLogger(nom_logger)
logger.setLevel(logging.DEBUG)
prefix = "pix_main_log_file"
#fh=logging.handlers.RotatingFileHandler(prefix + "_log.txt", 'a',
1000000,10)
fh=logging.FileHandler("main_log.txt")
fh.setLevel(logging.DEBUG)
#formater = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s
- %(message)s")
formater = logging.Formatter("%(asctime)s - %(levelname)s -
%(message)s")
fh.setFormatter(formater)
logger.addHandler(fh)
#logger.info("***************************** debut")
#message = main(url,conf_file,logger)
#message="hello seb"
if str(level).lower() == "info" :
print "logger info"
logger.info(str(message))
elif str(level).lower() =="error":
print "logger error"
logger.error(str(message))
elif str(level).lower()=="warning" :
print "logger warning"
logger.warning(str(message))
elif str(level).lower() =="critical":
print "logger critical"
logger.critical(str(message))
elif str(level).lower() == "exception":
print "logger exception"
logger.exception(str(message))
else :
logger.info("niveau inconnu "+str(message))

print "_________",message
#print dir(logger)
return

*************************************************
Example of the log file with duplicated line.
**************************************************

2007-01-08 18:26:19,578 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___init_rs232initrs232_openC OM1
2007-01-08 18:26:19,578 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___run____thread lance
2007-01-08 18:26:19,578 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___run____thread lance
2007-01-08 18:26:32,015 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:32,015 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:32,015 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:51:26,562 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___init_rs232initrs232_openC OM1
2007-01-09 08:51:26,733 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___run____thread lance
2007-01-09 08:51:26,733 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___run____thread lance
2007-01-09 08:51:39,453 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:39,453 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:39,453 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:48,280 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:48,280 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:48,280 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:48,280 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
**************************
Program calling the log :
***************************
import appel_log
......

message = "something"
nom_function_actuelle= str(sys._getframe().f_code.co_filename)
+"___"+str(sys._getframe().f_code.co_name)
appel_log.write_log("info",nom_function_actuelle+m essage)
.................

Jan 9 '07 #1
3 1587
seb wrote:
Hi, I am writing to a file some basic information using the logging
module. It is working but in the log file some line are printed
several time. I had put some print debugging messages in the logging
function (so they appear on the consile) and they are called once only.
Obviously there is some understantding of the logging module that I am
missing.

My simple logging program (see below) is called by several processes.
In this way I can collect the information from various sources (and not
use the network enabled logging module)

I am using python 2.4 on WinXP SP2.

Do you have any idea ? Thanks in advance.
I copied your two scripts into "appel_main.py" and "appel_log.py" and
when I run, I get a single line in main_log.txt:

2007-01-09 10:01:42,766 - INFO -C:\temp\appel_main.py___?something

On the console is printed:

time.asctime() Tue Jan 09 10:04:25 2007 received level= info message=
C:\temp\appel_main.py___?something
logger info
_________ C:\temp\appel_main.py___?something

I didn't see any repetitions. Perhaps it's to do with multiple
processes in your environment?

If you call write_log several times you will add a handler several
times - this is not a normal pattern of use. You should separate setup
of Loggers/Handlers/Formatters from actually logging events via
debug(), info() etc.

Best regards,

Vinay Sajip

Jan 9 '07 #2
"seb" <se************@laposte.netwrote in message
news:11**********************@11g2000cwr.googlegro ups.com...
Hi, I am writing to a file some basic information using the logging
module. It is working but in the log file some line are printed
several time. I had put some print debugging messages in the logging
function (so they appear on the consile) and they are called once only.
Obviously there is some understantding of the logging module that I am
missing.

My simple logging program (see below) is called by several processes.
In this way I can collect the information from various sources (and not
use the network enabled logging module)

I am using python 2.4 on WinXP SP2.

Do you have any idea ? Thanks in advance.

Seb.
A quick tally of log messages by timestamp and comment gives this data:

('2007-01-08 18:26:19,578', '___init_rs232initrs232_openCOM1') : 1
('2007-01-08 18:26:19,578', '___run____thread lance') : 2
('2007-01-08 18:26:32,015', '___test1TEST 1 = OK') : 3
('2007-01-08 18:26:42,483', '___test1TEST 1 = OK') : 4
('2007-01-08 18:26:53,750', '___test1TEST 1 = OK') : 5
('2007-01-08 18:27:03,092', '___test1TEST 1 = OK') : 6
('2007-01-08 18:27:13,671', '___test1TEST 1 = OK') : 7
('2007-01-08 18:27:14,796', '___run___fin dans le run car continue = 0') : 8
('2007-01-08 18:27:14,890', "___stopthread demande d'arret") : 9
('2007-01-09 08:51:26,562', '___init_rs232initrs232_openCOM1') : 1
('2007-01-09 08:51:26,733', '___run____thread lance') : 2
('2007-01-09 08:51:39,453', '___test1TEST 1 = OK') : 3
('2007-01-09 08:51:48,280', '___test1TEST 1 = OK') : 4
('2007-01-09 08:51:58,750', '___test1TEST 1 = OK') : 5
('2007-01-09 08:52:09,812', '___test1TEST 1 = OK') : 6
('2007-01-09 08:52:19,078', '___test1TEST 1 = OK') : 7
('2007-01-09 08:52:22,078', '___run___fin dans le run car continue = 0') : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret") : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret ") : 1

Does this suggest anything to you?

-- Paul
(BTW, here is the pyparsing program I used to do this analysis)

from pyparsing import
Word,nums,Combine,alphas,oneOf,Literal,SkipTo,rest OfLine

# create pyparsing grammar definition for a log line
date = Word(nums,exact=4)+'-'+Word(nums,exact=2)+'-'+Word(nums,exact=2)
time = Word(nums,exact=2)+':'+Word(nums,exact=2)+':'+Word (nums,exact=2)+ \
','+Word(nums,exact=3)
timestamp = Combine(date + ' ' + time)
severity = oneOf( ["INFO","WARNING"] ) # not complete, but enough for this
data
backslash = Literal("\\")
fileref = Combine(Word(alphas,exact=1)+":" + backslash + SkipTo(".py") +
".py")
logline = ( timestamp.setResultsName("timestamp") + "-" +
severity.setResultsName("severity") + "-" +
fileref.setResultsName("fileref") +
restOfLine.setResultsName("comment") )

# create list of ParseResults, with addressable log line elements by results
name
logEntries = [ logline.parseString(line) for line in logdata ]

# tally up log lines by timestamp and comment
tallyByTimestamp = {}
for entry in logEntries:
tallyKey = (entry.timestamp, entry.comment)
tallyByTimestamp[tallyKey] = tallyByTimestamp.get(tallyKey,0) + 1

for ts in sorted( tallyByTimestamp.items() ):
print "%s : %d" % ts
Jan 9 '07 #3
seb
Hi,

Thanks for the help.
Meanwhile I have written the logging function from scratch and it works
without the multiple lines.

This means that the multiple line copy is not due to the multiple
processes (or thread) trying to access the log file but to something
else.

Thanks.
Sebastien.

the new function :
import logging
import time

def write_log(level, message):
nom_logger="main_log_file.txt"
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s %(levelname)s %(message)s',
filename=nom_logger,
filemode='w')

if str(level).lower() == "info" :
logging.info(str(message))
elif str(level).lower() =="error":
logging.error(str(message))
elif str(level).lower()=="warning" :
logging.warning(str(message))
elif str(level).lower() =="critical":
logging.critical(str(message))
elif str(level).lower() == "exception":
logging.exception(str(message))
else :
logging.INFO(str(message))

return



Paul McGuire a écrit :
"seb" <se************@laposte.netwrote in message
news:11**********************@11g2000cwr.googlegro ups.com...
Hi, I am writing to a file some basic information using the logging
module. It is working but in the log file some line are printed
several time. I had put some print debugging messages in the logging
function (so they appear on the consile) and they are called once only.
Obviously there is some understantding of the logging module that I am
missing.

My simple logging program (see below) is called by several processes.
In this way I can collect the information from various sources (and not
use the network enabled logging module)

I am using python 2.4 on WinXP SP2.

Do you have any idea ? Thanks in advance.

Seb.

A quick tally of log messages by timestamp and comment gives this data:

('2007-01-08 18:26:19,578', '___init_rs232initrs232_openCOM1') : 1
('2007-01-08 18:26:19,578', '___run____thread lance') : 2
('2007-01-08 18:26:32,015', '___test1TEST 1 = OK') : 3
('2007-01-08 18:26:42,483', '___test1TEST 1 = OK') : 4
('2007-01-08 18:26:53,750', '___test1TEST 1 = OK') : 5
('2007-01-08 18:27:03,092', '___test1TEST 1 = OK') : 6
('2007-01-08 18:27:13,671', '___test1TEST 1 = OK') : 7
('2007-01-08 18:27:14,796', '___run___fin dans le run car continue = 0') : 8
('2007-01-08 18:27:14,890', "___stopthread demande d'arret") : 9
('2007-01-09 08:51:26,562', '___init_rs232initrs232_openCOM1') : 1
('2007-01-09 08:51:26,733', '___run____thread lance') : 2
('2007-01-09 08:51:39,453', '___test1TEST 1 = OK') : 3
('2007-01-09 08:51:48,280', '___test1TEST 1 = OK') : 4
('2007-01-09 08:51:58,750', '___test1TEST 1 = OK') : 5
('2007-01-09 08:52:09,812', '___test1TEST 1 = OK') : 6
('2007-01-09 08:52:19,078', '___test1TEST 1 = OK') : 7
('2007-01-09 08:52:22,078', '___run___fin dans le run car continue = 0') : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret") : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret ") : 1

Does this suggest anything to you?

-- Paul
(BTW, here is the pyparsing program I used to do this analysis)

from pyparsing import
Word,nums,Combine,alphas,oneOf,Literal,SkipTo,rest OfLine

# create pyparsing grammar definition for a log line
date = Word(nums,exact=4)+'-'+Word(nums,exact=2)+'-'+Word(nums,exact=2)
time = Word(nums,exact=2)+':'+Word(nums,exact=2)+':'+Word (nums,exact=2)+ \
','+Word(nums,exact=3)
timestamp = Combine(date + ' ' + time)
severity = oneOf( ["INFO","WARNING"] ) # not complete, but enough for this
data
backslash = Literal("\\")
fileref = Combine(Word(alphas,exact=1)+":" + backslash + SkipTo(".py") +
".py")
logline = ( timestamp.setResultsName("timestamp") + "-" +
severity.setResultsName("severity") + "-" +
fileref.setResultsName("fileref") +
restOfLine.setResultsName("comment") )

# create list of ParseResults, with addressable log line elements by results
name
logEntries = [ logline.parseString(line) for line in logdata ]

# tally up log lines by timestamp and comment
tallyByTimestamp = {}
for entry in logEntries:
tallyKey = (entry.timestamp, entry.comment)
tallyByTimestamp[tallyKey] = tallyByTimestamp.get(tallyKey,0) + 1

for ts in sorted( tallyByTimestamp.items() ):
print "%s : %d" % ts
Jan 9 '07 #4
New Post

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
Logging with callers filename and line number
by: Tero Saarni | last post by:
I'm using Logging in my library module for producing trace log of certain events. For being more useful these log entries should be associated with the filename and line number of *users* code...
Python
3
[csv module] duplication of end of line character in output file generated
by: simon.alexandre | last post by:
Hi all, I use csv module included in python 2.3. I use the writer and encouter the following problem: in my output file (.csv) there is a duplication of the end of line character, so when I open...
Python
6
using iostream library how do I display output to console as well as write to a file.
by: radnoraj | last post by:
Hi, I am sucessfull in redirecting console output to a file. but in this case nothing is displayed on the console, cout output is written to file without display. how do write the output to...
C / C++
6
Long: DB2 crash because of logging failure
by: Burkhard Schultheis | last post by:
As I wrote last week, we have a problem with a DB2 V8 on Linux. Here is what is in db2diag.log during online backup: Starting a full database backup. 2004-04-01-02.33.54.760164 ...
DB2 Database
20
Faking inheritance in VBA to remove code duplication
by: Steve Jorgensen | last post by:
A while back, I started boning up on Software Engineering best practices and learning about Agile programming. In the process, I've become much more committed to removing duplication in code at a...
Microsoft Access / VBA
6
Delete a line from a text file
by: tomtown.net | last post by:
Hello I'm trying to get a single line removed from a text file using a search pattern (pretty simple: if line contains "NODE1") -> remove line). To achieve this I's like to operate with only the...
C# / C Sharp
61
how to remove code duplication
by: arnuld | last post by:
I have created a program which creates and renames files. I have described everything in comments. All I have is the cod-duplication. function like fopen, sprint and fwrite are being called again...
C / C++
7
Missing Web log Entry's for file downloads using ashx
by: =?Utf-8?B?QU9UWCBTYW4gQW50b25pbw==?= | last post by:
Hi, I have been using the code (some of it has been removed for simplicity) below to allow authenticated (using ASP.NET membership database) users to get a file from their archive area. It...
ASP.NET
4
assembling dynamic pages, nested duplication of header-trailermaterial
by: henry | last post by:
Folks: Using Dreamweaver CS3... Consider a home page, "index.php" which conditionally REQUIREs one of 'N' HTML files of pure content. All site styles are specified in a master CSS file,...
PHP
1
Cloud Servers without Credit Card and Email Registration: A Simpler Way to Get on the Cloud
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
General
0
One-click Importing Excel Data into a*Database
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
Microsoft Excel
0
Easy Steps to Fix "Canon Printer Won't Connect to WiFi Network"
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
General
0
How to turn on java script in a villaon keypad mobile phone
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
Java
0
Basic Javascript concepts
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
Javascript
0
Merging data from multiple Excel files
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
Data Management
0
Migrating Website to Cloud - Emmanuel Katto
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
General
1
Looking to do Android software development, any suggestions? Is flutter better?
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
General
0
How to build RAID in BIOS?
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
Computer Hardware

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.

BYTES.COM © 2024
About Bytes
Terms Of Use
Privacy Policy
Sitemap
Advertise on Bytes:
Post a Job
Sponsored Posts
Platinum & Gold Sponsors
Hire Now!