By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,092 Members | 1,546 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,092 IT Pros & Developers. It's quick & easy.

line duplication using logging to file

P: n/a
seb
Hi, I am writing to a file some basic information using the logging
module. It is working but in the log file some line are printed
several time. I had put some print debugging messages in the logging
function (so they appear on the consile) and they are called once only.
Obviously there is some understantding of the logging module that I am
missing.

My simple logging program (see below) is called by several processes.
In this way I can collect the information from various sources (and not
use the network enabled logging module)

I am using python 2.4 on WinXP SP2.

Do you have any idea ? Thanks in advance.

Seb.

*****************************************
The very simple "logging program ":
*****************************************
import logging, logging.handlers
import time

def write_log(level, message):
# Utilisation de l'API pour le Handler global
print "time.asctime()",time.asctime(),"received level=",level,"message
=",message
nom_logger="main_log_file"
logger=logging.getLogger(nom_logger)
logger.setLevel(logging.DEBUG)
prefix = "pix_main_log_file"
#fh=logging.handlers.RotatingFileHandler(prefix + "_log.txt", 'a',
1000000,10)
fh=logging.FileHandler("main_log.txt")
fh.setLevel(logging.DEBUG)
#formater = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s
- %(message)s")
formater = logging.Formatter("%(asctime)s - %(levelname)s -
%(message)s")
fh.setFormatter(formater)
logger.addHandler(fh)
#logger.info("***************************** debut")
#message = main(url,conf_file,logger)
#message="hello seb"
if str(level).lower() == "info" :
print "logger info"
logger.info(str(message))
elif str(level).lower() =="error":
print "logger error"
logger.error(str(message))
elif str(level).lower()=="warning" :
print "logger warning"
logger.warning(str(message))
elif str(level).lower() =="critical":
print "logger critical"
logger.critical(str(message))
elif str(level).lower() == "exception":
print "logger exception"
logger.exception(str(message))
else :
logger.info("niveau inconnu "+str(message))

print "_________",message
#print dir(logger)
return

*************************************************
Example of the log file with duplicated line.
**************************************************

2007-01-08 18:26:19,578 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___init_rs232initrs232_openC OM1
2007-01-08 18:26:19,578 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___run____thread lance
2007-01-08 18:26:19,578 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___run____thread lance
2007-01-08 18:26:32,015 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:32,015 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:32,015 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:42,483 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:26:53,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:03,092 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:13,671 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,796 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-08 18:27:14,890 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:51:26,562 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___init_rs232initrs232_openC OM1
2007-01-09 08:51:26,733 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___run____thread lance
2007-01-09 08:51:26,733 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version
simple\thread_RS232.py___run____thread lance
2007-01-09 08:51:39,453 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:39,453 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:39,453 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:48,280 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:48,280 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:48,280 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:48,280 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:51:58,750 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:09,812 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:19,078 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\Client_tests.py___test1TEST
1 = OK
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,078 - WARNING - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___run___fin
dans le run car continue = 0
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
2007-01-09 08:52:22,125 - INFO - C:\Documents and
Settings\test\Bureau\xmlrpc\version simple\thread_RS232.py___stopthread
demande d'arret
**************************
Program calling the log :
***************************
import appel_log
......

message = "something"
nom_function_actuelle= str(sys._getframe().f_code.co_filename)
+"___"+str(sys._getframe().f_code.co_name)
appel_log.write_log("info",nom_function_actuelle+m essage)
.................

Jan 9 '07 #1
Share this Question
Share on Google+
3 Replies


P: n/a
seb wrote:
Hi, I am writing to a file some basic information using the logging
module. It is working but in the log file some line are printed
several time. I had put some print debugging messages in the logging
function (so they appear on the consile) and they are called once only.
Obviously there is some understantding of the logging module that I am
missing.

My simple logging program (see below) is called by several processes.
In this way I can collect the information from various sources (and not
use the network enabled logging module)

I am using python 2.4 on WinXP SP2.

Do you have any idea ? Thanks in advance.
I copied your two scripts into "appel_main.py" and "appel_log.py" and
when I run, I get a single line in main_log.txt:

2007-01-09 10:01:42,766 - INFO -C:\temp\appel_main.py___?something

On the console is printed:

time.asctime() Tue Jan 09 10:04:25 2007 received level= info message=
C:\temp\appel_main.py___?something
logger info
_________ C:\temp\appel_main.py___?something

I didn't see any repetitions. Perhaps it's to do with multiple
processes in your environment?

If you call write_log several times you will add a handler several
times - this is not a normal pattern of use. You should separate setup
of Loggers/Handlers/Formatters from actually logging events via
debug(), info() etc.

Best regards,

Vinay Sajip

Jan 9 '07 #2

P: n/a
"seb" <se************@laposte.netwrote in message
news:11**********************@11g2000cwr.googlegro ups.com...
Hi, I am writing to a file some basic information using the logging
module. It is working but in the log file some line are printed
several time. I had put some print debugging messages in the logging
function (so they appear on the consile) and they are called once only.
Obviously there is some understantding of the logging module that I am
missing.

My simple logging program (see below) is called by several processes.
In this way I can collect the information from various sources (and not
use the network enabled logging module)

I am using python 2.4 on WinXP SP2.

Do you have any idea ? Thanks in advance.

Seb.
A quick tally of log messages by timestamp and comment gives this data:

('2007-01-08 18:26:19,578', '___init_rs232initrs232_openCOM1') : 1
('2007-01-08 18:26:19,578', '___run____thread lance') : 2
('2007-01-08 18:26:32,015', '___test1TEST 1 = OK') : 3
('2007-01-08 18:26:42,483', '___test1TEST 1 = OK') : 4
('2007-01-08 18:26:53,750', '___test1TEST 1 = OK') : 5
('2007-01-08 18:27:03,092', '___test1TEST 1 = OK') : 6
('2007-01-08 18:27:13,671', '___test1TEST 1 = OK') : 7
('2007-01-08 18:27:14,796', '___run___fin dans le run car continue = 0') : 8
('2007-01-08 18:27:14,890', "___stopthread demande d'arret") : 9
('2007-01-09 08:51:26,562', '___init_rs232initrs232_openCOM1') : 1
('2007-01-09 08:51:26,733', '___run____thread lance') : 2
('2007-01-09 08:51:39,453', '___test1TEST 1 = OK') : 3
('2007-01-09 08:51:48,280', '___test1TEST 1 = OK') : 4
('2007-01-09 08:51:58,750', '___test1TEST 1 = OK') : 5
('2007-01-09 08:52:09,812', '___test1TEST 1 = OK') : 6
('2007-01-09 08:52:19,078', '___test1TEST 1 = OK') : 7
('2007-01-09 08:52:22,078', '___run___fin dans le run car continue = 0') : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret") : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret ") : 1

Does this suggest anything to you?

-- Paul
(BTW, here is the pyparsing program I used to do this analysis)

from pyparsing import
Word,nums,Combine,alphas,oneOf,Literal,SkipTo,rest OfLine

# create pyparsing grammar definition for a log line
date = Word(nums,exact=4)+'-'+Word(nums,exact=2)+'-'+Word(nums,exact=2)
time = Word(nums,exact=2)+':'+Word(nums,exact=2)+':'+Word (nums,exact=2)+ \
','+Word(nums,exact=3)
timestamp = Combine(date + ' ' + time)
severity = oneOf( ["INFO","WARNING"] ) # not complete, but enough for this
data
backslash = Literal("\\")
fileref = Combine(Word(alphas,exact=1)+":" + backslash + SkipTo(".py") +
".py")
logline = ( timestamp.setResultsName("timestamp") + "-" +
severity.setResultsName("severity") + "-" +
fileref.setResultsName("fileref") +
restOfLine.setResultsName("comment") )

# create list of ParseResults, with addressable log line elements by results
name
logEntries = [ logline.parseString(line) for line in logdata ]

# tally up log lines by timestamp and comment
tallyByTimestamp = {}
for entry in logEntries:
tallyKey = (entry.timestamp, entry.comment)
tallyByTimestamp[tallyKey] = tallyByTimestamp.get(tallyKey,0) + 1

for ts in sorted( tallyByTimestamp.items() ):
print "%s : %d" % ts
Jan 9 '07 #3

P: n/a
seb
Hi,

Thanks for the help.
Meanwhile I have written the logging function from scratch and it works
without the multiple lines.

This means that the multiple line copy is not due to the multiple
processes (or thread) trying to access the log file but to something
else.

Thanks.
Sebastien.

the new function :
import logging
import time

def write_log(level, message):
nom_logger="main_log_file.txt"
logging.basicConfig(level=logging.DEBUG,
format='%(asctime)s %(levelname)s %(message)s',
filename=nom_logger,
filemode='w')

if str(level).lower() == "info" :
logging.info(str(message))
elif str(level).lower() =="error":
logging.error(str(message))
elif str(level).lower()=="warning" :
logging.warning(str(message))
elif str(level).lower() =="critical":
logging.critical(str(message))
elif str(level).lower() == "exception":
logging.exception(str(message))
else :
logging.INFO(str(message))

return



Paul McGuire a écrit :
"seb" <se************@laposte.netwrote in message
news:11**********************@11g2000cwr.googlegro ups.com...
Hi, I am writing to a file some basic information using the logging
module. It is working but in the log file some line are printed
several time. I had put some print debugging messages in the logging
function (so they appear on the consile) and they are called once only.
Obviously there is some understantding of the logging module that I am
missing.

My simple logging program (see below) is called by several processes.
In this way I can collect the information from various sources (and not
use the network enabled logging module)

I am using python 2.4 on WinXP SP2.

Do you have any idea ? Thanks in advance.

Seb.

A quick tally of log messages by timestamp and comment gives this data:

('2007-01-08 18:26:19,578', '___init_rs232initrs232_openCOM1') : 1
('2007-01-08 18:26:19,578', '___run____thread lance') : 2
('2007-01-08 18:26:32,015', '___test1TEST 1 = OK') : 3
('2007-01-08 18:26:42,483', '___test1TEST 1 = OK') : 4
('2007-01-08 18:26:53,750', '___test1TEST 1 = OK') : 5
('2007-01-08 18:27:03,092', '___test1TEST 1 = OK') : 6
('2007-01-08 18:27:13,671', '___test1TEST 1 = OK') : 7
('2007-01-08 18:27:14,796', '___run___fin dans le run car continue = 0') : 8
('2007-01-08 18:27:14,890', "___stopthread demande d'arret") : 9
('2007-01-09 08:51:26,562', '___init_rs232initrs232_openCOM1') : 1
('2007-01-09 08:51:26,733', '___run____thread lance') : 2
('2007-01-09 08:51:39,453', '___test1TEST 1 = OK') : 3
('2007-01-09 08:51:48,280', '___test1TEST 1 = OK') : 4
('2007-01-09 08:51:58,750', '___test1TEST 1 = OK') : 5
('2007-01-09 08:52:09,812', '___test1TEST 1 = OK') : 6
('2007-01-09 08:52:19,078', '___test1TEST 1 = OK') : 7
('2007-01-09 08:52:22,078', '___run___fin dans le run car continue = 0') : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret") : 8
('2007-01-09 08:52:22,125', "___stopthread demande d'arret ") : 1

Does this suggest anything to you?

-- Paul
(BTW, here is the pyparsing program I used to do this analysis)

from pyparsing import
Word,nums,Combine,alphas,oneOf,Literal,SkipTo,rest OfLine

# create pyparsing grammar definition for a log line
date = Word(nums,exact=4)+'-'+Word(nums,exact=2)+'-'+Word(nums,exact=2)
time = Word(nums,exact=2)+':'+Word(nums,exact=2)+':'+Word (nums,exact=2)+ \
','+Word(nums,exact=3)
timestamp = Combine(date + ' ' + time)
severity = oneOf( ["INFO","WARNING"] ) # not complete, but enough for this
data
backslash = Literal("\\")
fileref = Combine(Word(alphas,exact=1)+":" + backslash + SkipTo(".py") +
".py")
logline = ( timestamp.setResultsName("timestamp") + "-" +
severity.setResultsName("severity") + "-" +
fileref.setResultsName("fileref") +
restOfLine.setResultsName("comment") )

# create list of ParseResults, with addressable log line elements by results
name
logEntries = [ logline.parseString(line) for line in logdata ]

# tally up log lines by timestamp and comment
tallyByTimestamp = {}
for entry in logEntries:
tallyKey = (entry.timestamp, entry.comment)
tallyByTimestamp[tallyKey] = tallyByTimestamp.get(tallyKey,0) + 1

for ts in sorted( tallyByTimestamp.items() ):
print "%s : %d" % ts
Jan 9 '07 #4

This discussion thread is closed

Replies have been disabled for this discussion.