Connecting Tech Pros Worldwide Forums | Help | Site Map
bytes > topic > perl > answers

Extract lines from a file

Member
  
Join Date: Sep 2008
Posts: 35
#1: Sep 2 '09
I have this log ...

Expand|Select|Wrap|Line Numbers
  1. [ERROR] [02 Sep 2009 11:33:17] [Trace] [WebContainer : 54] - infrastructure:ID_UNHANDLED: An un-handled server exception occu
  2. rred. Please contact your administrator.
  3.         at .util.internal.HandleException.getRemoteException(HandleException.java(Compiled Code))
  4.         at .util.internal.HandleException.getRemoteException(HandleException.java(Compiled Co
  5. nested exception is:
  6. infrastructure:RUN_ID_RUNTIME: A runtime exception occurred: javax.transaction.RollbackException.
  7.  at java.lang.reflect.Method.invoke(Method.java(Compiled Code))
  8.         at com.ibm.rmi.util.ProxyUtil$2.run(ProxyUtil.java(Compiled Code))
  9.         at java.security.AccessController.doPrivileged1(Native Method)
  10.         at java.security.AccessController.doPrivileged(AccessController.java(Compiled Code))
  11.         at com.ibm.rmi.util.ProxyUtil.invokeWithPrivilege(ProxyUtil.java(Compiled Code))
  12.         at com.ibm.CORBA.iiop.ClientDelegate.invoke(ClientDelegate.ja
  13.         at .interfaces.Search_TH.invokeStub
  14. nested exception is:
  15. javax.transaction.RollbackException
  16.         at com.ibm.ws.Transaction.JTA.TransactionImpl.stage3CommitProcessing(TransactionImpl.java(Compiled Code))
  17.         at com.ibm.ws.Transaction.JTA.TransactionImpl.processCommit(TransactionImpl.java(Compiled Code))
  18.         at com.ibm.ws.Transaction.JTA.TransactionImpl.commit(TransactionImpl.java(Compiled Code))
  19.         at com.ibm.ws.Transaction.JTA.TranManagerImpl.commit(TranManagerImpl.java(Compiled Code))
  20.         at com.ibm.ws.Transaction.JTA.TranManagerSet.commit(TranManagerSet.java(Compiled Code))
  21.         at com.ibm.ws.Transaction.JTA.UserTransactionImpl.commit(UserTransactionImpl.java(Compiled Code))
  22.         at com.ibm.ejs.container.UserTransactionWrapper.commit(UserTransactionWrapper.java(Compiled Code))
  23.         at .util.transaction.ResourcesJTA.M-T00000(ResourcesJTA.java(Compiled Code))
  24.         at .util.transaction.TransactionInfo.commit(TransactionInfo.java(Compiled C
  25.  
  26. [ERROR] [02 Sep 2009 11:33:18] [Trace.Tools] [WebContainer : 61] - infrastructure:RUN_ID_RECORD_NOT_FOUND: Record not found.
  27. [ERROR] [02 Sep 2009 11:33:18] [Trace.Tools] [WebContainer : 61] - infrastructure:RUN_ID_RECORD_NOT_FOUND: Record not found.
  28.  
I need to input it into a perl script that takes 2 arguments, an input file, i.e. the log above and an output file, that will contain the output,
eg. Rollback.pl input.log report.log

First, I need to search for the string "UNHANDLED" and within that stack trace, I need to find the string "RollbackException". If the "RollbackException" string is found, I need to find the "_TH" string, that's within "RollbackException".
Finally I need to print the date that's on the "UNHANDLED" line and the line that contains the "_TH" string.

The "UNHANDLED" string could contain more than 1 "RollbackException" string but I need to find just the 1st instance of "RollbackException" and then within that find the first instance of "_TH".

The delimiters could be [ERROR].

The date and "_TH" string should then be output to the "report.log" file.

numberwhun's Avatar
Site Moderator
  
Join Date: May 2007
Location: New Hampshire
Posts: 2,572
#2: Sep 3 '09

re: Extract lines from a file

Quote:

Originally Posted by joeferns79 View Post

I have this log ...

Expand|Select|Wrap|Line Numbers
  1. [ERROR] [02 Sep 2009 11:33:17] [Trace] [WebContainer : 54] - infrastructure:ID_UNHANDLED: An un-handled server exception occu
  2. rred. Please contact your administrator.
  3.         at .util.internal.HandleException.getRemoteException(HandleException.java(Compiled Code))
  4.         at .util.internal.HandleException.getRemoteException(HandleException.java(Compiled Co
  5. nested exception is:
  6. infrastructure:RUN_ID_RUNTIME: A runtime exception occurred: javax.transaction.RollbackException.
  7.  at java.lang.reflect.Method.invoke(Method.java(Compiled Code))
  8.         at com.ibm.rmi.util.ProxyUtil$2.run(ProxyUtil.java(Compiled Code))
  9.         at java.security.AccessController.doPrivileged1(Native Method)
  10.         at java.security.AccessController.doPrivileged(AccessController.java(Compiled Code))
  11.         at com.ibm.rmi.util.ProxyUtil.invokeWithPrivilege(ProxyUtil.java(Compiled Code))
  12.         at com.ibm.CORBA.iiop.ClientDelegate.invoke(ClientDelegate.ja
  13.         at .interfaces.Search_TH.invokeStub
  14. nested exception is:
  15. javax.transaction.RollbackException
  16.         at com.ibm.ws.Transaction.JTA.TransactionImpl.stage3CommitProcessing(TransactionImpl.java(Compiled Code))
  17.         at com.ibm.ws.Transaction.JTA.TransactionImpl.processCommit(TransactionImpl.java(Compiled Code))
  18.         at com.ibm.ws.Transaction.JTA.TransactionImpl.commit(TransactionImpl.java(Compiled Code))
  19.         at com.ibm.ws.Transaction.JTA.TranManagerImpl.commit(TranManagerImpl.java(Compiled Code))
  20.         at com.ibm.ws.Transaction.JTA.TranManagerSet.commit(TranManagerSet.java(Compiled Code))
  21.         at com.ibm.ws.Transaction.JTA.UserTransactionImpl.commit(UserTransactionImpl.java(Compiled Code))
  22.         at com.ibm.ejs.container.UserTransactionWrapper.commit(UserTransactionWrapper.java(Compiled Code))
  23.         at .util.transaction.ResourcesJTA.M-T00000(ResourcesJTA.java(Compiled Code))
  24.         at .util.transaction.TransactionInfo.commit(TransactionInfo.java(Compiled C
  25.  
  26. [ERROR] [02 Sep 2009 11:33:18] [Trace.Tools] [WebContainer : 61] - infrastructure:RUN_ID_RECORD_NOT_FOUND: Record not found.
  27. [ERROR] [02 Sep 2009 11:33:18] [Trace.Tools] [WebContainer : 61] - infrastructure:RUN_ID_RECORD_NOT_FOUND: Record not found.
  28.  
I need to input it into a perl script that takes 2 arguments, an input file, i.e. the log above and an output file, that will contain the output,
eg. Rollback.pl input.log report.log

First, I need to search for the string "UNHANDLED" and within that stack trace, I need to find the string "RollbackException". If the "RollbackException" string is found, I need to find the "_TH" string, that's within "RollbackException".
Finally I need to print the date that's on the "UNHANDLED" line and the line that contains the "_TH" string.

The "UNHANDLED" string could contain more than 1 "RollbackException" string but I need to find just the 1st instance of "RollbackException" and then within that find the first instance of "_TH".

The delimiters could be [ERROR].

The date and "_TH" string should then be output to the "report.log" file.

Ok, so what have you tried thus far? Please show your code.

Regards,

Jeff
Member
  
Join Date: Sep 2008
Posts: 35
#3: Sep 3 '09

re: Extract lines from a file

Expand|Select|Wrap|Line Numbers
  1.  
  2. use strict;
  3.  
  4.  
  5. ####################################################################
  6. # constants
  7. ####################################################################
  8.  
  9. use constant COUNT   => 0;
  10. use constant MAX     => 1;
  11. use constant MIN     => 2;
  12. use constant TOTAL   => 3;
  13. use constant AVERAGE => 4;
  14. use constant FCOUNT   => 5;
  15. use constant FMAX   => 6;
  16. use constant FMIN   => 7;
  17. use constant FTOTAL  => 8;
  18. use constant FAVERAGE   => 9;
  19.  
  20. ####################################################################
  21. # genreport
  22. ####################################################################
  23.  
  24. sub genreport {
  25.    my ($logfile, $reportfile) = @_;
  26.    my ($line, $elapsed, $facade, %calls);
  27.    my (@fields, $count, $min, $max, $total, $average,$fcount,$fmin,$fmax,$ftotal,$faverage);
  28.    my (@list, $totalcalls);
  29.    my ($falseflag);
  30.    my ($statdate,$server);
  31.    my ($starttime,$error,$first);
  32.   open(LOGFILE, "<$logfile")
  33.       or die "Can't open $logfile: $!";
  34.  
  35.    open(REPORTFILE, ">$reportfile")
  36.       or die "Can't open $reportfile: $!";
  37.  
  38.    $totalcalls = 0;
  39.    while ($line = <LOGFILE>) {
  40.       chomp $line;
  41.       $falseflag= 0;
  42.       $statdate = substr($line,9,11);
  43.       $starttime = substr($line,21,8);
  44.     if($line =~(/UNHANDLED/))
  45.      {
  46.  while ($line =<LOGFILE>){
  47.  if($line =~(/RollbackException/))
  48.       {
  49.    $falseflag = 1;
  50. while ($line =<LOGFILE>){
  51.  if($line =~(/_TH/)) {
  52.          $line = substr($line,3,length($line));
  53.          printf "%s, %s,%s\n",$statdate,$starttime,$line;
  54.          last;
  55.         }
  56.       }
  57.     }
  58.  }
  59. }
  60. }
  61. }
  62. ####################################################################
  63. # main
  64. ####################################################################
  65.  
  66. if (@ARGV != 2) {
  67.    die "usage: servercalls.pl <logfile> <reportfile>";
  68. }
  69.  
  70. my ($logfile, $reportfile) = @ARGV;
  71.  
  72. genreport($logfile, $reportfile);
  73.  
  74.  
KevinADC's Avatar
Expert
  
Join Date: Jan 2007
Location: Southern California USA
Posts: 4,091
#4: Sep 3 '09

re: Extract lines from a file

How come you have not added the filehandle name where you want to redirect output to a file like Jeff showed you?
Member
  
Join Date: Sep 2008
Posts: 35
#5: Sep 3 '09

re: Extract lines from a file

Sorry, the one I made a change to was another script. Here's this one with the change....

Expand|Select|Wrap|Line Numbers
  1. use strict; 
  2.  
  3.  
  4. #################################################################### 
  5. # constants 
  6. #################################################################### 
  7.  
  8. use constant COUNT   => 0; 
  9. use constant MAX     => 1; 
  10. use constant MIN     => 2; 
  11. use constant TOTAL   => 3; 
  12. use constant AVERAGE => 4; 
  13. use constant FCOUNT   => 5; 
  14. use constant FMAX   => 6; 
  15. use constant FMIN   => 7; 
  16. use constant FTOTAL  => 8; 
  17. use constant FAVERAGE   => 9; 
  18.  
  19. #################################################################### 
  20. # genreport 
  21. #################################################################### 
  22.  
  23. sub genreport { 
  24.    my ($logfile, $reportfile) = @_; 
  25.    my ($line, $elapsed, $facade, %calls); 
  26.    my (@fields, $count, $min, $max, $total, $average,$fcount,$fmin,$fmax,$ftotal,$faverage); 
  27.    my (@list, $totalcalls); 
  28.    my ($falseflag); 
  29.    my ($statdate,$server); 
  30.    my ($starttime,$error,$first); 
  31.   open(LOGFILE, "<$logfile") 
  32.       or die "Can't open $logfile: $!"; 
  33.  
  34.    open(REPORTFILE, ">$reportfile") 
  35.       or die "Can't open $reportfile: $!"; 
  36.  
  37.    $totalcalls = 0; 
  38.    while ($line = <LOGFILE>) { 
  39.       chomp $line; 
  40.       $falseflag= 0; 
  41.       $statdate = substr($line,9,11); 
  42.       $starttime = substr($line,21,8); 
  43.     if($line =~(/UNHANDLED/)) 
  44.      { 
  45.  while ($line =<LOGFILE>){ 
  46.  if($line =~(/RollbackException/)) 
  47.       { 
  48.    $falseflag = 1; 
  49. while ($line =<LOGFILE>){ 
  50.  if($line =~(/_TH/)) { 
  51.          $line = substr($line,3,length($line)); 
  52.          printf REPORTFILE "%s, %s,%s\n",$statdate,$starttime,$line; 
  53.          last; 
  54.         } 
  55.       } 
  56.     } 
  57.  } 
  58. } 
  59. } 
  60. } 
  61. #################################################################### 
  62. # main 
  63. #################################################################### 
  64.  
  65. if (@ARGV != 2) { 
  66.    die "usage: servercalls.pl <logfile> <reportfile>"; 
  67. } 
  68.  
  69. my ($logfile, $reportfile) = @ARGV; 
  70.  
  71. genreport($logfile, $reportfile); 
  72.  
  73.  
Member
  
Join Date: Sep 2008
Posts: 35
#6: Sep 3 '09

re: Extract lines from a file

Somehow, the line that's supposed to be printed just once is getting repeated 4 times, since the line with the "_TH" string is present in 4 locations under the stack trace.

i.e. the input log looks like this...

Expand|Select|Wrap|Line Numbers
  1.  
  2. [ERROR] [02 Sep 2009 11:33:17] [Trace] [WebContainer : 54] - infrastructure:ID_UNHANDLED: An un-handled server exception occu 
  3. rred. Please contact your administrator. 
  4.         at .util.internal.HandleException.getRemoteException(HandleException.java(Compiled Code)) 
  5.         at .util.internal.HandleException.getRemoteException(HandleException.java(Compiled Co 
  6. at .interfaces.Search_TH.invokeStub 
  7. nested exception is: 
  8. infrastructure:RUN_ID_RUNTIME: A runtime exception occurred: javax.transaction.RollbackException. 
  9.  at java.lang.reflect.Method.invoke(Method.java(Compiled Code)) 
  10.         at com.ibm.rmi.util.ProxyUtil$2.run(ProxyUtil.java(Compiled Code)) 
  11.         at java.security.AccessController.doPrivileged1(Native Method) 
  12.         at java.security.AccessController.doPrivileged(AccessController.java(Compiled Code)) 
  13.         at com.ibm.rmi.util.ProxyUtil.invokeWithPrivilege(ProxyUtil.java(Compiled Code)) 
  14.         at com.ibm.CORBA.iiop.ClientDelegate.invoke(ClientDelegate.ja 
  15.         at .interfaces.Search_TH.invokeStub 
  16. nested exception is: 
  17. javax.transaction.RollbackException 
  18.         at com.ibm.ws.Transaction.JTA.TransactionImpl.stage3CommitProcessing(TransactionImpl.java(Compiled Code)) 
  19.         at com.ibm.ws.Transaction.JTA.TransactionImpl.processCommit(TransactionImpl.java(Compiled Code)) 
  20.         at com.ibm.ws.Transaction.JTA.TransactionImpl.commit(TransactionImpl.java(Compiled Code)) 
  21.         at com.ibm.ws.Transaction.JTA.TranManagerImpl.commit(TranManagerImpl.java(Compiled Code)) 
  22.         at com.ibm.ws.Transaction.JTA.TranManagerSet.commit(TranManagerSet.java(Compiled Code)) 
  23.         at com.ibm.ws.Transaction.JTA.UserTransactionImpl.commit(UserTransactionImpl.java(Compiled Code)) 
  24.         at com.ibm.ejs.container.UserTransactionWrapper.commit(UserTransactionWrapper.java(Compiled Code)) 
  25.         at .util.transaction.ResourcesJTA.M-T00000(ResourcesJTA.java(Compiled Code)) 
  26.         at .util.transaction.TransactionInfo.commit(TransactionInfo.java(Compiled C 
  27.   at .interfaces.Search_TH.invokeStub 
  28. nested exception is: 
  29. javax.transaction.RollbackException 
  30.         at com.ibm.ws.Transaction.JTA.TransactionImpl.stage3CommitProcessing(TransactionImpl.java(Compiled Code)) 
  31.         at com.ibm.ws.Transaction.JTA.TransactionImpl.processCommit(TransactionImpl.java(Compiled Code)) 
  32.         at com.ibm.ws.Transaction.JTA.TransactionImpl.commit(TransactionImpl.java(Compiled Code)) 
  33.         at com.ibm.ws.Transaction.JTA.TranManagerImpl.commit(TranManagerImpl.java(Compiled Code)) 
  34.         at com.ibm.ws.Transaction.JTA.TranManagerSet.commit(TranManagerSet.java(Compiled Code)) 
  35.         at com.ibm.ws.Transaction.JTA.UserTransactionImpl.commit(UserTransactionImpl.java(Compiled Code)) 
  36.         at com.ibm.ejs.container.UserTransactionWrapper.commit(UserTransactionWrapper.java(Compiled Code)) 
  37.         at .util.transaction.ResourcesJTA.M-T00000(ResourcesJTA.java(Compiled Code)) 
  38.         at .util.transaction.TransactionInfo.commit(TransactionInfo.java(Compiled C 
  39.   at .interfaces.Search_TH.invokeStub 
  40. nested exception is: 
  41. javax.transaction.RollbackException 
  42.         at com.ibm.ws.Transaction.JTA.TransactionImpl.stage3CommitProcessing(TransactionImpl.java(Compiled Code)) 
  43.         at com.ibm.ws.Transaction.JTA.TransactionImpl.processCommit(TransactionImpl.java(Compiled Code)) 
  44.         at com.ibm.ws.Transaction.JTA.TransactionImpl.commit(TransactionImpl.java(Compiled Code)) 
  45.         at com.ibm.ws.Transaction.JTA.TranManagerImpl.commit(TranManagerImpl.java(Compiled Code)) 
  46.         at com.ibm.ws.Transaction.JTA.TranManagerSet.commit(TranManagerSet.java(Compiled Code)) 
  47.         at com.ibm.ws.Transaction.JTA.UserTransactionImpl.commit(UserTransactionImpl.java(Compiled Code)) 
  48.         at com.ibm.ejs.container.UserTransactionWrapper.commit(UserTransactionWrapper.java(Compiled Code)) 
  49.         at .util.transaction.ResourcesJTA.M-T00000(ResourcesJTA.java(Compiled Code)) 
  50.         at .util.transaction.TransactionInfo.commit(TransactionInfo.java(Compiled C 
  51.   at .interfaces.Search_TH.invokeStub 
  52.  
  53. [ERROR] [02 Sep 2009 11:33:18] [Trace.Tools] [WebContainer : 61] - infrastructure:RUN_ID_RECORD_NOT_FOUND: Record not found. 
  54. [ERROR] [02 Sep 2009 11:33:18] [Trace.Tools] [WebContainer : 61] - infrastructure:RUN_ID_RECORD_NOT_FOUND: Record not found.
  55.  
  56.  
I want to extract the line that contains the "_TH" string only from the 1st instance after finding the 1st "RollbackException" string.
Reply

« previous question | next question »

Similar Perl bytes

/bytes/development

/bytes/it

Forums
Visit our community forums for general discussions and latest on Bytes

Our staff and community areas have moved to bytes.com/forums/

/bytes/about

We are a network of experts and professionals in IT and software development that help one another with answers to tough questions and share insights. Get the best answers to your questions from over 226,501 network members.

dev questions:
algorithms | asp | asp.net | c/c++ | coldfusion | c# | flash | html/css | java | javascript |
mobile dev | .net | perl | php | python | ruby | software dev | vb.net | visual basic | xml
it questions:
access | apache | careers | db2 | iis | macosx | mysql | networking | oracle | postgresql | sql server | unix | windows

About Bytes | Copyright 1995-2009 BYTES. All rights Reserved