473,385 Members | 1,192 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

home > topics > perl > questions > extract lines from a file

Join Bytes to post your question to a community of 473,385 software developers and data experts.

Extract lines from a file

I have this log ...

Expand|Select|Wrap|Line Numbers
  1. [ERROR] [02 Sep 2009 11:33:17] [Trace] [WebContainer : 54] - infrastructure:ID_UNHANDLED: An un-handled server exception occu
  2. rred. Please contact your administrator.
  3.         at .util.internal.HandleException.getRemoteException(HandleException.java(Compiled Code))
  4.         at .util.internal.HandleException.getRemoteException(HandleException.java(Compiled Co
  5. nested exception is:
  6. infrastructure:RUN_ID_RUNTIME: A runtime exception occurred: javax.transaction.RollbackException.
  7.  at java.lang.reflect.Method.invoke(Method.java(Compiled Code))
  8.         at com.ibm.rmi.util.ProxyUtil$2.run(ProxyUtil.java(Compiled Code))
  9.         at java.security.AccessController.doPrivileged1(Native Method)
  10.         at java.security.AccessController.doPrivileged(AccessController.java(Compiled Code))
  11.         at com.ibm.rmi.util.ProxyUtil.invokeWithPrivilege(ProxyUtil.java(Compiled Code))
  12.         at com.ibm.CORBA.iiop.ClientDelegate.invoke(ClientDelegate.ja
  13.         at .interfaces.Search_TH.invokeStub
  14. nested exception is:
  15. javax.transaction.RollbackException
  16.         at com.ibm.ws.Transaction.JTA.TransactionImpl.stage3CommitProcessing(TransactionImpl.java(Compiled Code))
  17.         at com.ibm.ws.Transaction.JTA.TransactionImpl.processCommit(TransactionImpl.java(Compiled Code))
  18.         at com.ibm.ws.Transaction.JTA.TransactionImpl.commit(TransactionImpl.java(Compiled Code))
  19.         at com.ibm.ws.Transaction.JTA.TranManagerImpl.commit(TranManagerImpl.java(Compiled Code))
  20.         at com.ibm.ws.Transaction.JTA.TranManagerSet.commit(TranManagerSet.java(Compiled Code))
  21.         at com.ibm.ws.Transaction.JTA.UserTransactionImpl.commit(UserTransactionImpl.java(Compiled Code))
  22.         at com.ibm.ejs.container.UserTransactionWrapper.commit(UserTransactionWrapper.java(Compiled Code))
  23.         at .util.transaction.ResourcesJTA.M-T00000(ResourcesJTA.java(Compiled Code))
  24.         at .util.transaction.TransactionInfo.commit(TransactionInfo.java(Compiled C
  25.  
  26. [ERROR] [02 Sep 2009 11:33:18] [Trace.Tools] [WebContainer : 61] - infrastructure:RUN_ID_RECORD_NOT_FOUND: Record not found.
  27. [ERROR] [02 Sep 2009 11:33:18] [Trace.Tools] [WebContainer : 61] - infrastructure:RUN_ID_RECORD_NOT_FOUND: Record not found.
  28.  
I need to input it into a perl script that takes 2 arguments, an input file, i.e. the log above and an output file, that will contain the output,
eg. Rollback.pl input.log report.log

First, I need to search for the string "UNHANDLED" and within that stack trace, I need to find the string "RollbackException". If the "RollbackException" string is found, I need to find the "_TH" string, that's within "RollbackException".
Finally I need to print the date that's on the "UNHANDLED" line and the line that contains the "_TH" string.

The "UNHANDLED" string could contain more than 1 "RollbackException" string but I need to find just the 1st instance of "RollbackException" and then within that find the first instance of "_TH".

The delimiters could be [ERROR].

The date and "_TH" string should then be output to the "report.log" file.
Sep 2 '09 #1
5 3384
numberwhun
3,509 Expert Mod 2GB
@joeferns79
Ok, so what have you tried thus far? Please show your code.

Regards,

Jeff
Sep 2 '09 #2
Expand|Select|Wrap|Line Numbers
  1.  
  2. use strict;
  3.  
  4.  
  5. ####################################################################
  6. # constants
  7. ####################################################################
  8.  
  9. use constant COUNT   => 0;
  10. use constant MAX     => 1;
  11. use constant MIN     => 2;
  12. use constant TOTAL   => 3;
  13. use constant AVERAGE => 4;
  14. use constant FCOUNT   => 5;
  15. use constant FMAX   => 6;
  16. use constant FMIN   => 7;
  17. use constant FTOTAL  => 8;
  18. use constant FAVERAGE   => 9;
  19.  
  20. ####################################################################
  21. # genreport
  22. ####################################################################
  23.  
  24. sub genreport {
  25.    my ($logfile, $reportfile) = @_;
  26.    my ($line, $elapsed, $facade, %calls);
  27.    my (@fields, $count, $min, $max, $total, $average,$fcount,$fmin,$fmax,$ftotal,$faverage);
  28.    my (@list, $totalcalls);
  29.    my ($falseflag);
  30.    my ($statdate,$server);
  31.    my ($starttime,$error,$first);
  32.   open(LOGFILE, "<$logfile")
  33.       or die "Can't open $logfile: $!";
  34.  
  35.    open(REPORTFILE, ">$reportfile")
  36.       or die "Can't open $reportfile: $!";
  37.  
  38.    $totalcalls = 0;
  39.    while ($line = <LOGFILE>) {
  40.       chomp $line;
  41.       $falseflag= 0;
  42.       $statdate = substr($line,9,11);
  43.       $starttime = substr($line,21,8);
  44.     if($line =~(/UNHANDLED/))
  45.      {
  46.  while ($line =<LOGFILE>){
  47.  if($line =~(/RollbackException/))
  48.       {
  49.    $falseflag = 1;
  50. while ($line =<LOGFILE>){
  51.  if($line =~(/_TH/)) {
  52.          $line = substr($line,3,length($line));
  53.          printf "%s, %s,%s\n",$statdate,$starttime,$line;
  54.          last;
  55.         }
  56.       }
  57.     }
  58.  }
  59. }
  60. }
  61. }
  62. ####################################################################
  63. # main
  64. ####################################################################
  65.  
  66. if (@ARGV != 2) {
  67.    die "usage: servercalls.pl <logfile> <reportfile>";
  68. }
  69.  
  70. my ($logfile, $reportfile) = @ARGV;
  71.  
  72. genreport($logfile, $reportfile);
  73.  
  74.  
Sep 3 '09 #3
KevinADC
4,059 Expert 2GB
How come you have not added the filehandle name where you want to redirect output to a file like Jeff showed you?
Sep 3 '09 #4
Sorry, the one I made a change to was another script. Here's this one with the change....

Expand|Select|Wrap|Line Numbers
  1. use strict; 
  2.  
  3.  
  4. #################################################################### 
  5. # constants 
  6. #################################################################### 
  7.  
  8. use constant COUNT   => 0; 
  9. use constant MAX     => 1; 
  10. use constant MIN     => 2; 
  11. use constant TOTAL   => 3; 
  12. use constant AVERAGE => 4; 
  13. use constant FCOUNT   => 5; 
  14. use constant FMAX   => 6; 
  15. use constant FMIN   => 7; 
  16. use constant FTOTAL  => 8; 
  17. use constant FAVERAGE   => 9; 
  18.  
  19. #################################################################### 
  20. # genreport 
  21. #################################################################### 
  22.  
  23. sub genreport { 
  24.    my ($logfile, $reportfile) = @_; 
  25.    my ($line, $elapsed, $facade, %calls); 
  26.    my (@fields, $count, $min, $max, $total, $average,$fcount,$fmin,$fmax,$ftotal,$faverage); 
  27.    my (@list, $totalcalls); 
  28.    my ($falseflag); 
  29.    my ($statdate,$server); 
  30.    my ($starttime,$error,$first); 
  31.   open(LOGFILE, "<$logfile") 
  32.       or die "Can't open $logfile: $!"; 
  33.  
  34.    open(REPORTFILE, ">$reportfile") 
  35.       or die "Can't open $reportfile: $!"; 
  36.  
  37.    $totalcalls = 0; 
  38.    while ($line = <LOGFILE>) { 
  39.       chomp $line; 
  40.       $falseflag= 0; 
  41.       $statdate = substr($line,9,11); 
  42.       $starttime = substr($line,21,8); 
  43.     if($line =~(/UNHANDLED/)) 
  44.      { 
  45.  while ($line =<LOGFILE>){ 
  46.  if($line =~(/RollbackException/)) 
  47.       { 
  48.    $falseflag = 1; 
  49. while ($line =<LOGFILE>){ 
  50.  if($line =~(/_TH/)) { 
  51.          $line = substr($line,3,length($line)); 
  52.          printf REPORTFILE "%s, %s,%s\n",$statdate,$starttime,$line; 
  53.          last; 
  54.         } 
  55.       } 
  56.     } 
  57.  } 
  58. } 
  59. } 
  60. } 
  61. #################################################################### 
  62. # main 
  63. #################################################################### 
  64.  
  65. if (@ARGV != 2) { 
  66.    die "usage: servercalls.pl <logfile> <reportfile>"; 
  67. } 
  68.  
  69. my ($logfile, $reportfile) = @ARGV; 
  70.  
  71. genreport($logfile, $reportfile); 
  72.  
  73.  
Sep 3 '09 #5
Somehow, the line that's supposed to be printed just once is getting repeated 4 times, since the line with the "_TH" string is present in 4 locations under the stack trace.

i.e. the input log looks like this...

Expand|Select|Wrap|Line Numbers
  1.  
  2. [ERROR] [02 Sep 2009 11:33:17] [Trace] [WebContainer : 54] - infrastructure:ID_UNHANDLED: An un-handled server exception occu 
  3. rred. Please contact your administrator. 
  4.         at .util.internal.HandleException.getRemoteException(HandleException.java(Compiled Code)) 
  5.         at .util.internal.HandleException.getRemoteException(HandleException.java(Compiled Co 
  6. at .interfaces.Search_TH.invokeStub 
  7. nested exception is: 
  8. infrastructure:RUN_ID_RUNTIME: A runtime exception occurred: javax.transaction.RollbackException. 
  9.  at java.lang.reflect.Method.invoke(Method.java(Compiled Code)) 
  10.         at com.ibm.rmi.util.ProxyUtil$2.run(ProxyUtil.java(Compiled Code)) 
  11.         at java.security.AccessController.doPrivileged1(Native Method) 
  12.         at java.security.AccessController.doPrivileged(AccessController.java(Compiled Code)) 
  13.         at com.ibm.rmi.util.ProxyUtil.invokeWithPrivilege(ProxyUtil.java(Compiled Code)) 
  14.         at com.ibm.CORBA.iiop.ClientDelegate.invoke(ClientDelegate.ja 
  15.         at .interfaces.Search_TH.invokeStub 
  16. nested exception is: 
  17. javax.transaction.RollbackException 
  18.         at com.ibm.ws.Transaction.JTA.TransactionImpl.stage3CommitProcessing(TransactionImpl.java(Compiled Code)) 
  19.         at com.ibm.ws.Transaction.JTA.TransactionImpl.processCommit(TransactionImpl.java(Compiled Code)) 
  20.         at com.ibm.ws.Transaction.JTA.TransactionImpl.commit(TransactionImpl.java(Compiled Code)) 
  21.         at com.ibm.ws.Transaction.JTA.TranManagerImpl.commit(TranManagerImpl.java(Compiled Code)) 
  22.         at com.ibm.ws.Transaction.JTA.TranManagerSet.commit(TranManagerSet.java(Compiled Code)) 
  23.         at com.ibm.ws.Transaction.JTA.UserTransactionImpl.commit(UserTransactionImpl.java(Compiled Code)) 
  24.         at com.ibm.ejs.container.UserTransactionWrapper.commit(UserTransactionWrapper.java(Compiled Code)) 
  25.         at .util.transaction.ResourcesJTA.M-T00000(ResourcesJTA.java(Compiled Code)) 
  26.         at .util.transaction.TransactionInfo.commit(TransactionInfo.java(Compiled C 
  27.   at .interfaces.Search_TH.invokeStub 
  28. nested exception is: 
  29. javax.transaction.RollbackException 
  30.         at com.ibm.ws.Transaction.JTA.TransactionImpl.stage3CommitProcessing(TransactionImpl.java(Compiled Code)) 
  31.         at com.ibm.ws.Transaction.JTA.TransactionImpl.processCommit(TransactionImpl.java(Compiled Code)) 
  32.         at com.ibm.ws.Transaction.JTA.TransactionImpl.commit(TransactionImpl.java(Compiled Code)) 
  33.         at com.ibm.ws.Transaction.JTA.TranManagerImpl.commit(TranManagerImpl.java(Compiled Code)) 
  34.         at com.ibm.ws.Transaction.JTA.TranManagerSet.commit(TranManagerSet.java(Compiled Code)) 
  35.         at com.ibm.ws.Transaction.JTA.UserTransactionImpl.commit(UserTransactionImpl.java(Compiled Code)) 
  36.         at com.ibm.ejs.container.UserTransactionWrapper.commit(UserTransactionWrapper.java(Compiled Code)) 
  37.         at .util.transaction.ResourcesJTA.M-T00000(ResourcesJTA.java(Compiled Code)) 
  38.         at .util.transaction.TransactionInfo.commit(TransactionInfo.java(Compiled C 
  39.   at .interfaces.Search_TH.invokeStub 
  40. nested exception is: 
  41. javax.transaction.RollbackException 
  42.         at com.ibm.ws.Transaction.JTA.TransactionImpl.stage3CommitProcessing(TransactionImpl.java(Compiled Code)) 
  43.         at com.ibm.ws.Transaction.JTA.TransactionImpl.processCommit(TransactionImpl.java(Compiled Code)) 
  44.         at com.ibm.ws.Transaction.JTA.TransactionImpl.commit(TransactionImpl.java(Compiled Code)) 
  45.         at com.ibm.ws.Transaction.JTA.TranManagerImpl.commit(TranManagerImpl.java(Compiled Code)) 
  46.         at com.ibm.ws.Transaction.JTA.TranManagerSet.commit(TranManagerSet.java(Compiled Code)) 
  47.         at com.ibm.ws.Transaction.JTA.UserTransactionImpl.commit(UserTransactionImpl.java(Compiled Code)) 
  48.         at com.ibm.ejs.container.UserTransactionWrapper.commit(UserTransactionWrapper.java(Compiled Code)) 
  49.         at .util.transaction.ResourcesJTA.M-T00000(ResourcesJTA.java(Compiled Code)) 
  50.         at .util.transaction.TransactionInfo.commit(TransactionInfo.java(Compiled C 
  51.   at .interfaces.Search_TH.invokeStub 
  52.  
  53. [ERROR] [02 Sep 2009 11:33:18] [Trace.Tools] [WebContainer : 61] - infrastructure:RUN_ID_RECORD_NOT_FOUND: Record not found. 
  54. [ERROR] [02 Sep 2009 11:33:18] [Trace.Tools] [WebContainer : 61] - infrastructure:RUN_ID_RECORD_NOT_FOUND: Record not found.
  55.  
  56.  
I want to extract the line that contains the "_TH" string only from the 1st instance after finding the 1st "RollbackException" string.
Sep 3 '09 #6

Sign in to post your reply or Sign up for a free account.

Similar topics

1
Q: Extract only text lines visible in webbrowser?
by: Flemse | last post by:
I need to web query for further processing, f.ex http://moneycentral.msn.com/investor/external/excel/quotes.asp?SYMBOL=F,MSFT,DE I use <?php...
PHP
9
C: extract string from file
by: Sharon | last post by:
hi, I want to extract a string from a file, if the file is like this: 1 This is the string 2 3 4 how could I extract the string, starting from the 10th position (i.e. "T") and...
C / C++
10
Extract a function from C code?
by: Robert Schultz | last post by:
I have a C/C++ file that I simply want to 'extract' a function from. Something like: extract <function name> <c or cpp file> I want it to return from the beginning of the function, to the end. ...
C / C++
4
php script to filter a text file and extract lines starting with keyword?
by: anne001 | last post by:
Hi For a class, students are going to run an experiment on line. Each time a subject runs, his/her data is appended to one giant text file. Their own data set will be just one line starting with...
PHP
8
extract data from file
by: nick | last post by:
Hi all can any one please tell me what is wrong in this code?? I'm new to deal with text files and extract data. i'm trying to look for data in a text file (3~4 pages) some lines start with a...
C / C++
8
extract certain values from file with re
by: Fabian Braennstroem | last post by:
Hi, I would like to remove certain lines from a log files. I had some sed/awk scripts for this, but now, I want to use python with its re module for this task. Actually, I have two different...
Python
7
Extract zip file from email attachment
by: erikcw | last post by:
Hi all, I'm trying to extract zip file (containing an xml file) from an email so I can process it. But I'm running up against some brick walls. I've been googling and reading all afternoon, and...
Python
4
[OpenCV] Extract en recognize barcode from image file.
by: kurapix | last post by:
Hi there!!! I've been working on recognizing EAN barcodes from images. Yes it's about recognizing barcodes from images but from bars not from the digit with the help of OCR software but with...
C / C++
45
extract all hotmail email addresses in a file and store in separatefile
by: Dennis | last post by:
Hi, I have a text file that contents a list of email addresses like this: "foo@yahoo.com" "tom@hotmail.com" "jerry@gmail.com" "tommy@apple.com" I like to
C / C++
18
how to extract part of HTML page
by: Ecka | last post by:
Hi everyone, I'm trying to write a PHP script that connects to a bank's currency convertor page using cURL and that part works fine. The issue is that I end up with a page that includes a lot...
PHP
1
Cloud Servers without Credit Card and Email Registration: A Simpler Way to Get on the Cloud
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
General
0
Easy Steps to Fix "Canon Printer Won't Connect to WiFi Network"
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
General
0
How to turn on java script in a villaon keypad mobile phone
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
Java
0
Basic Javascript concepts
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
Javascript
0
Batch import of multiple excel files into the database
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
Data Management
0
BarryA
Navigating the Data Structures and Algorithms (DSA)
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
Algorithms / Advanced Math
1
Looking to do Android software development, any suggestions? Is flutter better?
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
General
1
Is that possible of reading the .csv file in column wise and the column have different lengths ?
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
C / C++
0
How to build RAID in BIOS?
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
Computer Hardware

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.

BYTES.COM © 2024
About Bytes
Terms Of Use
Privacy Policy
Sitemap
Advertise on Bytes:
Post a Job
Sponsored Posts
Platinum & Gold Sponsors
Hire Now!