I am new to Perl and I am trying for figure out how to solve this
problem. If anyone can give me some suggestions, I would greatly
appreciate it.
I am trying to read a log file and generate some statistics from it.
For simplicity purposes (only) I have edited some of my logs and code.
Below is an example of a log file which has 3 columns (separated by
commas). The first column contains a time stamp, the second column
contains a numeric identifier and the last column contains a NYSE
Symbol:
10:00,123,KO
10:00,124,KO
10:00,123,KO
10:00,123,KO
10:00,125,T
10:00,125,T
10:20,123,KO
10:20,123,KO
10:20,126,YY
10:20,123,KO
10:20,129,PP
10:40,145,YY
10:40,147,MM
11:00,123,KO
11:00,124,KO
11:00,123,KO
11:00,123,KO
11:00,125,T
11:00,125,T
11:20,123,KO
I am trying to determine at each time interval, how many times the
numeric identifier appears. For example, I would like my output to
look something like this:
TIME NUMERIC IDENTIFIER OCCURENCES SYMBOL
10:00 123 3 KO
10:00 124 1 KO
10:00 125 2 TO
10:20 123 3 KO
10:20 126 1 YY
10:20 129 1 PP
10:40 145 1 YY
10:40 147 1 MM
11:00 123 3 KO
11:00 125 2 KO
11:00 124 1 KO
11:20 123 1 KO
Please keep in mind that my log file contains roughly 70,000 rows of
data
I have been working on this some time and I am ABLE to determine the
total number of messages in the time period (see below), but I am
UNABLE to further break it out by numeric identifier. Can anyone
recommend how I would do this? Below is how far I have gotten in the
code so far:
SAMPLE OF CODE:
$samplelog = "test2.log";
open (IN, "$samplelog");
while ($rcd = <IN>){
@fields = split(/,/,$rcd);
$time{$fields[0]}++;
}
foreach $key (sort keys(%time)) {
print "$key $time{$key} \n";
}
Thank you 2 2761
In article <ef*************************@posting.google.com> , scadav
<dk*******@yahoo.com> wrote: I am new to Perl and I am trying for figure out how to solve this problem. If anyone can give me some suggestions, I would greatly appreciate it.
I am trying to read a log file and generate some statistics from it. For simplicity purposes (only) I have edited some of my logs and code.
Below is an example of a log file which has 3 columns (separated by commas). The first column contains a time stamp, the second column contains a numeric identifier and the last column contains a NYSE Symbol:
[data snipped (see below program, below)]
I am trying to determine at each time interval, how many times the numeric identifier appears. For example, I would like my output to look something like this:
TIME NUMERIC IDENTIFIER OCCURENCES SYMBOL 10:00 123 3 KO 10:00 124 1 KO 10:00 125 2 TO 10:20 123 3 KO 10:20 126 1 YY 10:20 129 1 PP 10:40 145 1 YY 10:40 147 1 MM 11:00 123 3 KO 11:00 125 2 KO 11:00 124 1 KO 11:20 123 1 KO
Please keep in mind that my log file contains roughly 70,000 rows of data
I have been working on this some time and I am ABLE to determine the total number of messages in the time period (see below), but I am UNABLE to further break it out by numeric identifier. Can anyone recommend how I would do this? Below is how far I have gotten in the code so far: SAMPLE OF CODE:
$samplelog = "test2.log"; open (IN, "$samplelog");
while ($rcd = <IN>){ @fields = split(/,/,$rcd); $time{$fields[0]}++;
}
foreach $key (sort keys(%time)) { print "$key $time{$key} \n"; }
Use the entire record as your key. Use chomp first to remove the
newline at the end. Then split apart the record, which is also the key
to the hash, to do the printing:
Jim 48% cat scadav.pl
#!/usr/local/bin/perl
use strict;
use warnings;
my @fields;
my %time;
while (my $rcd = <DATA>){
chomp($rcd);
$time{$rcd}++;
}
foreach my $entry (sort keys(%time)) {
my($tim,$num,$id) = split(/,/,$entry);
printf " %5s %3d %3d %s\n", $tim, $num, $time{$entry}, $id;
}
__END__
10:00,123,KO
10:00,124,KO
10:00,123,KO
10:00,123,KO
10:00,125,T
10:00,125,T
10:20,123,KO
10:20,123,KO
10:20,126,YY
10:20,123,KO
10:20,129,PP
10:40,145,YY
10:40,147,MM
11:00,123,KO
11:00,124,KO
11:00,123,KO
11:00,123,KO
11:00,125,T
11:00,125,T
11:20,123,KO
Jim 49% ./scadav.pl
10:00 123 3 KO
10:00 124 1 KO
10:00 125 2 T
10:20 123 3 KO
10:20 126 1 YY
10:20 129 1 PP
10:40 145 1 YY
10:40 147 1 MM
11:00 123 3 KO
11:00 124 1 KO
11:00 125 2 T
11:20 123 1 KO
Jim 50%
If you wish to break down your data in different ways or sort, then you
need to consider more complicated solutions such as defining a
hash-of-hashes or a hash-of-arrays to store your multi-level data.
Some more pointers for those new to Perl:
1. Always put 'use strict' and 'use warnings' at the beginning of your
program, then declare all of your variables with 'my' or 'our' or use a
package name for globals.
2. Always check the results of an open call (and all other system
calls, as well):
open (IN, $samplelog) or die("Can't open $samplelog: $!";
3. There is no need to put double-quotes around $samplelog in the above.
4. Post further miscellaneous Perl questions to comp.lang.perl.misc, as
this newsgroup is defunct, but be sure and check the guidelines for
that newsgroup before doing so: http://mail.augustmail.com/~tadmc/cl...uidelines.html
Jim Gibson <jg*****@mail.arc.nasa.gov> wrote in
news:020720041255540732%jg*****@mail.arc.nasa.gov: In article <ef*************************@posting.google.com> , scadav <dk*******@yahoo.com> wrote:
I am new to Perl and I am trying for figure out how to solve this problem. If anyone can give me some suggestions, I would greatly appreciate it.
I am trying to read a log file and generate some statistics from it. For simplicity purposes (only) I have edited some of my logs and code.
Below is an example of a log file which has 3 columns (separated by commas). The first column contains a time stamp, the second column contains a numeric identifier and the last column contains a NYSE Symbol: [data snipped (see below program, below)]
I am trying to determine at each time interval, how many times the numeric identifier appears. For example, I would like my output to look something like this:
TIME NUMERIC IDENTIFIER OCCURENCES SYMBOL 10:00 123 3 KO 10:00 124 1 KO 10:00 125 2 TO 10:20 123 3 KO 10:20 126 1 YY 10:20 129 1 PP 10:40 145 1 YY 10:40 147 1 MM 11:00 123 3 KO 11:00 125 2 KO 11:00 124 1 KO 11:20 123 1 KO
Please keep in mind that my log file contains roughly 70,000 rows of data
I have been working on this some time and I am ABLE to determine the total number of messages in the time period (see below), but I am UNABLE to further break it out by numeric identifier. Can anyone recommend how I would do this? Below is how far I have gotten in the code so far: SAMPLE OF CODE:
$samplelog = "test2.log"; open (IN, "$samplelog");
while ($rcd = <IN>){ @fields = split(/,/,$rcd); $time{$fields[0]}++;
}
foreach $key (sort keys(%time)) { print "$key $time{$key} \n"; }
Use the entire record as your key. Use chomp first to remove the newline at the end. Then split apart the record, which is also the key to the hash, to do the printing:
Jim 48% cat scadav.pl #!/usr/local/bin/perl
use strict; use warnings;
my @fields; my %time;
while (my $rcd = <DATA>){ chomp($rcd); $time{$rcd}++; }
foreach my $entry (sort keys(%time)) { my($tim,$num,$id) = split(/,/,$entry); printf " %5s %3d %3d %s\n", $tim, $num, $time{$entry}, $id; } __END__ 10:00,123,KO 10:00,124,KO 10:00,123,KO 10:00,123,KO 10:00,125,T 10:00,125,T 10:20,123,KO 10:20,123,KO 10:20,126,YY 10:20,123,KO 10:20,129,PP 10:40,145,YY 10:40,147,MM 11:00,123,KO 11:00,124,KO 11:00,123,KO 11:00,123,KO 11:00,125,T 11:00,125,T 11:20,123,KO
Jim 49% ./scadav.pl 10:00 123 3 KO 10:00 124 1 KO 10:00 125 2 T 10:20 123 3 KO 10:20 126 1 YY 10:20 129 1 PP 10:40 145 1 YY 10:40 147 1 MM 11:00 123 3 KO 11:00 124 1 KO 11:00 125 2 T 11:20 123 1 KO Jim 50%
If you wish to break down your data in different ways or sort, then you need to consider more complicated solutions such as defining a hash-of-hashes or a hash-of-arrays to store your multi-level data.
Some more pointers for those new to Perl:
1. Always put 'use strict' and 'use warnings' at the beginning of your program, then declare all of your variables with 'my' or 'our' or use a package name for globals.
2. Always check the results of an open call (and all other system calls, as well):
open (IN, $samplelog) or die("Can't open $samplelog: $!";
3. There is no need to put double-quotes around $samplelog in the above.
4. Post further miscellaneous Perl questions to comp.lang.perl.misc, as this newsgroup is defunct, but be sure and check the guidelines for that newsgroup before doing so:
http://mail.augustmail.com/~tadmc/cl...uidelines.html
Thank you for your assistance and suggestions it is much appreciated. This discussion thread is closed Replies have been disabled for this discussion. Similar topics
13 posts
views
Thread by Wayne Folta |
last post: by
|
2 posts
views
Thread by Daniel Tonks |
last post: by
|
reply
views
Thread by Mike Chirico |
last post: by
|
9 posts
views
Thread by Dieter Vanderelst |
last post: by
|
2 posts
views
Thread by kavikailash |
last post: by
|
10 posts
views
Thread by stylecomputers |
last post: by
|
6 posts
views
Thread by Henry Law |
last post: by
|
2 posts
views
Thread by MK |
last post: by
| | | | | | | | | | | |