Searching files by matching filename pattern and concatenating contents of the files

[joeferns79]

Hi,
I wanted to write a Perl script that searches a given folder for all files that have filenames based on the previous day's date.

eg. if the filenames of the files in the said folder are ....
server1_Statistics_0.20090618T124050
server1_Statistics_0.20090619T080652
server1_Statistics_0.20090220T124502
server1_Statistics_0.20090621T105927
server1_Statistics_0.20090621T105638
server1_Statistics_0.20090621T105611
server1_Statistics_0.20090621T105518
server1_Statistics_0.20090621T105108

find all files with yesterday's date(20090621) and concatenate their contents into a single file with the filename "server1_Statistics_20090621"

[RonB]

What have you tried?

What errors and/or warnings are you receiving?

What portion of the task do you not know how to accomplish?

[joeferns79]

I am relatively new to Perl and we have it installed on AIX. I have worked with the same on Linux and when I tried getting the previous day's date itself, I am having trouble...

#!/usr/bin/perl
 
$today=`date --date='1 day ago' +%Y%m%d`;
 
 

I tried using the following command, but I was having trouble parsing the columns, to get only the date ...

perl -e 'print localtime(time() - 86400) . "\n"'
 

Even after the date has been captured, the month is shown in words, whereas I want it to match the filenames which are numerals.

[RonB]

Use the strftime function from the POSIX module.

use POSIX 'strftime';
 
my $date = strftime("%Y%m%d", localtime(time - 86400));
print $date;
 

[joeferns79]

That worked fine, thanks.

I am now trying to include the output from that into a search for filenames containing yesterday's date.

#!/usr/bin/perl
$hostname=`hostname`;
chomp($hostname);
#$today=`perl -e 'print localtime(time() - 86400) . "\n"'`;
#chomp($today);
 
#!/usr/bin/perl
 
use POSIX 'strftime';
 
my $date = strftime("%Y%m%d", localtime(time - 86400)). "\n";
#print $date;
 
$wldir="/usr/IBM/WebSphere/AppServer/logs/test";
#$wlarc=$wldir/$date;
 
#print "Archiving $wldir directory\n";
do_dir($wldir);
 
 
 
sub do_dir {
    my $dir = shift;
    opendir(D, $dir);
    my @f = readdir(D);
    closedir(D);
    foreach my $file (@f)
       {
        my $filename = $dir . '/' . $file;
        if ($file eq '.' || $file eq '..')
          {
          }
        elsif (-d $filename)
          {
            # depending on your needs you can do subdirs
            do_dir($filename);
          }
        else {
            # do something with $filename, like ...
            if ($filename =~ m/$date/)
              {
              print "Concatenating $filename\n";
              `cat $filename >> $date`;
              }
             }
      }
  }
 
exit(0);
 
 
 

Even though it doesn't throw an error, it doesn't concatenate either.

[joeferns79]

I tried the following script as well, to no avail...

 
#!/usr/bin/perl
 
use POSIX 'strftime';
 
my $date = strftime("%Y%m%d", localtime(time - 86400)). "\n";
#print $date;
 
$wldir="/usr/IBM/WebSphere/AppServer/logs/test";
#$wlarc=$wldir/$date;
 
do_dir($wldir);
 
 
 
sub do_dir {
    my $dir = shift;
    opendir(D, $dir);
    my @f = readdir(D);
    closedir(D);
    foreach my $file (@f)
       {
        my $filename = $dir . '/' . $file;
        if ($file eq '.' || $file eq '..')
          {
          }
        elsif (-d $filename)
          {
            # depending on your needs you can do subdirs
            do_dir($filename);
          }
        else {
            # do something with $filename, like ...
            if ($filename =~ m/server1(.*)\.$date(.*)$/)
              {
              print "Concatenating $filename\n";
              `cat $filename >> $date`;
              }
             }
      }
  }
 
 

If I use ...

if ($filename =~ m/server1(.*)$/)
 

it works but that takes the contents of all the files in the DIR.

[RonB]

Why are you appending "\n" to the date? That is the main problem, but there are others, such as "useless use of cat".

perldoc -f open
perldoc -f glob

[joeferns79]

Thanks, I got it working once I removed the "\n". I was actually testing it out and forgot to remove it earlier. Thanks, again.

[RonB]

You should look at using:

File::Find - Traverse a directory tree.
http://search.cpan.org/~nwclark/perl...b/File/Find.pm

or

File::Find::Rule - Alternative interface to File::Find
http://search.cpan.org/~rclamp/File-...e/Find/Rule.pm