Connecting Tech Pros Worldwide Help | Site Map
bytes > topic > perl > answers

Searching files by matching filename pattern and concatenating contents of the files

  #1  
Old June 22nd, 2009, 02:33 PM
Member
  
Join Date: Sep 2008
Posts: 35
Hi,
I wanted to write a Perl script that searches a given folder for all files that have filenames based on the previous day's date.

eg. if the filenames of the files in the said folder are ....
server1_Statistics_0.20090618T124050
server1_Statistics_0.20090619T080652
server1_Statistics_0.20090220T124502
server1_Statistics_0.20090621T105927
server1_Statistics_0.20090621T105638
server1_Statistics_0.20090621T105611
server1_Statistics_0.20090621T105518
server1_Statistics_0.20090621T105108

find all files with yesterday's date(20090621) and concatenate their contents into a single file with the filename "server1_Statistics_20090621"
  #2  
Old June 22nd, 2009, 04:18 PM
Member
  
Join Date: Jun 2009
Posts: 43

re: Searching files by matching filename pattern and concatenating contents of the files

What have you tried?

What errors and/or warnings are you receiving?

What portion of the task do you not know how to accomplish?
  #3  
Old June 22nd, 2009, 04:39 PM
Member
  
Join Date: Sep 2008
Posts: 35

re: Searching files by matching filename pattern and concatenating contents of the files

I am relatively new to Perl and we have it installed on AIX. I have worked with the same on Linux and when I tried getting the previous day's date itself, I am having trouble...

Expand|Select|Wrap|Line Numbers
  1. #!/usr/bin/perl
  2.  
  3. $today=`date --date='1 day ago' +%Y%m%d`;
  4.  
  5.  
I tried using the following command, but I was having trouble parsing the columns, to get only the date ...

Expand|Select|Wrap|Line Numbers
  1. perl -e 'print localtime(time() - 86400) . "\n"'
  2.  
Even after the date has been captured, the month is shown in words, whereas I want it to match the filenames which are numerals.
Last edited by eWish; June 23rd, 2009 at 05:09 AM. Reason: Fixed code tags
  #4  
Old June 22nd, 2009, 05:01 PM
Member
  
Join Date: Jun 2009
Posts: 43

re: Searching files by matching filename pattern and concatenating contents of the files

Use the strftime function from the POSIX module.

Expand|Select|Wrap|Line Numbers
  1. use POSIX 'strftime';
  2.  
  3. my $date = strftime("%Y%m%d", localtime(time - 86400));
  4. print $date;
  5.  
  #5  
Old June 22nd, 2009, 06:31 PM
Member
  
Join Date: Sep 2008
Posts: 35

re: Searching files by matching filename pattern and concatenating contents of the files

That worked fine, thanks.

I am now trying to include the output from that into a search for filenames containing yesterday's date.

Expand|Select|Wrap|Line Numbers
  1. #!/usr/bin/perl
  2. $hostname=`hostname`;
  3. chomp($hostname);
  4. #$today=`perl -e 'print localtime(time() - 86400) . "\n"'`;
  5. #chomp($today);
  6.  
  7. #!/usr/bin/perl
  8.  
  9. use POSIX 'strftime';
  10.  
  11. my $date = strftime("%Y%m%d", localtime(time - 86400)). "\n";
  12. #print $date;
  13.  
  14. $wldir="/usr/IBM/WebSphere/AppServer/logs/test";
  15. #$wlarc=$wldir/$date;
  16.  
  17. #print "Archiving $wldir directory\n";
  18. do_dir($wldir);
  19.  
  20.  
  21.  
  22. sub do_dir {
  23.     my $dir = shift;
  24.     opendir(D, $dir);
  25.     my @f = readdir(D);
  26.     closedir(D);
  27.     foreach my $file (@f)
  28.        {
  29.         my $filename = $dir . '/' . $file;
  30.         if ($file eq '.' || $file eq '..')
  31.           {
  32.           }
  33.         elsif (-d $filename)
  34.           {
  35.             # depending on your needs you can do subdirs
  36.             do_dir($filename);
  37.           }
  38.         else {
  39.             # do something with $filename, like ...
  40.             if ($filename =~ m/$date/)
  41.               {
  42.               print "Concatenating $filename\n";
  43.               `cat $filename >> $date`;
  44.               }
  45.              }
  46.       }
  47.   }
  48.  
  49. exit(0);
  50.  
  51.  
  52.  
Even though it doesn't throw an error, it doesn't concatenate either.
Last edited by eWish; June 23rd, 2009 at 05:10 AM. Reason: Fixed code tags [code][/code]
  #6  
Old June 22nd, 2009, 07:26 PM
Member
  
Join Date: Sep 2008
Posts: 35

re: Searching files by matching filename pattern and concatenating contents of the files

I tried the following script as well, to no avail...


Expand|Select|Wrap|Line Numbers
  1.  
  2. #!/usr/bin/perl
  3.  
  4. use POSIX 'strftime';
  5.  
  6. my $date = strftime("%Y%m%d", localtime(time - 86400)). "\n";
  7. #print $date;
  8.  
  9. $wldir="/usr/IBM/WebSphere/AppServer/logs/test";
  10. #$wlarc=$wldir/$date;
  11.  
  12. do_dir($wldir);
  13.  
  14.  
  15.  
  16. sub do_dir {
  17.     my $dir = shift;
  18.     opendir(D, $dir);
  19.     my @f = readdir(D);
  20.     closedir(D);
  21.     foreach my $file (@f)
  22.        {
  23.         my $filename = $dir . '/' . $file;
  24.         if ($file eq '.' || $file eq '..')
  25.           {
  26.           }
  27.         elsif (-d $filename)
  28.           {
  29.             # depending on your needs you can do subdirs
  30.             do_dir($filename);
  31.           }
  32.         else {
  33.             # do something with $filename, like ...
  34.             if ($filename =~ m/server1(.*)\.$date(.*)$/)
  35.               {
  36.               print "Concatenating $filename\n";
  37.               `cat $filename >> $date`;
  38.               }
  39.              }
  40.       }
  41.   }
  42.  
  43.  
If I use ...

Expand|Select|Wrap|Line Numbers
  1. if ($filename =~ m/server1(.*)$/)
  2.  
it works but that takes the contents of all the files in the DIR.
Last edited by eWish; June 23rd, 2009 at 05:11 AM. Reason: Fixed code tags [code][/code]
  #7  
Old June 22nd, 2009, 08:02 PM
Member
  
Join Date: Jun 2009
Posts: 43

re: Searching files by matching filename pattern and concatenating contents of the files

Why are you appending "\n" to the date? That is the main problem, but there are others, such as "useless use of cat".

perldoc -f open
perldoc -f glob
  #8  
Old June 22nd, 2009, 08:14 PM
Member
  
Join Date: Sep 2008
Posts: 35

re: Searching files by matching filename pattern and concatenating contents of the files

Thanks, I got it working once I removed the "\n". I was actually testing it out and forgot to remove it earlier. Thanks, again.
  #9  
Old June 22nd, 2009, 08:18 PM
Member
  
Join Date: Jun 2009
Posts: 43

re: Searching files by matching filename pattern and concatenating contents of the files

You should look at using:

File::Find - Traverse a directory tree.
http://search.cpan.org/~nwclark/perl...b/File/Find.pm

or

File::Find::Rule - Alternative interface to File::Find
http://search.cpan.org/~rclamp/File-...e/Find/Rule.pm
Reply

« previous question | next question »

/bytes/development

/bytes/it

/bytes/about

We are a network of experts and professionals in IT and software development that help one another with answers to tough questions and share insights. Get the best answers to your questions from over 225,662 network members.

dev questions:
algorithms | asp | asp.net | c/c++ | coldfusion | c# | flash | html/css | java | javascript |
mobile dev | .net | perl | php | python | ruby | software dev | vb.net | visual basic | xml
it questions:
access | apache | careers | db2 | iis | macosx | mysql | networking | oracle | postgresql | sql server | unix | windows

About Bytes | Copyright 1995-2009 BYTES. All rights Reserved