Perl script will not sort files from August to Dec 2007 for some reason

[jonathan184]

Hi I have a perl script, basically what it is suppose to do is check a folder with files. Now the files are checked using a timestamp with the command ls -l so the timestamp in this format is checked. Now what the script does is it checks the time stamp and creates a year folder if it does not exist and then creates a month folder if it does not exist and puts the respective files in the month folders. If the files are created this month then it leaves it in the folder it checks and moves others that are older to year, month folders.

Now I have tested the script and it works great . The script is doing what it suppose to but for some reason it is not processing files with timestamps from Aug 2007 to Dec 2007
It creates from Jan to July 2007 and all the other years and months but for 2007, for some reason these files disappear. Now I created an exception if it does nto go in any of the folders b put in exception and this does not happen.

Could somebody help me out in troublleshooting this or is this a bug. I am not sure where to check.


This also uses a sort_config.txt file
the file has info like this and you could make your own file if you want

So basically in the text file but info in a line like this
sourcepath destinationpath pathtologfile
e.g
/home/unix/sourcewherefilesare /home/destinationmonthsandyears /home/log20.txt

Make sure and separate the paths by a space and if you want to add comments
put # at the start of the line.

Here is the code

#!/usr/bin/perl
 
#Config File: sort_config.txt
#Purpose: To sort files by create date and to archive them in respecitive 
#folders by years and months.
#Date: 01-17-2008
 
use File::Copy;
 
($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime();
 
$logdateTime2 = sprintf("%02d-%02d-%04d", $mon + 1, $mday, $year + 1900, $hour, $min, $sec);
 
# Get Current Date Time
@datepcs = split(/\s+/, scalar localtime);
$dateTime = "@datepcs[1]";
 
#open config file
open(HAN, "sort_config.txt") || die " Unable to run date command $!";
   while ($configline = <HAN>) 
   {
      $_ = $configline;
 
      # This next if statement is to remove lines from the array that start 
      # with #
      chomp($configline);
 
      # s/#.*//;            # ignore comments by erasing them
      # next if /^(\s)*$/;  # skip blank lines
      next if ($_ =~ /^(#|$)/);
 
      # Putting config file elements in an array
      push @array1, [ split ];
   }
close HAN;
 
$i = 0;
foreach (@array1) 
{
  # Processing configuration file
  # Record Start Time for the Log
  ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime();
  $logdateTime = sprintf("%02d-%02d-%04d %02d:%02d:%02d", $mon + 1, $mday, $year + 1900, $hour, $min, $sec);
  $sourcedir1 = $array1[$i][0];
  $destdir1 = $array1[$i][1];
  $log = $array1[$i][2];
 
  chomp($sourcedir1);
  chomp($destdir1);
  chomp($log);
 
  $line1= 'Start Date/Time | Files Processed | Duration';
 
  #IF log file does not exist create the log with the title header.
  if (-e "$log") {}
  else 
  {
    open(NEWLOG, "> $log") || die ("Could not NEWLOG open file $!");
    print NEWLOG "$line1\n";
    close (NEWLOG);
  };
 
  ###print Header on LOG
  open(INFILE2,"< $log") || die("Could not open INFILE2 File!");
  $header=<INFILE2>;
  chomp($header);
 
  if($header eq "$line1") {}
  else
  {
    open(OUTFILE2, "> $log") || die ("Could not OUTFILE2 open file $!");
    print OUTFILE2 "$line1\n";
    print OUTFILE2 "$header\n";
    while (<INFILE2>)
    {
      print OUTFILE2 "$_";
    }
    close OUTFILE2;
  }
  close INFILE2;
 
  ### Adding log info
  open (LOG2, ">>$log") || die "error recording log $!";
  &sort("$sourcedir1", "$destdir1");
 
  #Converting time and date formats from the subroutine below
  my ($y, $m, $d, $hh, $mm, $ss) = (localtime)[5,4,3,2,1,0]; $y += 1900; $m++;
  my $iso_now = sprintf("%d-%02d-%02d %02d:%02d:%02d", $y, $m, $d, $hh, $mm, $ss);
 
  #Calculating time difference
  my $timeDiffStr = &timeDiff( date1 => $logdateTime, date2 => $iso_now );
 
  #Printing date time and Time difference and files processed to the log 
  print LOG2 "$logdateTime | $rc | $timeDiffStr   ", "\n";
  close (LOG2);
 
  ++$i;
} # Close array1 loop
 
 
# Start Subroutine to create folders and files in them.
sub sort 
{
  #Run the date command to parse to get the year from the system.
  $yearcmd = "date |";
  open (ZING, "$yearcmd") || die " Unable to run date command $!";
  $_ = <ZING>;
  $_ = split;
  $system_year = $_[5]; # Year showing up on the UNIX system
  chomp($system_year);
  $system_month = $_[1]; # Month showing up on the UNIX system
  chomp($system_month);
  chdir("$sourcedir1");
  $test = "ls -l |";
  open (INFO, "$test") || die " Unable to run list command $!";
  @rows = <INFO>;
  $rc = -1; # So result will not include the first line in the ls command
  foreach (@rows) 
  {
    $_ = split; # Split the line output in elements
    $year = "$_[7]"; # element to show the year
    chomp($year);
    $month = "$_[5]"; # Shows element of month
    chomp($month);
 
   #print "This is the time of the file $year \n and this is the year on the 
   #system **** $system_year ** \n";
 
   if ($year =~ m/\:/)
   {
     #Checking for files for this month
     if ($year =~ m/\:/ && ($month eq $dateTime)) 
     {}
     elsif (-d "$destdir1/$system_year") #Check for files for this year only 
     {
       if (-d "$destdir1/$system_year/$system_month")
       {
         move("$_[8]", "$destdir1/$system_year/$system_month");
       }
       else 
         {
           # Create new dir if it does not exist
           mkdir("$destdir1/$system_year/$system_month") or die "Can't open $srcdir: $!";
           move("$_[8]", "$destdir1/$system_year/$system_month");
         }
     } 
     else { 
            #Create new dir if it does not exist
            mkdir("$destdir1/$system_year") or die "Can't open $srcdir: $!";
            mkdir("$destdir1/$system_year/$system_month") or die "Can't open $srcdir: $!";
            move("$_[8]", "$destdir1/$system_year/$system_month");
          }
   }
   else #else 1
   {
     if (-d "$destdir1/$year")
     {
       if (-d "$destdir1/$year/$month")
       {
         move("$_[8]", "$destdir1/$year/$month");
       }
       else 
          {
          mkdir("$destdir1/$year/$month") or die "Can't create sort dir $!";
              move("$_[8]", "$destdir1/$year/$month");
          }          
     }
     else 
     {
        # Create new dir if it does not exist
        mkdir("$destdir1/$year") or die "Can't create sort dir $!";
        mkdir("$destdir1/$year/$month") or die "Can't create sort dir $!";
 
        # Check if directory was created now
        if (-d "$destdir1/$year/$month") 
        {
           move("$_[8]", "$destdir1/$year/$month");  
        }
        else
        {
           mkdir("$destdir1/exception") or die "Can't create sort dir $!";
           move("$_[8]", "$destdir1/exception");
        }
 
     }
   } #close else 1
     ++$rc; 
 
  }   # close loop 
 
   close(ZING);
   close(INFO);
 
} # close subroutine
 
 
# To calculate the time difference
sub timeDiff (%)
{
  my %args = @_;
  my @offset_days = qw(0 31 59 90 120 151 181 212 243 273 304 334);
  my $year1  = substr($args{'date1'}, 0, 4);
  my $month1 = substr($args{'date1'}, 5, 2);
  my $day1   = substr($args{'date1'}, 8, 2);
  my $hh1    = substr($args{'date1'},11, 2) || 0;
  my $mm1    = substr($args{'date1'},14, 2) || 0;
  my $ss1    = substr($args{'date1'},17, 2) if (length($args{'date1'}) > 16);
  $ss1  ||= 0;
  my $year2  = substr($args{'date2'}, 0, 4);
  my $month2 = substr($args{'date2'}, 5, 2);
  my $day2   = substr($args{'date2'}, 8, 2);
  my $hh2    = substr($args{'date2'},11, 2) || 0;
  my $mm2    = substr($args{'date2'},14, 2) || 0;
  my $ss2    = substr($args{'date2'},17, 2) if (length($args{'date2'}) > 16);
  $ss2  ||= 0;
  my $total_days1 = $offset_days[$month1 - 1] + $day1 + 365 * $year1;
  my $total_days2 = $offset_days[$month2 - 1] + $day2 + 365 * $year2;
  my $days_diff   = $total_days2 - $total_days1;
  my $seconds1 = $total_days1 * 86400 + $hh1 * 3600 + $mm1 * 60 + $ss1;
  my $seconds2 = $total_days2 * 86400 + $hh2 * 3600 + $mm2 * 60 + $ss2;
  my $ssDiff = $seconds2 - $seconds1;
  my $dd     = int($ssDiff / 86400);
  my $hh     = int($ssDiff /  3600) - $dd *    24;
  my $mm     = int($ssDiff /    60) - $dd *  1440 - $hh *   60;
  my $ss     = int($ssDiff /     1) - $dd * 86400 - $hh * 3600 - $mm * 60;
 
  #"$dd Tage $hh Std. $mm Min.";
  "$hh Hours $mm Mins $ss secs";
}
 

[KevinADC]

Can you narrow down the scope of the problem? Thats too much code to look through.

[jonathan184]

Ok from line 129 to 183 is where it actually checks the timestampa nd the year and month folder creations and where it moves the files. Now after posting this I did like some print statements and when it was creating 2007 Jan to Jul in the print statements in the loop it just stopped at July in the loop in print statement it did not print it created Aug to Dec. So this is very strange. Its like it just stopped but all the other years it runs fine for these.

[KevinADC]

I don't know, I am really not inclined to look over code written without using "strict". One small typo in a variable name could be the problem, or not. Using "stict" is very important, it helps a lot to insure your code is good if not the logic.

Your code is also severly tortured, it is written in a manner that makes it too hard to follow the flow and logic, all those if/elsif/if/else blocks are a nightmare to try and read and follow.

I humbly suggest an entire rewrite from scratch. This has to be much easier to accomplish than how you have gone about it.

[KevinADC]

I tried, but your script is so confusing I could not understand it. You really have to re-examine your approach and re-write the script. What you have right now is a nightmare of unnecessary complexity.