467,114 Members | 1,367 Online
Bytes | Developer Community
Ask Question

Home New Posts Topics Members FAQ

Post your question to a community of 467,114 developers. It's quick & easy.

problem adding column to array

Oh perl masters, please show me the way!!! Alright, my issue in a nut shell is... I have a folder packed full of csv files. In each csv file there are 2 fields (NAME, VALUE). I need to combine all these files into one file BUT here is where I am stuck.. each csv is named by a specific data/24time (ie 20080220013000.csv).
As i loop through these csv's I need to add the file name as a column, so I would end up with 3 fields in the combined new csv file (NAME,VALUE,DATETIME) this is so i can differentiate between the files when I go to do some final analysis.
Right now I am just trying to add the DATETIME to one csv (will work on looping through all the files after I get this working) here is what i've managed to come up with so far (ps it does't work either!)

Expand|Select|Wrap|Line Numbers
  1. #!/usr/local/bin/perl
  2. #use strict;
  3. use warnings;
  4. use tie::File;
  5. my $InDir="D:\\input";
  6. my $OutDir="D:\\output";
  7. my $file_count=0;
  8. my $directory_count=0;
  9. my $file_out="temp_out.csv";
  10.  
  11.   opendir(DIR, $InDir);
  12.     LINE: while(my $FILE = readdir(DIR)) {
  13.       next LINE if($FILE =~ /^\.\.?/);
  14.       ## check to see if it is a directory
  15.       if(-d "$FILE"){
  16.         $directory_count++;
  17.         }
  18.       else {
  19.         $file_count++;
  20.       }
  21.     }
  22.   closedir(DIR);
  23.  
  24. my @dirSen;  
  25.   opendir(DIR, $InDir) or die "Unable to open dir $InDir: $!\n";
  26.   @dirSen = sort readdir(DIR);
  27.   my $dir_sen_contents = $dirSen[2];
  28.   close(DIR);
  29.   #$dir_sen_contents is now the first file of sensor directory
  30.   #print "$InDir\\$dir_sen_contents\n";
  31.  
  32. open (FILE, "$InDir\\$dir_sen_contents") || die "Cannot write to $dir_sen_contents\n"; 
  33. open (TMP, ">$OutDir\\$file_out") || die "Cannot write to $file_out\n"; 
  34. my @TMP=(<FILE>);
  35. my $tmp2=@TMP;
  36. for(my $i = 0; $i < $tmp2; $i +=1) {
  37.   $TMP[$i][3] = "$dir_sen_contents"}
  38. print @TMP;
  39.  close FILE;
  40.  close TMP;
ps. i get an error Can't use string ("NAME,VALUE") as an array ref while "stricts ref" in use at line 37
Any assistance will be greatly appreciated.
Thanks ahead of time
Eric
Jun 10 '08 #1
  • viewed: 1720
Share:
1 Reply
Solved albeit backwards... and with the help of a textpad macro to tidy at the end...
Expand|Select|Wrap|Line Numbers
  1. #!/usr/local/bin/perl
  2. use strict;
  3. use warnings;
  4. use tie::File;
  5. use File::Copy;
  6.  
  7.  
  8. my $pid;
  9. die "cant fork $!\n" unless defined($pid=fork());
  10.  
  11.  if($pid) {
  12.  print "Here we go\n";
  13.  my $in='go';
  14.    while ($in ne 'stop') {
  15.    print "$in is your input\n";
  16.        if($in eq 'stop') {
  17.        print "exiting.........\n";
  18.        }
  19.    }
  20.  }
  21.  else {
  22. LOOP1:
  23.   sleep 1;
  24.   {move_files()}
  25.  
  26. goto LOOP1;
  27.  
  28.  }
  29.  
  30.  
  31. sub move_files { 
  32.  
  33. my $InDir="D:\\input";
  34. my $TmpDir="D:\\temp";
  35. my $OutDir="D:\\output";
  36. my $file_count=0;
  37. my $directory_count=0;
  38. my $file_out="combined_data.csv";
  39.  
  40.   opendir(DIR, $InDir);
  41.     LINE: while(my $FILE = readdir(DIR)) {
  42.       next LINE if($FILE =~ /^\.\.?/);
  43.       ## check to see if it is a directory
  44.       if(-d "$FILE"){
  45.         $directory_count++;
  46.         }
  47.       else {
  48.         $file_count++;
  49.       }
  50.     }
  51.   closedir(DIR);
  52.  
  53. my @dirSen;  
  54.   opendir(DIR, $InDir) or die "Unable to open dir $InDir: $!\n";
  55.   @dirSen = sort readdir(DIR);
  56.   my $dir_sen_contents = $dirSen[2];
  57.   close(DIR);
  58.   #$dir_sen_contents is now the first file of sensor directory
  59.  
  60. my $tmpyear=substr $dir_sen_contents,6,4;
  61. my $tmpmonth=substr $dir_sen_contents,10,2;
  62. my $tmpday=substr $dir_sen_contents,12,2;
  63. my $tmphour=substr $dir_sen_contents,14,2;
  64. my $tmpminute=substr $dir_sen_contents,16,2;
  65. my $tmpsecond=substr $dir_sen_contents,18,2;
  66. my $field=$tmpyear."-".$tmpmonth."-".$tmpday." ".$tmphour.":".$tmpminute.":".$tmpsecond;
  67. my $movefrom="$InDir\\$dir_sen_contents";
  68. my $moveto="$TmpDir\\$dir_sen_contents";
  69. print $movefrom;
  70. print $moveto;
  71.  
  72. open (FILE, "$InDir\\$dir_sen_contents") || die "Cannot write to $dir_sen_contents\n"; 
  73. open (TMP, ">>$OutDir\\$file_out") || die "Cannot write to $dir_sen_contents\n"; 
  74.   while (<FILE>) {
  75.     print TMP ("$field",",$_");
  76.     }
  77.  
  78. close FILE;
  79. close TMP;
  80. move ("$movefrom", "$moveto") or die "failed to move"; 
  81. }
would still definitely love to see someones code that isn't as cluttered as mine if they have the time :)
Jun 10 '08 #2

Post your reply

Sign in to post your reply or Sign up for a free account.

Similar topics

2 posts views Thread by Marcus | last post: by
33 posts views Thread by Geoff Berrow | last post: by
1 post views Thread by Simon Harvey | last post: by
1 post views Thread by tangus via DotNetMonster.com | last post: by
2 posts views Thread by ajikoe@gmail.com | last post: by
17 posts views Thread by Sri | last post: by
11 posts views Thread by dennis.sprengers@gmail.com | last post: by
60 posts views Thread by Bill Cunningham | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.