473,385 Members | 1,267 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

Timed Proceedure

58
Hey all, I struggled with this code for the last couple days and finally got it working. Its still a bit noisy and could most definitely be refined, but thought I would share it.
Basically, the script is being run on an ftp server, where we are sending 3 files to a read only folder, waiting for a group on the outside to download those files, process them, and upload 9 results files to another write directory, repeat til all files have been processed. The timer is run at 10 seconds. Hope this helps someone else in the future with a similar project.
Cheers,
Eric
Expand|Select|Wrap|Line Numbers
  1. #!/usr/local/bin/perl
  2. use strict;
  3. use warnings;
  4. use File::Copy;
  5. my $InDir = "D:\\data\\eclipse\\workspace\\contents";
  6. my $file_count_old=-1;
  7. my $pid;
  8. die "cant fork $!\n" unless defined($pid=fork());
  9.  
  10.  if($pid) {
  11.  print "Here we go\n";
  12.  my $in=<STDIN>;
  13.    while ($in ne 'stop') {
  14.    $in=<STDIN>;
  15.    chomp $in;
  16.    print "$in is your input\n";
  17.        if($in eq 'stop') {
  18.        print "exiting.........\n";
  19.        }
  20.    }
  21.  }
  22.  else {
  23. LOOP1:
  24.   sleep 10;
  25.   {move_files()}
  26.  
  27. goto LOOP1;
  28.  
  29.  }
  30.  
  31. sub move_files {
  32.   my $SenDir = "D:\\data\\eclipse\\workspace\\contents\\sen";
  33.   my $SptDir = "D:\\data\\eclipse\\workspace\\contents\\spt";
  34.   my $WeaDir = "D:\\data\\eclipse\\workspace\\contents\\wea";
  35.   my $OutDir = "D:\\data\\eclipse\\workspace\\contents\\out";
  36.  
  37.   my @dirSen;  
  38.     opendir(DIR, $SenDir) or die "Unable to open dir $SenDir: $!\n";
  39.     @dirSen = sort readdir(DIR);
  40.     my $dir_sen_contents = $dirSen[2];
  41.     close(DIR);
  42.     #$dir_sen_contents is now the first file of sen directory
  43.  
  44.   my @dirSpt;  
  45.     opendir(DIR, $SptDir) or die "Unable to open dir $SptDir: $!\n";
  46.     @dirSpt = sort readdir(DIR);
  47.     my $dir_spt_contents = $dirSpt[2];
  48.     close(DIR);
  49.     #$dir_spt_contents is now the first file of spt directory
  50.  
  51.   my @dirWea;  
  52.     opendir(DIR, $WeaDir) or die "Unable to open dir $WeaDir: $!\n";
  53.     @dirWea = sort readdir(DIR);
  54.     my $dir_wea_contents = $dirWea[2];
  55.     close(DIR);
  56.     #$dir_wea_contents is now the first file of wea directory
  57.  
  58.   my $directory_count=0;
  59.   my $file_count=0;
  60.  
  61. opendir(DIR, $InDir);
  62.   LINE: while(my $FILE = readdir(DIR)) {
  63.     next LINE if($FILE =~ /^\.\.?/);
  64.     ## check to see if it is a directory
  65.     if(-d "$FILE"){
  66.       $directory_count++;
  67.     }
  68.     else {
  69.       $file_count++;
  70.     }
  71.   }
  72. closedir(DIR);
  73.  
  74. my $divisor = 9;
  75. my $remainder = $file_count % $divisor;
  76. my $FilesLeft = ($divisor - $remainder);
  77. my $tmp_sen_in ="$SptDir\\$dir_spt_contents";
  78. my $tmp_spt_in="$SenDir\\$dir_sen_contents";
  79. my $tmp_wea_in="$WeaDir\\$dir_wea_contents";
  80. my $tmp_sen_out="$OutDir\\$dir_spt_contents";
  81. my $tmp_spt_out="$OutDir\\$dir_sen_contents";
  82. my $tmp_wea_out="$OutDir\\$dir_wea_contents";
  83. my $file_out="temp";
  84. my $nav_date=substr $dir_spt_contents,1,6;
  85. (my $sec,my $min,my $hour,my $mday,my $mon,my $year,my $wday,my $yday,my $isdst)=localtime(time);
  86. open (FILE, ">>$file_out") || die "Cannot write to $file_out\n";
  87.  
  88. if (($remainder==0) && ($file_count_old<$file_count)){
  89.   $file_count_old=$file_count;
  90.   print "$file_count_old\n";
  91.   move ($tmp_sen_in, $tmp_sen_out) or die "failed to move";
  92.   move ($tmp_spt_in, $tmp_spt_out) or die "failed to move";
  93.   move ($tmp_wea_in, $tmp_wea_out) or die "failed to move";
  94.   print "it worked! we moved it at :",$year+1900,"-",$mon+1,"-",$mday," ",$hour,":",$min,":",$sec,"\n\n";
  95.   while(<FILE>){
  96.     print "$dir_spt_contents$dir_sen_contents$dir_wea_contents.txt";
  97.   }
  98.   close (FILE);
  99.   rename "$file_out","$OutDir\\$file_out\_$nav_date" || die "Cannot update $file_out\n";
  100. }
  101. else {
  102.   print "They are currently uploading: $FilesLeft files left don't move em at ",$year+1900,"-",$mon+1,"-",$mday," ",$hour,":",$min,":",$sec,"\n"
  103.   }
  104. }
  105. kill ("TERM",$pid); 
Apr 17 '08 #1
1 1060
numberwhun
3,509 Expert Mod 2GB
Hey all, I struggled with this code for the last couple days and finally got it working. Its still a bit noisy and could most definitely be refined, but thought I would share it.
Basically, the script is being run on an ftp server, where we are sending 3 files to a read only folder, waiting for a group on the outside to download those files, process them, and upload 9 results files to another write directory, repeat til all files have been processed. The timer is run at 10 seconds. Hope this helps someone else in the future with a similar project.
Cheers,
Eric
Expand|Select|Wrap|Line Numbers
  1. #!/usr/local/bin/perl
  2. use strict;
  3. use warnings;
  4. use File::Copy;
  5. my $InDir = "D:\\data\\eclipse\\workspace\\contents";
  6. my $file_count_old=-1;
  7. my $pid;
  8. die "cant fork $!\n" unless defined($pid=fork());
  9.  
  10.  if($pid) {
  11.  print "Here we go\n";
  12.  my $in=<STDIN>;
  13.    while ($in ne 'stop') {
  14.    $in=<STDIN>;
  15.    chomp $in;
  16.    print "$in is your input\n";
  17.        if($in eq 'stop') {
  18.        print "exiting.........\n";
  19.        }
  20.    }
  21.  }
  22.  else {
  23. LOOP1:
  24.   sleep 10;
  25.   {move_files()}
  26.  
  27. goto LOOP1;
  28.  
  29.  }
  30.  
  31. sub move_files {
  32.   my $SenDir = "D:\\data\\eclipse\\workspace\\contents\\sen";
  33.   my $SptDir = "D:\\data\\eclipse\\workspace\\contents\\spt";
  34.   my $WeaDir = "D:\\data\\eclipse\\workspace\\contents\\wea";
  35.   my $OutDir = "D:\\data\\eclipse\\workspace\\contents\\out";
  36.  
  37.   my @dirSen;  
  38.     opendir(DIR, $SenDir) or die "Unable to open dir $SenDir: $!\n";
  39.     @dirSen = sort readdir(DIR);
  40.     my $dir_sen_contents = $dirSen[2];
  41.     close(DIR);
  42.     #$dir_sen_contents is now the first file of sen directory
  43.  
  44.   my @dirSpt;  
  45.     opendir(DIR, $SptDir) or die "Unable to open dir $SptDir: $!\n";
  46.     @dirSpt = sort readdir(DIR);
  47.     my $dir_spt_contents = $dirSpt[2];
  48.     close(DIR);
  49.     #$dir_spt_contents is now the first file of spt directory
  50.  
  51.   my @dirWea;  
  52.     opendir(DIR, $WeaDir) or die "Unable to open dir $WeaDir: $!\n";
  53.     @dirWea = sort readdir(DIR);
  54.     my $dir_wea_contents = $dirWea[2];
  55.     close(DIR);
  56.     #$dir_wea_contents is now the first file of wea directory
  57.  
  58.   my $directory_count=0;
  59.   my $file_count=0;
  60.  
  61. opendir(DIR, $InDir);
  62.   LINE: while(my $FILE = readdir(DIR)) {
  63.     next LINE if($FILE =~ /^\.\.?/);
  64.     ## check to see if it is a directory
  65.     if(-d "$FILE"){
  66.       $directory_count++;
  67.     }
  68.     else {
  69.       $file_count++;
  70.     }
  71.   }
  72. closedir(DIR);
  73.  
  74. my $divisor = 9;
  75. my $remainder = $file_count % $divisor;
  76. my $FilesLeft = ($divisor - $remainder);
  77. my $tmp_sen_in ="$SptDir\\$dir_spt_contents";
  78. my $tmp_spt_in="$SenDir\\$dir_sen_contents";
  79. my $tmp_wea_in="$WeaDir\\$dir_wea_contents";
  80. my $tmp_sen_out="$OutDir\\$dir_spt_contents";
  81. my $tmp_spt_out="$OutDir\\$dir_sen_contents";
  82. my $tmp_wea_out="$OutDir\\$dir_wea_contents";
  83. my $file_out="temp";
  84. my $nav_date=substr $dir_spt_contents,1,6;
  85. (my $sec,my $min,my $hour,my $mday,my $mon,my $year,my $wday,my $yday,my $isdst)=localtime(time);
  86. open (FILE, ">>$file_out") || die "Cannot write to $file_out\n";
  87.  
  88. if (($remainder==0) && ($file_count_old<$file_count)){
  89.   $file_count_old=$file_count;
  90.   print "$file_count_old\n";
  91.   move ($tmp_sen_in, $tmp_sen_out) or die "failed to move";
  92.   move ($tmp_spt_in, $tmp_spt_out) or die "failed to move";
  93.   move ($tmp_wea_in, $tmp_wea_out) or die "failed to move";
  94.   print "it worked! we moved it at :",$year+1900,"-",$mon+1,"-",$mday," ",$hour,":",$min,":",$sec,"\n\n";
  95.   while(<FILE>){
  96.     print "$dir_spt_contents$dir_sen_contents$dir_wea_contents.txt";
  97.   }
  98.   close (FILE);
  99.   rename "$file_out","$OutDir\\$file_out\_$nav_date" || die "Cannot update $file_out\n";
  100. }
  101. else {
  102.   print "They are currently uploading: $FilesLeft files left don't move em at ",$year+1900,"-",$mon+1,"-",$mday," ",$hour,":",$min,":",$sec,"\n"
  103.   }
  104. }
  105. kill ("TERM",$pid); 
Is there a question associated with this or just a code sharing exercise?

Regards,

Jeff
Apr 18 '08 #2

Sign in to post your reply or Sign up for a free account.

Similar topics

6
by: John Webb | last post by:
Hope someone can help. I am trying to write a stored proceedure to display sales activity by month and then sum all the columbs. The problem is that our sales year starts in April and end in...
0
by: BobC | last post by:
One of Microsoft's VB code samples is a solution called Logger (or WinLogger). It has a file associated with the "Logging" project called ILog.vb. In the module, interface or whatever it's called...
1
by: Melissa Meyer via SQLMonster.com | last post by:
How do you set a stored proceedure for automatic execution? -- Message posted via http://www.sqlmonster.com
0
by: pmclinn | last post by:
In the .net designer wrote a proceedure named 'Fireitup'. How do I call on this proceedure from client javascript.
6
by: Daniel Walzenbach | last post by:
Hi, I have a web application which sometimes throws an “out of memory” exception. To get an idea what happens I traced some values using performance monitor and got the following values (for...
2
by: Problematic coder | last post by:
I have a form based application that inputs values into various oracle tables, after it has done this I want to call a stored proceedure in Oracle which is already written to do more work on this...
9
by: Problematic coder | last post by:
The intention is to call a stored proceedure which sets flags in the database, this requires no parameters and the page does not need an output from the stored proceedure, though is this is...
0
by: ig.martix | last post by:
I'm a bit new to asp. What I'm looking to doing is having three input boxes on a page. As an example I would like for the user to input a employee ID in the first input box and then wither...
4
by: Mick Walker | last post by:
Hi Everyone, I am stumped here. I have the following stored proceedure:P CREATE PROCEDURE . @SupplierSKU varchar(50), @RetVal int AS Select @Retval = count(*) from dbo.ImportLines Where =...
8
by: jthep | last post by:
Is there a way to create a stored procedure where more than one type of parameter can be entered? For example, a user can enter a int type, char(4) type, or nothing in the same stored proceedure.
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.