473,574 Members | 11,734 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Perl FTP issue

4 New Member
I am somewhat a Perl Noob, but I've done some programming with my animal books at my side. This issue, however doesn't make sense to me. I am grabbing data via FTP corresponding to data I've already downloaded (this is a pre-processing front end to some other fortran code). However, in the part indicated below (the second FTP execution), I get a login failure, even both ftp sites allow anonymous FTP.

Any thoughts or suggestions on how I might run this down would be much appreciated.

Sincerely,
Todd

Code pasted below
----------------------------

#!/usr/bin/perl

use Net::FTP;

$ftp_username = 'anonymous';
$ftp_passwd = 'BLOCKED - this would normally be my email'

my ($tsec, $tmin, $thour, $mday, $mmon, $yyear, $wday, $yday, $isdst) = localtime time;
$yyear += 1900;

#initialize arrays with strings

@month_str = ('01', '02', '03', '04', '05', '06', '07', '08', '09', '10', '11', '12');
@day_str = ('01', '02', '03', '04', '05', '06', '07', '08', '09', '10', '11', '12', '13', '14', '15', '16', '17', '18', '19', '20', '21', '22', '23', '24', '25', '26', '27', '28', '29', '30', '31');

# set local AMSU data directory

$AMSUdir = 'data/2005';

chdir $AMSUdir or die "cannot change dir to $AMSU_dir: $!";

@AMSUfiles = glob "*.eos";

# loop through each file to grab relevant ancillary data

foreach $AMSU (@AMSUfiles) {

# do not grab more data if this swath is in same day as last swath

if ($AMSU ne $hold_name) {

# capture the year and the doy

if ($AMSU =~ /\w+(\d{4})\.(\d +)\w\d+\w\d+\w\ d+\w+\.eos/) {

$year = $1*1;
$doy = $2;

# convert dayofyear to date

if ($year % 4 eq 0) {
@days_by_month = (31, 60, 91, 121, 152, 182, 213, 244, 274, 305, 335, 366);
} else {
@days_by_month = (31, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334, 365);
}

$month = 1;
$dd = $2*1;

foreach $dm (@days_by_month ) {
if ($2 <= $dm) {
$day = $dd;
} else {
$dd = $2 - $dm;
$month++;
}

}

$mo_string = $month_str[$month-1];
$day_string = $day_str[$day-1];

# get the SSMI data

$ssmi_dir = "/ssmi/f15/bmaps_v06/y$year/m$mo_string/";
$ssmi_filename = "f15_$year$mo_s tring$day_strin gv6.gz";

$ssmi_site = 'ftp.ssmi.com';
$ftp_obj = Net::FTP -> new ($ssmi_site, Timeout => 20) or
die "Cannot access $ssmi_site via FTP\n";

$ftp_obj -> login($ftp_user name, $ftp_passwd) or
die "Invalid user name and/or password SSMI\n";

$ftp_obj -> binary;

$ftp_obj -> cwd ("$ssmi_dir" ) or
die "Cannot access directory $ssmi_dir\n";

$ftp_obj -> get("$ssmi_file name");

$ftp_obj -> quit;

# get the SST data

$sst_dir = "/sst/daily/tmi_amsre/";
$sst_filename = "tmi_amsre.fusi on.$year.$doy.v 02.gz";

$sst_site = 'ftp.discover-earth.org';
$ftp_obj -> new ($sst_site, Timeout=>20) or
die "Cannot access $sst_site via FTP\n";

# this next line throws out an invalid login error

$ftp_obj -> login($ftp_user name, $ftp_passwd) or
die "Invalid user name and/or password SST\n";

$ftp_obj -> binary;

$ftp_obj -> cwd ("$sst_dir") or
die "Cannot access directory $sst_dir\n";

$ftp_obj -> get("$sst_filen ame");

$ftp_obj -> quit;

}
}

# set this string for next time through

$hold_name = $AMSU;

# write out the relevant data

print "$year $doy $AMSU $ssmi_filename $sst_filename\n ";

}

# make a new data for all data

$data_dir = "data_$thour$tm in_$yday_$yyear ";
mkdir $data_dir, 0777;

# move all the data

foreach my $file (glob "*.eos") {

my $newfile = "$data_dir/$file";
rename $file, $newfile;

}

foreach my $file (glob "*.gz") {

my $newfile = "$data_dir/$file";
rename $file, $newfile;

}
Nov 18 '06 #1
6 5744
GunnarH
83 New Member
# this next line throws out an invalid login error

$ftp_obj -> login($ftp_user name, $ftp_passwd) or
die "Invalid user name and/or password SST\n";
Then ask Perl why it failed. See in the Net::FTP docs (the SYNOPSIS section) how it can be done.
Nov 18 '06 #2
psutoad
4 New Member
Then ask Perl why it failed. See in the Net::FTP docs (the SYNOPSIS section) how it can be done.
That's not helpful - nothing in the synopsis tells me anything more than I already know - that I am unable to log in during the second ftp event. Adding the $ftp_obj -> message to this step returns "connection closed in line 83".

Like I said, I'm relatively new to perl, but I've been programming for 20 years...


So is this something about the FTP package? Something with the use of the variables? Something else? I've been trying stuff for 4 hours.

-T
Nov 18 '06 #3
miller
1,089 Recognized Expert Top Contributor
psutoad. I would like to help you, but I must first pick on one thing with regard to your perl programming practices. It is very important that you include the line "use strict;" at the beginning of any perl script that you write. This will force you to always declare your variables with "my", and in this way encourage you to think about the scope of your variables, instead of treating everything as global. Additionally, it as the benefit of helping you find spelling errors immediately, of which you currently have 4 in your script.

Take the below code that I spent 10 minutes reformatting for you. There are a lot more ways that I can clean up the code, but I woudl like you to discover the spelling errors on your own so that you can fix them. After you've done that, I will continue trying to help you debug your problem.

Expand|Select|Wrap|Line Numbers
  1. #!/usr/bin/perl
  2.  
  3. use Net::FTP;
  4.  
  5. use strict;
  6.  
  7. my $ftp_username = 'anonymous';
  8. my $ftp_passwd   = 'BLOCKED - this would normally be my email';
  9.  
  10. my ( $tsec, $tmin, $thour, $mday, $mmon, $yyear, $wday, $yday, $isdst ) =
  11.   localtime time;
  12. $yyear += 1900;
  13.  
  14. # set local AMSU data directory
  15. my $AMSUdir = 'data/2005';
  16.  
  17. chdir $AMSUdir or die "cannot change dir to $AMSU_dir: $!";
  18.  
  19. my @AMSUfiles = glob "*.eos";
  20.  
  21. # loop through each file to grab relevant ancillary data
  22. my $hold_name = '';
  23.  
  24. foreach my $AMSU (@AMSUfiles) {
  25.  
  26.     my $year = '';
  27.     my $doy = '';
  28.     my $ssmi_filename = '';
  29.     my $sst_filename = '';
  30.  
  31.     # do not grab more data if this swath is in same day as last swath
  32.  
  33.     if ( $AMSU ne $hold_name ) {
  34.  
  35.         # capture the year and the doy
  36.  
  37.         if ( $AMSU =~ /\w+(\d{4})\.(\d+)\w\d+\w\d+\w\d+\w+\.eos/ ) {
  38.  
  39.             my $year = $1 * 1;
  40.             my $doy  = $2;
  41.  
  42.             # convert dayofyear to date
  43.  
  44.             my @days_by_month = ( $year % 4 eq 0 )
  45.                 ? ( 31, 60, 91, 121, 152, 182, 213, 244, 274, 305, 335, 366 )
  46.                 : ( 31, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334, 365 );
  47.  
  48.             my $month = 1;
  49.             my $dd    = $2 * 1;
  50.  
  51.             my $day;
  52.             foreach my $dm (@days_by_month) {
  53.                 if ( $2 <= $dm ) {
  54.                     $day = $dd;
  55.                 }
  56.                 else {
  57.                     $dd = $2 - $dm;
  58.                     $month++;
  59.                 }
  60.  
  61.             }
  62.  
  63.             my $mo_string  = sprintf "%02s", $month;
  64.             my $day_string = sprintf "%02s", $day;
  65.  
  66.             # get the SSMI data
  67.  
  68.             my $ssmi_dir      = "/ssmi/f15/bmaps_v06/y$year/m$mo_string/";
  69.             my $ssmi_filename = "f15_$year$mo_string$day_stringv6.gz";
  70.  
  71.             my $ssmi_site = 'ftp.ssmi.com';
  72.             my $ftp_obj = Net::FTP->new( $ssmi_site, Timeout => 20 )
  73.               or die "Cannot access $ssmi_site via FTP\n";
  74.  
  75.             $ftp_obj->login( $ftp_username, $ftp_passwd )
  76.               or die "Invalid user name and/or password SSMI\n";
  77.  
  78.             $ftp_obj->binary;
  79.  
  80.             $ftp_obj->cwd("$ssmi_dir")
  81.               or die "Cannot access directory $ssmi_dir\n";
  82.  
  83.             $ftp_obj->get("$ssmi_filename");
  84.  
  85.             $ftp_obj->quit;
  86.  
  87.             # get the SST data
  88.  
  89.             my $sst_dir      = "/sst/daily/tmi_amsre/";
  90.             my $sst_filename = "tmi_amsre.fusion.$year.$doy.v02.gz";
  91.  
  92.             my $sst_site = 'ftp.discover-earth.org';
  93.             $ftp_obj->new( $sst_site, Timeout => 20 )
  94.               or die "Cannot access $sst_site via FTP\n";
  95.  
  96.             # this next line throws out an invalid login error
  97.  
  98.             $ftp_obj->login( $ftp_username, $ftp_passwd )
  99.               or die "Invalid user name and/or password SST\n";
  100.  
  101.             $ftp_obj->binary;
  102.  
  103.             $ftp_obj->cwd("$sst_dir")
  104.               or die "Cannot access directory $sst_dir\n";
  105.  
  106.             $ftp_obj->get("$sst_filename");
  107.  
  108.             $ftp_obj->quit;
  109.  
  110.         }
  111.     }
  112.  
  113.     # set this string for next time through
  114.  
  115.     $hold_name = $AMSU;
  116.  
  117.     # write out the relevant data
  118.  
  119.     print "$year $doy $AMSU $ssmi_filename $sst_filename\n";
  120.  
  121. }
  122.  
  123. # make a new data for all data
  124.  
  125. my $data_dir = "data_$thour$tmin_$yday_$yyear";
  126. mkdir $data_dir, 0777;
  127.  
  128. # move all the data
  129.  
  130. foreach my $file ( glob "*.eos" ) {
  131.     my $newfile = "$data_dir/$file";
  132.     rename $file, $newfile;
  133. }
  134.  
  135. foreach my $file ( glob "*.gz" ) {
  136.     my $newfile = "$data_dir/$file";
  137.     rename $file, $newfile;
  138. }
  139.  
Good night for now.

Note. If you want to explicitly specify a variable in a interpolated string, then you should include it in brackets. "my ${var}is crowded". That reads the variable $var, where as "my $varis crowded" reads the varaible $varis.

Just repost the fixed code, and I will continue helping you as soon as I wake up.
Nov 18 '06 #4
psutoad
4 New Member
I would like to help you, but I must first pick on one thing with regard to your perl programming practices. It is very important that you include the line "use strict;" at the beginning of any perl script that you write. This will force you to always declare your variables with "my", and in this way encourage you to think about the scope of your variables, instead of treating everything as global. Additionally, it as the benefit of helping you find spelling errors immediately, of which you currently have 4 in your script.

Take the below code that I spent 10 minutes reformatting for you. There are a lot more ways that I can clean up the code, but I woudl like you to discover the spelling errors on your own so that you can fix them. After you've done that, I will continue trying to help you debug your problem.
Wow, I really appreciate not only the effort, but this commenton 'use strict'. In my Perl books they talk about this in later chapters, but it didn't seem necessary to me and I'd never used the strict command in any scripts I've made over the past 10 years. I will use it from now on, though I really don't understand what purpose it serves other than to find spelling errors.. I will read more on this...

So this is my reworked code - I incorported most of your code reformatting, but not all. There's also a slightly different reordering of commands and a slightly different FTP block. But, the problem remains. Whenever I try to open a second ftp connection within the same loop, it fails every time. I've tried several different combinations... I've changed the $ftp_obj variable to a different one for each FTP block, I've tried troubleshooting every variable by printing them between every command and they all look ok.

Again many thanks for the effort so far, it was more than I expected. I just can't get my head around what I'm doing wrong with these FTP calls.. any continued help is greatly appreciated.

Expand|Select|Wrap|Line Numbers
  1. #!/usr/bin/perl
  2.  
  3. use Net::FTP;
  4. use strict;
  5.  
  6. #### USER INPUT PARAMETERS
  7.  
  8. my $ftp_passwd = 'my@email.address'  # your email for anonymous FTP
  9.  
  10. my $AMSUdir = '/home/ellis/work/classwork/AT652/DREAM/test/'; # the directory where your AMSU data is located
  11.  
  12. ########### DO NOT CHANGE ANYTHING BELOW THIS LINE ###################
  13.  
  14. my $ftp_username = 'anonymous'; # for anonymous ftp
  15.  
  16. # get current time for naming files and directories
  17.  
  18. my ($tsec, $tmin, $thour, $mday, $mmon, $yyear, $wday, $yday, $isdst) = localtime time;
  19. $yyear += 1900;
  20.  
  21. # initialize output file name
  22.  
  23. my $out_file_name = "DREAMfiles_${thour}${tmin}_${yday}_${yyear}.dat";
  24. open OUTFILE, ">$out_file_name";
  25. $| = 1; # clear buffer after each output
  26.  
  27. # change to AMSU data directory and work here
  28.  
  29. chdir $AMSUdir or die "cannot change dir to $AMSUdir: $!";
  30.  
  31. # load in names of all existing AMSU data
  32.  
  33. my @AMSUfiles = glob "*.eos";
  34.  
  35. # output number of files (for IDL program)
  36.  
  37. print OUTFILE "$#AMSUfiles\n";
  38.  
  39. # make a new data directory for all data files
  40.  
  41. my $data_dir = "data_${thour}${tmin}_${yday}_${yyear}";
  42. mkdir $data_dir, 0777;
  43.  
  44. # output data directory name for IDL program
  45.  
  46. print OUTFILE "$data_dir\n";
  47.  
  48. # initialize holders
  49.  
  50. my $hold_doy = 0;
  51. my $hold_year = 0;
  52.  
  53. # loop through each AMSU file
  54.  
  55. foreach my $AMSU (@AMSUfiles) {
  56.  
  57.  my $ssmi_filename = '';
  58.  my $sst_filename = '';
  59.  my $ncep_uwind_name = '';
  60.  my $ncep_vwind_name = '';
  61.  my $year = '';
  62.  my $doy = '';
  63.  
  64.  # match file name to extract year and day of year
  65.  
  66.  if ($AMSU =~ /\w+(\d{4})\.(\d+)\w\d+\w\d+\w\d+\w+\.eos/) {
  67.  
  68.   my $year = $1 * 1; # converts year to number
  69.   my $doy = $2;
  70.  
  71.   # compare day of year to last time through - only download data if new day
  72.  
  73.   if ($doy !=  $hold_doy) {
  74.  
  75.    # convert dayofyear to date
  76.  
  77.    my @days_by_month = ($year % 4 eq 0)
  78.     ? (31, 60, 91, 121, 152, 182, 213, 244, 274, 305, 335, 366)
  79.     : (31, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334, 365);
  80.  
  81.    my $month = 1;
  82.    my $dd = $2*1;
  83.  
  84.    my $day;
  85.  
  86.    foreach my $dm (@days_by_month) {
  87.     if ($2 <= $dm) {
  88.      $day = $dd;
  89.     } else {
  90.      $dd = $2 - $dm;
  91.      $month++;
  92.     }
  93.  
  94.    }
  95.  
  96.    # grab strings corresponding to date of file
  97.  
  98.    my $mo_string = sprintf "%02s", $month;
  99.    my $day_string = sprintf "%02s", $day;
  100.  
  101.    # get the SSMI data
  102.  
  103.    my $ssmi_dir = "/ssmi/f15/bmaps_v06/y${year}/m${mo_string}/";
  104.    my $ssmi_filename = "f15_${year}${mo_string}${day_string}v6.gz";
  105.  
  106.    my $ssmi_site = 'ftp.ssmi.com';
  107.  
  108.    my $ftp_obj = Net::FTP -> new ($ssmi_site, Timeout => 20) or
  109.     die "Cannot access $ssmi_site via FTP\n";
  110.  
  111.    $ftp_obj -> login($ftp_username, $ftp_passwd) or
  112.     die "Invalid user name and/or password SSMI\n";
  113.  
  114.    $ftp_obj -> binary;
  115.  
  116.    $ftp_obj -> cwd ("$ssmi_dir") or
  117.     die "Cannot access directory $ssmi_dir\n";
  118.  
  119.    $ftp_obj -> get("$ssmi_filename") or die "Get failed $ssmi_filename";
  120.  
  121.    # get the SST data
  122.  
  123.    my $sst_dir = "/sst/daily/tmi_amsre/";
  124.    my $sst_filename = "tmi_amsre.fusion.${year}.${doy}.v02.gz";
  125.  
  126.    $ftp_obj -> binary;
  127.  
  128.    $ftp_obj -> cwd ("$sst_dir") or
  129.     die "Cannot access directory $sst_dir\n";
  130.  
  131.    $ftp_obj -> get("$sst_filename") or die "Cannot get SST file\n";
  132.  
  133.    $ftp_obj -> quit;
  134.  
  135.   }
  136.  
  137.   # if new year, grab NCEP data
  138.  
  139.   if ($year != $hold_year) {
  140.  
  141.    my $ncep_dir = '/Datasets/ncep.reanalysis.dailyavgs/surface/';
  142.    my $ncep_uwind_name = "uwnd.sig995.${year}.nc";
  143.    my $ncep_vwind_name = "vwnd.sig995.${year}.nc";
  144.  
  145.    my $ncep_site = "ftp.cdc.noaa.gov";
  146.  
  147.    my $ncep_obj = Net::FTP -> new ($ncep_site, Timeout => 20) or
  148.     die "Cannot access $ncep_site via FTP\n";
  149.  
  150.    print "$ncep_obj\n";
  151.  
  152.    $ncep_obj -> binary;
  153.  
  154.    print "$ncep_obj\n";
  155.    print "$ncep_dir\n";
  156.  
  157. # error is currently at this step - yet I can manually use FTP to access the directory...
  158.    $ncep_obj -> cwd ("$ncep_dir") or die "Cannot access directory $ncep_dir\n";
  159.  
  160.    $ncep_obj -> get("$ncep_uwind_name") or die "Get failed u-wind\n";
  161.    $ncep_obj -> get("$ncep_vwind_name") or die "Get failed v-wind\n";
  162.  
  163.    $ncep_obj -> quit;
  164.  
  165.   }
  166.  } 
  167.  
  168.  # hold the day of year and year for comparison to next iteration of loop
  169.  
  170.  $hold_doy = $doy;
  171.  $hold_year = $year;
  172.  
  173.  # write out the relevant data
  174.  
  175.  print OUTFILE "$year $doy $AMSU $ssmi_filename $sst_filename $ncep_uwind_name $ncep_vwind_name\n"; 
  176.  
  177. }
  178.  
  179. # move all data to new data directory
  180.  
  181. foreach my $file (glob "*.eos") {
  182.  
  183.  my $newfile = "$data_dir/$file";
  184.  rename $file, $newfile;
  185.  
  186. }
  187.  
  188. foreach my $file (glob "*.gz") {
  189.  
  190.  my $newfile = "$data_dir/$file";
  191.  rename $file, $newfile;
  192.  
  193. }
  194.  
  195. foreach my $file (glob "*.nc") {
  196.  
  197.  my $newfile = "$data_dir/$file";
  198.  rename $file, $newfile;
  199.  
  200. }
  201.  
  202. # end of file
  203.  
Many thanks
-T
Nov 21 '06 #5
miller
1,089 Recognized Expert Top Contributor
The main thing that is missing from your script now are the error messages as stated in http://perldoc.perl.or g/Net/FTP.html I went ahead and added them, and immediate was returned the following error:

"Cwd failed Please login with USER and PASS"

This is definitely beats the old error message as far as clarity. I therefore added the login statement and everything appears to work just fine. You'll also notice that I changed the formatting of the code some and discovered a couple other minor bugs. There are a lot of other enhancements that I could make to your script or suggestions of ones that you can make. But I think it's just important that you get it running for now.

The main lesson learned is that take advantage of all the debugging information that perl gives you, whether that is from the use of directives, or error messages provided by modules. Both were very useful in getting your script to a working state.

Happy Thanksgiving.

Expand|Select|Wrap|Line Numbers
  1. #!/usr/bin/perl
  2.  
  3. use Net::FTP;
  4. use strict;
  5.  
  6. #### USER INPUT PARAMETERS
  7.  
  8. my $ftp_passwd = 'my@email.address';  # your email for anonymous FTP
  9. # Was BUG: missing semicolon
  10.  
  11. my $AMSUdir = '/home/ellis/work/classwork/AT652/DREAM/test/'; # the directory where your AMSU data is located
  12.  
  13. ########### DO NOT CHANGE ANYTHING BELOW THIS LINE ###################
  14.  
  15. my $ftp_username = 'anonymous'; # for anonymous ftp
  16.  
  17. # get current time for naming files and directories
  18.  
  19. my ($tsec, $tmin, $thour, $mday, $mmon, $yyear, $wday, $yday, $isdst) = localtime time;
  20. $yyear += 1900;
  21. # REDUNDANT CODE - from http://perldoc.perl.org/functions/localtime.html
  22. # "If EXPR is omitted, localtime() uses the current time (localtime(time))."
  23.  
  24. # initialize output file name
  25.  
  26. my $out_file_name = "DREAMfiles_${thour}${tmin}_${yday}_${yyear}.dat";
  27. open OUTFILE, ">$out_file_name";
  28. my $old_fh = select(OUTFILE); $| = 1; select($old_fh); # clear buffer after each output
  29. # Was BUG: Cleared only STDOUT (not OUTFILE) - http://perldoc.perl.org/functions/select.html
  30.  
  31. # change to AMSU data directory and work here
  32.  
  33. chdir $AMSUdir or die "cannot change dir to $AMSUdir: $!";
  34.  
  35. # load in names of all existing AMSU data
  36.  
  37. my @AMSUfiles = glob "*.eos";
  38.  
  39. # output number of files (for IDL program)
  40.  
  41. print OUTFILE "$#AMSUfiles\n";
  42.  
  43. # make a new data directory for all data files
  44.  
  45. my $data_dir = "data_${thour}${tmin}_${yday}_${yyear}";
  46. mkdir $data_dir, 0777;
  47.  
  48. # output data directory name for IDL program
  49.  
  50. print OUTFILE "$data_dir\n";
  51.  
  52. # initialize holders
  53.  
  54. my $hold_doy = 0;
  55. my $hold_year = 0;
  56.  
  57. # loop through each AMSU file
  58.  
  59. foreach my $AMSU (@AMSUfiles) {
  60.  
  61.     my $ssmi_filename = '';
  62.     my $sst_filename = '';
  63.     my $ncep_uwind_name = '';
  64.     my $ncep_vwind_name = '';
  65.     my $year = '';
  66.     my $doy = '';
  67.  
  68.     # match file name to extract year and day of year
  69.  
  70.     if ($AMSU =~ /\w+(\d{4})\.(\d+)\w\d+\w\d+\w\d+\w+\.eos/) {
  71.  
  72.         my $year = $1 * 1; # converts year to number
  73.         my $doy = $2;
  74.  
  75.         # compare day of year to last time through - only download data if new day
  76.  
  77.         if ($doy !=  $hold_doy) {
  78.  
  79.             # convert dayofyear to date
  80.  
  81.             my @days_by_month = ($year % 4 eq 0)
  82.                 ? (31, 60, 91, 121, 152, 182, 213, 244, 274, 305, 335, 366)
  83.                 : (31, 59, 90, 120, 151, 181, 212, 243, 273, 304, 334, 365);
  84.  
  85.             my $month = 1;
  86.             my $dd = $2*1;
  87.  
  88.             my $day;
  89.  
  90.             foreach my $dm (@days_by_month) {
  91.                 if ($2 <= $dm) {
  92.                     $day = $dd;
  93.                 } else {
  94.                     $dd = $2 - $dm;
  95.                     $month++;
  96.                 }
  97.             }
  98.  
  99.             # grab strings corresponding to date of file
  100.             my $mo_string = sprintf "%02s", $month;
  101.             my $day_string = sprintf "%02s", $day;
  102.  
  103.             my $ssmi_site = 'ftp.ssmi.com';
  104.  
  105.             my $ftp = Net::FTP->new($ssmi_site, Timeout => 20)
  106.                 or die "Cannot connect to $ssmi_site: $@";
  107.             $ftp ->login($ftp_username, $ftp_passwd)
  108.                 or die "Login failed " , $ftp->message;
  109.             $ftp->binary;
  110.  
  111.             # get the SSMI data
  112.             $ftp->cwd("/ssmi/f15/bmaps_v06/y${year}/m${mo_string}/")
  113.                 or die "Cwd failed ", $ftp->message;
  114.             $ftp->get("f15_${year}${mo_string}${day_string}v6.gz")
  115.                 or die "Get failed ", $ftp->message;
  116.  
  117.             # get the SST data
  118.             $ftp->cwd("/sst/daily/tmi_amsre/")
  119.                 or die "Cwd failed", $ftp->message;
  120.             $ftp->get("tmi_amsre.fusion.${year}.${doy}.v02.gz")
  121.                 or die "Get failed ", $ftp->message;
  122.  
  123.             $ftp->quit;
  124.         }
  125.  
  126.         # if new year, grab NCEP data
  127.  
  128.         if ($year != $hold_year) {
  129.  
  130.             my $ncep_site = "ftp.cdc.noaa.gov";
  131.  
  132.             my $ftp = Net::FTP->new($ncep_site, Timeout => 20)
  133.                 or die "Cannot connect to $ncep_site: $@";
  134.             $ftp->login($ftp_username, $ftp_passwd)
  135.                 or die "Login failed " , $ftp->message;
  136.             $ftp->binary;
  137.             $ftp->cwd('/Datasets/ncep.reanalysis.dailyavgs/surface/')
  138.                 or die "Cwd failed ", $ftp->message;
  139.             $ftp->get("uwnd.sig995.${year}.nc")
  140.                 or die "Get failed ", $ftp->message;
  141.             $ftp->get("vwnd.sig995.${year}.nc")
  142.                 or die "Get failed ", $ftp->message;
  143.             $ftp->quit;
  144.  
  145.         }
  146.     }
  147.  
  148.     # hold the day of year and year for comparison to next iteration of loop
  149.  
  150.     $hold_doy = $doy;
  151.     $hold_year = $year;
  152.  
  153.     # write out the relevant data
  154.  
  155.     print OUTFILE "$year $doy $AMSU $ssmi_filename $sst_filename $ncep_uwind_name $ncep_vwind_name\n";
  156.  
  157. }
  158.  
  159. # move all data to new data directory
  160.  
  161. foreach my $file (glob "*.eos") {
  162.  
  163.     my $newfile = "$data_dir/$file";
  164.     rename $file, $newfile;
  165.  
  166. }
  167.  
  168. foreach my $file (glob "*.gz") {
  169.  
  170.     my $newfile = "$data_dir/$file";
  171.     rename $file, $newfile;
  172.  
  173. }
  174.  
  175. foreach my $file (glob "*.nc") {
  176.  
  177.     my $newfile = "$data_dir/$file";
  178.     rename $file, $newfile;
  179.  
  180. }
  181.  
  182. # end of file
  183.  
Nov 24 '06 #6
psutoad
4 New Member
I cannot thank you enough, both for your assistance is finding the bugs (yes, that error message was much more helpful), but more importantly for helping me take better advantage of all perl has to offer. I feel like I learned a great deal from you, which is significantly more valuable than just having code that works.

I apologize for my late reply, as I was on travel for the holiday. I hope your holidays are meaningful and enjoyable.

Thanks again,
-T
Nov 28 '06 #7

Sign in to post your reply or Sign up for a free account.

Similar topics

4
2396
by: Keith | last post by:
All: What is the difference between Perl (CGI) and PHP (Apache module)? I thought both used servers to direct the user to the appropiate Perl or PHP program in order to execute the program to interpret the user's Perl or PHP script respectively. Keith
6
5938
by: John Smith | last post by:
Hello, I have a rather odd question. My company is an all java/oracle shop. We do everything is Java... no matter what it is... parsing of text files, messaging, gui you name it. My question is this... is Perl so much better at parsing text files and outputing that we would see a substantial speed increase? We process about 10 million...
31
4753
by: surfunbear | last post by:
I've read some posts on Perl versus Python and studied a bit of my Python book. I'm a software engineer, familiar with C++ objected oriented development, but have been using Perl because it is great for pattern matching, text processing, and automated testing. Our company is really fixated on risk managnemt and the only way I can do enough...
3
4948
by: PzYon | last post by:
Hey 2gether! I'm trying to execute a PERL script on the web server when the user presses a button in an ASP.NET Web Application using Visual Basic. The most obvious solution for me seemed to be to use the 'Shell()'-command . This didn't work: Either I get an "File not found" error, or I get no error at all, but the PERL script isn't...
1
3619
by: smsabu2002 | last post by:
Hi, I am facing the build problem while installing the DBD-MySql perl module (ver 2.9008) using both GCC and CC compilers in HP-UX machine. For the Build using GCC, the compiler error is produced due to the unknown GCC compiler option "+DAportable". For the Build using CC, the preprocessor error is produced due to the recursive...
9
4508
by: Dieter Vanderelst | last post by:
Dear all, I'm currently comparing Python versus Perl to use in a project that involved a lot of text processing. I'm trying to determine what the most efficient language would be for our purposes. I have to admit that, although I'm very familiar with Python, I'm complete Perl noob (and I hope to stay one) which is reflected in my questions....
6
2990
by: surfivor | last post by:
I may be involved in a data migration project involving databases and creating XML feeds. Our site is PHP based, so I imagine the team might suggest PHP, but I had a look at the PHP documentation for one of the Pear modules for creating XML and it didn't look like much. I've used Perl XML:Twig and I have the impression that there is more Perl...
7
55097
numberwhun
by: numberwhun | last post by:
**NOTE: This article is written using the 5.8.8 Alpha2 release of Strawberry Perl. I am writing this article with much joy and glee. This is due to the fact that Active State no longer has a monopoly on the issue of Perl on the Windows platform As anyone who code's Perl on the Windows platform knows, the choice that you have (had) for Perl...
223
7181
by: Pilcrow | last post by:
Given that UNIX, including networking, is almost entirely coded in C, how come so many things are almost impossible in ordinary C? Examples: Network and internet access, access to UNIX interprocess controls and communication, locale determination, EBCDIC/ASCII discrimination, etc. Almost all of these are easy in Perl. Why isn't there a...
1
5020
by: rajpar | last post by:
Environment: Solaris (client + server) db2 version 7.2 latest fixpak (DB2 v7.1.0.111", "s050516" and "U803330") Compiler: gcc Here is my SP code executed on the client: CREATE PROCEDURE db2user.x (IN value INT) LANGUAGE SQL BEGIN INSERT INTO db2dba.t1 values (value);
0
7817
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
7738
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
8258
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
1
7833
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
6482
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
0
3759
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
0
3771
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1359
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
1082
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.