473,386 Members | 1,668 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,386 software developers and data experts.

Script not listing all the files in the dir

154 100+
Hi The script seems to be working ok with a list files to read a dir from but i got a dir with thousands of files and it does not search through all files, when i do a print of the read dir it comes up with less than ahindred. Could some one please help me, not sure which part is wrong in the code. Need it read and search the whole dir

Expand|Select|Wrap|Line Numbers
  1. #!/usr/bin/perl
  2. #use strict;
  3. use warnings;
  4. use File::Copy;
  5.  
  6. $| = 1;
  7.  
  8. # Contains a list of product numbers
  9. my $prodfile = "/home/prod.txt";
  10.  
  11. #Contains a bunch of flat files where i would take a product number and check line in each file of that dir and if they have a line containing the prod number it will print to the output file.
  12. my $srcdir = "/home/archive/";
  13. my $finalfile = "/home/output.txt";
  14.  
  15. # Check for command line arguments
  16. if ($#ARGV != 0) {
  17.     die "Usage: $0 You need to enter <search string> \n";
  18. } else {
  19.     #print "Arg : $#ARGV\t @ARGV \n";
  20.     $search = "ARGV[0]";
  21. }
  22.  
  23. print "##### Reading Line from File ##### \n\n\n\n";
  24. open(PRODFILE, "< $prodfile") or die (" Could not open Product File $!");
  25. while (my $product = <PRODFILE>) {
  26.     chomp($product);
  27.  
  28.     ##### Opens Dir and Lists files in archive dir #####
  29.     opendir(DIR, $srcdir) or die "Can't open $srcdir: $!";
  30.     while (my $files = readdir(DIR)) {
  31.         print " This is file $files \n";
  32.  
  33.         if ((-f == "$files") && (("$files")=~ /^$search/)) {
  34.             print " This product is : $product \n";
  35.  
  36.             ### Open file and searching for matching products ###
  37.             open(INFILE2,"< $srcdir/$files") || die(" Could not open INFILE2 File!");
  38.             open(OUTFILE2, ">> $finalfile") || die("Could not OUTFILE2 open file $!");
  39.  
  40.             # while (my $findprod = <INFILE2>) {
  41.             #    chomp($findprod);
  42.             #    if($product =~ /$findprod/) {
  43.             #        print OUTFILE2 "$findprod \n";
  44.             #        print "******* Match Found: $findprod ********* \n\n"
  45.             #    }
  46.  
  47.             #### Checking in line in the file and comaring with prod.txt to see if there is a match ######
  48.             while (my $findprod = <INFILE2>) {
  49.                 chomp($findprod);
  50.                 # print "qq~if ($product =~ /$findprod/)\n~";
  51.  
  52.                 if ($findprod =~ /$product\|/) {
  53.                     print "******* Match Found: $findprod ********* \n\n";
  54.                     sleep 2;
  55.                     print OUTFILE2 "$findprod \n";
  56.                 }
  57.             }
  58.         }
  59.     }
  60. }
  61.  
  62. close(DIR);
  63. close OUTFILE2;
  64. close INFILE2;
  65. close PRODFILE;
  66.  
Mar 15 '07 #1
14 1752
KevinADC
4,059 Expert 2GB
this line jumped out at me:

Expand|Select|Wrap|Line Numbers
  1. if ((-f == "$files") && (("$files")=~ /^$search/)) {
besides having excessive parenthesis , maybe this:

Expand|Select|Wrap|Line Numbers
  1. (-f == "$files") 
should be:

Expand|Select|Wrap|Line Numbers
  1. (-f $files) 
the wole line is better written as:

Expand|Select|Wrap|Line Numbers
  1. if (-f $files  &&  $files =~ /^$search/) {
you never need to put double-quotes around a single scalar variable and the over use of parenthesis can lead to problems if you don't understand the implications of using them correctly/incorrectly.
Mar 15 '07 #2
jonathan184
154 100+
Interesting The reason i did it like that was I thought i had to separate each condition. Great thanks for the help I will try that?
Mar 15 '07 #3
jonathan184
154 100+
Hi Kevin
ok the strange thing is when I coded the correct way you told me.
Expand|Select|Wrap|Line Numbers
  1. if (-f $files  &&  $files =~ /^$search/) {
it did not come up with errors but the script stopped at after listing the files with the print statement i used
Expand|Select|Wrap|Line Numbers
  1. print " This product is : $product \n";
so it listed some of the files and went back to the command prompt like the script was done. Nothing came up and it did not print any lines or output file.

Now just to experiment i tried
Expand|Select|Wrap|Line Numbers
  1.     Expand|Select|Wrap|Line Numbers
  •  
  •     
  •  
  • if (-f == $files  &&  $files =~ /^$search/) {
  •  
  •     
  •  
  •  
  • I got these errors repeating untilt he end of the script :

    Argument "PMD555_3Com.20060806" isn't numeric in numeric eq (==) at compare.pl line 41, <PRODFILE> line 1.
    Use of uninitialized value in numeric eq (==) at compare.pl line 41, <PRODFILE> line 1.
    This is file mig_sbl_customer_06844035000.txt

    But it looked like it was searching through the lines which it suppose to. Since it is not reading all the files in the dir to search through its not finding everything because some files are showing up in the readdir loop and not all. Its crazy.
    Mar 15 '07 #4
    KevinADC
    4,059 Expert 2GB
    do this for a sanity check:

    Expand|Select|Wrap|Line Numbers
    1. if (-f $files) {
    remove the $files =~ /^$search/ condition, then run the code and see if it prints. If it does, you know the regexp is causing the problem.

    This is basic debugging, removing some or all conditions and testing them one at a time until you are able to determine which if any is the culprit.
    Mar 15 '07 #5
    KevinADC
    4,059 Expert 2GB
    I'll check back later, I have some work to do right now :(
    Mar 15 '07 #6
    jonathan184
    154 100+
    Ok I tried that but it still stopping at the print statement above the if statement.



    do this for a sanity check:

    Expand|Select|Wrap|Line Numbers
    1. if (-f $files) {
    remove the $files =~ /^$search/ condition, then run the code and see if it prints. If it does, you know the regexp is causing the problem.

    This is basic debugging, removing some or all conditions and testing them one at a time until you are able to determine which if any is the culprit.
    Mar 15 '07 #7
    jonathan184
    154 100+
    For experimental purposes I tried

    Expand|Select|Wrap|Line Numbers
    1. if($files) {
    and it did go through to the rest of the code and did the print statements.

    So I guess you are right something was messing it up.

    NP Catch you laters when you get some time.
    Mar 15 '07 #8
    KevinADC
    4,059 Expert 2GB
    this is in your original code, see any problem with it?

    Expand|Select|Wrap|Line Numbers
    1. $search = "ARGV[0]";
    Mar 16 '07 #9
    jonathan184
    154 100+
    yes the quotes but itried it without the quotes and got the same issue.

    this is in your original code, see any problem with it?

    Expand|Select|Wrap|Line Numbers
    1. $search = "ARGV[0]";
    Mar 16 '07 #10
    KevinADC
    4,059 Expert 2GB
    do you really want $search to literally equal the string ARGV[0] or the value of scalar $ARGV[0]?

    You should never double-quote single scalars, "strict" would have picked that up and warned of a bareword on that line and alerted you to the fact you omitted the $ . Double-quoting single scalars is totally unnecessary and only slows down your script because it forces perl to make copy of the scalar that you never use. Double-quotes are a string operator, which allows constructs like this to work:

    Expand|Select|Wrap|Line Numbers
    1. my $foo = 'foo';
    2. my $bar = 'bar';
    3. my $foobar = "$foo$bar";
    Mar 16 '07 #11
    jonathan184
    154 100+
    Sorry for the late reply Kevin.

    I guess I learned it as not literally use " " and if it using literally use ' '
    Would this be correct?
    Mar 20 '07 #12
    miller
    1,089 Expert 1GB
    Sorry for the late reply Kevin.

    I guess I learned it as not literally use " " and if it using literally use ' '
    Would this be correct?
    Hi Jonathan,

    No that is not correct.

    What Kevin was hoping that you would recognize was the missing $ in front of your variable. Currently you're simply assigning the constant "ARGV[0]" to $search, instead of the variable $ARGV[0], which is the first parameter sent at the command line:

    Expand|Select|Wrap|Line Numbers
    1. my $search = $ARGV[0];
    2.  
    This is why it is bad to enclose variables in quotes, as it keeps "use strict;" from being able to detect the error.

    - Miller
    Mar 20 '07 #13
    jonathan184
    154 100+
    I see I understand now. Thank you
    Mar 20 '07 #14
    KevinADC
    4,059 Expert 2GB
    jonathan184

    as Miller said, the missing $ was the specific problem. The bigger problem is the use of double-quotes when you should not be using them. Quotes and parenthesis are often over used by new perl coders, often because of poor perl tutorials or picking up bad habits by looking at other peoples bad code. Hopefully you learned a valuable lesson. This is one situation where "strict" was not able to save you from yourself because of the improper use of quotes.
    Mar 20 '07 #15

    Sign in to post your reply or Sign up for a free account.

    Similar topics

    2
    by: cwhite | last post by:
    hello all i'm having problems trying to find information on how to write an asp script that lists the files in a directoy what i am looking to do is just list the 5 most recent files and list...
    2
    by: Xarky | last post by:
    Hi, I would like to list all the folders and files on my harddrive in two seperate parts of the window (one part for folders, other part for files). Also, when clicking one of the folders...
    0
    by: Rob | last post by:
    The value of argument 'searchType' (0) is invalid for Enum type 'SearchOption'. Parameter name: searchType A little lost here, Latest VBExpress beta, new to VB. I am trying to list files in a...
    3
    by: simonharrison | last post by:
    Hello everyone. Hopefully someone can point me in the right direction here. I'm wanting to write a script to open microsoft word and adobe pdf documents . Here is a little background: At the...
    1
    by: TahoeKid | last post by:
    I need to modify and assemble the assembler listing files generated from the VS 2005 IDE. A test generated listing file did not assmeble 'as is'. Has anyone tried this? It seems to me the...
    1
    by: sunishvk | last post by:
    I am using CTreeCtrl in my project and i want to list all the files and subfolders in a folder recursively in CTreeCtrl. I am stuck with this.. I want the code to list all the files and subfolders in...
    3
    by: Jrdman | last post by:
    hi. i wrote that code to list the files existed in "c:\" but it doesn't seem to work cuz when i excute it it dosn't list all the exitsed files in "c:\" can someone tell me what's wrong ?...
    2
    by: Mike P | last post by:
    Is it possible to write some code to print to the screen a listing of all files within a directory? I want to show the path of each file on screen and enable the user to download any of the files....
    2
    by: Jay Garland | last post by:
    How can I list 20 files in a directory and have multiple pages of 20 files? Like you see on memberlists, there's 10 pages but 20 members per page.
    0
    by: Charles Arthur | last post by:
    How do i turn on java script on a villaon, callus and itel keypad mobile phone
    0
    by: aa123db | last post by:
    Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
    0
    by: emmanuelkatto | last post by:
    Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
    0
    BarryA
    by: BarryA | last post by:
    What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
    1
    by: nemocccc | last post by:
    hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
    1
    by: Sonnysonu | last post by:
    This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
    0
    by: Hystou | last post by:
    There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
    0
    marktang
    by: marktang | last post by:
    ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
    0
    Oralloy
    by: Oralloy | last post by:
    Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...

    By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

    To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.