Searching for Keywords in Files in a Dir

[jonathan184]

script to search for keywords in files in a dir

Basically the script consists of a file with part numbers

I got a dir with files I want to search the files line by line and if a line has the part number I want it to print that line to an outfile file.

I did this script but it does not find matching products.
I aded an else statement and it only prints the else statements alone when i commented out the else statements nothing prints tot output file.

This is the script i am working with hopefully someone could show where i am going wrong. Thanks in advance.

#!/usr/bin/perl
$| = 1;
 
use File::Copy;
# Contains a list of product numbers
my $prodfile = "d:\\prod\\automated_Script\\testfile\\prod.txt";
 
#Contains a bunch of flat files where i would take a product number and check line in each file of that dir and if they have a line containing the prod number it will print to the output file.
my $srcdir = "d:\\prod\\automated_Script\\archivetest";
my $finalfile = "d:\\prod\\automated_Script\\testfile\\output.txt";
 
##### Listing files in archive dir #####
opendir(DIR, $srcdir) or die "Can't open $srcdir: $!";
@files = grep {!/^\.+$/} readdir(DIR);
close(DIR);
print @files;
 
if (!@files) {
    print "No files in dir.\n\n";
    last;
}
 
##### Reading Line from File #####
open(PRODFILE, "< $prodfile") or die (" Could not open Product File $!");
while ($product = <PRODFILE>) {
    chomp($product);
    foreach $files (@files) {
        print " This product is : $product \n";
 
        ### Open file and searching for matching products ###
 
        open(INFILE2,"< $srcdir\\$files") || die(" Could not open INFILE2 File!");
        $findprod=<INFILE2>;
        chomp($findprod);
 
        open(OUTFILE2, ">> $finalfile") || die ("Could not OUTFILE2 open file $!");
 
        while (<INFILE2>) {
            if ($product =~ /$findprod/) {
                print OUTFILE2 "$findprod";
            #} else {
            #    print OUTFILE2 "Product not found \n";
            }
 
            close OUTFILE2;
        }
        close INFILE2;
 
        print "Product search completed";
    }
}
close PRODFILE;
 

[KevinADC]

is this a one line file?

        open(INFILE2,"< $srcdir\\$files") || die(" Could not open INFILE2 File!");
        $findprod=<INFILE2>;
        chomp($findprod);
 

[jonathan184]

no it has muliple lines

[KevinADC]

try this:

#!/usr/bin/perl
use strict;
use warnings;
$| = 1;
 
use File::Copy;
# Contains a list of product numbers
my $prodfile = "d:\\prod\\automated_Script\\testfile\\prod.txt";
 
#Contains a bunch of flat files where i would take a product number and check line in each file of that dir and if they have a line containing the prod number it will print to the output file.
my $srcdir = "d:\\prod\\automated_Script\\archivetest";
my $finalfile = "d:\\prod\\automated_Script\\testfile\\output.txt";
 
##### Listing files in archive dir #####
opendir(DIR, $srcdir) or die "Can't open $srcdir: $!";
my @files = grep {!/^\.+$/} readdir(DIR);
close(DIR);
print @files;
 
if (!@files) {
    print "No files in dir.\n\n";
}
 
##### Reading Line from File #####
open(PRODFILE, "< $prodfile") or die (" Could not open Product File $!");
while (my $product = <PRODFILE>) {
    chomp($product);
    foreach my $files (@files) {
        print " This product is : $product \n";
 
        ### Open file and searching for matching products ###
 
        open(INFILE2,"< $srcdir\\$files") || die(" Could not open INFILE2 File!");
        open(OUTFILE2, ">> $finalfile") || die ("Could not OUTFILE2 open file $!");
 
      while (my $findprod = <INFILE2>) {
         chomp($findprod);
            if ($product =~ /$findprod/) {
                print OUTFILE2 "$findprod\n";
            #} else {
            #    print OUTFILE2 "Product not found \n";
            }
      }
    }
   print "Product search completed for: $product\n ";
}
close OUTFILE2;
close INFILE2;
close PRODFILE;

you can use forward slashes in your path statements even if you are using windows:

my $prodfile = 'd:/prod/automated_Script/testfile/prod.txt';

[jonathan184]

The script ran but it did not print the lines containing the matching part numbers.
When i searched the output file the part number could not be found in any of the lines. There are other products than the ones i was searching for.

Pretty strange.

[jonathan184]

Hi Kevin, It looks like the comparison string is not working , I did a print statement. The lines do not have anything relating to the product number i am trying to find. Is ther somthing wrong with my comparison string?

while (my $findprod = <INFILE2>) {
         chomp($findprod);
            if ($product =~ /$findprod/) {
                print OUTFILE2 "$findprod \n";
                 print "******* Match Found: $findprod ********* \n\n"

try this:

#!/usr/bin/perl
use strict;
use warnings;
$| = 1;
 
use File::Copy;
# Contains a list of product numbers
my $prodfile = "d:\\prod\\automated_Script\\testfile\\prod.txt";
 
#Contains a bunch of flat files where i would take a product number and check line in each file of that dir and if they have a line containing the prod number it will print to the output file.
my $srcdir = "d:\\prod\\automated_Script\\archivetest";
my $finalfile = "d:\\prod\\automated_Script\\testfile\\output.txt";
 
##### Listing files in archive dir #####
opendir(DIR, $srcdir) or die "Can't open $srcdir: $!";
my @files = grep {!/^\.+$/} readdir(DIR);
close(DIR);
print @files;
 
if (!@files) {
    print "No files in dir.\n\n";
}
 
##### Reading Line from File #####
open(PRODFILE, "< $prodfile") or die (" Could not open Product File $!");
while (my $product = <PRODFILE>) {
    chomp($product);
    foreach my $files (@files) {
        print " This product is : $product \n";
 
        ### Open file and searching for matching products ###
 
        open(INFILE2,"< $srcdir\\$files") || die(" Could not open INFILE2 File!");
        open(OUTFILE2, ">> $finalfile") || die ("Could not OUTFILE2 open file $!");
 
      while (my $findprod = <INFILE2>) {
         chomp($findprod);
            if ($product =~ /$findprod/) {
                print OUTFILE2 "$findprod\n";
            #} else {
            #    print OUTFILE2 "Product not found \n";
            }
      }
    }
   print "Product search completed for: $product\n ";
}
close OUTFILE2;
close INFILE2;
close PRODFILE;

you can use forward slashes in your path statements even if you are using windows:

my $prodfile = 'd:/prod/automated_Script/testfile/prod.txt';

[KevinADC]

Your best bet is to add as many "print" lines as you can to "watch" the data and script as it's being processed. This often results in catching the error in the logic of code that runs but does not do what you expect it to.

[jonathan184]

After putting the print statements i think i troubleshoot it down to the if statement not work and it just prints all the lines from the extract files and not doing what the if statement said. IS there something I am doing wrong?

      while (my $findprod = <INFILE2>) {
         chomp($findprod);
            if ($product =~ /$findprod/) {
                print OUTFILE2 "$findprod \n";
                 print "******* Match Found: $findprod ********* \n\n"}
      }

[KevinADC]

do this:

      while (my $findprod = <INFILE2>) {
         chomp($findprod);
         print qq~if ($product =~ /$findprod/)\n~; 
            if ($product =~ /$findprod/) {
                print OUTFILE2 "$findprod \n";
                 print "******* Match Found: $findprod ********* \n\n"}
      }

and post some of the lins that get printed

[jonathan184]

Hi Kevin


Ok this is crazy I ran the code you gave just put the print statement quotes

qq~if (3CRTP0400C96C-ME =~ /EA|0223-000-151-1|PKG,END CAP,PROTEUS||||0.000|0.000|KG|0.000||0.000|0.000|0.000||90|N/A||| | ||/)
~******* Match Found: EA|0223-000-151-1|PKG,END CAP,PROTEUS||||0.000|0.000|KG|0.000||0.000|0.000|0.000||90|N/A||| | || ********* 

This is one of the lines it is printing fromt he extract file.

Now the number it is checking would be 0223-000-151-1 and as you can see 3CRTP0400C96C-ME does not match 0223-000-151-1. For some reason it comes up as a match. The only thing that did not match was the header but every other line showed as match which is wrong should only match about two lines that was in the extract but all lines are coming up as above.

I created a test extract file with the 3CRTP0400C96C-ME dummy lines and did not use the original extract format and the script worked.
as you could see below. I am puzzled what in the format of the orginal file could be causing it not to print the correct records only. Any ideas would be greatly appreciated.

qq~if (3CRTP0400CF96C-KR =~ /This is the products header/)
~qq~if (3CRTP0400CF96C-KR =~ /EA|3CRTP0400C96C-ME|vfdggfgdfg|gfdgdfgf/)
~qq~if (3CRTP0400CF96C-KR =~ /EA|34RTP0400CF96C-KR|fdgfgdfgdfgdfgfd|fdgdfgdfgf/)
~qq~if (3CRTP0400CF96C-KR =~ /EA|3SRTP0400C96C-ME|vfdggfgdfg|gfdgdfgf/)
~qq~if (3CRTP0400CF96C-KR =~ /EA|3LRTP0400CF96C-KR|fdgfgdfgdfgdfgfd|fdgdfgdfgf/)
~qq~if (3CRTP0400CF96C-KR =~ /EA|3URTP0400C96C-ME|vfdggfgdfg|gfdgdfgf/)
~qq~if (3CRTP0400CF96C-KR =~ /EA|3HRTP0400CF96C-KR|fdgfgdfgdfgdfgfd|fdgdfgdfgf/)
~qq~if (3CRTP0400CF96C-KR =~ /EA|3YRTP0400C96C-ME|vfdggfgdfg|gfdgdfgf/)
~qq~if (3CRTP0400CF96C-KR =~ /EA|3HRTP0400CF96C-KR|fdgfgdfgdfgdfgfd|fdgdfgdfgf/)
~qq~if (3CRTP0400CF96C-KR =~ /EA|3DRTP0400C96C-ME|vfdggfgdfg|gfdgdfgf/)
~qq~if (3CRTP0400CF96C-KR =~ /EA|3CRTP0400CF96C-KR|fdgfgdfgdfgdfgfd|fdgdfgdfgf/)
~******* Match Found: EA|3CRTP0400CF96C-KR|fdgfgdfgdfgdfgfd|fdgdfgdfgf *********