New to Perl and have basic array question

[svancouw]

I just started teaching myself Perl a couple days ago, and am looking forward to what this will give me for my Linux administration skills. Any help on this will be appreciated.

I ran into a problem in writing a simple script for searching for an entry in a file, then erasing the entire line. I know this can be done with grep -v in the Linux command line, but I felt I needed the practice.

The problem I'm having is that the script works, but when it finds and erases a match, it ignores the next match. I've already shown this to a friend who said it's a "weird way" to go about it, but he doesn't work much with arrays.

#!/usr/bin/perl
 
use strict;
use warnings;
 
open(INFILE, "test") or die "Can't open file.";
my @array = <INFILE>;
close(INFILE);
 
my $number_removed = 0;
my $loops = 0; #Used to tell splice() what part of the array to remove
 
foreach my $l (@array) {
    if ($l =~ /SHOREWALL/i) { #Search for the shorewall string
        splice(@array, $loops, 1); #Remove the part of the array that's
        $number_removed += 1; #being searched
    }
    $loops += 1; #Each time a line is searched increment this value to
} #...allow splice to remove the correct line
 
open(OUTFILE, ">test_result"); #Print the remaining @array to a file
print OUTFILE @array;
close(OUTFILE);
 
print "$number_removed entries were removed.\n"
 

[miller]

Greetings and welcome to perl.

Yes, your method of implementation is more difficult than it needs to be, but this will make for good practice. The main bug with your current implementation is the fact that you should not be incrementing the index when you remove an element from the array, becuase you just reduced the size by one. So here is your current code so fixed:

#!/usr/bin/perl
 
use strict;
use warnings;
 
open(INFILE, "test") or die "Can't open test: $!";
my @array = <INFILE>;
close(INFILE);
 
my $number_removed = 0;
my $index = 0; #Used to tell splice() what part of the array to remove
 
foreach my $l (@array) {
    if ($l =~ /SHOREWALL/i) { #Search for the shorewall string
        splice(@array, $index, 1); #Remove the part of the array that's
        $number_removed++; #being searched
    } else {
        $index++; #Each time a line is searched increment this value to
    }
} #...allow splice to remove the correct line
 
# Print the remaining @array to a file
open(OUTFILE, ">test_result") or die "Can't open test_result: $!";
print OUTFILE @array;
close(OUTFILE);
 
print "$number_removed entries were removed.\n"
 

However, there are two methods for file manipulation that you should probably learn for this. First of all, your entire loop procedure could be simplified by the use of the grep perl function.

#!/usr/bin/perl
 
use strict;
use warnings;
 
open(INFILE, "test") or die "Can't open test: $!";
my @array = <INFILE>;
close(INFILE);
 
my $oldsize = @array;
 
# Remove all noted lines.
@array = grep {!/SHOREWALL/i} @array;
 
my $number_removed = $oldsize - @array;
 
# Print the remaining @array to a file
open(OUTFILE, ">test_result") or die "Can't open test_result: $!";
print OUTFILE @array;
close(OUTFILE);
 
print "$number_removed entries were removed.\n"
 

Also, you can take advantage of the Tie::File core module in order to simply edit the file in place. The use of your for loop and use of splice then comes in handy in a real way.

#!/usr/bin/perl
 
use Tie::File;
 
use strict;
use warnings;
 
tie my @array, 'Tie::File', "test" or die "Can't open test: $!";
 
my $number_removed = 0;
 
for (my $index = 0; $index <= $#array;) {
    if ($array[$index] =~ /SHOREWALL/i) {
        splice @array, $index, 1;
        $number_removed++;
    } else {
        $index++;
    }
}
 
untie @array;
 

Also, as a general rule, whenever you find that you're keeping track of the index of an array, then you should use a for statement instead of a foreach. If you're just accessing the elements, then foreach is the way to go.

There are other methods to accomplish what you're trying to do, but these are two good ones to get you started on your quest for more tools.

- Miller

[svancouw]

Miller, thank you very much for your corrections and suggestions. I will look at this closely and see what I can learn from it, and will use these techniques in more practice programs.

[miller]

Your welcome. Good luck.

- Miller

[svancouw]

Miller, while I am sure your suggestion will be of great use, the corrected code you gave me still has the same symptom of skipping entries in the file. I created a new pl script and pasted the contents from your post.

In your response, you said that I shouldn't be incrementing the index, but does not your code do this as well? I don't suppose you have any other thoughts on this matter?

These are the contents of the file "test" that I used, with most lines numbered that helped me see what was going wrong.

SHOREWALL rocks
SHOREWALL SUCKS
George
Steve
3
4SHOREWALL
5SHOREWALL
6
7SHOREWALL
8SHOREWALL
9SHOREWALL
10
11SHOREWALL
12SHOREWALL
13SHOREWALL
14SHOREWALL
15
16SHOREWALL
17hi there bob
18SHOREWALL
19SHOREWALL
20hi
21there
22SHOREWALL
23bob
24go
25 away
26SHOREWALL
27i
28do
29not
30like
31mysteries
32


-Sean

[miller]

Greetings Sean,

You're right. The first code that I gave you doesn't work. The question is, can you figure out why? I know why, but the best way for you to learn would be to work through the steps. Let's see if I can help you:

Try adding some debugging statements to the script and see what results. Something like this is rather illuminating. It outputs the current

foreach my $l (@arraymirror) {
    print "$index - $l";
    if ($l =~ /SHOREWALL/i) { #Search for the shorewall string
        splice(@array, $index, 1); #Remove the part of the array that's
        $number_removed++; #being searched
        print "     Removed\n";
    } else {
        $index++; #Each time a line is searched increment this value to
    }
} #...allow splice to remove the correct line
 

After viewing the output we see that some of the entries in the @array are being skipped. Immediately we ask why this is? I think we can presume that it has something to do with the fact that we are modifying the array that we are looping on. We could check perdoc for answers about foreach. However, it's try something else. How about just replacing the array that we're looping on with a duplicate array.

my @staticarray = @array;
foreach my $l (@staticarray) {
    print "$index - $l";
    if ($l =~ /SHOREWALL/i) { #Search for the shorewall string
        splice(@array, $index, 1); #Remove the part of the array that's
        $number_removed++; #being searched
        print "     Removed\n";
    } else {
        $index++; #Each time a line is searched increment this value to
    }
} #...allow splice to remove the correct line
 

Ta-da! It works. Now obviously this is rather annoying. So now I go back to my earlier advice that I gave you. The other two methods of implementation do in fact work, and it's the third one that you should take note of. I told you that if you're ever accessing the index of an array, then you should just loop through the array by index instead of using foreach. The exact code that I provided that will can be used and will work. That gives us this

use strict;
use warnings;
 
open(INFILE, "test") or die "Can't open test: $!";
my @array = <INFILE>;
close(INFILE);
 
my $number_removed = 0;
 
for (my $index = 0; $index <= $#array;) {
    if ($array[$index] =~ /SHOREWALL/i) {
        splice @array, $index, 1;
        $number_removed++;
    } else {
        $index++;
    }
}
 
# Print the remaining @array to a file
open(OUTFILE, ">test_result") or die "Can't open test_result: $!";
print OUTFILE @array;
close(OUTFILE);
 
print "$number_removed entries were removed.\n"
 

Output:

>perl scratch.pl
16 entries were removed.
 

Good luck in your further studies.
- Miller