filter something from file or string

[tuananh87vn]

hi,
I'm dealing with a problem in which I have to scan through a short text and print out all ip addresses within it.

assume I have a text stored in a file named 'test', and I write a small perl program which read all lines of the text and print out any ip address:

open FILE, "test" or die $!;

open FILE, "header" or die $!;
 
print "IP addresses found:\n";
while ($line = <FILE>){
 
        if ($line=~ /(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/) {
                print $1;
                print "\n";
        }
}
 

There is a problem, however, that the perl program can print only 1 ip on each line. it's ok if there are all ip addresses in the same line are similar, yet if there are different ips on 1 line, we only get 1 ip but miss the others.

example:
"Hello this is the msg from 192.168.1.1 blah blah... 192.168.1.2
This is the second text from 192.168.1.3 blah blah..."

will print out only 192.168.1.1 and 192.168.1.3, and we'll miss the 192.168.1.2

any suggestion :-?

thanx :D

[Ganon11]

Try the /g modifier on your regexp to progressively search. You'll have to fiddle with your code a bit, but it shouldn't be too bad.

[tuananh87vn]

Try the /g modifier on your regexp to progressively search. You'll have to fiddle with your code a bit, but it shouldn't be too bad.

err, how is it? perl is pretty new to me though I ppl say it's just like php. is there any more optimal way to read thru the whole text and pick up what we want?

[numberwhun]

err, how is it? perl is pretty new to me though I ppl say it's just like php. is there any more optimal way to read thru the whole text and pick up what we want?

What Ganon is referring to is taking your regular expression and making it lok like this:

$line=~ /(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/g
 

The /g will apply globally. If you look at any Perl regex tutorial, they should explain the characters after the regex. They are simply options to the regex as a whole.

Also, it looks like you are cycling through the file fine, line by line. Its a slightly slower way to do it, but slurping the entire file into an array or something tends to be ram heavy for larger files.

Regards,

Jeff

[tuananh87vn]

it still doesnot work, the new regular expression can only print the first thing that matches the pattern, then it ignore the rest of that line. does anybody know how to force it read until the end of line and print whatever matching.

is there any way to read the whole text but not line by line? can we read word by word instead? does a string work in this case?

[nithinpes]

it still doesnot work, the new regular expression can only print the first thing that matches the pattern, then it ignore the rest of that line. does anybody know how to force it read until the end of line and print whatever matching.

is there any way to read the whole text but not line by line? a string works in this case?

That should work. But, remember that though /g forces the pattern match to match all occurences, the if() loop will execute only once. You should be using while() loop.

while ($line=~ /(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/g) {
                print $1;
                print "\n";
        }
 
 

- Nithin

[nithinpes]

is there any way to read the whole text but not line by line? can we read word by word instead? does a string work in this case?

Inorder to read whole text from the file, you need to undefine input record separator ($/) which is "\n" by default ,before reading from file.

$/ = "";
while(<FILE>)  {  ## read entire file
 

[tuananh87vn]

it works like a charm when while loop is used instead of if. I see the problem: 'if' will quit immediately as soon as a match is found, yet 'while' continues until reaching the end of line :D

actually I see there's no need to set $\=""; still works well;

thx so much!

[nithinpes]

actually I see there's no need to set $\=""; still works well;

thx so much!

Yes, offcourse. That was just to answer your second question.

[tuananh87vn]

errr, another problem raised:
assume that there are some similar ip addresses found duplicated in the text: ip1, ip1, ip2, ip2, ip2, ip3... how can we filter and get each ip address printed once only (i.e eliminate the duplicated ips)

[asedt]

errr, another problem raised:
assume that there are some similar ip addresses found duplicated in the text: ip1, ip1, ip2, ip2, ip2, ip3... how can we filter and get each ip address printed once only (i.e eliminate the duplicated ips)

I know nothing about perl, but I think that the solution is to putt them in a list or array and then do something like:

http://www.perlmonks.org/?node_id=604547

http://perldoc.perl.org/perlfaq4.htm...st-or-array%3f

[numberwhun]

asedt is correct. See the Perlmonks link and you will see the following code in one of the responses there:

my @array  = ("abc","def","abc","ghi","ghi","abc","jklm","abc","def");
my %hash   = map { $_ => 1 } @array;
my @unique = keys %hash;
 

So, you would simply create an array that contains all of the IPs, then the hash would be created from that array. Since the keys in an array have to be unique, it will not create a new key. In the end, print the list of keys.

Regards,

Jeff