to count the number of bigrams

humaid

hi,guys i have done a program to count the number of bigrams.
i have taken a input file by using @ARGV[0],then icounted the number of lines in the file,using the split function i splited the sentence and pushed it in an array.and counted how many times the word is repeated.similarly,this is for single word,similarly i have paired two words from the array,and now i want to count the number of occurence of this,please help me out of this..........the code is here.

Expand|Select|Wrap|Line Numbers

 
#!/user/bin/perl

$file=@ARGV[0];

undef $/;

open (FH,"$file");
 
@f=<FH>;

$n=<FH>;

close (FH);

print "$n\n";

$a=@f;

print "the number of lines in the file is:$a\n";
 
foreach $lin(@f)

{

    chomp($lin);

 push(@b,split (/\s+/,$lin));
 
}

print "@b \n";
 
$c=@b;
 
print "total numbers is $c\n";
 
for $i(0 .. $#b-1)

{

    $bigram=($b[$i] . " " . $b[$i+1]);

    print "$bigram\n";
 
}

till this part iam getting the bigrams,but from here onwards i have to calculate
it that how many times it appers in pair.

Expand|Select|Wrap|Line Numbers

 
$count=0;

while($n =~ /this is/)

{

$count += "$n";

$count++;

}

print "to play=$count\n";

numberwhun

Humaid,

It is common practice on this site that when you post code to the forums that code tags should surround your code. I have added them to this post for you. This is the second time I have done this for you. I ask that after this, you please diligently apply the code tags to the code in your posts.

If you do not know how to use them, please refer to the "REPLY GUIDELINES" to the right of the posting window.

Regards,

Jeff

KevinADC

humaid,

Is this school or course work of some kind?

numberwhun

Well, considering in his last post that he said he was "told to do" the program, I would say that all the stuff he is doing is school related.

Regards,

Jeff

humaid

jeff,first of all sorry for the code tags here after i'll see to it,and next thing is that iam not a student,iam working in a research center where i was doing program in java for 3 years,but now i was shifted to a search engine project for a regional language,for which i have to study perl.my first posts were just a practise,the day i posted my first question was my third day in perl.

in this iam trying to train the machine which it can differentiate the domains by knowing the repititive word,now iam trying for bigrams then trigrams and so on.

KevinADC

First thing you need to do is compile all your perl programs using the "strict" and "warnings" pragmas;

Expand|Select|Wrap|Line Numbers

 use strict;

use warnings;

Don't bother to continue learning perl unless you do. These two pragmas will save you so much problems you have no idea. Start using them immediately even if you do not understand what the benefit is. Eventually you will understand.

Don't continue with the bad habit of double-quoting single scalars in perl like you did here:

open (FH,"$file");

this is asking for trouble. Using double-quotes like that can introduce very hard to find bugs in your perl code that not even strict and warnings will help you to find. It also slows down your perl program because the double-quotes forces perl to make an annonymous copy of whats inside the double-quotes. Since the scalar is already defined this is unecessary.

Don't use $a or $b in your perl programs except for sorting with the sort() function. They are special variables that should never be declared with "my" in your perl programs. Just don't use them except for sorting.

I am pretty sure there are ngram modules on CPAN. If you are going to learn perl you need to know about CPAN and how to install modules unless you have tech support that does that for you. You still need to know about CPAN though.

http://search.cpan.org/

I think thats enough for now.

humaid

thanks kevin,i'll remember your words.the problem iam facing is
iam getting the bigrams in an array,and iam printing it, now i want to count that a particular bigram occurs how many time.
shall i use regex to match all the bigram with the $n value or something else.

KevinADC

You would use a regexp if you are dealing with patterns. You would use a string operator if you are dealing with strings. If you are counting one set of ngrams a simple scalar will work. if you are counting multiple sets of ngrams a hash would be better.

humaid

thanks kevin i'll try out.

humaid

thanks kevin i tried it out and got the result.here is the code

Expand|Select|Wrap|Line Numbers

 
foreach $w(@c)

{

    $hash{$w}++;

}

foreach $w1(keys(%hash))

{
 
print "$w1 => $hash{$w1}\n";

}

humaid

hey guys i have used the code tags but y is it not like it.

numberwhun

You need to look at the example's given in the "REPLY GUIDELINES" on the right side of your screen when replying.

Just like the QUOTE tags, the CODE tags start with a [ character and they end with a ] character. In between you would have something like code=perl. That will do what you want. The end tag ( there should always be an end tag, just like the QUOTE tag, should be /code between the square brackets. If you reply to this message, just look at the tags I have used below:

Expand|Select|Wrap|Line Numbers

 
some code

some more code

I have also fixed the post above. :-)

Regards,

Jeff

humaid

Expand|Select|Wrap|Line Numbers

  
foreach $w(@c)

{

    $hash{$w}++;

}

foreach $w1(keys(%hash))
 
print "$w1 => $hash{$w1}\n";

}

is it like this........
sorry jeff i have edited this..

numberwhun

Exactly! Thank you for your diligence!

Regards,

Jeff

to count the number of bigrams

Post your reply

Similar topics

Question stats

	Ask Question
Home Questions Articles Browse Topics Latest Top Members FAQ