Word Count not Counting Right...

[KWSW]

Got this assignment due a few weeks later and since I am done with the up coming assignment, decided to try out the next one early rather than rush later. First part of it requires me to do a character count on a text document which i will have to use huffman coding to encode.

Below is my code that i created to count and display the characters and its frequency in the document.

import java.io.*;
import java.util.*;
 
public class SourceModel 
{
 
 
    public static void main(String [] Args)
    {
 
        int j;
        String str;
 
        Map<Integer, Integer> m = new HashMap<Integer, Integer>();                                
 
        try
        {
            BufferedReader in = new BufferedReader(new FileReader("DecOfInd.txt"));             
 
            while ((str = in.readLine()) != null) 
            {
                for(int i = 0; i < str.length(); i++)
                {                        
 
                    if(!m.containsKey((int)str.charAt(i)))
                    {
                        m.put((int)str.charAt(i),1);
                    }
                    else
                    {
                        j = m.get((int)str.charAt(i));
                        j++;                
 
                        m.put((int)str.charAt(i),j);
 
                    }
 
                }
            }            
 
        }
        catch (IOException e) 
        {
 
        }                        
 
        System.out.println();
        System.out.println(m.size() + " distinct letters:");
        System.out.println(m);
 
        int count = 0;
 
        for(int i :m.keySet()) 
        {
            System.out.println(i + " = " + m.get(i));
            count += m.get(i);
        }
        System.out.println("Total Number of Characters: "+count);
 
    }
}
 

The code has no errors and from the first run, it looks like everything went ok with it displaying the characters and the count. But just to make sure I got it right, I used the unix command "wc" on the document and it seems I have missing characters or something.

Unix gives:
$ wc DecOfInd.txt
29 1369 8458 DecOfInd.txt

My program gives:
Total Number of Characters: 8429

Any kind soul would like to help me find out whats wrong with my code as I seem to have 29 missing characters.

Many thanks in advance.
Kenneth :)

[r035198x]

Why did you use Map<Integer, Integer> instead of Map<Character, Integer> ?

[KWSW]

Why did you use Map<Integer, Integer> instead of Map<Character, Integer> ?

Was trying to use the Ascii code as the key.

[JosAH]

Unix gives:
$ wc DecOfInd.txt
29 1369 8458 DecOfInd.txt

My program gives:
Total Number of Characters: 8429

Any kind soul would like to help me find out whats wrong with my code as I seem to have 29 missing characters.

Many thanks in advance.
Kenneth :)

wc also counts the end of line characters (there are 29 lines in your file). Your
method doesn't (it reads entire Strings and removes the \n characters), so
8429+29 == 8458

kind regards,

Jos

[r035198x]

wc also counts the end of line characters (there are 29 lines in your file). Your
method doesn't (it reads entire Strings and removes the \n characters), so
8429+29 == 8458

kind regards,

Jos

Duh .

[JosAH]

Duh .

<vigorously starts kicking dust towards the general direction of r035198x/>

:-P

kind regards,

Jos ;-)

[prometheuzz]

@OP: and if you run your code on other OS-es, you can get even different results. On Windows for example, the line separator is made out of two characters: \r\n.

[KWSW]

ah thanks for that info... hmmm will read() work instead since it doesn't read lines like readline()?

[JosAH]

ah thanks for that info... hmmm will read() work instead since it doesn't read lines like readline()?

Yup, that's the way to go because you want to reproduce the original file content
after Huffman decompression so you should take care of every single byte when
you compress the stuff. readLine() is a nono here, or you may want to add
those end-of-line characters yourself to the Huffman tables/tree.

kind regards,

Jos

[KWSW]

Yup, that's the way to go because you want to reproduce the original file content
after Huffman decompression so you should take care of every single byte when
you compress the stuff. readLine() is a nono here, or you may want to add
those end-of-line characters yourself to the Huffman tables/tree.

kind regards,

Jos

ok thanks for the heads up... will give it a try tmr... :)

[KWSW]

With the advise here, I have modified my code to write to a file the fequency of each character in its ascii code.

import java.io.*;
import java.util.*;
 
public class SourceModel 
{
 
 
    public static void main(String [] Args)
    {        
 
        Map<Integer,Integer> m = new HashMap<Integer,Integer>();
 
        String sourceFile = "DecOfInd.txt";
        String outputFile = "DisProb_"+sourceFile;
 
        String str = "";
 
        try 
        {
 
            File source = new File(sourceFile);
            FileInputStream in = new FileInputStream(source);
 
 
            int size = (int)source.length();            
            byte[] text = new byte[size];
 
            System.out.println("The size of the file is "+source.length());
 
            int b = in.read(text);
            int count;
 
            for (int i = 0; i < size ; i++) 
            {
                if(m.containsKey((int)text[i]))
                {
                    count = m.get((int)text[i]);
                    count++;
                    m.put((int)text[i],count);
                }
                else
                {
                    m.put((int)text[i],1);
                }
            }
 
            in.close();
        }       
 
        catch (IOException e) 
        {
 
        }
 
        // Writting Character Fequency To File
 
        try
        {
            BufferedWriter out = new BufferedWriter(new FileWriter(outputFile));                        
 
            int k = 0;
 
            for(int i : m.keySet()) 
            {                                                
                k += m.get(i);                                                
                out.write(i + " = " + m.get(i) + '\n');
            }
 
            out.write("Total Number of Characters: "+k+'\n');
 
            out.close();
        }
        catch (IOException e) 
        {
 
        }
 
 
        System.out.println();
        System.out.println(m.size() + " distinct letters:");
        System.out.println(m);
 
        int j = 0; int feq;
 
        for(int i : m.keySet()) 
        {            
            j += m.get(i);
            System.out.println(i + " = " + m.get(i));
        }
 
        System.out.println("Total Number of Characters: "+j);                           
 
    }
}

While it runs fine on my machine(win xp), it seems to be giving me problems over at the school's server...

$ javac -classpath . SourceModel.java      
SourceModel.java:11: not a statement
        Map<Integer,Integer> m = new HashMap<Integer,Integer>();
           ^
SourceModel.java:11: ';' expected
        Map<Integer,Integer> m = new HashMap<Integer,Integer>();
                   ^
SourceModel.java:63: ';' expected
            for(int i : m.keySet()) 
                      ^
SourceModel.java:72: illegal start of expression
        }
        ^
SourceModel.java:85: ';' expected
        for(int i : m.keySet()) 
                  ^
SourceModel.java:93: illegal start of expression
    }
    ^
SourceModel.java:91: ';' expected
        System.out.println("Total Number of Characters: "+j);                           
                                                             ^
7 errors

Is there something I need to do to make it unix friendly?

[JosAH]

While it runs fine on my machine(win xp), it seems to be giving me problems over at the school's server...

Is there something I need to do to make it unix friendly?

It has nothing to do with Unix or not: your school machine is running Java 1.4
or earlier; your home machine is running Java 1.5 or later.

kind regards,

Jos

[KWSW]

It has nothing to do with Unix or not: your school machine is running Java 1.4
or earlier; your home machine is running Java 1.5 or later.

kind regards,

Jos

Thanks for the quick reply... time to email my lecturer...