how to read large CSV file

jx2

hi everyone
i'm trying to read data from CSV file file is about 1MB (or bigger)
i figured out how to read it quickly from the disc but it takes 3000 milliseconds to change it into arrays so my question is
is there a faster way of doing it?
thats my class:

Expand|Select|Wrap|Line Numbers

 
public class ReadWrite {
 
    private static String filename = "myFile.csv";

    private static File file;

    private static Scanner in;

    private static String str="";

    private int[] time =     new int[100000];

    private float[] value =  new float[100000];

    private String [] date = new String[100000];

    private String[] update =new String[100000];
 
    public static int count = 0;

    public boolean hasNext = true;
 
    static private String str2="";
 
    /** Creates a new instance of ReadWrite */

    public ReadWrite() throws FileNotFoundException, IOException {

        file = new File(filename);

        fileReader = new FileReader(file);
 
        readFile();
 
        in = new Scanner(this.str);

        in.useDelimiter("(\t|\n)");
 
    }
 // thats where the problems are this method is to slow

// slower then PHP :/  

//(in php i would use explode("\n",str) and explode("\t",str)) 

    public void read(){

        while(in.hasNextInt()){

            time[count] = in.nextInt() ;

            value[count] = in.nextFloat();

            date[count] = in.next();

            update[count] = in.next();

            count++;

        }

    }

 ------------------------------------end of problems -----------------------------

    public String toString(int i){

        return time[i] + " " +value[i] +" "+ date[i];

    }
 
    FileReader fileReader;

    void readFile() throws IOException{

        while(hasNext){

            char [] c = new char[4096];

            if(fileReader.read(c) > -1 ){

                str += String.valueOf(c);

            }else{

                hasNext = false;                

            }

        }

    }
 
}

BigDaddyLH

I don't know about the speed of code, but first copying the entire file into a string is a whacky thing to do. Why not just read from the file?

Expand|Select|Wrap|Line Numbers

in = new Scanner(new File(filename));

Also, your use of static fields needs to be rethought, but on another day.

Finally, why are you loading the entire file into memory at once? Can you avoid this?

jx2

Quote:

Originally Posted by BigDaddyLH

I don't know about the speed of code, but first copying the entire file into a string is a whacky thing to do. Why not just read from the file?

Expand|Select|Wrap|Line Numbers

in = new Scanner(new File(filename));

Also, your use of static fields needs to be rethought, but on another day.

Finally, why are you loading the entire file into memory at once? Can you avoid this?

actualy i used scanner but it was the slowest one
perhaps i 'm missing something importent

Expand|Select|Wrap|Line Numbers