which api/method to use for concatenating multiple large files using least RAM?

[anyaservices]

Hi

I have multiple files (between 50-100) that basically contain simple text. Each file ranges in size from few MB to 50MB.

I have to concatenate all these files to produce one single file that contains the content of all the files-- and I have to do this in a machine that has quite less RAM-- I can use max 512MB RAM for the JVM.

How do I do this? Which API/method do I use to accomplish concatenation of multiple files while using least values?

[Jwalker]

try this

public void concatFile() {
        PrintWriter pw = new PrintWriter(new FileOutputStream("C:/temp/out/out.txt"));
        File file = new File("C:/temp/");
        File[] files = file.listFiles();
        for (int i = 0; i < files.length; i++) {
                BufferedReader br = new BufferedReader(new FileReader(files[i].getPath()));
                String line = br.readLine();
                while (line != null) {
                        pw.println(line);
                        line = br.readLine();
                }
                br.close();
        }
        pw.close();
}
 

simply read line by line, each file

[Anas Mosaad]

You may leverage the operating system capabilities to implement this task. Windows copy command can do this utlra fast.
For example, if you have 4 files to concatenate (T1.txt... T4.txt) into one file T5.txt, the following command can make it for you.

copy /B T1.txt+T2.txt+T3.txt+T4.txt T5.txt

Linux version would be:

cat T1.txt T2.txt T3.txt T4.txt > T5.txt

I'm expecting you to know how to execute OS commands from Java.

Good luck.