469,332 Members | 6,675 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,332 developers. It's quick & easy.

How do you read large text files in Java >20MB

N002213F
I have a Java text reader program that can easily read small text, but seems fails to read larger files >20MB. It seems to truncate the contents it reads.

I there a way of making sure all the data is read.

Thanks in advance.
Oct 16 '08 #1
6 8815
r035198x
13,262 8TB
If you are sure there are no inefficiencies with the code, try using the classes in the nio package. They try to speed up things by using the OS itself.
Oct 16 '08 #2
Some sample code, hope its helps;


Expand|Select|Wrap|Line Numbers
  1. int startPos = "2008-10-15 13:00:00,640 ".length();
  2. BufferedReader br = new BufferedReader(new FileReader("c:\\temp\\test-prices_bulk.log"));
  3. PrintWriter pw = new PrintWriter("c:\\temp\\test-prices_NASDAQ.log");
  4. String line = null;
  5. while ((line = br.readLine()) != null) {
  6.     int endPos = line.indexOf("|");
  7.     if(endPos == -1 || endPos < startPos){
  8.         System.out.println("| not found or occurs earlier that expected.");
  9.         continue;
  10.     }
  11.  
  12.     String contractCode = line.substring(startPos, endPos);
  13.     if(contractCode.indexOf("NASDAQ") != -1){
  14.         pw.println(line);
  15.     }
  16. }
  17. pw.close();
  18.  
Oct 16 '08 #3
Found the problem, but please promise you won't yell at me. I hadn't closed the PrintWriter.

The maximum file size a Java program can read is dependent on the maximum memory you allocate to it when its starts. ~20MB is fairly small considering the default is 64MB
Oct 16 '08 #4
r035198x
13,262 8TB
Found the problem, but please promise you won't yell at me. I hadn't closed the PrintWriter.

The maximum file size a Java program can read is dependent on the maximum memory you allocate to it when its starts. ~20MB is fairly small considering the default is 64MB
<Yelling>
But your line 17 ...
</Yelling>
Oct 16 '08 #5
@WideEyed
Surprisingly i added it on the notepad when i was assembling the sample code, there is a lot that occurrs in between and didn't want to include it.

@Aha
I guess i should just stick to the notepad.
Oct 16 '08 #6
@I_Know_these_guys_from_somewhere
This mistake reminds me of "No Problem"!!

that is the effect of copy and paste sipatha! But nice going.
Oct 17 '08 #7

Post your reply

Sign in to post your reply or Sign up for a free account.

Similar topics

4 posts views Thread by Michi | last post: by
3 posts views Thread by Chris | last post: by
1 post views Thread by bongoo | last post: by
23 posts views Thread by ShaneO | last post: by
10 posts views Thread by =?Utf-8?B?RGFu?= | last post: by
1 post views Thread by Mohamed Yousef | last post: by
1 post views Thread by CARIGAR | last post: by
reply views Thread by suresh191 | last post: by
reply views Thread by Marylou17 | last post: by
1 post views Thread by Marylou17 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.