derby - java.lang.OutOfMemoryError Java heap space

[jamborta]

im trying to iterate through a large database (over 5000 entries), and printing out the entries
using this code:

String sql = "SELECT "+column+ " ,clickurl from " + tableName;
stmt = conn.prepareStatement(sql);
ResultSet results=stmt.executeQuery();
 
while(results.next()){
 
String clickurl = results.getString(2);
System.out.println(" Test - " + clickurl);
}
 

but after the 4000. entry java runs out of memory.

i dont know if i understand the system correctly, but the results are stored in the a ResultSet variable already, and iterating through it shouldn't require more memory.

i can't figure out what the problem is.

[Atli]

Hi.

Seeing as this is more of a Java problem than a MySQL problem, I've moved it over to the Java Forum.

[Nepomuk]

im trying to iterate through a large database (over 5000 entries), and printing out the entries
using this code:

String sql = "SELECT "+column+ " ,clickurl from " + tableName;
stmt = conn.prepareStatement(sql);
ResultSet results=stmt.executeQuery();
 
while(results.next()){
 
String clickurl = results.getString(2);
System.out.println(" Test - " + clickurl);
}
 

but after the 4000. entry java runs out of memory.

It will probably help, to put

String clickurl = "";

before the loop and just

clickurl = results.getString(2);

in it. That way, you don't create a new string every time the loop is called, instead it's overwritten. Otherwise, the garbage collector would have to clean it up and it doesn't seem to be doing that in time.

i dont know if i understand the system correctly, but the results are stored in the a ResultSet variable already, and iterating through it shouldn't require more memory.

I think you're wrong with that. As I understand it, you're creating a new copy of a string every time the loop is called. It's like

public class CompareStrings {
    public static void main(String[] args) {
        StringWrapper sw = new StringWrapper();
        String hi = sw.getWord();
        System.out.println(hi == sw.hello);
    }
}
 
class StringWrapper {
    public String hello = "Hello";
    public String getWord() {return new String(hello);}
}

Check it, it will return false. That's why your program runs out of memory.

Of course, you could just do this:

while(results.next()){
 
System.out.println(" Test - " + results.getString(2));
}

That won't work of course, if you need the String clickurl somewhere else.

Greetings,
Nepomuk

[JosAH]

It will probably help, to put

String clickurl = "";

before the loop and just

clickurl = results.getString(2);

in it. That way, you don't create a new string every time the loop is called, instead it's overwritten. Otherwise, the garbage collector would have to clean it up and it doesn't seem to be doing that in time.

The gc always runs when it is desparately needed; one local String doesn't
increase the consumed memory much; the gc will clean it up. I use Derby
myself and just conducted a small test: 100,000 rows without any problems
so I guess the problem is somewhere else ...

kind regards,

Jos

[JosAH]

ps. probably your cache size is too small; e.g. set this property:

derby.storage.pageCacheSize    =    4M
 

kind regards,

Jos

[jamborta]

hi,
thanks for the suggestions. i tried to run it without creating a string, but it still runs out of memory, although slightly later.
where can i set derby.storage.pageCacheSize?

thanks

[JosAH]

where can i set derby.storage.pageCacheSize?

Put it in your System.properties before you start Derby or use a command line
parameter -Dderby.storage.pageCacheSize=4M

kind regards,

Jos

[jamborta]

Hi Jos,

thanks for the reply, I added these lines (-J-Dderby.storage.pageSize=8m -J-Dderby.storage.pageCacheSize=8m) to netbeans.conf, it shows up as an argument, but doesnt seem to effect the application. it still runs out of memory, i was trying to play with the numbers, but nothing happened.

(i have 4gb memory assigned 512/512 heap/non heap memory, that should be enough i guess)

[JosAH]

thanks for the reply, I added these lines (-J-Dderby.storage.pageSize=8m -J-Dderby.storage.pageCacheSize=8m) to netbeans.conf, it shows up as an argument, but doesnt seem to effect the application. it still runs out of memory, i was trying to play with the numbers, but nothing happened.

I suspect the bug to be somewhere else; as I wrote, I did a little test with 100,000
rows and it ran fine, i.e. insert them and select them all. Would it be possible to
show a bit of (relevant) code? Or, alternatively, use the Runtime.freeMemory()
method in the relevant portions of your code and see what happens; also make
sure that you close Derby's resources (result sets, statements etc.)

kind regards,

Jos