ResultSet limit

[pvinod]

Hi,

This is vinod.

i have tried to read a 10 lacks records with column count 14 from the database at a single time. But my application shows OUT OF MEMORY error.

Any help?

my id : <id removed per posting guidelines>

Thanks,
Vinod

[JosAH]

Hi,

This is vinod.

i have tried to read a 10 lacks records with column count 14 from the database at a single time. But my application shows OUT OF MEMORY error.

Any help?

my id : <id removed as per posting guidelines>

Thanks,
Vinod

How much is 'a 10 lacks'?

kind regards,

Jos

[pvinod]

10,00,000


Thanks,
vinod

[JosAH]

10,00,000


Thanks,
vinod

So that's one million records with 14 columns each? It depends a bit on the length
of the columns but I can see that you get an OutOfMemoryError ... there are a
few -X flags that increase the JVM's heap size but I think that's just postponing
the execution. What do you want to load that many records in memory at once?

kind regards,

Jos

[pvinod]

So that's one million records with 14 columns each? It depends a bit on the length
of the columns but I can see that you get an OutOfMemoryError ... there are a
few -X flags that increase the JVM's heap size but I think that's just postponing
the execution. What do you want to load that many records in memory at once?

kind regards,

Jos

Thanks for ur response Jos.

I need to import data from one database to another database. Please give me some way to avoid this Outof Memory error.

Regards,
Vinod

[JosAH]

Thanks for ur response Jos.

I need to import data from one database to another database. Please give me some way to avoid this Outof Memory error.

Regards,
Vinod

You might try to specify the Statement.setFetchSize() on your Statement

kind regards,

Jos

[pvinod]

You might try to specify the Statement.setFetchSize() on your Statement

kind regards,

Jos

I have tried this setFetchSize() method. But I got error message(Invalid fetch size).


Thanks,
Vinod

[JosAH]

I have tried this setFetchSize() method. But I got error message(Invalid fetch size).

Thanks,
Vinod

So what happens if you experiment a bit with different fetch sizes?

kind regards,

Jos

[pvinod]

So what happens if you experiment a bit with different fetch sizes?

kind regards,

Jos

Hi,
I got the following error.

rs.setFetchSize(50);

java.sql.SQLException: Invalid Fetch Size


Thanks,
Vinod

[JosAH]

Hi,
I got the following error.

rs.setFetchSize(50);

java.sql.SQLException: Invalid Fetch Size


Thanks,
Vinod

Can you show us a bit of relevant code please? What database are you using?

kind regards,

Jos

[praveen2gupta]

Hi
The ResultSet is having a limit of 1000 results. You can not access unlimited number of records from a singlr sql query. So you are getting errors

[JosAH]

The ResultSet is having a limit of 1000 results.

That all depends on the database vendor and its JDBC driver.

kind regards,

Jos

[pvinod]

That all depends on the database vendor and its JDBC driver.

kind regards,

Jos

Hi,

I have tried to read 3,00,000 records. I was able to do this.

Please advise. Is any other way to perform this.

Thanks,
Vinod

[JosAH]

Hi,

I have tried to read 3,00,000 records. I was able to do this.

Please advise. Is any other way to perform this.

Thanks,
Vinod

I don't know if you followed an other identical thread but my observations are:

1) users are not willing to scroll through so much data. They want it in reasonable
chunks through which they can navigate.
2) you can't collect that much database because of memory constraints.

So my tip is to 'page' through all the data. Depending on your database and its
SQL dialect you might check the 'offset/limit' statement parts or the 'numrows'
statement part or whatever your database supports.

A few buttons for navigational purposes and just a bit of data in the Session
object can do the rest. Don't try to retrieve and show all the data in one piece.

kind regards,

Jos

[pvinod]

I don't know if you followed an other identical thread but my observations are:

1) users are not willing to scroll through so much data. They want it in reasonable
chunks through which they can navigate.
2) you can't collect that much database because of memory constraints.

So my tip is to 'page' through all the data. Depending on your database and its
SQL dialect you might check the 'offset/limit' statement parts or the 'numrows'
statement part or whatever your database supports.

A few buttons for navigational purposes and just a bit of data in the Session
object can do the rest. Don't try to retrieve and show all the data in one piece.

kind regards,

Jos

thanks Jos. Let me try

vinod

[vineetsharma]

Please use this this will help you in storing the bulk data without memory dependency



Statement stmt = con.createStatement(java.sql.ResultSet.TYPE_FORWAR D_ONLY,java.sql.ResultSet.CONCUR_READ_ONLY);
stmt.setFetchSize(Integer.MIN_VALUE);