Streaming large result sets with MySQL
- by configurator
I'm developing a spring application that uses large MySQL tables. When loading large tables, I get an OutOfMemoryException, since the driver tries to load the entire table into application memory.
I tried using
statement.setFetchSize(Integer.MIN_VALUE);
but then every ResultSet I open hangs on close(); looking online I found that that happens because it tries loading any unread rows before closing the ResultSet, but that is not the case since I do this:
ResultSet existingRecords = getTableData(tablename);
try {
while (existingRecords.next()) {
// ...
}
} finally {
existingRecords.close(); // this line is hanging, and there was no exception in the try clause
}
The hangs happen for small tables (3 rows) as well, and if I don't close the RecordSet (which happened in one method) then connection.close() hangs.
