Streaming large result sets with MySQL

Posted by configurator on Stack Overflow See other posts from Stack Overflow or by configurator
Published on 2010-03-15T13:16:19Z Indexed on 2010/03/15 13:19 UTC
Read the original article Hit count: 166

Filed under:
|
|
|

I'm developing a spring application that uses large MySQL tables. When loading large tables, I get an OutOfMemoryException, since the driver tries to load the entire table into application memory.

I tried using

statement.setFetchSize(Integer.MIN_VALUE);

but then every ResultSet I open hangs on close(); looking online I found that that happens because it tries loading any unread rows before closing the ResultSet, but that is not the case since I do this:

ResultSet existingRecords = getTableData(tablename);
try {
    while (existingRecords.next()) {
        // ...
    }
} finally {
    existingRecords.close(); // this line is hanging, and there was no exception in the try clause
}

The hangs happen for small tables (3 rows) as well, and if I don't close the RecordSet (which happened in one method) then connection.close() hangs.

© Stack Overflow or respective owner

Related posts about mysql

Related posts about streaming