What is the best way to iterate over a large result set in JDBC/iBatis 3?
- by paul_sns
We're trying to iterate over a large number of rows from the database and convert those into objects. Behavior will be as follows:
Result will be sorted by sequence id, a new object will be created when sequence id changes. The object created will be sent to an external service and will sometimes have to wait before sending another one (which means the next set of data will not be immediately used)
We already have invested code in iBatis 3 so an iBatis solution will be the best approach for us (we've tried using RowBounds but haven't seen how it does the iteration under the hood).
We'd like to balance minimizing memory usage and reducing number of DB trips.
We're also open to pure JDBC approach but we'd like the solution to work on different databases.
UPDATE:
We need to make as few calls to DB as possible (1 call would be the ideal scenario) while also preventing the application to use too much memory. Are there any other solutions out there for this type of problem may it be pure JDBC or any other technology?
Thanks and hope to hear your insights on this.
