executorservice to read data from database in chuncks and run process on them
- by TazMan
I'm trying to write a process that would read data from a database and upload it onto a cloud datastore.
How can I decide the partition strategy of the data? I want to query the table in chunks and process each chunk in 10 threads. Each thread basically will send the data to an individual node on a 10 node cluster on the cloud..
Where in the below multi threading code will the dataquery to extract and send 10 concurrent requests for uploading data to cloud would be?
public class Caller {
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(10);
for (int i = 0; i < 10; i++) {
Runnable worker = new DomainCDCProcessor(i);
executor.execute(worker);
}
executor.shutdown();
while (!executor.isTerminated()) {
}
System.out.println("Finished all threads");
}
}
