Optimize CUDA with Thrust in a loop
- by macs
Given the following piece of code, generating a kind of code dictionary with CUDA using thrust (C++ template library for CUDA):
thrust::device_vector<float> dCodes(codes->begin(), codes->end());
thrust::device_vector<int> dCounts(counts->begin(), counts->end());
thrust::device_vector<int> newCounts(counts->size());
for (int i = 0; i < dCodes.size(); i++) {
float code = dCodes[i];
int count = thrust::count(dCodes.begin(), dCodes.end(), code);
newCounts[i] = dCounts[i] + count;
//Had we already a count in one of the last runs?
if (dCounts[i] > 0) {
newCounts[i]--;
}
//Remove
thrust::detail::normal_iterator<thrust::device_ptr<float> > newEnd = thrust::remove(dCodes.begin()+i+1, dCodes.end(), code);
int dist = thrust::distance(dCodes.begin(), newEnd);
dCodes.resize(dist);
newCounts.resize(dist);
}
codes->resize(dCodes.size());
counts->resize(newCounts.size());
thrust::copy(dCodes.begin(), dCodes.end(), codes->begin());
thrust::copy(newCounts.begin(), newCounts.end(), counts->begin());
The problem is, that i've noticed multiple copies of 4 bytes, by using CUDA visual profiler. IMO this is generated by
The loop counter i
float code, int count and dist
Every access to i and the variables noted above
This seems to slow down everything (sequential copying of 4 bytes is no fun...).
So, how i'm telling thrust, that these variables shall be handled on the device? Or are they already?
Using thrust::device_ptr seems not sufficient for me, because i'm not sure whether the for loop around runs on host or on device (which could also be another reason for the slowliness).
