Fastest way to read data from a lot of ASCII files
- by Alsenes
Hi guys, for a college exercise that I've already submitted I needed to read a .txt file wich contained a lot of names of images(1 in each line).
Then I needed to open each image as an ascii file, and read their data(images where in ppm format), and do a series of things with them.
The things is, I noticed my program was taking 70% of the time in the reading the data from the file part, instead of in the other calculations that I was doing (finding number of repetitions of each pixel with a hash table, finding diferents pixels beetween 2 images etc..), which I found quite odd to say the least.
This is how the ppm format looks like:
P3 //This value can be ignored when reading the file, because all image will be correctly formatted
4 4
255 //This value can be also ignored, will be always 255.
0 0 0 0 0 0 0 0 0 15 0 15
0 0 0 0 15 7 0 0 0 0 0 0
0 0 0 0 0 0 0 15 7 0 0 0
15 0 15 0 0 0 0 0 0 0 0 0
This is how I was reading the data from the files:
ifstream fdatos;
fdatos.open(argv[1]); //Open file with the name of all the images
const int size = 128;
char file[size]; //Where I'll get the image name
Image *img;
while (fdatos >> file) { //While there's still images anmes left, continue
ifstream fimagen;
fimagen.open(file); //Open image file
img = new Image(fimagen); //Create new image object with it's data file
………
//Rest of the calculations whith that image
………
delete img; //Delete image object after done
fimagen.close(); //Close image file after done
}
fdatos.close();
And inside the image object read the data like this:
const int tallafirma = 100;
char firma[tallafirma];
fich_in >> std::setw(100) >> firma; // Read the P3 part, can be ignored
int maxvalue, numpixels;
fich_in >> height >> width >> maxvalue; // Read the next three values
numpixels = height*width;
datos = new Pixel[numpixels];
int r,g,b; //Don't need to be ints, max value is 256, so an unsigned char would be ok.
for (int i=0; i<numpixels; i++) {
fich_in >> r >> g >> b;
datos[i] = Pixel( r, g ,b);
}
//This last part is the slow one,
//I thing I should be able to read all this data in one single read
//to buffer or something which would be stored in an array of unsigned chars,
//and then I'd only need to to do:
//buffer[0] -> //Pixel 1 - Red data
//buffer[1] -> //Pixel 1 - Green data
//buffer[2] -> //Pixel 1 - Blue data
So, any Ideas? I think I can improve it quite a bit reading all to an array in one single call, I just don't know how that is done.
Also, is it posible to know how many images will be in the "index file"? Is it posiible to know the number of lines a file has?(because there's one file name per line..)
Thanks!!
