php parsing csv with ftell
- by Robert82
I have a 500mb csv file with over 500,000 lines, each with 80 fields. I am using fget to process the file line by line.
$col1 = array();
while (($row = fgetcsv($handle, 1000, ",")) !== FALSE) {
$col1[] = $row[0];
}
Because of an execution time limit on the PHP file by my hosting provider (120 seconds), I can't process the whole file in one run.
I tried using ftell() and fseek() to remember the last position for restart. The trouble is, sometimes the ftell() position is in the middle of a row, and resuming means missing the first half of the row.
Is there an elegant way to know the last line successfully processed, and resume from the one after it? I realize I can do a simple counter, and then loop through to that point again, but that would produce diminishing returns on the rows I can process towards the end of the file.
Is there something like ftell() and fseek() that would work in my case? Or a way to limit ftell() to return the pointer for the end of the previous line?
