Is there a way to circumvent Python list.append() becoming progressively slower in a loop as the lis
- by Deniz
I have a big file I'm reading from, and convert every few lines to an instance of an Object.
Since I'm looping through the file, I stash the instance to a list using list.append(instance), and then continue looping.
This is a file that's around ~100MB so it isn't too large, but as the list grows larger, the looping slows down progressively. (I print the time for each lap in the loop).
This is not intrinsic to the loop ~ when I print every new instance as I loop through the file, the program progresses at constant speed ~ it is only when I append them to a list it gets slow.
My friend suggested disabling garbage collection before the while loop and enabling it afterward & making a garbage collection call.
Did anyone else observe a similar problem with list.append getting slower? Is there any other way to circumvent this?
