Fastest way to convert file from latin1 to utf-8 in python.
- by xsaero00
I need fastest way to convert files from latin1 to utf-8 in python. The files are large ~ 2G. ( I am moving DB data ). So far I have
import codecs
infile = codecs.open(tmpfile, 'r', encoding='latin1')
outfile = codecs.open(tmpfile1, 'w', encoding='utf-8')
for line in infile:
outfile.write(line)
infile.close()
outfile.close()
but it is still slow. The conversion takes one fourth of the whole migration time.
I could also use a linux command line utility if it is faster than native python code.
