Replacing whitespace with sed in a CSV (to use w/ postgres copy command)
        Posted  
        
            by 
                Wells
            
        on Stack Overflow
        
        See other posts from Stack Overflow
        
            or by Wells
        
        
        
        Published on 2012-04-14T23:20:45Z
        Indexed on 
            2012/04/14
            23:30 UTC
        
        
        Read the original article
        Hit count: 245
        
I iterate through a collection of CSV files in bash, running:
iconv --from-code=ISO-8859-1 --to-code=UTF-8 ${FILE} | \
    sed -e 's/\"//g' | \
    sed -e 's/, /,/g' \
    > ${FILE}.utf8
Running iconv to fix UTF-8 characters, then the first sed call removes the double quote characters, and the final sed call is supposed to remove leading and trailing whitespace around the commas.
HOWEVER, I still have a line like this in the saved file:
FALSE,,,, 2.40,,
The COPY command in postgres is kind of dumb, so it thinks " 2.40" is not valid syntax for a numeric value.
Where am I going wrong w/ my processing of the CSV file? Thanks!
© Stack Overflow or respective owner