How to Import a CSV file containing multiple sections into R?

Posted by PaulHurleyuk on Stack Overflow See other posts from Stack Overflow or by PaulHurleyuk
Published on 2010-04-13T10:14:35Z Indexed on 2010/04/13 12:02 UTC
Read the original article Hit count: 289

Filed under:

import

I want to import the contents of a csv file into R, the csv file contains multiple sections of data vertically, seperated by blank lines and asterisks. For example

********************************************************
* SAMPLE DATA ******************************************
********************************************************
Name, DOB, Sex
Rod, 1/1/1970, M
Jane, 5/7/1980, F
Freddy, 9.12,1965, M

*******************************************************
*  Income Data ****************************************
*******************************************************
Name, Income
Rod, 10000
Jane, 15000
Freddy, 7500

I would like to import this into R as two seperate dataframes. Currently I'm manually cutting the csv file up into smaller files, but I think I could do it using read.csv and the skip and nrows settings of read.csv, If I could work out where the secion breaks are.

This gives me a logical TRUE for every blank line

ifelse(readLines("DATA.csv")=="",TRUE,FALSE)

I'm hoping someone has already solved this problem.

Related posts about csv

DBD::CSV: How can I generate different behavior with the two f_ext-options ".csv" and ".csv/r"?

as seen on Stack Overflow - Search for 'Stack Overflow'
This is from the DBD::File-documentation: f_ext This attribute is used for setting the file extension where (CSV) files are opened. There are several possibilities. DBI:CSV:f_dir=data;f_ext=.csv In this case, DBD::File will open only table.csv if both table.csv and table exist… >>> More
SUPER CSV write bean to CSV.

as seen on Stack Overflow - Search for 'Stack Overflow'
Here is my class, public class FreebasePeopleResults { public String intendedSearch; public String weight; public Double heightMeters; public Integer age; public String type; public String parents; public String profession; public String alias; public String children; public String siblings; public… >>> More
export to csv using fastercsv and CSV::Writer (Ruby on Rails)

as seen on Stack Overflow - Search for 'Stack Overflow'
What am I trying to do: export data to csv. I have a form which allows user to select the format (from a drop down menu). So based on the selection of the format the ouput is displayed using a ajax call. Works fine for html but when I select the format as csv I don't see any pop up on the screen… >>> More
How can i make changes to this file Encoding?

as seen on Super User - Search for 'Super User'
I have these 3 files 21/08/2014 07:15 PM 122 Tw2AWK.csv 21/08/2014 07:15 PM 125 Tw2Notepad.csv 21/08/2014 07:15 PM 119 Tw2REPL.csv C:\myfilesfile Tw2AWK.csv TwREPL.csv Tw2Notepad.csv Tw2AWK.csv; UTF-8 Unicode text, with CRLF line terminators Tw2REPL… >>> More
Unix sort keys cause performance problems

as seen on Super User - Search for 'Super User'
My data: It's a 71 MB file with 1.5 million rows. It has 6 fields All six fields combine to form a unique key - so that's what I need to sort on. Sort statement: sort -t ',' -k1,1 -k2,2 -k3,3 -k4,4 -k5,5 -k6,6 -o output.csv input.csv The problem: If I sort without keys, it takes 30 seconds… >>> More

Developer IT