I would like to import this dataset into a df. I'm trying to convert this SAS support file to R code using read.fwf approach
Define variables name and lenght as described in the SAS file
vars <- c('SEQN', 'HYK1A', 'HYK1B', 'HYK2A', 'HYK2B', 'HYK3CG', 'HYK3DG', 'HYK6SG', 'HYK8SG', 'HYK3CH', 'HYK3DH', 'HYK6SH', 'HYK8SH', 'HYK3CI', 'HYK3DI', 'HYK6SI', 'HYK8SI', 'HYK3CJ', 'HYK3DJ', 'HYK6SJ', 'HYK8SJ', 'HYK3CK', 'HYK3DK', 'HYK6SK', 'HYK8SK', 'HYK3CL', 'HYK3DL', 'HYK6SL', 'HYK8SL', 'HYK3CM', 'HYK3DM', 'HYK6SM', 'HYK8SM', 'HYK3CN', 'HYK3DN', 'HYK6SN', 'HYK8SN', 'HYK3CO', 'HYK3DO', 'HYK6SO', 'HYK8SO', 'HYK3CP', 'HYK3DP', 'HYK6SP', 'HYK8SP', 'HYK9DG', 'HYK9EG', 'HYK9FG', 'HYK11AG', 'HYK12SG', 'HYK9DH', 'HYK9EH', 'HYK9FH', 'HYK11AH', 'HYK12SH', 'HYK9DI', 'HYK9EI', 'HYK9FI', 'HYK11AI', 'HYK12SI', 'HYK9DJ', 'HYK9EJ', 'HYK9FJ', 'HYK11AJ', 'HYK12SJ', 'HYK9DK', 'HYK9EK', 'HYK9FK', 'HYK11AK', 'HYK12SK', 'HYK9DL', 'HYK9EL', 'HYK9FL', 'HYK11AL', 'HYK12SL', 'HYK9DM', 'HYK9EM', 'HYK9FM', 'HYK11AM', 'HYK12SM', 'HYK9DN', 'HYK9EN', 'HYK9FN', 'HYK11AN', 'HYK12SN', 'HYK9DO', 'HYK9EO', 'HYK9FO', 'HYK11AO', 'HYK12SO')
len <-c(7, 3, 3, 3, 3, 3, 3, 4, 4, 3, 3, 4, 4, 3, 3, 4, 4, 3, 3, 4, 4, 3, 3, 4, 4, 3, 3, 4, 4, 3, 3, 4, 4, 3, 3, 4, 4, 3, 3, 4, 4, 3, 3, 4, 4, 4, 4, 4, 6, 4, 4, 4, 4, 6, 4, 4, 4, 4, 6, 4, 4, 4, 4, 6, 4, 4, 4, 4, 6, 4, 4, 4, 4, 6, 4, 4, 4, 4, 6, 4, 4, 4, 4, 6, 4, 4, 4, 4, 6, 4)
retrieve DF from the web
df <- read.fwf(".../you.dat",
widths = len, header = FALSE, n=10, stringsAsFactors = TRUE)
names(df) <- vars
Visualize the DF
head(df)
Honestly, I don't trust this DF. I'm getting too many NAs
Update after @42- illuminating answer. Faster way
I improved my code easily using SAScii library and it works.
However, I found something faster with lower system expenses here.
library(readr)
library(data.table)
#Parse SAS file
vars <- parse.SAScii(".../you.sas")
setDT(vars) #convert to data.table
#read to data frame
huge.df <- read_fwf(".../you.dat",
fwf_widths(dput(vars[,width]),
col_names=(dput(vars[,varname]))),
progress = interactive()
)
A few years ago I also tried doing that and eventually cobbled together a method, but in the meantime, @AnthonyDamico wrote the
SAScii-package and all these efforts are now unnecessary. Thank you, Anthony, for all your excellent efforts at making public use data available to R users. I suspect you've saved the Kaiser Foundation hundreds of thousands of dollars over the course of your employment and consulting career.I'm afraid there are quite a few NA's and appears your code was mostly successful.
I only see these as having possibly invalid data and they appear to have not been loaded correctly:
These appear to be ICD-9-CM codes that should have been read into R with character format, but for which there were not the proper
$entries in the INPUT statements in the CDC SAS code. You can get a full set of diagnostics with the "read.SAScii"-function:You can check the counts of non-NA values against the values reported in the codebook at: https://wwwn.cdc.gov/nchs/data/nhanes3/1a/YOUTH-acc.pdf
This is a partial screenshot of page 210 of that codebook and shows that there should be all NA's for the three "HYK__" variables on that page:
Note added for amusement. This is actually now a counter-example of sorts to this R fortune. There are quite a few items in that package referencing SAS (often in a less than totally favorable light) and it took me 7 or 8 tries before the one I was looking for was brought up: