fread
{data.table} 难以处理带有部分引用的行的制表符分隔文件。我找不到解决方法,因为它会自动处理引号(因此没有 quote
参数,就像 read.csv
一样)。这说明:
str1 = 'L1\tsome\tunquoted\tstuff\nL2\tsome\t"half" quoted\tstuff\nL3\tthis\t"should work"\tok thought'
str2 = gsub('"', '', str1)
fread(str2, sep='\t', header=F, skip=0L)
# V1 V2 V3 V4
# 1: L1 some unquoted stuff
# 2: L2 some half quoted stuff
# 3: L3 this should work ok thought
fread(str1, sep='\t', header=F, skip=0L)
# Error in fread(str1, sep = "\t", header = F, skip = 0L) :
# Expected sep (' ') but '
# ' ends field 3 on line 1 when detecting types: L2 some "half" quoted stuff
有没有办法解决这个问题,除了在原始文件上进行查找/替换?
最佳答案
怎么样stringi
反而?很容易弄清楚,而且非常有效。还有一个功能stri_read_lines()
, 用于从文件中读取/拆分行。
library(stringi)
as.data.frame(stri_split_fixed(stri_split_lines1(str1), "\t", simplify = TRUE))
# V1 V2 V3 V4
# 1 L1 some unquoted stuff
# 2 L2 some "half" quoted stuff
# 3 L3 this "should work" ok thought
如果您需要说服这是一种比
read.*()
更有效的方法,看看将上述方法应用于解析为 30k 行的扁平字符串时的时序。您还可以通过调整 as.data.frame()
中的参数来进一步加快速度。 .对于此示例,stringi
方法的速度大约是 read.table()
的两倍.str1 <- "L1\tsome\tunquoted\tstuff\nL2\tsome\t\"half\" quoted\tstuff\nL3\tthis\t\"should work\"\tok thought"
library(stringi)
library(microbenchmark)
write(stri_flatten(rep(str1, 1e5), collapse = "\n"))
file.info("data")[1]
# size
# data 8400000
microbenchmark(
stringi = {
mat <- stri_split_fixed(stri_read_lines("data"), "\t", simplify = TRUE)
out <- as.data.frame(mat)
},
read.table = {
out2 <- read.table("data", sep = "\t", quote = "\n")
},
times = 3L,
unit = "relative"
)
# Unit: relative
# expr min lq mean median uq max neval cld
# stringi 1.000000 1.000000 1.000000 1.000000 1.00000 1.000000 3 a
# read.table 2.074071 2.111722 1.997857 2.148897 1.96356 1.808365 3 b
identical(out, out2)
# [1] TRUE
关于r - fread 读取错误 "Expected sep (' ') but..",我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/28970082/