我的文件如下:
doc1 = very good, very bad, you are great
doc2 = very bad, good restaurent, nice place to visit
我想让我的语料库与 ,
分开这样我的决赛DocumentTermMatrix
变成:
terms
docs very good very bad you are great good restaurent nice place to visit
doc1 tf-idf tf-idf tf-idf 0 0
doc2 0 tf-idf 0 tf-idf tf-idf
我知道,如何计算DocumentTermMatrix
单个单词但不知道如何制作语料库 separated for each phrase
在 R. 中的解决方案 R
是首选,但解决方案在 Python
也很受欢迎。
我试过的是:
> library(tm)
> library(RWeka)
> BigramTokenizer <- function(x) NGramTokenizer(x, Weka_control(min = 1, max = 3))
> options(mc.cores=1)
> texts <- c("very good, very bad, you are great","very bad, good restaurent, nice place to visit")
> corpus <- Corpus(VectorSource(texts))
> a <- TermDocumentMatrix(corpus, control = list(tokenize = BigramTokenizer))
> as.matrix(a)
我得到:
Docs
Terms 1 2
bad good restaurent 0 1
bad you are 1 0
good restaurent nice 0 1
good very bad 1 0
nice place to 0 1
place to visit 0 1
restaurent nice place 0 1
very bad good 0 1
very bad you 1 0
very good very 1 0
you are great 1 0
我想要的不是单词的组合,而是我在矩阵中显示的短语。
最佳答案
这是使用 qdap
+ tm
包的一种方法:
library(qdap); library(tm); library(qdapTools)
dat <- list2df(list(doc1 = "very good, very bad, you are great",
doc2 = "very bad, good restaurent, nice place to visit"), "text", "docs")
x <- sub_holder(", ", dat$text)
m <- dtm(wfm(x$unhold(gsub(" ", "~~", x$output)), dat$docs) )
weightTfIdf(m)
inspect(weightTfIdf(m))
## A document-term matrix (2 documents, 5 terms)
##
## Non-/sparse entries: 4/6
## Sparsity : 60%
## Maximal term length: 19
## Weighting : term frequency - inverse document frequency (normalized) (tf-idf)
##
## Terms
## Docs good restaurent nice place to visit very bad very good you are great
## doc1 0.0000000 0.0000000 0 0.3333333 0.3333333
## doc2 0.3333333 0.3333333 0 0.0000000 0.0000000
您也可以一举完成并返回一个 DocumentTermMatrix
但这可能更难理解:
x <- sub_holder(", ", dat$text)
apply_as_tm(t(wfm(x$unhold(gsub(" ", "~~", x$output)), dat$docs)),
weightTfIdf, to.qdap=FALSE)
关于r - 用短语构建语料库,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/24038498/