我正在尝试使用 word2vec来自 Python 中 gensim 自然语言处理库的模块。
文档说要初始化模型:
from gensim.models import word2vec
model = Word2Vec(sentences, size=100, window=5, min_count=5, workers=4)
gensim 希望输入的句子是什么格式？我有原始文本
"the quick brown fox jumps over the lazy dogs"
"Then a cop quizzed Mick Jagger's ex-wives briefly."
etc.
我需要向 word2fec 发送什么额外的处理？
更新:这是我尝试过的方法。当它加载句子时，我什么也得不到。
>>> sentences = ['the quick brown fox jumps over the lazy dogs',
"Then a cop quizzed Mick Jagger's ex-wives briefly."]
>>> x = word2vec.Word2Vec()
>>> x.build_vocab([s.encode('utf-8').split( ) for s in sentences])
>>> x.vocab
{}
最佳答案
A list of utf-8 sentences .您还可以从磁盘流式传输数据。
确保是utf-8，然后拆分:
sentences = [ "the quick brown fox jumps over the lazy dogs",
"Then a cop quizzed Mick Jagger's ex-wives briefly." ]
word2vec.Word2Vec([s.encode('utf-8').split() for s in sentences], size=100, window=5, min_count=5, workers=4)
关于python - 如何将句子加载到Python gensim中？，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/20362993/