doc2vec的Python简单实现？

Question

我正在尝试从 gensim 实现 doc2vec，但遇到一些错误，并且网络上没有足够的文档或帮助。 这是我的工作代码的一部分:

from gensim.models import Doc2Vec
from gensim.models.doc2vec import LabeledSentence

class LabeledLineSentence(object):
    def __init__(self, filename):
        self.filename = filename
    def __iter__(self):
        with open(self.filename, 'r') as f:
            for uid, line in enumerate(f):
                print LabeledSentence(line.split(), tags=['TXT_%s' % uid])
                yield LabeledSentence(words=line.split(), tags=['TXT_%s' % uid])

sentences = LabeledLineSentence('myfile.txt')

我的 txt 文件是什么样的:

  1 hi how are you
  2 hi how are you
  3 hi how are you
  4 its such a great day
  5 its such a great day
  6 its such a great day
  7 i like dogs
  8 i like cats
  9 i like snakes
 10 the ice cream was yummy
 11 the cake was awesome  

初始化模型

model = Doc2Vec(alpha=0.025, min_alpha=0.025, size=50, window=5, min_count=5,
                dm=1, workers=8, sample=1e-5)       

打印输出示例:

LabeledSentence(['hi', 'how', 'are', 'you'], ['TXT_0'])
LabeledSentence(['hi', 'how', 'are', 'you'], ['TXT_1'])
LabeledSentence(['hi', 'how', 'are', 'you'], ['TXT_2'])
LabeledSentence(['its', 'such', 'a', 'great', 'day'], ['TXT_3'])
LabeledSentence(['its', 'such', 'a', 'great', 'day'], ['TXT_4'])

这是错误所在:

for epoch in range(500):
    try:
        print 'epoch %d' % (epoch)
        model.train(sentences)
        model.alpha *= 0.99
        model.min_alpha = model.alpha
    except (KeyboardInterrupt, SystemExit):
        break

RuntimeError: you must first build vocabulary before training the model

知道为什么吗？

Answer 1

调用 model.build_vocab 将修复该错误。

请参阅本教程 https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/doc2vec-lee.ipynb