python - 使用 Mallet Perplexity 进行 Gensim 主题建模

Question

我正在对哈佛图书馆的书名和主题进行主题建模。

我使用 Gensim Mallet Wrapper 与 Mallet 的 LDA 进行建模。 当我尝试获取 Coherence 和 Perplexity 值以查看模型有多好时，困惑度无法计算，但出现以下异常。 如果我使用 Gensim 的内置 LDA 模型而不是 Mallet，我不会收到相同的错误。 我的语料库包含 700 万多个文档，长度最多为 50 个单词，平均 20 个单词。因此文档很短。

以下是我的代码的相关部分:

# TOPIC MODELING

from gensim.models import CoherenceModel
num_topics = 50

# Build Gensim's LDA model
lda_model = gensim.models.ldamodel.LdaModel(corpus=corpus,
                                       id2word=id2word,
                                       num_topics=num_topics,
                                       random_state=100,
                                       update_every=1,
                                       chunksize=100,
                                       passes=10,
                                       alpha='auto',
                                       per_word_topics=True)

# Compute Perplexity
print('\nPerplexity: ', lda_model.log_perplexity(corpus))  
# a measure of how good the model is. lower the better.

Perplexity: -47.91929228302663

# Compute Coherence Score
coherence_model_lda = CoherenceModel(model=lda_model, 
texts=data_words_trigrams, dictionary=id2word, coherence='c_v')
coherence_lda = coherence_model_lda.get_coherence()
print('\nCoherence Score: ', coherence_lda)

Coherence Score: 0.28852857563541856

LDA给出的分数没有问题。现在我用 MALLET 对相同的词袋进行建模

# Building LDA Mallet Model
mallet_path = '~/mallet-2.0.8/bin/mallet' # update this path
ldamallet = gensim.models.wrappers.LdaMallet(mallet_path, 
corpus=corpus, num_topics=num_topics, id2word=id2word)

# Convert mallet to gensim type
mallet_model = 
gensim.models.wrappers.ldamallet.malletmodel2ldamodel(ldamallet)

# Compute Coherence Score
coherence_model_ldamallet = CoherenceModel(model=mallet_model, 
texts=data_words_trigrams, dictionary=id2word, coherence='c_v')
coherence_ldamallet = coherence_model_ldamallet.get_coherence()
print('\nCoherence Score: ', coherence_ldamallet)

Coherence Score: 0.5994123896865993

然后我询问 Perplexity 值并得到以下警告和 NaN 值。

# Compute Perplexity
print('\nPerplexity: ', mallet_model.log_perplexity(corpus))

/app/app-py3/lib/python3.5/site-packages/gensim/models/ldamodel.py:1108: RuntimeWarning: invalid value encountered in multiply score += np.sum((self.eta - _lambda) * Elogbeta)

Perplexity: nan

/app/app-py3/lib/python3.5/site-packages/gensim/models/ldamodel.py:1109: RuntimeWarning: invalid value encountered in subtract score += np.sum(gammaln(_lambda) - gammaln(self.eta))

我意识到这是一个非常 Gensim 特定的问题，需要对此功能有更深入的了解: gensim.models.wrappers.ldamallet.malletmodel2ldamodel(ldamallet)

因此，如果您对警告和 Gensim 域有任何评论，我将不胜感激。

Answer 1

我不认为 Mallet 包装器实现了困惑函数。正如 Radims answer 中提到的，困惑度显示到标准输出:

AFAIR, Mallet displays the perplexity to stdout -- would that be enough for you? Capturing these values programmatically should be possible too, but I haven't looked into that. Hopefully Mallet has some API call for perplexity eval too, but it's certainly not included in the wrapper.

我刚刚在样本语料库上运行了它，并且 LL/token 确实在每次迭代后都会被打印出来:

LL/代币:-9.45493

困惑度 = 2^(-LL/token) = 701.81