什么会导致损失model.get_latest_training_loss()
每个时期增加?
用于训练的代码:
class EpochSaver(CallbackAny2Vec):
'''Callback to save model after each epoch and show training parameters '''
def __init__(self, savedir):
self.savedir = savedir
self.epoch = 0
os.makedirs(self.savedir, exist_ok=True)
def on_epoch_end(self, model):
savepath = os.path.join(self.savedir, "model_neg{}_epoch.gz".format(self.epoch))
model.save(savepath)
print(
"Epoch saved: {}".format(self.epoch + 1),
"Start next epoch ... ", sep="\n"
)
if os.path.isfile(os.path.join(self.savedir, "model_neg{}_epoch.gz".format(self.epoch - 1))):
print("Previous model deleted ")
os.remove(os.path.join(self.savedir, "model_neg{}_epoch.gz".format(self.epoch - 1)))
self.epoch += 1
print("Model loss:", model.get_latest_training_loss())
def train():
### Initialize model ###
print("Start training Word2Vec model")
workers = multiprocessing.cpu_count()/2
model = Word2Vec(
DocIter(),
size=300, alpha=0.03, min_alpha=0.00025, iter=20,
min_count=10, hs=0, negative=10, workers=workers,
window=10, callbacks=[EpochSaver("./checkpoints")],
compute_loss=True
)
输出:
时代(1 到 20)的损失:
Model loss: 745896.8125
Model loss: 1403872.0
Model loss: 2022238.875
Model loss: 2552509.0
Model loss: 3065454.0
Model loss: 3549122.0
Model loss: 4096209.75
Model loss: 4615430.0
Model loss: 5103492.5
Model loss: 5570137.5
Model loss: 5955891.0
Model loss: 6395258.0
Model loss: 6845765.0
Model loss: 7260698.5
Model loss: 7712688.0
Model loss: 8144109.5
Model loss: 8542560.0
Model loss: 8903244.0
Model loss: 9280568.0
Model loss: 9676936.0
我究竟做错了什么?
语言阿拉伯语。
作为来自 DocIter 的输入 - 带有 token 的列表。
最佳答案
在 gensim 3.6.0 之前,报告的损失值可能不是很合理,只会将每次调用的计数重置为 train()
,而不是每个内部纪元。此问题即将进行一些修复:
https://github.com/RaRe-Technologies/gensim/pull/2135
同时,先前值与最新值之间的差异可能更有意义。在这种情况下,您的数据表明第一个时期的总损失为 745896,而最后一个时期的总损失为 (9676936-9280568=) 396,368——这可能表明所希望的进展类型。
关于python - 训练期间损失不会减少(Word2Vec,Gensim),我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/52038651/