python - 如何保存 Keras 的训练历史以进行交叉验证(循环)？

Question

For a cross validation, how to save the training history of different training set and cross validation set? I thought 'a' appending mode of pickle write will works but actually it didn't work. If possible, could you please also instruct me the way to save all the models, right now I can only save the last trained model with model.save(file).

historyfile = 'history.pickle'
f = open(historyfile,'w')
f.close()
ind = 0
save = {}
for train, test in kfold.split(input,output):
    ind = ind+1
    #create model
    model = model_FCN()
    # fit the model
    history = model.fit(input[list(train)], output[list(train)], batch_size = 16, epochs = 100, verbose =1, validation_data =(input[list(test)],output[list(test)]))
    #save to file 
    try:
        f = open(historyfile,'a') ## appending mode??
        save['cv'+ str(ind)]= history.history
        pickle.dump(save, f, pickle.HIGHEST_PROTOCOL)
        f.close()
    except Exception as e:
        print('Unable to save data to', historyfile, ':', e)

    scores = model.evaluate(MR_patch[list(test)], CT_patch[list(test)], verbose=0)
    print("%s: %.2f" % (model.metrics_names[1], scores[1]))
    cvscores.append(scores[1])
    print("cross validation stage: " + str(ind))

print("%.2f (+/- %.2f)" % (np.mean(cvscores), np.std(cvscores)))

Answer 1

要在特定训练的每个时期后保存模型并验证数据，您可以使用 Callback:

例如:

from keras.callbacks import ModelCheckpoint
import os

output_directory = '' # here should be path to output directory    
model_checkpoint = ModelCheckpoint(os.path.join(output_directory , 'weights.{epoch:02d}-{val_loss:.2f}.hdf5'))
model.fit(input[list(train)],
          output[list(train)],
          batch_size=16,
          epochs=100,
          verbose=1,
          validation_data=(input[list(test)],output[list(test)]),
          callbacks=[model_checkpoint])

在每个纪元之后，您的模型将保存在文件中。您可以在文档 (https://keras.io/callbacks/) 中找到有关此回调的更多信息

如果你想保存在每个折叠上训练的模型，你可以简单地在你的 for 循环中添加 model.save(file) :

model.fit(input[list(train)],
          output[list(train)],
          batch_size=16,
          epochs=100,
          verbose=1,
          validation_data=(input[list(test)],output[list(test)]))
model.save(os.path.join(output_directory, 'fold_{}_model.hdf5'.format(ind)))

保存历史: 您可以保存一次历史记录，而无需在每个循环中将其附加到文件中。在 for 循环之后，你应该得到带有键(你的折叠标记)和值(每个折叠的历史记录)的字典，并像这样保存这个字典:

f = open(historyfile, 'wb')
pickle.dump(save, f, pickle.HIGHEST_PROTOCOL)
f.close()