我正在尝试使用 Keras 对我的文本进行情感分析,使用示例 imdb_lstm.py但我不知道如何测试它。 我将模型和权重存储到文件中,如下所示:
model = model_from_json(open('my_model_architecture.json').read())
model.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
model.load_weights('my_model_weights.h5')
results = model.evaluate(X_test, y_test, batch_size=32)
但我当然不知道 X_test
和 y_test
应该是什么样子。有人可以帮助我吗?
最佳答案
首先,将数据集拆分为测试
、有效
和训练
并进行一些预处理:
from tensorflow import keras
print('load data')
(x_train, y_train), (x_test, y_test) = keras.datasets.imdb.load_data(num_words=10000)
word_index = keras.datasets.imdb.get_word_index()
print('preprocessing...')
x_train = keras.preprocessing.sequence.pad_sequences(x_train, maxlen=256)
x_test = keras.preprocessing.sequence.pad_sequences(x_test, maxlen=256)
x_val = x_train[:10000]
y_val = y_train[:10000]
x_train = x_train[10000:]
y_train = y_train[10000:]
如您所见,我们还加载了 word_index
,因为稍后我们需要它来将句子转换为整数序列。
第二,定义您的模型:
print('build model')
model = keras.Sequential()
model.add(keras.layers.Embedding(10000, 16))
model.add(keras.layers.LSTM(100))
model.add(keras.layers.Dense(16, activation='relu'))
model.add(keras.layers.Dense(1, activation='sigmoid'))
model.compile(optimizer='adam',
loss='binary_crossentropy',
metrics=['accuracy'])
print('train model')
model.fit(x_train,
y_train,
epochs=5,
batch_size=512,
validation_data=(x_val, y_val),
verbose=1)
最后,保存
并加载
您的模型:
print('save trained model...')
model.save('sentiment_keras.h5')
del model
print('load model...')
from keras.models import load_model
model = load_model('sentiment_keras.h5')
您可以使用test-set
评估您的模型:
print('evaluation')
evaluation = model.evaluate(x_test, y_test, batch_size=512)
print('Loss:', evaluation[0], 'Accuracy:', evaluation[1])
如果您想在全新的句子上测试模型,您可以这样做:
sample = 'this is new sentence and this very bad bad sentence'
sample_label = 0
# convert input sentence to tokens based on word_index
inps = [word_index[word] for word in sample.split() if word in word_index]
# the sentence length should be the same as the input sentences
inps = keras.preprocessing.sequence.pad_sequences([inps], maxlen=256)
print('Accuracy:', model.evaluate(inps, [sample_label], batch_size=1)[1])
print('Sentiment score: {}'.format(model.predict(inps)[0][0]))
关于python - Keras 使用 LSTM 进行情感分析如何测试,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/36991552/