python - 如何在 Tensorflow 中为预取数据集绘制混淆矩阵

Question

我试图用以下代码为我的图像分类器绘制一个混淆矩阵，但我收到一条错误消息:“PrefetchDataset”对象没有属性“类”

Y_pred = model.predict(validation_dataset)
y_pred = np.argmax(Y_pred, axis=1)

print('Confusion Matrix')
print(confusion_matrix(validation_dataset.classes, y_pred)) # ERROR message generated

Answer 1

免责声明:这不适用于混洗数据集。我会尽快更新这个答案。
您可以使用 tf.stack连接所有数据集值。像这样:

true_categories = tf.concat([y for x, y in test_dataset], axis=0)

为了重现性，假设您有一个数据集、一个神经网络和一个训练循环:

import tensorflow_datasets as tfds
import tensorflow as tf
from sklearn.metrics import confusion_matrix

data, info = tfds.load('iris', split='train',
                       as_supervised=True,
                       shuffle_files=True,
                       with_info=True)

AUTOTUNE = tf.data.experimental.AUTOTUNE

train_dataset = data.take(120).batch(4).prefetch(buffer_size=AUTOTUNE)
test_dataset = data.skip(120).take(30).batch(4).prefetch(buffer_size=AUTOTUNE)

model = tf.keras.Sequential([
    tf.keras.layers.Dense(8, activation='relu'),
    tf.keras.layers.Dense(16, activation='relu'),
    tf.keras.layers.Dense(info.features['label'].num_classes, activation='softmax')
    ])

model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', 
              metrics='accuracy')

history = model.fit(train_dataset, validation_data=test_dataset, epochs=50, verbose=0)

现在您的模型已经拟合，您可以预测测试集:

y_pred = model.predict(test_dataset)

array([[2.2177568e-05, 3.0841196e-01, 6.9156587e-01],
       [4.3539176e-06, 1.2779665e-01, 8.7219906e-01],
       [1.0816366e-03, 9.2667454e-01, 7.2243840e-02],
       [9.9921310e-01, 7.8686583e-04, 9.8775059e-09]], dtype=float32)

这将是一个 (n_samples, 3)数组，因为我们正在处理三个类别。我们想要一个 (n_samples, 1) sklearn.metrics.confusion_matrix 的数组，所以取 argmax:

predicted_categories = tf.argmax(y_pred, axis=1)

<tf.Tensor: shape=(30,), dtype=int64, numpy=
array([2, 2, 2, 0, 2, 2, 2, 2, 1, 1, 2, 0, 0, 2, 1, 1, 1, 2, 0, 2, 1, 2,
       1, 0, 2, 0, 1, 2, 1, 0], dtype=int64)>

那么，我们可以把所有的y来自预取数据集的值:

true_categories = tf.concat([y for x, y in test_dataset], axis=0)

[<tf.Tensor: shape=(4,), dtype=int64, numpy=array([1, 1, 1, 0], dtype=int64)>,
 <tf.Tensor: shape=(4,), dtype=int64, numpy=array([2, 2, 2, 2], dtype=int64)>,
 <tf.Tensor: shape=(4,), dtype=int64, numpy=array([1, 1, 1, 0], dtype=int64)>,
 <tf.Tensor: shape=(4,), dtype=int64, numpy=array([0, 2, 1, 1], dtype=int64)>,
 <tf.Tensor: shape=(4,), dtype=int64, numpy=array([1, 2, 0, 2], dtype=int64)>,
 <tf.Tensor: shape=(4,), dtype=int64, numpy=array([1, 2, 1, 0], dtype=int64)>,
 <tf.Tensor: shape=(4,), dtype=int64, numpy=array([2, 0, 1, 2], dtype=int64)>,
 <tf.Tensor: shape=(2,), dtype=int64, numpy=array([1, 0], dtype=int64)>]

然后，您已准备好获取混淆矩阵:

confusion_matrix(predicted_categories, true_categories)

array([[ 9,  0,  0],
       [ 0,  9,  0],
       [ 0,  2, 10]], dtype=int64)

(9 + 9 + 10) / 30 = 0.933是准确率分数。对应于model.evaluate(test_dataset) :

8/8 [==============================] - 0s 785us/step - loss: 0.1907 - accuracy: 0.9333

结果也与sklearn.metrics.classification_report一致:

              precision    recall  f1-score   support
           0       1.00      1.00      1.00         8
           1       0.82      1.00      0.90         9
           2       1.00      0.85      0.92        13
    accuracy                           0.93        30
   macro avg       0.94      0.95      0.94        30
weighted avg       0.95      0.93      0.93        30

这是整个代码:

import tensorflow_datasets as tfds
import tensorflow as tf
from sklearn.metrics import confusion_matrix

data, info = tfds.load('iris', split='train',
                       as_supervised=True,
                       shuffle_files=True,
                       with_info=True)

AUTOTUNE = tf.data.experimental.AUTOTUNE

train_dataset = data.take(120).batch(4).prefetch(buffer_size=AUTOTUNE)
test_dataset = data.skip(120).take(30).batch(4).prefetch(buffer_size=AUTOTUNE)

model = tf.keras.Sequential([
    tf.keras.layers.Dense(8, activation='relu'),
    tf.keras.layers.Dense(16, activation='relu'),
    tf.keras.layers.Dense(info.features['label'].num_classes, activation='softmax')
    ])

model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', 
              metrics='accuracy')

history = model.fit(train_dataset, validation_data=test_dataset, epochs=50, verbose=0)

y_pred = model.predict(test_dataset)

predicted_categories = tf.argmax(y_pred, axis=1)

true_categories = tf.concat([y for x, y in test_dataset], axis=0)

confusion_matrix(predicted_categories, true_categories)