请参阅下面的 EDIT ，最初的帖子现在几乎没有任何意义，但问题仍然存在。
我开发了一个神经网络来语义分割图像。我研究了各种损失函数(分类交叉熵 (CCE)、权重 CCE、焦点损失、tversky 损失、jaccard 损失、焦点 tversky 损失等)，这些函数试图处理高度偏斜的类表示，但都没有产生预期的效果。我的顾问提到尝试创建一个自定义损失函数，该函数忽略特定类别的误报(但仍会惩罚误报)。
我有一个 6 类问题，我的网络设置为在单热编码的真实数据中工作/使用。因此，我的损失函数将接受两个张量 y_true, y_pred ，形状为 (batch, row, col, class) (当前为 (8, 128, 128, 6) )。为了能够利用我已经探索过的损失，我想更改 y_pred 以将特定类别(第 0 个类别)的预测值设置为始终正确。这就是 y_true == class 0 设置 y_pred == class 0 的地方，否则什么都不做。
由于 tensorflow 张量是不可变的，我花了太多时间试图创建这个损失函数。我的第一次尝试(我是通过我对 numpy 的经验而获得的)
def weighted_categorical_crossentropy_ignore(weights):
weights = K.variable(weights)
def loss(y_true, y_pred):
y_pred[tf.where(y_true == [1, 0, 0, 0, 0, 0])] = [1, 0, 0, 0, 0, 0]
# Scale predictions so that the class probs of each sample sum to 1
y_pred /= K.sum(y_pred, axis=-1, keepdims=True)
# Clip to prevent NaN's and Inf's
y_pred = K.clip(y_pred, K.epsilon(), 1 - K.epsilon())
loss = y_true * K.log(y_pred) * weights
loss = -K.sum(loss, -1)
return loss
return loss
y_pred 所以这次尝试失败了。我最终创建了一些怪物，试图通过迭代 [batch, row, col] 并进行比较来“构建”张量。虽然这种(ese)尝试在技术上没有失败，但他们从未真正开始训练。我认为计算损失需要几分钟的时间。经过多次失败的努力，我开始尝试在 SSCCE 中以纯
numpy 执行必要的计算。但是保持认知我基本上仅限于实例化“简单”张量(即 ones 、 zeros )并且只执行“简单”操作，如逐元素乘法、加法和整形。于是我来到了这个 SSCCEimport numpy as np
from tensorflow.keras.utils import to_categorical
# Generate the "images" at random
true_flat = np.argmax(np.random.rand(1, 2, 2, 4), axis=3).astype('int')
true = to_categorical(true_flat, num_classes=4).astype('int')
pred_flat = np.argmax(np.random.rand(1, 2, 2, 4), axis=3).astype('int')
pred = to_categorical(pred_flat, num_classes=4).astype('int')
print('True:\n', true_flat)
print('Pred:\n', pred_flat)
# Create a mask representing an all "class 0" image
class_zero_label = np.array([1, 0, 0, 0])
czl_all = class_zero_label * np.ones(true.shape).astype('int')
# Mask both the truth and pred to locate class 0 pixels
czl_true_locs = czl_all * true
czl_pred_locs = czl_all * pred
# Subtract to create "addition" matrix
a = (czl_true_locs - czl_pred_locs) * czl_true_locs
print('a:\n', a)
# Do this
m = ((a + 1) - (a * 2))
print('m - ', m.shape, ':\n', m)
# Pull the front entry from 'm' and "expand" its value
#x = (m[:, :, :, 0].flatten() * np.ones(pred.shape).astype('int')).T.reshape(pred.shape)
m_front = m[:, :, :, 0]
print('m_front - ', m_front.shape, ':\n', m_front)
#m_flat = m_front.flatten()
m_flat = m_front.reshape(m_front.shape[0], m_front.shape[1]*m_front.shape[2])
print('m_flat - ', m_flat.shape, ':\n', m_flat)
m_expand = m_flat * np.ones(pred.shape).astype('int')
print('m_expand - ', m_expand.shape, ':\n', m_expand)
m_trans = m_expand.T
m_fixT = m_trans.reshape(pred.shape)
print('m_fixT - ', m_fixT.shape, ':\n', m_fixT)
m = m_fixT
print('m:\n', m.shape)
# Perform the math as described
pred = (pred * m) + a
print('Pred:\n', np.argmax(pred, axis=3))
[0, 0, 0, 0, 0, 0] 无关.然后，加法矩阵将向量 [1, 0, 0, 0, 0, 0] 添加到每个归零的位置。最终，这将实现将每个真正的 0 类像素的预测值设置为正确的目标。问题是这个 SSCCE 没有完全转化为 tensorflow 操作。第一个问题是乘法矩阵的生成，它在
batch_size > 1 时没有正确定义。我想没关系，只是为了看看它是否有效，我会分解 tf.unstack y_true 和 y_pred 张量并对其进行迭代。这导致我对我的损失函数进行了当前的实例化def weighted_categorical_crossentropy_ignore(weights):
weights = K.variable(weights)
def loss(y_true, y_pred):
y_true_un = tf.unstack(y_true)
y_pred_un = tf.unstack(y_pred)
y_pred_new = []
for i in range(0, y_true.shape[0]):
yt = y_true_un[i]
yp = y_pred_un[i]
# Pred:
# [[[0 3] * [[[1 0] + [[[0 1] = [[[0 0]
# [3 1]]] [[1 1]]] [[0 0]]] [[3 1]]]
# If we multiple pred by a tensor which zeros out only incorrect class 0 labelleling
# Then add class zero to those zero'd out locations
# We can negate the effect of mis-classified class 0 pixels but still punish for
# incorrectly predicted class 0 labels for other classes.
# Create a mask respresenting an all "class 0" image
class_zero_label = K.variable([1.0, 0.0, 0.0, 0.0, 0.0, 0.0])
czl_all = class_zero_label * K.ones(yt.shape)
# Mask both true and pred to locate class 0 pixels
czl_true = czl_all * yt
czl_pred = czl_all * yp
# Subtract to create "addition matrix"
a = czl_true - czl_pred
# Do this.
m = ((a + 1) - (a * 2.))
# And this.
x = K.flatten(m[:, :, 0])
x = x * K.ones(yp.shape)
x = K.transpose(x)
x = K.reshape(x, yp.shape)
# Voila.
ypnew = (yp * x) + a
y_pred_new.append(ypnew)
y_pred_new = tf.concat(y_pred_new, 0)
# Continue calculating weighted categorical crossentropy
# -------------------------------------------------------
# Scale predictions so that the class probs of each sample sum to 1
y_pred_new /= K.sum(y_pred_new, axis=-1, keepdims=True)
# Clip to prevent NaN's and Inf's
y_pred_new = K.clip(y_pred_new, K.epsilon(), 1 - K.epsilon())
loss = y_true * K.log(y_pred_new) * weights
loss = -K.sum(loss, -1)
return loss
return loss
numpy 和 tensorflow 之间的行为存在明显差异x = K.flatten(m[:, :, 0])
x = x * K.ones(yp.shape)
m_flat = m_front.flatten()
m_expand = m_flat * np.ones(pred.shape).astype('int')
所以在这一点上，我觉得我已经深入研究了穴居人编码，我无法摆脱它。我必须想象有一些简单的方法类似于我最初尝试执行所描述的行为。
所以，我想我的直接问题是 如何实现
y_pred[tf.where(y_true == [1, 0, 0, 0, 0, 0])] = [1, 0, 0, 0, 0, 0]
编辑: 在摸索了很多之后，我终于确定了如何在
.numpy()、y_true 张量上调用 y_pred 以使用 numpy 的 0x1045 设置操作 0x1046t 的 791045 次“7” tf.compat.v1.enable_eager_execution 到 run_eagerly=True )。这使我基本上实现了概述的第一次尝试
def weighted_categorical_crossentropy_ignore(weights):
weights = K.variable(weights)
def loss(y_true, y_pred):
yp = y_pred.numpy()
yt = y_true.numpy()
yp[np.nonzero(np.all(yt == [1, 0, 0, 0, 0, 0], axis=3))] = [1, 0, 0, 0, 0, 0]
# Continue calculating weighted categorical crossentropy
# -------------------------------------------------------
# Scale predictions so that the class probs of each sample sum to 1
yp /= K.sum(yp, axis=-1, keepdims=True)
# Clip to prevent NaN's and Inf's
yp = K.clip(yp, K.epsilon(), 1 - K.epsilon())
loss = y_true * K.log(yp) * weights
loss = -K.sum(loss, -1)
return loss
return loss
Model().compile(...)(或此后使用它)，我显然已经“破坏”了通过网络的路径/流。基于尝试 y_pred.numpy() 时的错误ValueError: No gradients provided for any variable: ['conv3d/kernel:0', <....>
yp = tf.convert_to_tensor(yp)
EDIT2: 好吧，从这个 SO Answer 看来，我实际上不能使用
.fit 来编码 numpy() ， y_true 到 1y_pred 到 9这必然会“破坏”网络路径，因此无法计算梯度。结果我已经意识到
numpy 我可以 run_eagerly=True 我的 tf.Variable/y_true 并执行分配。所以在纯 tensorflow 中我试图再次重新创建相同的代码def weighted_categorical_crossentropy_ignore(weights):
weights = K.variable(weights)
def loss(y_true, y_pred):
# yp = y_pred.numpy().copy()
# yt = y_true.numpy().copy()
# yp[np.nonzero(np.all(yt == [1, 0, 0, 0, 0, 0], axis=3))] = [1, 0, 0, 0, 0, 0]
yp = K.variable(y_pred)
yt = K.variable(y_true)
#np.all
x = K.all(yt == [1, 0, 0, 0, 0, 0], axis=3)
#np.nonzero
ne = tf.not_equal(x, tf.constant(False))
y = tf.where(ne)
# Perform the desired operation
yp[y] = [1, 0, 0, 0, 0, 0]
# Continue calculating weighted categorical crossentropy
# -------------------------------------------------------
# Scale predictions so that the class probs of each sample sum to 1
#yp /= K.sum(yp, axis=-1, keepdims=True) # Cannot use \= on tf.var, must use var = var /
yp = yp / K.sum(yp, axis=-1, keepdims=True)
# Clip to prevent NaN's and Inf's
yp = K.clip(yp, K.epsilon(), 1 - K.epsilon())
loss = y_true * K.log(yp) * weights
loss = -K.sum(loss, -1)
return loss
return loss
y_pred 时相同的问题；无法计算梯度。所以我似乎又回到了广场 1。EDIT3: 使用提出的解决方案 by gobrewers14 in the answer posted below 但根据我对问题的了解进行了修改，我产生了这个损失函数
def weighted_categorical_crossentropy_ignore(weights):
weights = K.variable(weights)
def loss(y_true, y_pred):
print('y_true.shape: ', y_true.shape)
print('y_pred.shape: ', y_pred.shape)
# Generate modified y_pred where all truly class0 pixels are correct
y_true_class0_indicies = tf.where(tf.math.equal(y_true, [1., 0., 0., 0., 0., 0.]))
y_pred_updates = tf.repeat([
[1.0, 0.0, 0.0, 0.0, 0.0, 0.0]],
repeats=y_true_class0_indicies.shape[0],
axis=0)
yp = tf.tensor_scatter_nd_update(y_pred, y_true_class0_indicies, y_pred_updates)
# Continue calculating weighted categorical crossentropy
# -------------------------------------------------------
# Scale predictions so that the class probs of each sample sum to 1
yp /= K.sum(yp, axis=-1, keepdims=True)
# Clip to prevent NaN's and Inf's
yp = K.clip(yp, K.epsilon(), 1 - K.epsilon())
loss = y_true * K.log(yp) * weights
loss = -K.sum(loss, -1)
return loss
return loss
.numpy() 的形状为 y_true(即“平面”类表示，而不是单热编码表示 [8, 128, 128] )，我首先打印 [8, 128, 128, 6] 和 y_true 输入的形状y_true.shape: (8, 128, 128, 6)
y_pred.shape: (8, 128, 128, 6)
y_pred 的尾部提供的网络输出形状是conv2d_18 (Conv2D) (None, 128, 128, 6) 1542 dropout_5[0][0]
__________________________________________________________________________________________________
activation_9 (Activation) (None, 128, 128, 6) 0 conv2d_18[0][0]
==================================================================================================
Total params: 535,551,494
Trainable params: 535,529,478
Non-trainable params: 22,016
__________________________________________________________________________________________________
model.summary 替换原来的 tf.math.equal(y_true, 0) 来处理单热编码的情况。根据我目前对提议的解决方案的理解(经过约 10 分钟的检查)，我认为这应该可行。尽管在尝试训练模型时会引发以下异常InvalidArgumentError: Inner dimensions of output shape must match inner dimensions of updates shape. Output: [8,128,128,6] updates: [684584,6] [Op:TensorScatterUpdate]
tf.math.equal(y_true, [1., 0., 0., 0., 0., 0.]) 的产生产生了一个具有“太多”元素的“折叠”张量。我理解使用 y_pred_updates 的动机，但它的具体用途似乎不正确。我认为它应该根据我对 tf.repeat 的理解产生形状为 (8, 128, 128, 6) 的张量。我认为这很可能只是基于在调用 tf.tensor_scatter_nd_update 期间对 repeats 和 axis 的选择。
最佳答案
如果我正确理解你的问题，你正在寻找这样的东西:
import tensorflow as tf
# batch of true labels
y_true = tf.constant([5, 0, 1, 3, 4, 0, 2, 0], dtype=tf.int64)
# batch of class probabilities
y_pred = tf.constant(
[
[0.34670502, 0.04551039, 0.14020428, 0.14341979, 0.21430719, 0.10985339],
[0.25681055, 0.14013883, 0.19890164, 0.11124421, 0.14526634, 0.14763844],
[0.09199252, 0.21889475, 0.1170236 , 0.1929019 , 0.20311192, 0.17607528],
[0.3246354 , 0.23257554, 0.15549366, 0.17282239, 0.00000001, 0.11447308],
[0.16502093, 0.13163856, 0.14371352, 0.19880624, 0.23360236, 0.12721846],
[0.27362782, 0.21408406, 0.10917682, 0.13135742, 0.10814326, 0.16361059],
[0.20697299, 0.23721898, 0.06455399, 0.11071447, 0.18990229, 0.19063729],
[0.10320242, 0.22173141, 0.2547973 , 0.2314068 , 0.07063974, 0.11822232]
], dtype=tf.float32)
# find the indices in the batch where the true label is the class 0
indices = tf.where(tf.math.equal(y_true, 0))
# create a tensor with the number of updates you want to replace in `y_pred`
updates = tf.repeat(
[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0]],
repeats=indices.shape[0],
axis=0)
# insert the updates into `y_pred` at the specified indices
modified_y_pred = tf.tensor_scatter_nd_update(y_pred, indices, updates)
print(modified_y_pred)
# tf.Tensor(
# [[0.34670502, 0.04551039, 0.14020428, 0.14341979, 0.21430719, 0.10985339],
# [1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000],
# [0.09199252, 0.21889475, 0.1170236 , 0.1929019 , 0.20311192, 0.17607528],
# [0.3246354 , 0.23257554, 0.15549366, 0.17282239, 0.00000001, 0.11447308],
# [0.16502093, 0.13163856, 0.14371352, 0.19880624, 0.23360236, 0.12721846],
# [1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000],
# [0.20697299, 0.23721898, 0.06455399, 0.11071447, 0.18990229, 0.19063729],
# [1.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000, 0.00000000]],
# shape=(8, 6), dtype=tf.float32)
modified_y_pred ，可用于微分。编辑:
使用面具可能更容易做到这一点。
例子:
# these arent normalized to 1 but you get the point
probs = tf.random.normal([2, 4, 4, 6])
# raw labels per pixel
labels = tf.random.uniform(
shape=[2, 4, 4],
minval=0,
maxval=6,
dtype=tf.int64)
# your labels are already one-hot encoded
labels = tf.one_hot(labels, 6)
# boolean mask where classes are `0`
# converting back to int labels with argmax for purposes of
# using `tf.math.equal`. Matching on `[1, 0, 0, 0, 0, 0]` is
# potentially buggy; matching on an integer is a lot more
# explicit.
mask = tf.math.equal(tf.math.argmax(labels, -1), 0)[..., None]
# flip the mask to zero out the pixels across channels where
# labels are zero
probs *= tf.cast(tf.math.logical_not(mask), tf.float32)
# multiply the mask by the one-hot labels, and add back
# to the already masked probabilities.
probs += labels * tf.cast(mask, tf.float32)
关于python - Keras 自定义损失函数在语义分割期间忽略特定类的漏报？，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/66084314/