Tensorflow 在构建 RNN 层时使用参数“sequence_length”支持动态长度序列，其中模型在序列大小=“sequence_length”之后不再学习序列，即返回零向量。
但是，https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/seq2seq.py#L890 处的成本函数如何？是否需要修改以遇到屏蔽序列，以便仅在实际序列而不是整个填充序列上计算成本和困惑度？
def sequence_loss_by_example(logits, targets, weights, average_across_timesteps=True, softmax_loss_function=None, name=None):
if len(targets) != len(logits) or len(weights) != len(logits):
raise ValueError("Lengths of logits, weights, and targets must be the same "
"%d, %d, %d." % (len(logits), len(weights), len(targets)))
with ops.op_scope(logits + targets + weights, name,
"sequence_loss_by_example"):
log_perp_list = []
for logit, target, weight in zip(logits, targets, weights):
if softmax_loss_function is None:
# TODO(irving,ebrevdo): This reshape is needed because
# sequence_loss_by_example is called with scalars sometimes, which
# violates our general scalar strictness policy.
target = array_ops.reshape(target, [-1])
crossent = nn_ops.sparse_softmax_cross_entropy_with_logits(
logit, target)
else:
crossent = softmax_loss_function(logit, target)
log_perp_list.append(crossent * weight)
log_perps = math_ops.add_n(log_perp_list)
if average_across_timesteps:
total_size = math_ops.add_n(weights)
total_size += 1e-12 # Just to avoid division by 0 for all-0 weights.
log_perps /= total_size
return log_perps
最佳答案
该函数已经支持通过使用权重来计算动态序列长度的成本。只要确保“填充目标”的权重为 0，这些步骤的交叉熵就会被推至 0:
log_perp_list.append(crossent * weight)
总大小也将仅反射(reflect)非填充步骤:
total_size = math_ops.add_n(weights)
如果您用零填充，导出权重的一种方法如下:
weights = tf.sign(tf.abs(model.targets))
(请注意，您可能需要将其转换为与目标相同的类型)
关于python - 如何修改填充向量的 seq2seq 成本函数？，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/38502366/