我正在 tensorflow 中实现 DeepMind 的 DQN 算法,并在调用 optimizer.minimize(self.loss) 的行上遇到此错误:
ValueError:没有为任何变量提供渐变...
通过阅读有关此错误的其他帖子,我发现这意味着损失函数不依赖于任何用于设置模型的张量,但在我的代码中我看不出这是怎么回事。 qloss()
函数显然依赖于对 predict()
函数的调用,而该函数又依赖于所有层张量来进行计算。
最佳答案
我发现问题在于,在我的 qloss()
函数中,我从张量中提取值,对它们进行操作并返回值。虽然这些值确实取决于张量,但它们本身并未封装在张量中,因此 TensorFlow 无法判断它们取决于图中的张量。
我通过更改 qloss()
解决了这个问题,以便它直接对张量进行操作并返回一个张量。这是新功能:
def qloss(actions, rewards, target_Qs, pred_Qs):
"""
Q-function loss with target freezing - the difference between the observed
Q value, taking into account the recently received r (while holding future
Qs at target) and the predicted Q value the agent had for (s, a) at the time
of the update.
Params:
actions - The action for each experience in the minibatch
rewards - The reward for each experience in the minibatch
target_Qs - The target Q value from s' for each experience in the minibatch
pred_Qs - The Q values predicted by the model network
Returns:
A list with the Q-function loss for each experience clipped from [-1, 1]
and squared.
"""
ys = rewards + DISCOUNT * target_Qs
#For each list of pred_Qs in the batch, we want the pred Q for the action
#at that experience. So we create 2D list of indeces [experience#, action#]
#to filter the pred_Qs tensor.
gather_is = tf.squeeze(np.dstack([tf.range(BATCH_SIZE), actions]))
action_Qs = tf.gather_nd(pred_Qs, gather_is)
losses = ys - action_Qs
clipped_squared_losses = tf.square(tf.minimum(tf.abs(losses), 1))
return clipped_squared_losses
关于python - TensorFlow: 'ValueError: No gradients provided for any variable',我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/37889125/