加载保存的 Tensorflow 模型后,我无法运行前向传播函数。我能够成功提取权重,但是当我尝试将新输入传递给前向 Prop 函数时,它会抛出“尝试使用未初始化的值”错误。
我的占位符如下:
x = tf.placeholder('int64', [None, 4], name='input') # Number of examples x features
y = tf.placeholder('int64', [None, 1], name='output') # Number of examples x output
正向 Prop 功能:
def forwardProp(x, y):
embedding_mat = tf.get_variable("EM", shape=[total_vocab, e_features], initializer=tf.random_normal_initializer(seed=1))
# m x words x total_vocab * total_vocab x e_features = m x words x e_features
# embed_x = tf.tensordot(x, tf.transpose(embedding_mat), axes=[[2], [0]])
# embed_y = tf.tensordot(y, tf.transpose(embedding_mat), axes=[[2], [0]])
embed_x = tf.gather(embedding_mat, x) # m x words x e_features
embed_y = tf.gather(embedding_mat, y) # m x words x e_features
#print("Shape of embed x", embed_x.get_shape())
W1 = tf.get_variable("W1", shape=[n1, e_features], initializer=tf.random_normal_initializer(seed=1))
B1 = tf.get_variable("b1", shape=[1, 4, n1], initializer=tf.zeros_initializer())
# m x words x e_features * e_features x n1 = m x words x n1
Z1 = tf.add(tf.tensordot(embed_x, tf.transpose(W1), axes=[[2], [0]]), B1, )
A1 = tf.nn.tanh(Z1)
W2 = tf.get_variable("W2", shape=[n2, n1], initializer=tf.random_normal_initializer(seed=1))
B2 = tf.get_variable("B2", shape=[1, 4, n2], initializer=tf.zeros_initializer())
# m x words x n1 * n1 x n2 = m x words x n2
Z2 = tf.add(tf.tensordot(A1, tf.transpose(W2), axes=[[2], [0]]), B2)
A2 = tf.nn.tanh(Z2)
W3 = tf.get_variable("W3", shape=[n3, n2], initializer=tf.random_normal_initializer(seed=1))
B3 = tf.get_variable("B3", shape=[1, 4, n3], initializer=tf.zeros_initializer())
# m x words x n2 * n2 x n3 = m x words x n3
Z3 = tf.add(tf.tensordot(A2, tf.transpose(W3), axes=[[2], [0]]), B3)
A3 = tf.nn.tanh(Z3)
# Convert m x words x n3 to m x n3
x_final = tf.reduce_mean(A3, axis=1)
y_final = tf.reduce_mean(embed_y, axis=1)
return x_final, y_final
返回 Prop 功能:
def backProp(X_index, Y_index):
x_final, y_final = forwardProp(x, y)
cost = tf.nn.l2_loss(x_final - y_final)
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
init = tf.global_variables_initializer()
saver = tf.train.Saver()
total_batches = math.floor(m/batch_size)
with tf.Session() as sess:
sess.run(init)
for epoch in range(epochs):
batch_start = 0
for i in range(int(m/batch_size)):
x_hot = X_index[batch_start: batch_start + batch_size]
y_hot = Y_index[batch_start: batch_start + batch_size]
batch_start += batch_size
_, temp_cost = sess.run([optimizer, cost], feed_dict={x: x_hot, y: y_hot})
print("Cost at minibatch: ", i , " and epoch ", epoch, " is ", temp_cost)
if m % batch_size != 0:
x_hot = X_index[batch_start: batch_start+m - (batch_size*total_batches)]
y_hot = Y_index[batch_start: batch_start+m - (batch_size*total_batches)]
_, temp_cost = sess.run([optimizer, cost], feed_dict={x: x_hot, y: y_hot})
print("Cost at minibatch: (beyond floor) and epoch ", epoch, " is ", temp_cost)
# Saving the model
save_path = saver.save(sess, "./model_neural_embeddingV1.ckpt")
print("Model saved!")
通过调用预测函数重新加载模型:
def predict_search():
# Initialize variables
total_features = 4
extra = len(word_to_indice)
query = input('Enter your query')
words = word_tokenize(query)
# For now, it will throw an error if a word not present in dictionary is present
features = [word_to_indice[w.lower()] for w in words]
len_features = len(features)
X_query = []
Y_query = [[0]] # Dummy variable, we don't care about the Y query while doing prediction
if len_features < total_features:
features += [extra] * (total_features - len_features)
elif len_features > total_features:
features = features[:total_features]
X_query.append(features)
X_query = np.array(X_query)
print(X_query)
Y_query = np.array(Y_query)
# Load the model
init_global = tf.global_variables_initializer()
init_local = tf.local_variables_initializer()
#X_final, Y_final = forwardProp(x, y)
with tf.Session() as sess:
sess.run(init_global)
sess.run(init_local)
saver = tf.train.import_meta_graph('./model_neural_embeddingV1.ckpt.meta')
saver.restore(sess, './model_neural_embeddingV1.ckpt')
print("Model loaded")
print("Loaded variables are: ")
print(tf.trainable_variables())
print(sess.graph.get_operations())
embedMat = sess.run('EM:0') # Get the word embedding matrix
W1 = sess.run('W1:0')
b1 = sess.run('b1:0')
W2 = sess.run('W2:0')
b2 = sess.run('B2:0')
print(b2)
W3 = sess.run('W3:0')
b3 = sess.run('B3:0')
**#This part is not working, calling forward prop gives an 'attempting to use uninitialized value' error.**
X_final = sess.run(forwardProp(x, y), feed_dict={x: X_query, y: Y_query})
print(X_final)
最佳答案
从元图加载后,您不小心使用 forwardProp
函数创建了一堆图变量,从而在无意中有效地复制了变量。
您应该重构代码,以遵循在创建 session 之前创建图形变量的最佳实践。
例如,在名为 build_graph
的函数中创建所有变量。您可以在创建 session 之前调用 build_graph
,但绝对不能在创建 session 之后调用。这将避免这样的困惑。
您几乎应该始终避免从 sess.run
调用函数,例如您正在执行的操作:
X_final = sess.run(forwardProp(x, y), feed_dict={x: X_query, y: Y_query})
您就是在以这种方式寻求错误。
请注意在 forwardProp(x, y)
中发生的情况,您正在创建 tensorflow 构造、所有权重和偏差。
但请注意,您是在这两行代码中创建的:
saver = tf.train.import_meta_graph('./model_neural_embeddingV1.ckpt.meta')
saver.restore(sess, './model_neural_embeddingV1.ckpt')
另一个选项(可能是您想要做的)是不使用import_meta_graph
。您可以创建所有 TensorFlow OP 和变量,然后运行 saver.restore 来恢复检查点,这会将检查点数据映射到您已创建的变量中。
请注意,这里的 tensorflow 实际上有 2 个选项,这有点令人困惑。您最终完成了这两件事(导入包含所有操作和变量的图表),并重新创建图表。你必须选择一个。
我通常选择第一个选项,不使用 import_meta_graph
,只需通过调用 build_graph
函数以编程方式重新创建图表。然后调用 saver.restore
引入检查点。当然,您将重复使用 build_graph
函数进行训练和推理时间,因此您最终会得到两次都是相同的图表。
关于tensorflow - 加载 tensorflow 模型后运行forward prop函数,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/49138484/