我目前正在尝试训练模型，并且我的输入管道是按照此答案 here 构建的。我想在每个时期后保存我的模型。但经过一些时期的训练后，训练崩溃了。我读到这是因为它将输入作为常量张量添加到图中。有建议的解决方案here使用 tf.placeholder 来解决问题。不幸的是它并不能解决我的问题。输入管道如下所示
....
filenames = [P_1]
dataset = tf.data.TFRecordDataset(filenames)
def _parse_function(example_proto):
keys_to_features = { 'data':tf.VarLenFeature(tf.float32)},
parsed_features = tf.parse_single_example(example_proto, keys_to_features)
return tf.sparse_tensor_to_dense(parsed_features['data'
# Parse the record into tensors.
dataset = dataset.map(_parse_function)
# Shuffle the dataset
dataset = dataset.shuffle(buffer_size=1000)
# Repeat the input indefinitly
dataset = dataset.repeat()
# Generate batches
dataset = dataset.batch(Batch_size)
# Create a one-shot iterator
iterator = dataset.make_one_shot_iterator()
data = iterator.get_next()
....
for i in range(epochs):
for ii in range(iteration):
image = sess.run(data)
....
saver.save(sess, 'filename')
错误消息如下所示
[libprotobuf FATAL external/protobuf_archive/src/google/protobuf/message_lite.cc:68] CHECK failed: (byte_size_before_serialization) == (byte_size_after_serialization): tensorflow.GraphDef was modified concurrently during serialization.
terminate called after throwing an instance of 'google::protobuf::FatalException'
what(): CHECK failed: (byte_size_before_serialization) == (byte_size_after_serialization): tensorflow.GraphDef was modified concurrently during serialization.
Aborted
最佳答案
问题看起来像是在 _parse_function 中。确保解析器以与创建 TFrecord 文件时相同的方式执行操作。例如，如果它们具有相同的数据类型等等
关于python - 由于保存模型导致训练崩溃 : "tensorflow.GraphDef was modified concurrently during serialization"，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/52717630/