tensorflow - 形状必须为 0 级，但为 1 级，parse_single_sequence_example

Question

在过去的几天里，我在将数据序列化为 tfrecord 格式，然后使用 parse_single_sequence 示例对其进行反序列化时遇到了问题。我正在尝试检索数据以与相当标准的 RNN 模型一起使用，但这是我第一次尝试使用 tfrecords 格式及其相关的管道。

这是一个重现我遇到的问题的玩具示例:

import tensorflow as tf
import tempfile
from IPython import embed

sequences = [[1, 2, 3], [4, 5, 1], [1, 2]]
label_sequences = [[0, 1, 0], [1, 0, 0], [1, 1]]

def make_example(sequence, labels):

    ex = tf.train.SequenceExample()

    sequence_length = len(sequence)
    ex.context.feature["length"].int64_list.value.append(sequence_length)

    fl_tokens = ex.feature_lists.feature_list["tokens"]
    fl_labels = ex.feature_lists.feature_list["labels"]
    for token, label in zip(sequence, labels):
        fl_tokens.feature.add().int64_list.value.append(token)
        fl_labels.feature.add().int64_list.value.append(label)
    return ex


writer = tf.python_io.TFRecordWriter('./test.tfrecords')
for sequence, label_sequence in zip(sequences, label_sequences):
    ex = make_example(sequence, label_sequence)
    writer.write(ex.SerializeToString())
writer.close()

tf.reset_default_graph()

file_name_queue = tf.train.string_input_producer(['./test.tfrecords'], num_epochs=None)

reader = tf.TFRecordReader()



context_features = {
    "length": tf.FixedLenFeature([], dtype=tf.int64)
}
sequence_features = {
    "tokens": tf.FixedLenSequenceFeature([], dtype=tf.int64),
    "labels": tf.FixedLenSequenceFeature([], dtype=tf.int64)
}

ex = reader.read(file_name_queue)

# Parse the example (returns a dictionary of tensors)
context_parsed, sequence_parsed = tf.parse_single_sequence_example(
    serialized=ex,
    context_features=context_features,
    sequence_features=sequence_features
)


context = tf.contrib.learn.run_n(context_parsed, n=1, feed_dict=None)
print(context[0])
sequence = tf.contrib.learn.run_n(sequence_parsed, n=1, feed_dict=None)
print(sequence[0])

关联的堆栈跟踪是:

Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/common_shapes.py", line 594, in call_cpp_shape_fn
status)
File "/usr/lib/python3.5/contextlib.py", line 66, in exit
next(self.gen)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors.py", line 463, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors.InvalidArgumentError: Shape must be rank 0 but is rank 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "my_test.py", line 51, in 
sequence_features=sequence_features
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/parsing_ops.py", line 640, in parse_single_sequence_example
feature_list_dense_defaults, example_name, name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/parsing_ops.py", line 837, in _parse_single_sequence_example_raw
name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_parsing_ops.py", line 285, in _parse_single_sequence_example
name=name)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 749, in apply_op
op_def=op_def)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2382, in create_op
set_shapes_for_outputs(ret)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1783, in set_shapes_for_outputs
shapes = shape_func(op)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/common_shapes.py", line 596, in call_cpp_shape_fn
raise ValueError(err.message)
ValueError: Shape must be rank 0 but is rank 1

我将其作为潜在问题发布在 github 上，尽管看起来我可能只是错误地使用了它:Tensorflow Github Issue 因此，在了解了背景信息后，我只是想知道我是否真的在这里犯了错误？任何正确方向的帮助将不胜感激，已经过去几天了，我的探索还没有成功。谢谢大家!

Answer 1

明白了，对我来说这是一个错误的假设。 tf.TFRecordReader.read(queue, name=None)当我假设它只返回值而不是 (key, value) 时，返回一个元组我直接将其传递到示例解析器中。