我正在尝试执行 (batch_size, time_steps, my_data) 类型的批处理
为什么在 flat_map
步骤中我得到 AttributeError: 'dict' object has no attribute 'batch'
x_train = np.random.normal(size=(60000, 768))
token_type_ids = np.ones(shape=(len(x_train)))
position_ids = np.random.normal(size=(x_train.shape[0], 5))
features_ds = tf.data.Dataset.from_tensor_slices({'inputs_embeds': x_train,
'token_type_ids': token_type_ids,
'position_ids': position_ids})
y_ds = tf.data.Dataset.from_tensor_slices(y_train)
ds = tf.data.Dataset.zip((features_ds, y_ds))
# result = list(ds.as_numpy_iterator())
result_ds = ds.window(size=time_steps, shift=time_steps, stride=1, drop_remainder=True). \
flat_map(lambda x, y: tf.data.Dataset.zip((x.batch(time_steps), y.batch(time_steps))))
知道问题出在哪里吗?以及如何解决?
最佳答案
您可以添加批处理作为单独的步骤:
x_train = np.random.normal(size=(60000, 768))
token_type_ids = np.ones(shape=(len(x_train)))
position_ids = np.random.normal(size=(x_train.shape[0], 5))
features_ds = tf.data.Dataset.from_tensor_slices({'inputs_embeds': x_train,
'token_type_ids': token_type_ids,
'position_ids': position_ids})
y_train = np.random.normal(size=(60000, 1))
y_ds = tf.data.Dataset.from_tensor_slices(y_train)
ds = tf.data.Dataset.zip((features_ds, y_ds))
result_ds = ds.window(size=time_steps, shift=time_steps, stride=1, drop_remainder=True).\
flat_map(lambda x, y: tf.data.Dataset.zip((x, y)))
time_steps=3
result_ds=result_ds.batch(time_steps)
for i in result_ds.take(1):
print(i)
关于tensorflow - tf.data WindowDataset flat_map 给出 'dict' 对象没有属性 'batch' 错误,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/63366229/