使用 ResNet50 预训练的权重我正在尝试构建一个分类器。代码库完全在 Keras 高级 Tensorflow API 中实现。完整代码发布在下面的 GitHub 链接中。
源代码:Classification Using RestNet50 Architecture
预训练模型的文件大小为94.7mb。
我加载了预训练的文件
new_model = Sequential()
new_model.add(ResNet50(include_top=False,
pooling='avg',
weights=resnet_weight_paths))
并拟合模型
train_generator = data_generator.flow_from_directory(
'path_to_the_training_set',
target_size = (IMG_SIZE,IMG_SIZE),
batch_size = 12,
class_mode = 'categorical'
)
validation_generator = data_generator.flow_from_directory(
'path_to_the_validation_set',
target_size = (IMG_SIZE,IMG_SIZE),
class_mode = 'categorical'
)
#compile the model
new_model.fit_generator(
train_generator,
steps_per_epoch = 3,
validation_data = validation_generator,
validation_steps = 1
)
在训练数据集中,我有两个文件夹 dog 和 cat,每个文件夹包含近 10,000 张图像。编译脚本时,出现以下错误
Epoch 1/1 2018-05-12 13:04:45.847298: W tensorflow/core/framework/allocator.cc:101] Allocation of 38535168 exceeds 10% of system memory. 2018-05-12 13:04:46.845021: W tensorflow/core/framework/allocator.cc:101] Allocation of 37171200 exceeds 10% of system memory. 2018-05-12 13:04:47.552176: W tensorflow/core/framework/allocator.cc:101] Allocation of 37171200 exceeds 10% of system memory. 2018-05-12 13:04:48.199240: W tensorflow/core/framework/allocator.cc:101] Allocation of 37171200 exceeds 10% of system memory. 2018-05-12 13:04:48.918930: W tensorflow/core/framework/allocator.cc:101] Allocation of 37171200 exceeds 10% of system memory. 2018-05-12 13:04:49.274137: W tensorflow/core/framework/allocator.cc:101] Allocation of 19267584 exceeds 10% of system memory. 2018-05-12 13:04:49.647061: W tensorflow/core/framework/allocator.cc:101] Allocation of 19267584 exceeds 10% of system memory. 2018-05-12 13:04:50.028839: W tensorflow/core/framework/allocator.cc:101] Allocation of 19267584 exceeds 10% of system memory. 2018-05-12 13:04:50.413735: W tensorflow/core/framework/allocator.cc:101] Allocation of 19267584 exceeds 10% of system memory.
有什么想法可以优化加载预训练模型的方式(或)摆脱此警告消息?
谢谢!
最佳答案
尝试将 batch_size 属性减少到一个小数字(如 1,2 或 3)。 示例:
train_generator = data_generator.flow_from_directory(
'path_to_the_training_set',
target_size = (IMG_SIZE,IMG_SIZE),
batch_size = 2,
class_mode = 'categorical'
)
关于python - TensorFlow 分配内存 : Allocation of 38535168 exceeds 10% of system memory,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/50304156/