我想使用 VGG 16 模型作为我的编码器构建一个 U-net 架构,并在 Imagenet 数据集上使用预训练权重。对于上采样,我想将编码器中某些层的输出与解码部分连接起来。几乎像这样(取自 TernausNet): TernausNet
到目前为止,我的模型如下所示:
from keras.applications.vgg16 import VGG16 as VGG16, preprocess_input
encode_model = VGG16(input_shape=(768,768,3), include_top=False, weights='imagenet')
encode_model.trainable = False
from keras import models, layers
input_img = layers.Input((768,768,3), name = 'RGB_Input')
# output and start upsampling
features = encode_model(input_img)
conv_1 = layers.Conv2D(512, (3,3), activation='relu', padding='same')(features)
up_conv = layers.Conv2DTranspose(256, (3,3), strides=(2,2), activation='relu', padding='same')(conv_1)
# first concatenation block
concat_1 = layers.concatenate([encode_model.get_layer('block5_conv3').output, up_conv], axis=-1, name='concat_1')
conv_2 = layers.Conv2D(512, (3,3), activation='relu', padding='same')(concat_1)
up_conv_2 = layers.Conv2DTranspose(256, (3,3), strides=(2,2), activation='relu', padding='same')(conv_2)
# second concatenation block
concat_2 = layers.concatenate([up_conv_2, encode_model.get_layer('block4_conv3').output])
conv_3 = layers.Conv2D(512, (3,3), activation='relu', padding='same')(concat_2)
up_conv_3 = layers.Conv2DTranspose(128, (3,3), strides=(2,2), activation='relu', padding='same')(conv_3)
# third concatenation block
concat_3 = layers.concatenate([up_conv_3, encode_model.get_layer('block3_conv3').output])
conv_4 = layers.Conv2D(256, (3,3), activation='relu', padding='same')(concat_3)
up_conv_4 = layers.Conv2DTranspose(64, (3,3), strides=(2,2), activation='relu', padding='same')(conv_4)
# fourth concatenation block
concat_4 = layers.concatenate([up_conv_4, encode_model.get_layer('block2_conv2').output])
conv_5 = layers.Conv2D(128, (3,3), activation='relu', padding='same')(concat_4)
up_conv_5 = layers.Conv2DTranspose(32, (3,3), strides=(2,2), activation='relu', padding='same')(conv_5)
# fifth concatenation block
concat_4 = layers.concatenate([up_conv_5, encode_model.get_layer('block1_conv2').output])
conv_6 = layers.Conv2D(128, (3,3), activation='sigmoid', padding='same')(concat_4)
final_model = models.Model(inputs=[input_img], outputs=[conv_6])
final_model.summary()
稍后我编译并拟合模型:
import keras.backend as K
from keras.optimizers import Adam
from keras.losses import binary_crossentropy
def dice_coef(y_true, y_pred, smooth=1):
intersection = K.sum(y_true * y_pred, axis=[1,2,3])
union = K.sum(y_true, axis=[1,2,3]) + K.sum(y_pred, axis=[1,2,3])
return K.mean( (2. * intersection + smooth) / (union + smooth),
axis=0)
def dice_p_bce(in_gt, in_pred):
return 1e-3*binary_crossentropy(in_gt, in_pred) - dice_coef(in_gt,
in_pred)
def true_positive_rate(y_true, y_pred):
return K.sum(K.flatten(y_true)*K.flatten(K.round(y_pred)))/K.sum(y_true)
final_model.compile(optimizer=Adam(1e-3, decay=1e-6), loss=dice_p_bce,
metrics=[dice_coef, 'binary_accuracy', true_positive_rate])
step_count = min(MAX_TRAIN_STEPS, balanced_train_df.shape[0]//BATCH_SIZE)
aug_gen = create_aug_gen(make_image_gen(balanced_train_df))
loss_history = [final_model.fit_generator(aug_gen,
steps_per_epoch=step_count,
epochs=NB_EPOCHS,
validation_data=(valid_x, valid_y),
callbacks=callbacks_list,
workers=1)]`
然后我得到以下错误:
Graph disconnected: cannot obtain value for tensor Tensor("input_4:0", shape=(?, 768, 768, 3), dtype=float32) at layer "input_4". The following previous layers were accessed without issue: []
我读到,当输入和输出不属于同一图表时,就会发生这种情况。我认为我正在连接两个图:
input_img = layers.Input((768,768,3), name = 'RGB_Input')
# output and start upsampling
features = encode_model(input_img)
conv_1 = layers.Conv2D(512, (3,3), activation='relu', padding='same')(features)
我的错误是什么?
最佳答案
我更新了我的答案以对应已编辑的问题
我认为问题出在这一行:
features = encode_model(input_img)
encode_model
是一个 keras 模型,您尝试将自定义输入层传递给它。这当然不是必需的。尝试以下操作:
# necessary imports
encode_model = VGG16(input_shape=(768,768,3), include_top=False, weights='imagenet')
conv_1 = layers.Conv2D(512, (3,3), activation='relu', padding='same')(encode_model.output) # need to pass the output tensor to the conv_1 layer
# define the rest of your model as you have it
关于python - 使用 VGG 16 作为特征提取器的类似 U-net 的架构 - 连接层的问题,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/52119988/