python - 将 Keras 中 VGG19 的部分层与 TimeDistributed 层一起使用

Question

我想将经过训练的 VGG19 模型的前 9 层与 TimeDistributed 层结合使用。但我收到一个 InvalidArgumentError。

def build_vgg(in_shape):
    vgg = VGG19(weights="imagenet")
    vgg.outputs = [vgg.layers[9].output]
    img = keras.Input(in_shape)
    img_features = vgg(img)
    return keras.Model(img, img_features)

vggmodel = build_vgg((50,50,3))
input_layer = keras.Input(batch_shape=(10,10,50,50,3))
h2 = keras.layers.wrappers.TimeDistributed(vggmodel)(input_layer)
model = keras.Model(input_layer,h2)
model.summary()

---

我收到此错误:

InvalidArgumentError                      Traceback (most recent call last)
~/.conda/envs/py3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in _create_c_op(graph, node_def, inputs, control_inputs)
   1566   try:
-> 1567     c_op = c_api.TF_FinishOperation(op_desc)
   1568   except errors.InvalidArgumentError as e:

InvalidArgumentError: Dimensions must be equal, but are 512 and 25088 for 'time_distributed_1/vgg19/fc1/MatMul' (op: 'MatMul') with input shapes: [10,512], [25088,4096].

Answer 1

首先，您的模型不应在 build_vgg 中使用额外的输入。您应该只采用您想要的张量。

其次，您应该使用兼容的输入形状。

第三，如果您要更改输入形状并加载 imagenet 权重，则不能包含 top:

def build_vgg(in_shape):
    vgg = VGG19(weights="imagenet", input_shape= in_shape, include_top = False)
    outputs = vgg.layers[9].output

    return keras.Model(vgg.input, outputs)

然后剩下的

vggmodel = build_vgg((50,50,3))
#vggmodel.summary()
input_layer = keras.Input(batch_shape=(10,10,50,50,3))
h2 = keras.layers.wrappers.TimeDistributed(vggmodel)(input_layer)
model = keras.Model(input_layer,h2)
model.summary()

model.predict(np.ones((10,10,50,50,3)))