tensorflow - 在简单的 colab 教程中使用 ELMo 交换 gnews

Question

我正在研究这个 colab 笔记本:

https://colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/tf2_text_classification.ipynb

我想用 ELMo 嵌入替换 gnews 旋转嵌入。

所以，更换

model = "https://tfhub.dev/google/tf2-preview/gnews-swivel-20dim/1"

和:

model = "https://tfhub.dev/google/elmo/2"

这里有一连串的变化，比如需要

tf.compat.v1.disable_eager_execution()

但我不了解成功进行此替换所需的图形形状。具体来说，我看到了。

#model = "https://tfhub.dev/google/tf2-preview/gnews-swivel-20dim/1"
model = "https://tfhub.dev/google/elmo/2"

elmo = hub.Module(model, trainable=True, name="{}_module".format("mymod"))

hub_layer = hub.KerasLayer(elmo, 
#                           output_shape=[3,20], 
#                           input_shape=(1,),
                           dtype=tf.string, 
                           trainable=True)

hub_layer(train_examples[:3])

生产

<tf.Tensor 'keras_layer_14/mymod_module_14_apply_default/truediv:0' shape=(3, 1024) dtype=float32>

这似乎很好。但:

model = tf.keras.Sequential()
model.add(hub_layer)
model.add(tf.keras.layers.Dense(16, activation='relu'))
model.add(tf.keras.layers.Dense(1, activation='sigmoid'))

# First, I have to build, because I no longer have eager executon.
model.build(input_shape=(None,1024))

model.summary()

然后这给出:

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

<ipython-input-54-8786753617e4> in <module>()
      4 model.add(tf.keras.layers.Dense(1, activation='sigmoid'))
      5 
----> 6 model.build(input_shape=(None,1024))
      7 
      8 model.summary()

18 frames

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in internal_convert_to_tensor_or_indexed_slices(value, dtype, name, as_ref)
   1381       raise ValueError(
   1382           "Tensor conversion requested dtype %s for Tensor with dtype %s: %r" %
-> 1383           (dtypes.as_dtype(dtype).name, value.dtype.name, str(value)))
   1384     return value
   1385   else:

ValueError: Tensor conversion requested dtype string for Tensor with dtype float32: 'Tensor("Placeholder_12:0", shape=(None, 1024), dtype=float32)'

图表尺寸还有什么变化，我该如何解决？

Answer 1

问题是 Keras 假设输入为 float32 :

conversion requested dtype string for Tensor with dtype float32

由于名称为“Placeholder_12:0”，您可以看出这是输入。占位符张量用于将数据输入模型。

型号hub_layer需要一个字符串输入，所以您需要做的就是添加一个 Input指定的层

model = tf.keras.Sequential()
#add an input layer
model.add(tf.keras.layers.Input(shape=tuple(),dtype=tf.string))
model.add(hub_layer)
model.add(tf.keras.layers.Dense(16, activation='relu'))
model.add(tf.keras.layers.Dense(1, activation='sigmoid'))

model.build(input_shape=(None,1024))

model.summary()

结果是:

Model: "sequential_4"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
keras_layer (KerasLayer)     (None, 1024)              93600852  
_________________________________________________________________
dense (Dense)                (None, 16)                16400     
_________________________________________________________________
dense_1 (Dense)              (None, 1)                 17        
=================================================================
Total params: 93,617,269
Trainable params: 16,417
Non-trainable params: 93,600,852
_________________________________________________________________

通过您的修改和上述修改，我能够使用 colab 笔记本进行训练。