python - 如何将 tensorflow 集线器模块保存到自定义路径/从自定义路径加载？

Question

tensorflow_hub库维护人员使用户可以轻松下载和使用预先训练的 tensorflow 模块，例如:

import tensorflow_hub as hub

embed = hub.Module("https://tfhub.dev/google/universal-sentence-encoder/1")

但是来自 sys.stderr看起来像是将模块本地保存到临时目录，即

INFO:tensorflow:Using /var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules to cache modules. INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_0:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_0 INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_1:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_1 INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_10:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_10 INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_11:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_11 INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_12:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_12 INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_13:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_13 INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_14:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_14 INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_15:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_15 INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_16:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_16 INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_2:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_2 INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_3:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_3 INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_4:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_4 INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_5:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_5 INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_6:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_6 INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_7:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_7 INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_8:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_8 INFO:tensorflow:Initialize variable module/Embeddings_en/sharded_9:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Embeddings_en/sharded_9 INFO:tensorflow:Initialize variable module/Encoder_en/DNN/ResidualHidden_0/weights:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Encoder_en/DNN/ResidualHidden_0/weights INFO:tensorflow:Initialize variable module/Encoder_en/DNN/ResidualHidden_1/weights:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Encoder_en/DNN/ResidualHidden_1/weights INFO:tensorflow:Initialize variable module/Encoder_en/DNN/ResidualHidden_2/weights:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Encoder_en/DNN/ResidualHidden_2/weights INFO:tensorflow:Initialize variable module/Encoder_en/DNN/ResidualHidden_3/projection:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Encoder_en/DNN/ResidualHidden_3/projection INFO:tensorflow:Initialize variable module/Encoder_en/DNN/ResidualHidden_3/weights:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with Encoder_en/DNN/ResidualHidden_3/weights INFO:tensorflow:Initialize variable module/SHARED_RANK_ANSWER/response_encoder_0/tanh_layer_0/bias:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with SHARED_RANK_ANSWER/response_encoder_0/tanh_layer_0/bias INFO:tensorflow:Initialize variable module/SHARED_RANK_ANSWER/response_encoder_0/tanh_layer_0/weights:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with SHARED_RANK_ANSWER/response_encoder_0/tanh_layer_0/weights INFO:tensorflow:Initialize variable module/SHARED_RANK_ANSWER/response_encoder_0/tanh_layer_1/bias:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with SHARED_RANK_ANSWER/response_encoder_0/tanh_layer_1/bias INFO:tensorflow:Initialize variable module/SHARED_RANK_ANSWER/response_encoder_0/tanh_layer_1/weights:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with SHARED_RANK_ANSWER/response_encoder_0/tanh_layer_1/weights INFO:tensorflow:Initialize variable module/SHARED_RANK_ANSWER/response_encoder_0/tanh_layer_2/bias:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with SHARED_RANK_ANSWER/response_encoder_0/tanh_layer_2/bias INFO:tensorflow:Initialize variable module/SHARED_RANK_ANSWER/response_encoder_0/tanh_layer_2/weights:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with SHARED_RANK_ANSWER/response_encoder_0/tanh_layer_2/weights INFO:tensorflow:Initialize variable module/SNLI/Classifier/LinearLayer/bias:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with SNLI/Classifier/LinearLayer/bias INFO:tensorflow:Initialize variable module/SNLI/Classifier/LinearLayer/weights:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with SNLI/Classifier/LinearLayer/weights INFO:tensorflow:Initialize variable module/SNLI/Classifier/tanh_layer_0/bias:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with SNLI/Classifier/tanh_layer_0/bias INFO:tensorflow:Initialize variable module/SNLI/Classifier/tanh_layer_0/weights:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with SNLI/Classifier/tanh_layer_0/weights INFO:tensorflow:Initialize variable module/global_step:0 from checkpoint b'/var/folders/j6/xczfl75n3sbfwpg4190gpb104vnlxt/T/tfhub_modules/c6f5954ffa065cdb2f2e604e740e8838bf21a2d3/variables/variables' with global_step

机器重新启动后，该模块将被删除并运行 hub.Module('...')再次代码将重新下载模块。

是否可以将模块保存到自定义目录，然后从自定义目录加载？

如果可能，如何将tensorflow hub模块保存到自定义路径/从自定义路径加载？

Answer 1

您可以从 url + '?tf-hub-format=compressed' 下载您需要的模型

我尝试下载 elmo 并且成功了

网址=https://tfhub.dev/google/elmo/2 + '?tf-hub-format=压缩'

例如:https://tfhub.dev/google/elmo/2?tf-hub-format=compressed

模型将以 tar 文件的形式下载到您的计算机上。

解压该文件后，它将包含 tfhub_module.pb