python - 记录由 tensorflow 服务模型服务的请求

Question

我已经使用 tesnorflow 服务构建了一个模型，并使用以下命令在服务器上运行它:-

bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server --port=9009 --model_name=ETA_DNN_Regressor --model_base_path=//apps/node-apps/tensorflow-models-repository/ETA

但现在此屏幕停滞不前，没有提供有关传入请求和响应的任何信息。 我尝试使用 TF_CPP_MIN_VLOG_LEVEL=1 标志。但现在它提供了如此多的输出，但仍然没有记录/监控传入的请求/响应。

请建议如何查看这些日志。

我面临的第二个问题是如何在后台运行这个进程并持续监控它。假设我关闭了控制台，那么这个进程也应该正在运行，以及如何再次重新连接该进程控制台并查看实时流量。

任何建议都会有所帮助。

Answer 1

对于基本的 HTTP 请求日志记录，您可以设置 TF_CPP_VMODULE=http_server=1 来为模块 http_server.cc 设置 VLOG 级别——这将为您提供一个非常简单的请求日志，显示传入的请求和一些基本的错误情况:

2020-08-26 10:42:47.225542: I tensorflow_serving/model_servers/http_server.cc:156] Processing HTTP request: POST /v1/models/mymodel:predict body: 761 bytes.
2020-08-26 10:44:32.472497: I tensorflow_serving/model_servers/http_server.cc:139] Ignoring HTTP request: GET /someboguspath
2020-08-26 10:51:36.540963: I tensorflow_serving/model_servers/http_server.cc:156] Processing HTTP request: GET /v1/someboguspath body: 0 bytes.
2020-08-26 10:51:36.541012: I tensorflow_serving/model_servers/http_server.cc:168] Error Processing HTTP/REST request: GET /v1/someboguspath Error: Invalid argument: Malformed request: GET /v1/someboguspath
2020-08-26 10:53:17.039291: I tensorflow_serving/model_servers/http_server.cc:156] Processing HTTP request: GET /v1/models/someboguspath body: 0 bytes.
2020-08-26 10:53:17.039456: I tensorflow_serving/model_servers/http_server.cc:168] Error Processing HTTP/REST request: GET /v1/models/someboguspath Error: Not found: Could not find any versions of model someboguspath
2020-08-26 11:01:43.466636: I tensorflow_serving/model_servers/http_server.cc:156] Processing HTTP request: POST /v1/models/mymodel:predict body: 755 bytes.
2020-08-26 11:01:43.473195: I tensorflow_serving/model_servers/http_server.cc:168] Error Processing HTTP/REST request: POST /v1/models/mymodel:predict Error: Invalid argument: Incompatible shapes: [1,38,768] vs. [1,40,768]
     [[{{node model/transformer/embeddings/add}}]]
2020-08-26 11:02:56.435942: I tensorflow_serving/model_servers/http_server.cc:156] Processing HTTP request: POST /v1/models/mymodel:predict body: 754 bytes.
2020-08-26 11:02:56.436762: I tensorflow_serving/model_servers/http_server.cc:168] Error Processing HTTP/REST request: POST /v1/models/mymodel:predict Error: Invalid argument: JSON Parse error: Missing a comma or ']' after an array element. at offset: 61

...您可以浏览 https://github.com/tensorflow/serving/blob/master/tensorflow_serving/model_servers/http_server.cc 中出现的 VLOG(1) << 以查看此模块中的所有日志记录语句。

对于 gRPC 可能有一些相应的模块，您可以类似地为其启用 VLOG — 我没有去寻找它。