python appengine-gcs-client 演示，本地开发服务器遇到 AccessTokenRefreshError(u'internal_failure',)

Question

我在使用 1.9.40(最新)SDK 的 dev_appserver.py 来获取 python appengine-gcs-client 演示时遇到问题。

我关注了Setting Up Google Cloud Storage和 App Engine and Google Cloud Storage Sample说明。

我为付费应用创建了默认存储桶，启用了计费功能并设置了非零的每日支出限额。我使用开发者控制台成功将文件上传到该存储桶。

我克隆了 GoogleCloudPlatform/appengine-gcs-client来自 github 的仓库。我将 python/src/cloudstorage 目录复制到 python/demo 目录中，现在看起来像这样:

dancorn-laptop.acasa:/home/dancorn/src/appengine-gcs-client/python> find demo/ | sort
demo/
demo/app.yaml
demo/blobstore.py
demo/cloudstorage
demo/cloudstorage/api_utils.py
demo/cloudstorage/api_utils.pyc
demo/cloudstorage/cloudstorage_api.py
demo/cloudstorage/cloudstorage_api.pyc
demo/cloudstorage/common.py
demo/cloudstorage/common.pyc
demo/cloudstorage/errors.py
demo/cloudstorage/errors.pyc
demo/cloudstorage/__init__.py
demo/cloudstorage/__init__.pyc
demo/cloudstorage/rest_api.py
demo/cloudstorage/rest_api.pyc
demo/cloudstorage/storage_api.py
demo/cloudstorage/storage_api.pyc
demo/cloudstorage/test_utils.py
demo/__init__.py
demo/main.py
demo/main.pyc
demo/README

这就是我执行devserver的方式以及尝试访问http://localhost:8080时报告的错误按照指示:

dancorn-laptop.acasa:/home/dancorn/src/appengine-gcs-client/python> /home/usr_local/google_appengine_1.9.40/dev_appserver.py demo
INFO     2016-08-04 01:07:51,786 sdk_update_checker.py:229] Checking for updates to the SDK.
INFO     2016-08-04 01:07:51,982 sdk_update_checker.py:257] The SDK is up to date.
INFO     2016-08-04 01:07:52,121 api_server.py:205] Starting API server at: http://localhost:50355
INFO     2016-08-04 01:07:52,123 dispatcher.py:197] Starting module "default" running at: http://localhost:8080
INFO     2016-08-04 01:07:52,124 admin_server.py:116] Starting admin server at: http://localhost:8000
INFO     2016-08-04 01:08:03,461 client.py:804] Refreshing access_token
INFO     2016-08-04 01:08:05,234 client.py:827] Failed to retrieve access token: {
  "error" : "internal_failure"
}
ERROR    2016-08-04 01:08:05,236 api_server.py:272] Exception while handling service_name: "app_identity_service"
method: "GetAccessToken"
request: "\n7https://www.googleapis.com/auth/devstorage.full_control"
request_id: "ccqdTObLrl"

Traceback (most recent call last):
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/tools/devappserver2/api_server.py", line 247, in _handle_POST
    api_response = _execute_request(request).Encode()
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/tools/devappserver2/api_server.py", line 186, in _execute_request
    make_request()
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/tools/devappserver2/api_server.py", line 181, in make_request
    request_id)
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/api/apiproxy_stub.py", line 131, in MakeSyncCall
    method(request, response)
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/api/app_identity/app_identity_defaultcredentialsbased_stub.py", line 192, in _Dynamic_GetAccessToken
    token = credentials.get_access_token()
  File "/home/usr_local/google_appengine_1.9.40/lib/oauth2client/oauth2client/client.py", line 689, in get_access_token
    self.refresh(http)
  File "/home/usr_local/google_appengine_1.9.40/lib/oauth2client/oauth2client/client.py", line 604, in refresh
    self._refresh(http.request)
  File "/home/usr_local/google_appengine_1.9.40/lib/oauth2client/oauth2client/client.py", line 775, in _refresh
    self._do_refresh_request(http_request)
  File "/home/usr_local/google_appengine_1.9.40/lib/oauth2client/oauth2client/client.py", line 840, in _do_refresh_request
    raise AccessTokenRefreshError(error_msg)
AccessTokenRefreshError: internal_failure

WARNING  2016-08-04 01:08:05,239 tasklets.py:468] suspended generator _make_token_async(rest_api.py:55) raised RuntimeError(AccessTokenRefreshError(u'internal_failure',))
WARNING  2016-08-04 01:08:05,240 tasklets.py:468] suspended generator get_token_async(rest_api.py:224) raised RuntimeError(AccessTokenRefreshError(u'internal_failure',))
WARNING  2016-08-04 01:08:05,240 tasklets.py:468] suspended generator urlfetch_async(rest_api.py:259) raised RuntimeError(AccessTokenRefreshError(u'internal_failure',))
WARNING  2016-08-04 01:08:05,240 tasklets.py:468] suspended generator run(api_utils.py:164) raised RuntimeError(AccessTokenRefreshError(u'internal_failure',))
WARNING  2016-08-04 01:08:05,240 tasklets.py:468] suspended generator do_request_async(rest_api.py:198) raised RuntimeError(AccessTokenRefreshError(u'internal_failure',))
WARNING  2016-08-04 01:08:05,241 tasklets.py:468] suspended generator do_request_async(storage_api.py:128) raised RuntimeError(AccessTokenRefreshError(u'internal_failure',))
ERROR    2016-08-04 01:08:05,241 main.py:62] AccessTokenRefreshError(u'internal_failure',)
Traceback (most recent call last):
  File "/home/dancorn/src/appengine-gcs-client/python/demo/main.py", line 43, in get
    self.create_file(filename)
  File "/home/dancorn/src/appengine-gcs-client/python/demo/main.py", line 89, in create_file
    retry_params=write_retry_params)
  File "/home/dancorn/src/appengine-gcs-client/python/demo/cloudstorage/cloudstorage_api.py", line 97, in open
    return storage_api.StreamingBuffer(api, filename, content_type, options)
  File "/home/dancorn/src/appengine-gcs-client/python/demo/cloudstorage/storage_api.py", line 697, in __init__
    status, resp_headers, content = self._api.post_object(path, headers=headers)
  File "/home/dancorn/src/appengine-gcs-client/python/demo/cloudstorage/rest_api.py", line 82, in sync_wrapper
    return future.get_result()
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
    self.check_success()
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/home/dancorn/src/appengine-gcs-client/python/demo/cloudstorage/storage_api.py", line 128, in do_request_async
    deadline=deadline, callback=callback)
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/home/dancorn/src/appengine-gcs-client/python/demo/cloudstorage/rest_api.py", line 198, in do_request_async
    follow_redirects=False)
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/home/dancorn/src/appengine-gcs-client/python/demo/cloudstorage/api_utils.py", line 164, in run
    result = yield tasklet(**kwds)
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/home/dancorn/src/appengine-gcs-client/python/demo/cloudstorage/rest_api.py", line 259, in urlfetch_async
    self.token = yield self.get_token_async()
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/home/dancorn/src/appengine-gcs-client/python/demo/cloudstorage/rest_api.py", line 224, in get_token_async
    self.scopes, self.service_account_id)
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/home/dancorn/src/appengine-gcs-client/python/demo/cloudstorage/rest_api.py", line 55, in _make_token_async
    token, expires_at = yield rpc
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/ext/ndb/tasklets.py", line 513, in _on_rpc_completion
    result = rpc.get_result()
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
    return self.__get_result_hook(self)
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/api/app_identity/app_identity.py", line 519, in get_access_token_result
    rpc.check_success()
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/api/apiproxy_stub_map.py", line 579, in check_success
    self.__rpc.CheckSuccess()
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/api/apiproxy_rpc.py", line 157, in _WaitImpl
    self.request, self.response)
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/ext/remote_api/remote_api_stub.py", line 201, in MakeSyncCall
    self._MakeRealSyncCall(service, call, request, response)
  File "/home/usr_local/google_appengine_1.9.40/google/appengine/ext/remote_api/remote_api_stub.py", line 235, in _MakeRealSyncCall
    raise pickle.loads(response_pb.exception())
RuntimeError: AccessTokenRefreshError(u'internal_failure',)
INFO     2016-08-04 01:08:05,255 module.py:788] default: "GET / HTTP/1.1" 200 249

当我看到尝试联系 Google 服务器时，我感到很惊讶，我本以为会使用伪造的、基于本地文件系统的模拟，基于 App Engine and Google Cloud Storage Sample 中的这些注释。说明:

• Using the client library with the development app server :

You can use the client library with the development server.

**Note**: Files saved locally are subject to the file size and naming conventions imposed by the local filesystem.

• app.yaml walkthrough :

You specify the project ID in the line application: your-app-id, replacing the value your-app-id. This value isn't used when running locally, but you must supply a valid project ID before deploying: the deployment utility reads this entry to determine where to deploy your app.

• Deploying the Sample ，第5步:

In your browser, visit https://.appspot.com; the application will execute on page load, just as it did when running locally. Only this time, the app will actually be writing to and reading from a real bucket.

我什至将真实应用的 ID 放入 app.yaml 文件中，但这没有任何区别。

我检查了已知的 GAE 问题，只发现了这个可能相关的问题，但是是在更旧的 SDK 版本上:

• Issue 11690 GoogleAppEngineLanucher 开发服务器中的 GloudStorage 错误

我检查了一些旧的 SDK 版本(1.9.30、1.9.35)，以防万一 - 也没有区别。

我的问题:

1. 当 GCS 客户端与 dev_appserver.py 一起使用时，如何使 GCS 客户端在本地运行(基于本地文件系统的伪造 GCS)？
2. 既然提到了，即使与 dev_appserver.py 一起使用，它也应该与真正的 GCS 一起工作，我需要做什么才能实现这一目标？ (不太重要，更多的是好奇)

Answer 1

实际上，原因是恕我直言，这是一个非常愚蠢的错误 - 无法从早期版本的 SDK(或相关包？)编写的本地文件中读取凭据，并且无法回退到更合适的操作，从而导致一个相当具有误导性的回溯，导致调查失败。

归功于这个答案:https://stackoverflow.com/a/35890078/4495081 (“尽管帖子中提到的错误是为了其他原因，最终触发了类似的最终结果)

删除 ~/.config/gcloud/application_default_credentias.json 文件后，演示使用本地文件系统成功完成。我真正的应用程序也运行良好。

我的第二个问题仍然存在，但我不太担心 - 就我个人而言，我认为在本地开发服务器上使用真正的 GCS 存储没有太大值(value) - 我必须在真正的登台 GAE 应用程序上进行测试无论如何，由于其他原因。