python-3.x - 由于调用者没有权限，从 S3 到 GCS 的 Google StorageTransfer 作业失败

Question

我尝试使用 API 将文件从 Amazon S3 传输到 Google Cloud Storage。我在 GCP 控制台上成功完成，但是当我使用服务帐户凭据编写 Python 脚本时，我收到了 HttpError 403 when requesting https://storagetransfer.googleapis.com/v1/transferJobs?alt=json returned "The caller does not have permission
此外，我的服务帐户已经是存储对象管理员。

我正在使用的脚本

import google.auth
import google_auth_httplib2
import google.oauth2.service_account
from googleapiclient.discovery import build


_DEFAULT_SCOPES = ('https://www.googleapis.com/auth/cloud-platform',)

credentials = (
                    google.oauth2.service_account.Credentials.from_service_account_file(
                        key_path, scopes=_DEFAULT_SCOPES)
                )

http = httplib2.Http()
http_authorized = google_auth_httplib2.AuthorizedHttp(
            credentials, http=http)

conn = build('storagetransfer', 'v1', http=http_authorized, cache_discovery=False)
now = datetime.datetime.utcnow()

project_id='projectid'
transfer_spec = {
    'awsS3DataSource': {
        'bucketName': 's3_bucket',
        'awsAccessKey': {
            'accessKeyId': 'key',
            'secretAccessKey': 'secret',
        }
    },
    'gcsDataSink': {
        'bucketName': 'bucket',
    },
    'objectConditions': {'includePrefixes': ['PREFIX']},
    'transferOptions': {'overwriteObjectsAlreadyExistingInSink': True},
}

transfer_job = {
            'status': 'ENABLED',
            'projectId': project_id,
            'transferSpec': transfer_spec,
            'schedule': {
                'scheduleStartDate': {
                    'day': now.day,
                    'month': now.month,
                    'year': now.year,
                },
                'scheduleEndDate': {
                    'day': now.day,
                    'month': now.month,
                    'year': now.year,
                }
            }
        }

job = conn.transferJobs().create(body=transfer_job).execute()

有谁知道缺少什么？

文档:https://cloud.google.com/storage-transfer/docs/reference/rest/v1/transferJobs/create

Answer 1

根据文档 here您需要分配以下权限 owner/editor/storagetransfer.admin/storagetransfer.user 之一。还有一个 python 脚本示例可能有助于将数据从 S3 传输到 GCS。