我已创建从 AWS S3 到 Google BigQuery 的 BigQuery 传输。它失败并出现以下错误,
No new files found matching "gs://bqdts-amazon_s3-prod-eu-w5jetqct8ohvcjih85apf7gvkbibvbkcj9o6l67/test/files"
但是,数据已成功从 S3 移动到 Google Cloud
Moving data from Amazon S3 to Google Cloud complete: Moved 10 object(s).
我还在 BigQuery 数据集中创建了一个表
请帮我解决这个问题
最佳答案
我之前也遇到过同样的问题,只有当我在 Amazon S3 URI 后面添加“*”时才有效。
示例:
之前(显示与您相同的错误): s3://mybucket/路径/
之后(工作成功): s3://mybucket/路径/*
在 General issues Amazon S3 传输页面上的部分显示了以下推荐操作:
Confirm that the Amazon S3 URI in the transfer configuration is correct.
If the transfer configuration was meant to load all files with a common prefix, ensure that the Amazon S3 URI ends with a wildcard. For example, to load all files in s3://my-bucket/my-folder/, the Amazon S3 URI in the transfer configuration must be s3://my-bucket/my-folder/*, not just s3://my-bucket/my-folder/.
关于amazon-web-services - BigQuery Transfer Service 不会从 S3 复制行,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/59563469/