amazon-web-services - 亚马逊网络服务 S3 请求限制

Question

我正在使用 AWS 运行一些数据处理。我在 EC2 中有 400 个 Spot 实例，每个实例有 4 个进程，它们都写入 S3 中的单个存储桶。我开始收到一个(显然不常见的)错误说:

503: Slow Down

有谁知道 S3 存储桶的实际请求限制是多少？我找不到任何关于它的 AWS 文档。

谢谢!

Answer 1

由于临时错误，AWS 记录了 503。它不反射(reflect)特定限制。
根据有关处理错误的“使用 Amazon S3 的最佳实践”部分 ( http://aws.amazon.com/articles/1904/ ):

500-series errors indicate that a request didn't succeed, but may be retried. Though infrequent, these errors are to be expected as part of normal interaction with the service and should be explicitly handled with an exponential backoff algorithm (ideally one that utilizes jitter). One such algorithm can be found at http://en.wikipedia.org/wiki/Truncated_binary_exponential_backoff.

Particularly if you suddenly begin executing hundreds of PUTs per second into a single bucket, you may find that some requests return a 503 "Slow Down" error while the service works to repartition the load. As with all 500 series errors, these should be handled with exponential backoff.

虽然不太详细，但 S3 错误响应文档确实包含 503 Slow Down ( http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html )。