我正在尝试解决问题。
我正在启动一个 Boto3 客户端，如下所示:
s3_client = boto3.client('s3')
然后我迭代许多文件并使用以下方式上传它们:
s3_client.upload_file()
我的问题是，我时不时地看到它暂停 60 秒，然后正常继续。
我尝试减少超时值，但我似乎得到的错误是:
botocore.exceptions.ConnectTimeoutError
或
urllib3.exceptions.ConnectTimeoutError
如果我将其保留在 60，最终它会成功并且不会出现任何错误。
我的问题是，这个错误意味着在执行upload_file时，它尝试向服务器发送put请求，但服务器没有响应该请求？
启动客户端时，是否建立了任何可能丢失的连接，或者只是存储凭据而与此问题无关？
非常感谢。
更新:添加更详细的日志:
CRITICAL Connect timeout
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 169, in _new_conn
conn = connection.create_connection(
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 96, in create_connection
raise err
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 86, in create_connection
sock.connect(sa)
TimeoutError: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/.local/lib/python3.10/site-packages/botocore/httpsession.py", line 439, in send
urllib_response = conn.urlopen(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 507, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
raise value
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 1012, in _validate_conn
conn.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 353, in connect
conn = self._new_conn()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 174, in _new_conn
raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError
During handling of the above exception, another exception occurred:
File "/home/user/.local/lib/python3.10/site-packages/boto3/s3/inject.py", line 143, in upload_file
return transfer.upload_file(
File "/home/user/.local/lib/python3.10/site-packages/boto3/s3/transfer.py", line 288, in upload_file
future.result()
File "/home/user/.local/lib/python3.10/site-packages/s3transfer/futures.py", line 103, in result
return self._coordinator.result()
File "/home/user/.local/lib/python3.10/site-packages/s3transfer/futures.py", line 266, in result
raise self._exception
File "/home/user/.local/lib/python3.10/site-packages/s3transfer/tasks.py", line 139, in __call__
return self._execute_main(kwargs)
File "/home/user/.local/lib/python3.10/site-packages/s3transfer/tasks.py", line 162, in _execute_main
return_value = self._main(**kwargs)
File "/home/user/.local/lib/python3.10/site-packages/s3transfer/upload.py", line 758, in _main
client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
File "/home/user/.local/lib/python3.10/site-packages/botocore/client.py", line 415, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/home/user/.local/lib/python3.10/site-packages/botocore/client.py", line 731, in _make_api_call
http, parsed_response = self._make_request(
File "/home/user/.local/lib/python3.10/site-packages/botocore/client.py", line 751, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/home/user/.local/lib/python3.10/site-packages/botocore/endpoint.py", line 107, in make_request
return self._send_request(request_dict, operation_model)
File "/home/user/.local/lib/python3.10/site-packages/botocore/endpoint.py", line 183, in _send_request
while self._needs_retry(attempts, operation_model, request_dict,
File "/home/user/.local/lib/python3.10/site-packages/botocore/endpoint.py", line 305, in _needs_retry
responses = self._event_emitter.emit(
File "/home/user/.local/lib/python3.10/site-packages/botocore/hooks.py", line 358, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/home/user/.local/lib/python3.10/site-packages/botocore/hooks.py", line 229, in emit
return self._emit(event_name, kwargs)
File "/home/user/.local/lib/python3.10/site-packages/botocore/hooks.py", line 212, in _emit
response = handler(**kwargs)
File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 194, in __call__
if self._checker(**checker_kwargs):
File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 267, in __call__
should_retry = self._should_retry(attempt_number, response,
File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 294, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 333, in __call__
checker_response = checker(attempt_number, response,
File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 233, in __call__
return self._check_caught_exception(
File "/home/user/.local/lib/python3.10/site-packages/botocore/retryhandler.py", line 376, in _check_caught_exception
raise caught_exception
File "/home/user/.local/lib/python3.10/site-packages/botocore/endpoint.py", line 249, in _do_get_response
http_response = self._send(request)
File "/home/user/.local/lib/python3.10/site-packages/botocore/endpoint.py", line 321, in _send
return self.http_session.send(request)
File "/home/user/.local/lib/python3.10/site-packages/botocore/httpsession.py", line 472, in send
raise ConnectTimeoutError(endpoint_url=request.url, error=e)
最佳答案
默认的 boto3 重试模式(旧版)无法正确处理许多错误/异常，包括 RequestThrottled、PriorRequestNotComplete、ConnectionError和BandwidthLimitExceeded。它可能表现为间歇性问题，伴有长时间延迟和重复的连接失败。使用 Standard retry mode 可能会更好相反，它也有指数退避。
import boto3
from botocore.client import Config as BotoConfig
TIMEOUT = 3
config = BotoConfig(connect_timeout=TIMEOUT, retries={"mode": "standard"})
client = boto3.client("s3", config=config)
关于python - Boto3 超时故障排除，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/72622237/