python - 如何使用 Python Backoff 库根据 http 错误代码进行重试？

Question

我正在尝试针对某些异常和特定的 http 错误代码 408 实现重试机制。

这是我当前的代码片段:

@backoff.on_exception(
    backoff.expo,
    (requests.exceptions.ReadTimeout, requests.exceptions.ConnectTimeout),
    max_tries=MAX_TIMEOUT_RETRY_COUNT)
def get_response_http_post(data, params, headers, url) -> dict:
    """ Returns the response as json for a http post request """
    response_dict = {}
    try:
        response = requests.get(url, data=data, params=params, headers=headers, timeout=REQUEST_TIMEOUT)
        response.raise_for_status()
        if response.text:
            response_dict = json.loads(response.text)
        if response.status_code:
            response_dict['status_code'] = response.status_code
        if response.reason:
            response_dict['reason'] = response.reason
        return response_dict
    except requests.exceptions.ReadTimeout as err:
        logger.exception(f'ReadTimeout: {err}')
        raise requests.exceptions.ReadTimeout
    except requests.exceptions.ConnectTimeout as err:
        logger.exception(f'ConnectTimeout: {err}')
        raise requests.exceptions.ConnectTimeout
    except requests.exceptions.ConnectionError as err:
        logger.exception(f'ConnectionError: {err}')
        raise requests.exceptions.ConnectionError
    except requests.exceptions.RequestException as err:
        logger.exception(f'Exception occurred while sending http request: {err}')
        raise requests.exceptions.RequestException

正如您所看到的，我正在重试两种类型的异常请求，并且我还需要对 http 代码 408 执行相同的操作。

如何使用相同的 backoff 库来实现它？或者还有其他办法吗？

我注意到该库有一个名为 @backoff.on_predicate 装饰器的东西，但不确定如何将它用于此用例。如有任何帮助，我们将不胜感激。

Answer 1

尝试使用giveup参数(关键字参数):

def fatal_code(e):
    return 400 <= e.response.status_code < 500

@backoff.on_exception(backoff.expo,
                      requests.exceptions.RequestException,
                      max_time=300,
                      giveup=fatal_code)
def get_url(url):
    return requests.get(url)

https://github.com/litl/backoff