python - 如何使用 pytest 只报告某种异常类型的失败？

Question

有没有一种方法可以告诉 pytest 忽略所有失败的测试，但我选择的一种异常类型。

例如，我希望 pytest 只告诉我有关引发 IndexError 的测试，而不告诉我其余的测试。

def test_a():
    raise IndexError()

def test_b():
    1/0 # this test would be ignored

def test_c():
    raise KeyError() # this test would also be ignored

请注意，我有 3000 多个测试，我无法编辑每个测试来添加标记。我只是想找到那些产生 IndexError 的。

Answer 1

Note that I have 3000+ tests and I cannot edit each test to add markers.

您可以动态添加标记。示例:在 conftest.py 中添加以下代码:

# conftest.py
import pytest

def pytest_collection_modifyitems(items):
    xfail_exceptions = (IndexError, KeyError)
    for item in items:
        item.add_marker(pytest.mark.xfail(raises=xfail_exceptions))

现在，所有测试都将自动标记，仅引发 xfail_exceptions 中未列出的异常。

您可以进一步扩展它并从命令行参数化异常:

import importlib
import pytest


def pytest_addoption(parser):
    parser.addoption('--ex', action='append', default=[], help='exception classes to xfail')


def class_from(name):
    modname, clsname = name.rsplit('.', 1)
    mod = importlib.import_module(modname)
    return getattr(mod, clsname)


def pytest_collection_modifyitems(items):
    xfail_exceptions = tuple((class_from(name) for name in pytest.config.getoption('--ex')))
    if xfail_exceptions:
        for item in items:
            item.add_marker(pytest.mark.xfail(raises=xfail_exceptions))

使用示例:

$ pytest --ex builtins.KeyError --ex builtins.IndexError -sv
================================== test session starts ====================================
platform darwin -- Python 3.6.4, pytest-3.7.3, py-1.5.4, pluggy-0.7.1
cachedir: .pytest_cache
rootdir: /Users/hoefling/projects/private/stackoverflow, inifile:
plugins: django-3.4.2
collected 3 items

test_spam.py::test_a xfail
test_spam.py::test_b FAILED
test_spam.py::test_c xfail

======================================== FAILURES =========================================
_________________________________________ test_b __________________________________________

    def test_b():
>       1/0 # this test would be ignored
E       ZeroDivisionError: division by zero

test_spam.py:5: ZeroDivisionError
========================== 1 failed, 2 xfailed in 0.08 seconds ============================