c++ - CMake 测试在测试失败时导致错误

Question

我一直在四处寻找解决方案，但似乎无法找出发生这种情况的原因。我将 CMake 用于一个小项目，我想添加单元测试 cmakes add_test。它在测试通过时运行良好，但在测试失败时，我会收到一条我无法解释的奇怪错误消息。

我做了一个最小的例子来说明会发生什么。

CMakeLists.txt

cmake_minimum_required (VERSION 2.6)

project (TestExample)

enable_testing()

add_executable(succeedExample succeed.cpp)
add_executable(failExample fail.cpp)

add_test(NAME succeedTest COMMAND succeedExample)
add_test(NAME failTest COMMAND failExample)

成功.cpp

int main()
{
    return 0;
}

失败.cpp

int main()
{
    return 1;
}

我使用 Visual Studio 2015，在构建 RUN_TESTS 目标时，出现以下错误:

Severity    Code    Description Project File    Line    Suppression State
Error   MSB3073 The command "setlocal
"C:\Program Files (x86)\CMake\bin\ctest.exe" --force-new-ctest-process -C Debug
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
:VCEnd" exited with code 8. RUN_TESTS   C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets  133 

然而，如果我将 fail.cpp 更改为 return 0; 我得到

1>------ Build started: Project: RUN_TESTS, Configuration: Debug Win32 ------
1>  Test project C:/Users/.../projects/cmaketestexample/build
1>      Start 1: succeedTest
1>  1/2 Test #1: succeedTest ......................   Passed    0.01 sec
1>      Start 2: failTest
1>  2/2 Test #2: failTest .........................   Passed    0.02 sec
1>
1>  100% tests passed, 0 tests failed out of 2
1>
1>  Total Test time (real) =   0.03 sec
========== Build: 1 succeeded, 0 failed, 1 up-to-date, 0 skipped ==========

任何人都可以指出解决此问题的解决方案吗？我是否遗漏了一些明显的东西？

谢谢:)

编辑

Visual Studio 版本为 14.0(Visual Studio 2015 社区)

Cmake 3.5.2 - 但我在 CMakeLists.txt 中将版本设置为 2.6。

EDIT2

在命令行上运行 ctest -C Debug:

Test project C:/Users/.../projects/cmaketestexample/build
    Start 1: succeedTest
1/2 Test #1: succeedTest ......................   Passed    0.02 sec
    Start 2: failTest
2/2 Test #2: failTest .........................***Failed    0.02 sec

50% tests passed, 1 tests failed out of 2

Total Test time (real) =   0.07 sec

The following tests FAILED:
          2 - failTest (Failed)
Errors while running CTest

将 fail.cpp 更改为

#include <iostream>
int main()
{
    std::cout << "Does it work now?" << std::endl;
    return 1;
}

没有区别。

EDIT3

在 Ubuntu 14.04.4 LTS 上出现了类似的错误 使用

• cmake 版本 3.2.2
• GNU Make 3.81
• g++ (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4

Answer 1

我已经为您提供了代码尝试，您看到的是 CMake 将 ctest 调用嵌入到的小脚本。并且 ctest 确实失败了——返回一个非零值——如果它正在执行的任何测试确实失败了。 ctest 的这种行为是故意在遇到错误时停止运行测试的任何脚本/构建。

可能的 error return codes ctest 是以下位的组合:

// provide some more detailed info on the return code for ctest
enum {
  UPDATE_ERRORS    = 0x01,
  CONFIGURE_ERRORS = 0x02,
  BUILD_ERRORS     = 0x04,
  TEST_ERRORS      = 0x08,
  MEMORY_ERRORS    = 0x10,
  COVERAGE_ERRORS  = 0x20,
  SUBMIT_ERRORS    = 0x40
};

所以我看到了以下选项:

1. 忽略奇怪的错误，你要找的输出就在错误信息的上面:

   1>------ Build started: Project: RUN_TESTS, Configuration: Debug Win32 ------
   1>  Test project C:/temp/StackOverflow/TestError/VS2015
   1>      Start 1: succeedTest
   1>  1/2 Test #1: succeedTest ......................   Passed    0.03 sec
   1>      Start 2: failTest
   1>  2/2 Test #2: failTest .........................***Failed    0.03 sec
   1>
   1>  50% tests passed, 1 tests failed out of 2
   1>
   1>  Total Test time (real) =   0.10 sec
   1>
   1>  The following tests FAILED:
   1>        2 - failTest (Failed)
   1>  Errors while running CTest
   1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(133,5): error MSB3073: The command "setlocal
   1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(133,5): error MSB3073: "C:\Program Files\CMake\bin\ctest.exe" --force-new-ctest-process -C Debug
   1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(133,5): error MSB3073: if %errorlevel% neq 0 goto :cmEnd
   1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(133,5): error MSB3073: :cmEnd
   1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(133,5): error MSB3073: endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
   1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(133,5): error MSB3073: :cmErrorLevel
   1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(133,5): error MSB3073: exit /b %1
   1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(133,5): error MSB3073: :cmDone
   1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(133,5): error MSB3073: if %errorlevel% neq 0 goto :VCEnd
   1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.CppCommon.targets(133,5): error MSB3073: :VCEnd" exited with code 8.
   ========== Build: 0 succeeded, 1 failed, 1 up-to-date, 0 skipped ==========
   

2. 在没有脚本的情况下在 VS 上添加您自己的自定义目标调用 ctest，仅给出一个错误:

   if (CMAKE_CONFIGURATION_TYPES)
       add_custom_target(
           RUN_CTEST
               COMMAND ${CMAKE_CTEST_COMMAND} -C $<CONFIGURATION> --force-new-ctest-process 
               WORKING_DIRECTORY "${CMAKE_BINARY_DIR}"
       )
   endif()
   

3. 通过将 ctest 调用移动到单独的脚本中(请参阅引用资料)，添加您自己的忽略返回值的自定义目标命令。

引用资料

• How can I ignore errors on an individual CMake command in my CMakeLists.txt?
• CMake: setting an environmental variable for ctest (or otherwise getting failed test output from ctest/make test automatically)