azure-devops - 当所有测试通过时，Azure DevOps Build Pipeline 中的 Chutzpah Jasmine 测试返回失败

Question

我有一个简单的 ASP.Net (MVC) 应用程序，其中包含一个在 Azure DevOps 中设置的单元测试项目。

单元测试项目有一个 chutzpah.json 文件和一个包含四个 jasmine.js 测试的文件。这些测试测试主 ASP.Net 项目中的文件。我安装了用于测试资源管理器的 Chutzpah 测试适配器，并且测试在 Visual Studio 2017 的测试资源管理器中显示、运行和传递。

在 DevOps 中，类似于此 blog ，我向构建管道(默认 ASP.Net 模板)添加了第二个 Visual Studio Test 任务，其中 测试文件 位于 **\chutzpah.json。此任务运行，日志显示测试通过，然后指出“测试运行失败”。这使得任务和构建失败。

输出为

2018-11-08T11:52:17.5347601Z ##[section]Starting: VsTest - Front End
2018-11-08T11:52:17.5354696Z ==============================================================================
2018-11-08T11:52:17.5354757Z Task         : Visual Studio Test
2018-11-08T11:52:17.5354793Z Description  : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
2018-11-08T11:52:17.5354871Z Version      : 2.142.12
2018-11-08T11:52:17.5354908Z Author       : Microsoft Corporation
2018-11-08T11:52:17.5354962Z Help         : [More information](https://go.microsoft.com/fwlink/?LinkId=835764)
2018-11-08T11:52:17.5354999Z ==============================================================================
2018-11-08T11:52:18.2871153Z SystemVssConnection exists true
2018-11-08T11:52:18.3290569Z SystemVssConnection exists true
2018-11-08T11:52:18.4136045Z Running tests using vstest.console.exe runner.
2018-11-08T11:52:18.4136143Z ======================================================
2018-11-08T11:52:18.4136511Z Test selector : Test assemblies
2018-11-08T11:52:18.4136597Z Test filter criteria : null
2018-11-08T11:52:18.4136924Z Search folder : D:\a\1\s
2018-11-08T11:52:18.4137559Z VisualStudio version selected for test execution : latest
2018-11-08T11:52:18.4259721Z Run in parallel : false
2018-11-08T11:52:18.4307979Z Run in isolation : false
2018-11-08T11:52:18.4309258Z Path to custom adapters : null
2018-11-08T11:52:18.4310180Z Other console options : null
2018-11-08T11:52:18.4310482Z Code coverage enabled : false
2018-11-08T11:52:18.4310627Z Diagnostics enabled : false
2018-11-08T11:52:18.4311321Z SystemVssConnection exists true
2018-11-08T11:52:18.4334452Z Run the tests locally using vstest.console.exe
2018-11-08T11:52:18.4334531Z ========================================================
2018-11-08T11:52:18.4347193Z Test selector : Test assemblies
2018-11-08T11:52:18.4351418Z Test assemblies : **\chutzpah.json
2018-11-08T11:52:18.4351642Z Test filter criteria : null
2018-11-08T11:52:18.4355360Z Search folder : D:\a\1\s
2018-11-08T11:52:18.4357017Z Run settings file : D:\a\1\s
2018-11-08T11:52:18.4358825Z Run in parallel : false
2018-11-08T11:52:18.4360300Z Run in isolation : false
2018-11-08T11:52:18.4382185Z Path to custom adapters : null
2018-11-08T11:52:18.4382534Z Other console options : null
2018-11-08T11:52:18.4384839Z Code coverage enabled : false
2018-11-08T11:52:18.4386257Z Diagnostics enabled : false
2018-11-08T11:52:18.4388538Z Rerun failed tests: false
2018-11-08T11:52:18.4391303Z VisualStudio version selected for test execution : latest
2018-11-08T11:52:18.6093014Z ========================================================
2018-11-08T11:52:19.1694099Z ======================================================
2018-11-08T11:52:20.3093149Z [command]"C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" @D:\a\_temp\be5d5f11-e34c-11e8-a8a9-2d59e72db6fd.txt
2018-11-08T11:52:20.4991286Z Microsoft (R) Test Execution Command Line Tool Version 15.8.0
2018-11-08T11:52:20.4991453Z Copyright (c) Microsoft Corporation.  All rights reserved.
2018-11-08T11:52:20.4991515Z 
2018-11-08T11:52:20.5016812Z vstest.console.exe 
2018-11-08T11:52:20.5016907Z "D:\a\1\s\TestMVCApplication.Tests\chutzpah.json"
2018-11-08T11:52:20.5016952Z /logger:"trx"
2018-11-08T11:52:20.5017011Z /TestAdapterPath:"D:\a\1\s"
2018-11-08T11:52:20.7387852Z Starting test execution, please wait...
2018-11-08T11:52:27.4042414Z Error: DEPRECATION: The catchExceptions option is deprecated and will be replaced with stopOnSpecFailure in Jasmine 3.0
2018-11-08T11:52:27.4042924Z    [{file:file:///D:/a/1/s/packages/Chutzpah.4.4.2/tools/ChutzpahTestHarnessFiles/jasmine/v2/jasmine.js,line:949,function:deprecated}]
2018-11-08T11:52:27.4044557Z While Running:D:\a\1\s\TestMVCApplication.Tests\js-tests\utilitiy-tests.js
2018-11-08T11:52:27.4044678Z 
2018-11-08T11:52:27.4767512Z Passed   app-funcs: add() Correctly adds 3 numbers
2018-11-08T11:52:27.4768451Z Passed   app-funcs: add() Correctly adds 2 numbers
2018-11-08T11:52:27.4768908Z Passed   app-funcs: add() Correctly adds 1 number
2018-11-08T11:52:27.4769290Z Passed   app-funcs: add() Does the impossible
2018-11-08T11:52:27.5069526Z Results File: D:\a\1\s\TestResults\VssAdministrator_factoryvm-az489_2018-11-08_11_52_27.trx
2018-11-08T11:52:27.5094303Z 
2018-11-08T11:52:27.5097933Z Total tests: 4. Passed: 4. Failed: 0. Skipped: 0.
2018-11-08T11:52:27.5098548Z Test Run Failed.
2018-11-08T11:52:27.5107869Z Test execution time: 6.5793 Seconds
2018-11-08T11:52:27.7938298Z ##[warning]Vstest failed with error. Check logs for failures. There might be failed tests.
2018-11-08T11:52:27.8666710Z ##[error]Error: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe failed with return code: 1
2018-11-08T11:52:27.9914771Z ##[error]VsTest task failed.
2018-11-08T11:52:28.0305446Z Publishing test results to test run '213'
2018-11-08T11:52:28.0305575Z Test results remaining: 4. Test run id: 213
2018-11-08T11:52:28.0578818Z ##[section]Async Command Start: Publish test results
2018-11-08T11:52:28.2630565Z Published Test Run : https://linneyit.visualstudio.com/T-Level%20Setup%20Test/_TestManagement/Runs#runId=213&_a=runCharts
2018-11-08T11:52:28.2630806Z ##[section]Async Command End: Publish test results
2018-11-08T11:52:28.2631502Z ##[section]Finishing: VsTest - Front End

我想知道这是否是由于它们是两个测试选择器 block (2018-11-08T11:52:18.4136511Z) 和 (2018-11-08T11:52:18.4347193Z )，第一个带有测试过滤条件:null，第二个带有测试程序集:**\chutzpah.json。

任务设置如下:

Visual Studio Test
Version: 2.*

Display name: VsTest - Front End
Select tests using: Test assemblies
Test files: **\chutzpah.json
Search folder: $(System.DefaultWorkingDirectory)
Test filter criteria:
[ ] Run only impacted tests
[ ] Test mix contains UI tests

Execution Options
Select test platform using: [x] Version [ ] Specific location
Test platform version: Latest
Settings file:
Override test run parameters:
Path to custom test adapters:
[ ] Run tests in parallel on multi-core machines
[ ] Run tests in isolation
[ ] Code coverage enabled
Other console options:
[x] Collect advanced diagnostics in case of catastrophic failures
Collect process dump and attach to test run report: On abort only
[ ] Rerun failed tests

Advanced execution options
Batch tests: Based on number of tests and agents
Batch options: [x] Automatically determine the batch size [ ] Specify a batch size
[ ] Do not distribute tests and replicate instead when multiple agents are used in the job

Reporting options
Test run title:
Build platform:
Build configuration:
[x] Upload test attachments

Control Options
[x] Enabled
[ ] Continue on error
Timeout: 0
Run this task: Only when all previous tasks have succeeded

Output Variables
Reference name:

此构建管道任务中的设置是否正确？关于任务失败原因的任何其他问题/可能性吗？

Answer 1

我认为构建会出现错误:

Error: DEPRECATION: The catchExceptions option is deprecated and will be replaced with stopOnSpecFailure in Jasmine 3.0

并从中报告错误状态。