java - Testng 6.9.10 assertEquals() 没有像它应该的那样失败

Question

所以我使用 TestNG 设置了一个测试用例。当我对应该失败的东西执行断言 equals 时，测试无论如何都会通过。我尝试将它们作为列表和字符串进行比较。实际结果为:

actual = [The license key was not valid., 
          The license key has expired and is no longer valid., 
          That email domain is not allowed., 
          The maximum number of allowed users for that license key has been reached. No other users may register unless a user is removed or the user limit for that key is increased., 
          That license key does not have a valid start date, the administrator must define an effective date for the license key.]

我正在使用的代码如下。实际和预期明显不同，但测试仍然通过。我正在使用 TestNG 6.9.10。我还没有找到任何地方有关于assertEquals 的错误。

import org.testng.annotations.*;
import static org.testng.Assert.assertEquals;
import static org.testng.Assert.assertTrue;

@Test
public void verifyLicencingErrorsTest() {
    List<String> expected = Arrays.asList(
            "The license key was not valid.",
            "The license key has expired and is no longer valid.",
            "The license key was not valid.",
            "The maximum number of allowed users for that license key has been reached. No other users may register unless a user is removed or the user limit for that key is increased.",
            "That license key does not have a valid start date, the administrator must define an effective date for the license key."
    );
    List<String> actual = createAccountPage.verifyLicencingErrors();

    try {
        assertEquals(actual.toString(), expected.toString(), "The actual alerts did not match what was expected");
    } catch (Error e) {
        verificationErrors.append(e.toString());
    }
}

我想我已经提供了所有必要的信息。如果我需要更多，请告诉我。

编辑:添加导入

Answer 1

根据 TestNG Assert documentation :

public static void assertEquals(java.lang.String actual,
                                java.lang.String expected,
                                java.lang.String message)

Asserts that two Strings are equal. If they are not, an AssertionError, with the given message, is thrown.
Parameters:
actual - the actual value
expected - the expected value
message - the assertion error message

问题是您捕获了Error，因此抛出的AssertionError不再到达框架并且测试被认为是正常的。如果您记录捕获的错误，您可以轻松地看到:

    try {
        assertEquals(actual.toString(), expected.toString(), "The actual alerts did not match what was expected");
    } catch (Error e) {
        log.error("Caught an [" + e.getClass().getName() + "]", e);
    }

...产量:

[ERROR] 2016-03-30 12:51:01,345 test.LauncherTest - Caught an [java.lang.AssertionError]
java.lang.AssertionError: The actual alerts did not match what was expected expected [[The license key was not valid., The license key has expired and is no longer valid., The license key was not valid., The maximum number of allowed users for that license key has been reached. No other users may register unless a user is removed or the user limit for that key is increased., That license key does not have a valid start date, the administrator must define an effective date for the license key.]] but found [[The license key was not valid., The license key has expired and is no longer valid., That email domain is not allowed., The maximum number of allowed users for that license key has been reached. No other users may register unless a user is removed or the user limit for that key is increased., That license key does not have a valid start date, the administrator must define an effective date for the license key]]
    at org.testng.Assert.fail(Assert.java:94)
    at org.testng.Assert.failNotEquals(Assert.java:496)
    at org.testng.Assert.assertEquals(Assert.java:125)
    at org.testng.Assert.assertEquals(Assert.java:178)
    at test.LauncherTest.verifyLicencingErrorsTest(LauncherTest.java:42)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:85)
    at org.testng.internal.Invoker.invokeMethod(Invoker.java:639)
    at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:821)
    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1131)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:124)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:108)
    at org.testng.TestRunner.privateRun(TestRunner.java:773)
    at org.testng.TestRunner.run(TestRunner.java:623)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:357)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:352)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:310)
    at org.testng.SuiteRunner.run(SuiteRunner.java:259)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1185)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1110)
    at org.testng.TestNG.run(TestNG.java:1018)
    at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
    at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204)
    at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175)
    at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:125)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)

在删除 catch block 或可能细化捕获的异常时，将按预期工作:

java.lang.AssertionError: The actual alerts did not match what was expected 
Expected :[The license key was not valid., The license key has expired and is no longer valid., The license key was not valid., The maximum number of allowed users for that license key has been reached. No other users may register unless a user is removed or the user limit for that key is increased., That license key does not have a valid start date, the administrator must define an effective date for the license key.]
Actual   :[The license key was not valid., The license key has expired and is no longer valid., That email domain is not allowed., The maximum number of allowed users for that license key has been reached. No other users may register unless a user is removed or the user limit for that key is increased., That license key does not have a valid start date, the administrator must define an effective date for the license key]
  <Click to see difference>


    at org.testng.Assert.fail(Assert.java:94)
    at org.testng.Assert.failNotEquals(Assert.java:496)
    at org.testng.Assert.assertEquals(Assert.java:125)
    at org.testng.Assert.assertEquals(Assert.java:178)
    at test.LauncherTest.verifyLicencingErrorsTest(LauncherTest.java:41)

===============================================
Custom suite
Total tests run: 1, Failures: 1, Skips: 0
===============================================