Test with failed expectation on value passed to mocked method is incorrectly considered risky

[sebastianbergmann]

final class C
{
    private string $foo;

    public function __construct(string $foo)
    {
        $this->foo = $foo;
    }
}

interface I
{
    public function m(C $c): void;
}

final class IssueXXXXTest extends TestCase
{
    public function testOne(): void
    {
        $i = $this->createMock(I::class);

        $i->expects($this->once())->method('m')->with(new C('bar'));

        $i->m(new C('baz'));
    }
}

Running the test shown above yields the output shown below:

PHPUnit 11.5.9-8-g6b2c09626d by Sebastian Bergmann and contributors.

Runtime:       PHP 8.4.4
Configuration: /usr/local/src/phpunit/phpunit.xml

F                                                                   1 / 1 (100%)

Time: 00:00.016, Memory: 12.00 MB

There was 1 failure:

1) PHPUnit\TestFixture\IssueXXXX\IssueXXXXTest::testOne
Expectation failed for method name is "m" when invoked 1 time
Parameter 0 for invocation PHPUnit\TestFixture\IssueXXXX\I::m(PHPUnit\TestFixture\IssueXXXX\C Object (...)): void does not match expected value.
Failed asserting that two objects are equal.
--- Expected
+++ Actual
@@ @@
 PHPUnit\TestFixture\IssueXXXX\C Object (
-    'foo' => 'bar'
+    'foo' => 'baz'
 )

/usr/local/src/phpunit/src/Framework/MockObject/Runtime/Matcher.php:110
/usr/local/src/phpunit/src/Framework/MockObject/Runtime/InvocationHandler.php:109
/usr/local/src/phpunit/tests/end-to-end/regression/XXXX/IssueXXXXTest.php:37
/usr/local/src/phpunit/src/Framework/TestCase.php:1240
/usr/local/src/phpunit/src/Framework/TestCase.php:514
/usr/local/src/phpunit/src/Framework/TestRunner/TestRunner.php:87
/usr/local/src/phpunit/src/Framework/TestCase.php:361
/usr/local/src/phpunit/src/Framework/TestSuite.php:369
/usr/local/src/phpunit/src/TextUI/TestRunner.php:64
/usr/local/src/phpunit/src/TextUI/Application.php:210

--

There was 1 risky test:

1) PHPUnit\TestFixture\IssueXXXX\IssueXXXXTest::testOne
This test did not perform any assertions

/usr/local/src/phpunit/tests/end-to-end/regression/XXXX/IssueXXXXTest.php:31

FAILURES!
Tests: 1, Assertions: 0, Failures: 1, Risky: 1.

The test is correctly considered to have failed. It is incorrectly considered to be risky as it did perform an assertion (expectations on mock objects are assertions).

final class IssueXXXXTest extends TestCase
{
    public function testOne(): void
    {
        $i = $this->createMock(I::class);

        $i->expects($this->once())->method('m')->with(new C('bar'));

        $i->m(new C('bar'));
    }
}

Running the test shown (which calls $i->m(new C('bar')); instead of $i->m(new C('baz'));) above yields the output shown below:

PHPUnit 11.5.9-8-g6b2c09626d by Sebastian Bergmann and contributors.

Runtime:       PHP 8.4.4
Configuration: /usr/local/src/phpunit/phpunit.xml

.                                                                   1 / 1 (100%)

Time: 00:00.018, Memory: 12.00 MB

OK (1 test, 1 assertion)

The expectation succeeds, is counted as an assertions, and the test is therefore not considered to be risky.