8. Risky Tests

PHPUnit can help you identify risky tests, for instance lying tests that give you a false sense of security.

Tests that are considered risky do not contribute to code coverage.

Useless Tests

By default, PHPUnit is strict about tests that do not test anything: tests that do not perform assertions and do not configure expectations on mock objects.

This check can be disabled by using the --dont-report-useless-tests option on the command-line or by setting beStrictAboutTestsThatDoNotTestAnything="false" in PHPUnit’s XML configuration file.

Unintentionally Covered Code

PHPUnit can be strict about unintentionally covered code. This check can be enabled by using the --strict-coverage option on the command-line or by setting beStrictAboutCoverageMetadata="true" in PHPUnit’s XML configuration file.

A test that is attributed with PHPUnit\Framework\Attributes\CoversClass or PHPUnit\Framework\Attributes\CoversFunction (or annotated with @covers) and that executes code which is not specified using PHPUnit\Framework\Attributes\CoversClass, PHPUnit\Framework\Attributes\CoversFunction, PHPUnit\Framework\Attributes\UsesClass, or PHPUnit\Framework\Attributes\UsesFunction (or using the @covers or @uses annotations) will be considered risky when this check is enabled and the size of the test is either small or unknown.

Furthermore, by setting requireCoverageMetadata="true" in PHPUnit’s XML configuration file, a test can be considered risky when it does not have code coverage metadata.

Output During Test Execution

PHPUnit can be strict about output during tests. This check can be enabled by using the --disallow-test-output option on the command-line or by setting beStrictAboutOutputDuringTests="true" in PHPUnit’s XML configuration file.

A test that emits output, for instance by invoking print in either the test code or the tested code, will be considered risky when this check is enabled.

Test Execution Timeout

PHPUnit can enforce a time limit for the execution of a test when the pcntl extension is available. The enforcing of this time limit can be enabled by using the --enforce-time-limit option on the command-line or by setting enforceTimeLimit="true" in PHPUnit’s XML configuration file.

A test that is attributed with PHPUnit\Framework\Attributes\Large (or annotated with @large) will be considered risky when it takes longer than 60 seconds to run. This timeout is configurable via the timeoutForLargeTests attribute in the XML configuration file.

A test that is attributed with PHPUnit\Framework\Attributes\Medium (or annotated with @medium) will be considered risky when it takes longer than 10 seconds to run. This timeout is configurable via the timeoutForMediumTests attribute in the XML configuration file.

A test that is attributed with PHPUnit\Framework\Attributes\Small (or annotated with @small) will be considered risky when it takes longer than 1 second to run. This timeout is configurable via the timeoutForSmallTests attribute in the XML configuration file.

Global State Manipulation

PHPUnit can be strict about tests that manipulate global state. This check can be enabled by using the --strict-global-state option on the command-line or by setting beStrictAboutChangesToGlobalState="true" in PHPUnit’s XML configuration file.