7. Incomplete and Skipped Tests

Incomplete Tests

When you are working on a new test case class, you might want to begin by writing empty test methods such as:

public function testSomething(): void
{
}

to keep track of the tests that you have to write. The problem with empty test methods is that they are interpreted as a success by the PHPUnit framework. This misinterpretation leads to the test reports being useless – you cannot see whether a test is actually successful or just not yet implemented. Calling $this->fail() in the unimplemented test method does not help either, since then the test will be interpreted as a failure. This would be just as wrong as interpreting an unimplemented test as a success.

If we think of a successful test as a green light and a test failure as a red light, we need an additional yellow light to mark a test as being incomplete or not yet implemented. PHPUnit\Framework\IncompleteTest is a marker interface for marking an exception that is raised by a test method as the result of the test being incomplete or currently not implemented. PHPUnit\Framework\IncompleteTestError is the standard implementation of this interface.

Example 7.1 shows a test case class, SampleTest, that contains one test method, testSomething(). By calling the convenience method markTestIncomplete() (which automatically raises an PHPUnit\Framework\IncompleteTestError exception) in the test method, we mark the test as being incomplete.

Example 7.1 Marking a test as incomplete

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class SampleTest extends TestCase
{
    public function testSomething(): void
    {
        // Optional: Test anything here, if you want.
        $this->assertTrue(true, 'This should already work.');

        // Stop here and mark this test as incomplete.
        $this->markTestIncomplete(
          'This test has not been implemented yet.'
        );
    }
}

An incomplete test is denoted by an I in the output of the PHPUnit command-line test runner, as shown in the following example:

$ phpunit --verbose SampleTest
PHPUnit 9.6.0 by Sebastian Bergmann and contributors.

I

Time: 0 seconds, Memory: 3.95Mb

There was 1 incomplete test:

1) SampleTest::testSomething
This test has not been implemented yet.

/home/sb/SampleTest.php:12
OK, but incomplete or skipped tests!
Tests: 1, Assertions: 1, Incomplete: 1.

Table 7.1 shows the API for marking tests as incomplete.

Table 7.1 API for Incomplete Tests
Method	Meaning
`void markTestIncomplete()`	Marks the current test as incomplete.
`void markTestIncomplete(string $message)`	Marks the current test as incomplete using `$message` as an explanatory message.

Skipping Tests

Not all tests can be run in every environment. Consider, for instance, a database abstraction layer that has several drivers for the different database systems it supports. The tests for the MySQL driver can only be run if a MySQL server is available.

Example 7.2 shows a test case class, DatabaseTest, that contains one test method, testConnection(). In the test case class’ setUp() template method we check whether the MySQLi extension is available and use the markTestSkipped() method to skip the test if it is not.

Example 7.2 Skipping a test

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class DatabaseTest extends TestCase
{
    protected function setUp(): void
    {
        if (!extension_loaded('mysqli')) {
            $this->markTestSkipped(
              'The MySQLi extension is not available.'
            );
        }
    }

    public function testConnection(): void
    {
        // ...
    }
}

A test that has been skipped is denoted by an S in the output of the PHPUnit command-line test runner, as shown in the following example:

$ phpunit --verbose DatabaseTest
PHPUnit 9.6.0 by Sebastian Bergmann and contributors.

S

Time: 0 seconds, Memory: 3.95Mb

There was 1 skipped test:

1) DatabaseTest::testConnection
The MySQLi extension is not available.

/home/sb/DatabaseTest.php:9
OK, but incomplete or skipped tests!
Tests: 1, Assertions: 0, Skipped: 1.

Table 7.2 shows the API for skipping tests.

Table 7.2 API for Skipping Tests
Method	Meaning
`void markTestSkipped()`	Marks the current test as skipped.
`void markTestSkipped(string $message)`	Marks the current test as skipped using `$message` as an explanatory message.

Skipping Tests using @requires

In addition to the above methods it is also possible to use the @requires annotation to express common preconditions for a test case.

A test can have multiple @requires annotations, in which case all requirements need to be met for the test to run.

Table 7.3 Possible @requires usages
Type	Possible Values	Examples	Another example
`PHP`	Any PHP version identifier along with an optional operator	@requires PHP 7.1.20	@requires PHP >= 7.2
`PHPUnit`	Any PHPUnit version identifier along with an optional operator	@requires PHPUnit 7.3.1	@requires PHPUnit < 8
`OS`	A regexp matching PHP_OS	@requires OS Linux	@requires OS WIN32\|WINNT
`OSFAMILY`	Any OS family	@requires OSFAMILY Solaris	@requires OSFAMILY Windows
`function`	Any valid parameter to function_exists	@requires function imap_open	@requires function ReflectionMethod::setAccessible
`extension`	Any extension name along with an optional version identifier and optional operator	@requires extension mysqli	@requires extension redis >= 2.2.0
`setting`	Name and value for a `php.ini` configuration setting	@requires setting date.timezone Europe/Berlin

The following operators are supported for PHP, PHPUnit, and extension version constraints: <, <=, >, >=, =, ==, !=, <>.

Versions are compared using PHP’s version_compare function. Among other things, this means that the = and == operator can only be used with complete X.Y.Z version numbers and that just X.Y will not work.

Example 7.3 Skipping test cases using @requires

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

/**
 * @requires extension mysqli
 */
final class DatabaseTest extends TestCase
{
    /**
     * @requires PHP >= 5.3
     */
    public function testConnection(): void
    {
        // Test requires the mysqli extension and PHP >= 5.3
    }

    // ... All other tests require the mysqli extension
}

If you are using syntax that doesn’t compile with a certain PHP Version look into the xml configuration for version dependent includes in The <testsuites> Element