3. The Command-Line Test Runner

The PHPUnit command-line test runner can be invoked through the phpunit command.

Running tests

The most common way to run tests is with a phpunit.xml configuration file (see Composing a Test Suite Using XML Configuration). When a configuration file is present in the current working directory, simply running phpunit without any arguments will execute all configured test suites:

$ ./tools/phpunit
PHPUnit 13.4.0 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.3
Configuration: /path/to/project/phpunit.xml

..                                                                  2 / 2 (100%)

Time: 00:00.077, Memory: 10.00 MB

OK (2 tests, 2 assertions)

You can also run the tests of a specific test source file by passing its path as an argument:

$ ./tools/phpunit tests/ExampleTest.php
PHPUnit 13.4.0 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.3

..                                                                  2 / 2 (100%)

Time: 00:00.077, Memory: 10.00 MB

OK (2 tests, 2 assertions)

When invoked as shown above, the PHPUnit command-line test runner will look for a ExampleTest.php source code file in the current working directory, load it, and expect to find an ExampleTest test case class. It will then run the tests found in that class.

PHPT tests

In addition to test classes, PHPUnit’s test runner can also execute PHPT tests. PHPT is a simple test format used by the PHP project itself. Files with the .phpt suffix are automatically discovered by the test runner (see the --test-suffix option in Selection).

This can be useful, for instance, for end-to-end tests of CLI tools or for getting basic regression tests in place for legacy codebases.

PHPT tests are documented in PHPT Tests.

Selecting tests

PHPUnit provides several command-line options for selecting which tests to run.

Filtering by test name

The --filter option selects tests whose name matches a pattern:

$ ./tools/phpunit --filter testDoesSomething
PHPUnit 13.4.0 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.3
Configuration: /path/to/project/phpunit.xml

.                                                                   1 / 1 (100%)

Time: 00:00.077, Memory: 10.00 MB

OK (1 test, 1 assertion)

The pattern is matched against the full test name, which has one of these forms:

Fully\Qualified\ClassName::testMethodName
Fully\Qualified\ClassName::testMethodName with data set #0
Fully\Qualified\ClassName::testMethodName with data set "named set"

Matching is a partial match (the pattern does not need to cover the entire name) and not anchored. For example, --filter testAdd also matches a test named testAddress.

PHPT tests (see PHPT tests) are never selected by --filter.

The --exclude-filter option accepts the same pattern syntax described below and removes matching tests from the run. It can be combined with --filter.

Shortcut syntax

For common cases you do not need to write a regular expression. The following shortcut forms are recognized and translated into an appropriate pattern internally:

testMethodName: Selects all tests whose name contains testMethodName.
ClassName::testMethodName: Selects a specific test method in a specific class.
testMethodName#3: Selects the test method together with data set #3 (a numeric data set key).
testMethodName#1-3: Selects the test method together with numeric data sets in the inclusive range 1 to 3.
#1-3: Selects numeric data sets in the inclusive range 1 to 3 across all test methods.
testMethodName@one plus one: Selects the test method together with the named data set "one plus one". Quote the argument so the shell preserves spaces.
testMethodName@one.*: Selects the test method together with any named data set that matches the regular expression one.*.
testMethodName#one plus one: Selects the test method together with the named data set "one plus one". This form is equivalent to testMethodName@one plus one and is accepted so that the test ID format produced by --list-test-ids can be passed directly to --filter. Quote the argument so the shell preserves spaces.

When the shortcut form is used, matching is case-insensitive.

Regular expression syntax

If the pattern begins with a non-alphanumeric character and is a valid PCRE regular expression, it is used verbatim (including its delimiters and modifiers). This form is useful when the shortcut syntax is not expressive enough, or when you need case-sensitive matching:

$ ./tools/phpunit --filter '/^App\\Domain\\.*Test::test/'

Caution

Fully-qualified class names contain backslashes, and backslashes are special in regular expressions. If you want to use --filter with a fully-qualified class name, either escape every backslash (App\\\\Domain\\\\FooTest) or wrap the pattern with explicit delimiters (/App\\\\Domain\\\\FooTest/). The filter is not passed through preg_quote(), so all regular expression metacharacters remain active.

Filtering by test suite

When you have multiple test suites defined in your XML configuration file, you can use the --testsuite option to run only the tests of a specific suite:

$ ./tools/phpunit --testsuite unit
PHPUnit 13.4.0 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.3
Configuration: /path/to/project/phpunit.xml

...............                                                    15 / 15 (100%)

Time: 00:00.100, Memory: 10.00 MB

OK (15 tests, 30 assertions)

The --exclude-testsuite option excludes a named test suite from the run.

The --list-suites option prints a list of all test suites defined in the XML configuration file without running any tests.

Filtering by group

The --group option runs only the tests that belong to a specified group. Groups are assigned to tests using the #[Group] attribute (see Group).

The --exclude-group option excludes tests belonging to a specified group.

Both options can be used more than once. A test is run when it is in any of the groups selected with --group, and it is not run when it is in any of the groups selected with --exclude-group:

$ ./tools/phpunit --group database --group network

The command shown above runs the tests that are in the group database, the tests that are in the group network, and the tests that are in both.

Selecting the tests that are in every one of several groups

Separating the names of groups with + selects the tests that are in all of them:

$ ./tools/phpunit --group database+slow

The command shown above runs only the tests that are in the group database and in the group slow. A test that is in only one of them is not run.

Any number of group names can be combined this way, and such a combination can be used wherever a single group name can be used. The two ways of combining groups can be mixed: each value of the option is a combination of groups a test has to be in, and a test is run when it matches at least one of these values.

$ ./tools/phpunit --group database+slow --group smoke

The command shown above runs the tests that are in both database and slow, as well as the tests that are in smoke.

--exclude-group works the same way:

$ ./tools/phpunit --exclude-group database+slow

The command shown above runs every test except the ones that are in both database and slow. A test that is only in database, or only in slow, is still run.

Caution

Because + combines group names, it cannot also be part of a group name. A group named database+slow cannot be selected: --group database+slow asks for the tests that are in database and in slow, not for the tests that are in the group whose name is database+slow.

PHPUnit emits a warning where such a group name is declared so that this does not go unnoticed. Only a + between two group names is treated as a combination, though: +name, name+, and + are ordinary group names and can be selected as before.

Filtering by covered unit

The --covers option runs only tests that declare they intend to cover a specified class or function. The --uses option runs only tests that declare they intend to use a specified class or function.

Filtering by test ID

A test ID is the canonical, fully qualified identifier of a single test. It has one of these forms:

Fully\Qualified\ClassName::testMethodName
Fully\Qualified\ClassName::testMethodName#0
Fully\Qualified\ClassName::testMethodName#named data set

Test IDs are matched strictly and exactly; they are not patterns.

The --run-test-id option runs the single test identified by the given test ID:

$ ./tools/phpunit --run-test-id 'PHPUnit\TestFixture\ExampleTest::testOne#named data set'

The --test-id-filter-file option runs the tests whose IDs are listed in the specified file, with one test ID per line. Together with --list-test-ids (see Listing tests) this enables a list/filter/run workflow that is useful, for example, for splitting a test suite across parallel CI jobs:

List all test IDs: phpunit --list-test-ids > test-ids.txt
Split or filter the file into batches
Run a batch: phpunit --test-id-filter-file batch-1.txt

Listing tests

The --list-tests option prints a list of all tests that would be executed without actually running them. This is useful for verifying that your selection options are working as expected.

The --list-test-ids option prints a list of all tests that would be executed as test IDs (one per line) without actually running them. The output of this option is suitable for use with --test-id-filter-file and --run-test-id (see Filtering by test ID).

The --list-tests-xml option writes this list in XML format to a specified file.

See Selection for the complete reference of all test selection options.

Outcome and Issues

PHPUnit separates the outcome (errored, failed, incomplete, skipped, or passed) of a test from the issues (considered risky, triggered a warning, …) of a test.

With regard to outcome, PHPUnit distinguishes between failures and errors. A test fails when an assertion failed. This is different from an unexpected exception or a PHP error that occur while a test is running. When this happens, the test errors.

Errors tend to be easier to fix than failures. If you have a big list of problems, it is best to tackle the errors first and see if you have any failures left when they are all fixed.

Output

The default output of PHPUnit’s test runner consists of up to four sections.

Runtime information

This section contains information about the PHPUnit version, PHP version, and PHPUnit’s XML configuration file.

This section can be disabled using the --no-output CLI option.

Progress

This section is printed to indicate progress while the tests are being run.

For each test run, the PHPUnit command-line tool prints one character to indicate progress:

.

Printed when a successful test has no issues

F

Printed when an assertion fails while running the test method

E

Printed when an error occurs while running the test method

W

Printed when the test triggered a warning

R

Printed when the test has been considered risky (see Risky Tests)

D

Printed when the test triggered a deprecation

N

Printed when the test triggered a notice

I

Printed when the test is marked as incomplete (see Incomplete tests)

S

Printed when the test was skipped (see Skipping tests)

This section can be disabled using the --no-progress and --no-output CLI options.

Test results

This section contains information about errors, failures, skipped tests, incomplete tests, and issues. By default, only information about errors and failures is printed.

This section is only printed when there are errors, failures, or issues to report. It can be disabled using the --no-results and --no-output CLI options.

Failure output

Whenever a test fails, PHPUnit tries its best to provide you with as much context as possible that can help to identify the problem.

Example 3.1 Output generated when an array comparison fails

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class ArrayDiffTest extends TestCase
{
    public function testEquality(): void
    {
        $this->assertSame(
            [1, 2,  3, 4, 5, 6],
            [1, 2, 33, 4, 5, 6],
        );
    }
}

Running the test shown above yields the output shown below:

./tools/phpunit tests/ArrayDiffTest.php
PHPUnit 13.4.0 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.11

F                                                                   1 / 1 (100%)

Time: 00:00, Memory: 32.33 MB

There was 1 failure:

1) ArrayDiffTest::testEquality
Failed asserting that two arrays are identical.
--- Expected
+++ Actual
@@ @@
 Array &0 [
     0 => 1,
     1 => 2,
-    2 => 3,
+    2 => 33,
     3 => 4,
     4 => 5,
     5 => 6,
 ]

/path/to/tests/ArrayDiffTest.php:8

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

In this example only one of the array values differs and the other values are shown to provide context on where the error occurred.

When the generated output would be long to read PHPUnit will split it up and provide a few lines of context around every difference.

Example 3.2 Output when an array comparison of a long array fails

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class LongArrayDiffTest extends TestCase
{
    public function testEquality(): void
    {
        $this->assertSame(
            [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2,  3, 4, 5, 6],
            [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 33, 4, 5, 6],
        );
    }
}

Running the test shown above yields the output shown below:

./tools/phpunit tests/LongArrayDiffTest.php
PHPUnit 13.4.0 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.11

F                                                                   1 / 1 (100%)

Time: 00:00, Memory: 32.33 MB

There was 1 failure:

1) LongArrayDiffTest::testEquality
Failed asserting that two arrays are identical.
--- Expected
+++ Actual
@@ @@
     11 => 0,
     12 => 1,
     13 => 2,
-    14 => 3,
+    14 => 33,
     15 => 4,
     16 => 5,
     17 => 6,
 ]

/path/to/tests/LongArrayDiffTest.php:8

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

Edge cases

When a comparison fails PHPUnit creates textual representations of the input values and compares those. Due to that implementation a diff might show more problems than actually exist.

This only happens when using assertEquals() or other “weak” comparison functions on arrays or objects.

Example 3.3 Edge case in the diff generation when using weak comparison

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class ArrayWeakComparisonTest extends TestCase
{
    public function testEquality(): void
    {
        $this->assertEquals(
            [1, 2, 3, 4, 5, 6],
            ['1', 2, 33, 4, 5, 6],
        );
    }
}

Running the test shown above yields the output shown below:

./tools/phpunit tests/ArrayWeakComparisonTest.php
PHPUnit 13.4.0 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.11

F                                                                   1 / 1 (100%)

Time: 00:00, Memory: 32.33 MB

There was 1 failure:

1) ArrayWeakComparisonTest::testEquality
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    0 => 1
+    0 => '1'
     1 => 2
-    2 => 3
+    2 => 33
     3 => 4
     4 => 5
     5 => 6
 )

/path/to/tests/ArrayWeakComparisonTest.php:8

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

In this example the difference in the first index between 1 and '1' is reported even though assertEquals() considers the values as a match.

Summary

This section contains a summary of the test suite execution.

When no test errored or failed then the summary will be OK
When a test errored then the summary will be ERRORED!
When a test failed and no test errored then the summary will be FAILED!
When there were no errors or failures, but a test was skipped, then the summary will be OK, but some tests were skipped!
When there were no errors, failures, or skipped tests, but there were issues, then the summary will be OK, but there were issues!

--fail-on-*

Please note that if you use CLI options such as --fail-on-deprecation, for example, or their XML configuration counterparts, then this only affects the test runner’s shell exit code. It does not have an effect on whether or not OK will be printed in the output’s summary section.

This section can be disabled using the --no-output CLI option.

Alternative output formats

PHPUnit supports alternative output formats that replace the default progress and result output.

TestDox

The --testdox option replaces the default result output with TestDox format. See TestDox for details.

Debug

The --debug option replaces the default progress and result output with detailed event-by-event information about the test execution process. When combined with --with-telemetry, timing and memory usage information is included.

TeamCity

The --teamcity option replaces the default progress and result output with TeamCity format. This is used for integration with PhpStorm or TeamCity.

Compact

The --compact option replaces the default progress and result output with a compact format that is stripped down to what is actionable: there is no progress section, no timing or memory usage line, and no use of color. When all tests pass, only the runtime information section and a single summary line are printed:

$ ./tools/phpunit --compact tests
PHPUnit 13.4.0 by Sebastian Bergmann and contributors.

Runtime: PHP 8.5.5

OK (4699 tests, 13576 assertions)

When tests fail or have other issues, each defect is printed as a clearly delimited block with the test name, the assertion message, and the file location:

$ ./tools/phpunit --compact tests
PHPUnit 13.4.0 by Sebastian Bergmann and contributors.

Runtime: PHP 8.5.5


--- FAILURE: ExampleTest::testSomething
Failed asserting that false is true.

/path/to/tests/ExampleTest.php:47

FAILURES (23 tests, 42 assertions, 1 failure)

The summary line begins with OK when all tests pass, FAILURES when at least one test failed, and ERRORS when at least one test errored. The counts that follow include the number of tests run and assertions made as well as, when non-zero, the number of errors, failures, deprecations, PHPUnit deprecations, warnings, PHPUnit warnings, notices, PHPUnit notices, skipped tests, incomplete tests, and risky tests.

Errors and failures are printed as soon as they occur, before the summary line. All other defects are listed after the summary line, in this order: errors, warnings, deprecations, and notices reported by PHPUnit itself, deprecations, warnings, notices, errors triggered by tests, risky tests, incomplete tests, skipped tests.

When the time limit for the test run configured with --timeout (see Limiting the duration of the test run) was exceeded, this is printed as a record of its own directly after the summary line, so that it cannot be mistaken for the output of a test:

ERRORS (23 tests, 42 assertions, 1 error)

--- TIME LIMIT EXCEEDED
The time limit of 300 seconds for the test run was exceeded.

By default, only errors, failures, risky tests, and the errors and warnings reported by PHPUnit itself are shown in detail. Like the default output, --compact respects the --display-* flags described under Controlling output: use --display-deprecations, --display-warnings, --display-notices, --display-errors, --display-incomplete, --display-skipped, --display-phpunit-deprecations, --display-phpunit-notices, or --display-all-issues to display additional details. Issues that are only counted in the summary line are not shown in detail unless the corresponding flag is used.

Compact output can also be activated by setting the PHPUNIT_COMPACT_OUTPUT environment variable to 1. This makes it easy to enable compact output globally without changing how PHPUnit is invoked, for example when running tests inside an AI-based coding assistant where every token of test output consumes context window budget.

Note

When both the PHPUNIT_COMPACT_OUTPUT environment variable and the --compact CLI option are set, compact output is used. The CLI option --compact cannot be combined with --teamcity, --testdox, --testdox-summary, --debug, or --no-output.

Controlling output

The following options control what is displayed in the test results section. By default, only information about errors and failures is printed. Use these options to display additional details:

--display-incomplete shows details for incomplete tests
--display-skipped shows details for skipped tests
--display-deprecations shows details for deprecations triggered by tests
--display-notices shows details for notices triggered by tests
--display-warnings shows details for warnings triggered by tests
--display-errors shows details for errors triggered by tests
--display-phpunit-deprecations shows details for deprecations reported by PHPUnit itself
--display-phpunit-notices shows details for notices reported by PHPUnit itself
--display-all-issues enables all of the above

The --colors=<flag> option controls the use of colors in terminal output. Accepted values are never, auto (uses colors when the terminal supports it), and always.

The --columns option controls the width of the progress output. Use --columns max to use the full terminal width.

The --reverse-list option prints defects in reverse order, showing the most recently encountered defects first.

See Reporting for the complete reference of all reporting options.

Interaction between output options

The options described above fall into two groups:

--no-output, --no-progress, and --no-results suppress sections of the default output format described in Output.
--testdox, --teamcity, --compact, and --debug select an alternative output format that replaces the default progress and result output.

The --no-* options apply to the default output format only. They do not suppress the output produced by an alternative output format. For example, --testdox --no-output will still produce TestDox output.

The --testdox-html and --testdox-text options are logging options (see Logging): they write a TestDox-formatted report to a file. They are not intended to replace or augment the console output. Use --testdox when you want TestDox-formatted output on the console.

Exit codes

The PHPUnit command-line test runner exits with an exit code that indicates the outcome of the test run. This is particularly important for Continuous Integration (CI) pipelines and scripts that need to act on the result of a test run.

0

All tests passed (no errors, no failures)

1

At least one test failed

2

At least one test errored

124

The time limit for the test run configured with --timeout was exceeded (see Limiting the duration of the test run)

By default, issues such as deprecations, notices, warnings, and risky tests do not affect the exit code. The --fail-on-* family of options changes this behaviour:

--fail-on-warning causes a failure exit code when any test triggers a warning
--fail-on-risky causes a failure exit code when any test is considered risky
--fail-on-deprecation causes a failure exit code when any test triggers a deprecation
--fail-on-notice causes a failure exit code when any test triggers a notice
--fail-on-skipped causes a failure exit code when any test is skipped
--fail-on-incomplete causes a failure exit code when any test is marked incomplete
--fail-on-empty-test-suite causes a failure exit code when no tests were executed
--fail-on-all-issues enables all of the above

These options are commonly used in CI pipelines to enforce strict quality standards:

$ ./tools/phpunit --fail-on-deprecation --fail-on-notice --fail-on-warning
PHPUnit 13.4.0 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.3
Configuration: /path/to/project/phpunit.xml

DW                                                                  2 / 2 (100%)

Time: 00:00.077, Memory: 10.00 MB

OK, but there were issues!
Tests: 2, Assertions: 2, Deprecations: 1, Warnings: 1.

--fail-on-all-issues and its XML configuration file counterpart, failOnAllIssues="true", take precedence over the fine-grained failOn* attributes of the XML configuration file: configuring a fine-grained failOn* attribute to false has no effect when failOnAllIssues is configured to true.

The fine-grained --fail-on-* options have --do-not-fail-on-* counterparts that can be used to override settings from the XML configuration file. A --do-not-fail-on-* option disables failing on the respective issue type and takes precedence over all --fail-on-* options and failOn* attributes, including --fail-on-all-issues and failOnAllIssues="true". The --do-not-fail-on-* options are therefore the only way to not fail on a specific issue type when failOnAllIssues="true" is configured.

For deprecations, the failure behaviour can be refined based on how a deprecation was triggered. The --fail-on-self-deprecation, --fail-on-direct-deprecation, and --fail-on-indirect-deprecation options (and their --do-not-fail-on-* counterparts) control the exit code separately for each trigger type. See Failing on deprecations based on how they are triggered for details.

See Execution for the complete reference.

Stopping early

When a test suite is large, it can be useful to stop execution as soon as a problem is encountered instead of waiting for the entire suite to finish.

The --stop-on-* family of options controls this:

--stop-on-defect[=<n>] stops after the first (or n-th) error, failure, warning, or risky test
--stop-on-error[=<n>] stops after the first (or n-th) error
--stop-on-failure[=<n>] stops after the first (or n-th) failure
--stop-on-warning[=<n>] stops after the first (or n-th) warning
--stop-on-risky[=<n>] stops after the first (or n-th) risky test
--stop-on-deprecation[=<n>] stops after the first (or n-th) deprecation
--stop-on-notice[=<n>] stops after the first (or n-th) notice
--stop-on-skipped[=<n>] stops after the first (or n-th) skipped test
--stop-on-incomplete[=<n>] stops after the first (or n-th) incomplete test

When used without a value, these options stop execution after the first occurrence. When a threshold n is specified (for example, --stop-on-failure=3), execution continues until n occurrences have been encountered.

See Execution for the complete reference.

Limiting the duration of the test run

The --timeout <sec> option sets a wall-clock limit, in seconds, for the entire test run:

$ ./tools/phpunit --timeout 300

The clock starts as soon as the configuration is known, so bootstrapping and loading the test suite count toward the limit. The limit is checked whenever a test has finished; no PHP extension is required for this. When the limit is exceeded, no further tests are started and the run ends the way a run that was stopped by a --stop-on-* option ends: the configured logs are written, the result is printed, and a message states that the time limit was exceeded.

When the pcntl extension is available, a test that is still running when the limit is exceeded is aborted. It is reported as an error whose stack trace shows where the test was stuck:

$ ./tools/phpunit --timeout 300
PHPUnit 13.4.0 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.5
Configuration: /path/to/project/phpunit.xml

.......E

The time limit of 300 seconds for the test run was exceeded.

Time: 05:00.012, Memory: 12.00 MB

There was 1 error:

1) ExampleTest::testSomething
PHPUnit\Runner\TimeLimit\TimeLimitExceededException: This test was aborted because the time limit of 300 seconds for the test run was exceeded

/path/to/tests/ExampleTest.php:47

ERRORS!
Tests: 8, Assertions: 7, Errors: 1.

Without the pcntl extension, the test that is running when the limit is exceeded is not aborted. The test run stops after it has finished. The same applies to tests that run in a separate process (see RunInSeparateProcess) and to PHPT tests, regardless of whether pcntl is available.

The shell exit code of a run that exceeded its time limit is 124, the value that GNU timeout(1) uses for a command that exceeded its time limit, regardless of whether the tests that did run passed (see Exit codes). Code coverage reports and the baseline are not generated for such a run, as for a run that was interrupted by a signal. The PHPUnit\Event\TestRunner\TimeLimitExceeded event (see Events) is emitted when the limit is exceeded.

--timeout limits the test run as a whole, whereas --enforce-time-limit limits individual tests based on their size (see Test Execution Timeout). The two options can be combined.

--timeout is intended for situations in which PHPUnit is invoked without a person watching the run, for instance by a CI pipeline or by an AI-based coding assistant. Such a caller cannot tell a hung test run from a long one. Without a time limit, its only option is to kill the process, which leaves it without a summary, an exit code, a log file, or any record of which test was running. With --timeout, the run ends on its own with all of these.

Logging

PHPUnit can write test results to log files in various formats:

--log-otr <file> writes test results in Open Test Reporting (OTR) XML format (recommended)
--log-junit <file> writes test results in JUnit XML format (legacy)
--log-teamcity <file> writes test results in TeamCity format (used by PhpStorm)
--testdox-html <file> writes test results in TestDox format as an HTML file
--testdox-text <file> writes test results in TestDox format as a plain text file
--log-events-text <file> writes all test runner events as plain text
--log-events-verbose-text <file> writes all test runner events with extended information

These logfiles are commonly used for CI tool integration and reporting.

The --testdox-html and --testdox-text options write a report to a file; they are not intended to replace or augment the console output. For TestDox-formatted console output use --testdox (see TestDox).

Logging can also be configured in the XML configuration file (see The <logging> Element).

The --no-logging option ignores all logging configured in the XML configuration file.

Test execution order

By default, PHPUnit executes the tests of each test suite in the order in which it discovered them. The --order-by option changes that: it configures a pipeline of reordering stages that is applied to the tests of every test suite, one stage after another.

Order

The order is the first stage of the pipeline. Exactly one order can be configured:

default

Tests are executed in the order in which they were discovered

duration-ascending

Tests are ordered by duration, fastest test first. This requires the test run history (enabled by default, see --record-test-run-history)

duration-descending

Tests are ordered by duration, slowest test first. This requires the test run history

modified-ascending

Tests are ordered by the time their source files were last modified, least recently modified test first

modified-descending

Tests are ordered by the time their source files were last modified, most recently modified test first

random

Tests are executed in random order. Use --random-order-seed to make the order reproducible

reverse

Tests are executed in reverse discovery order

size-ascending

Tests are ordered by size (small, medium, large, unknown)

size-descending

Tests are ordered by size in reverse (large, medium, small, unknown)

Ordering by modification time

--order-by modified-descending runs the tests you have worked on most recently first. Combined with --stop-on-failure, this shortens the feedback loop while a new test is being written.

The source files of a test are the files a change to which can change what the test does: the file that declares the test class, the files that declare the classes it extends, and the files that declare the traits any of them use. The most recent modification among them decides, so modifying a base test class or a trait moves every test class that uses it to the front as well. For a PHPT test, the .phpt file is the test. A test suite is as new as the newest test it contains.

Note

The modification time of a file is not a record of who changed what: checking out a branch, cloning a repository, installing dependencies, and copying or deploying files all set it to the time of that operation. On a continuous integration server, where every file is usually written at the same time, this order is therefore meaningless. It is intended for local development and should not be configured in a project’s XML configuration file.

Many file systems store modification times with a resolution of one second, so files that were written during the same second are equally new. Tests that are equally new keep the order that they had before this stage was applied.

Running previously defective tests first

The order can be followed by defects, for instance --order-by duration-ascending,defects. This adds a second stage that moves the tests that were defective during the previous test run to the front, so that a defect is reached as fast as possible. Tests that are not moved to the front keep the order that the first stage established.

Only tests that errored or failed count as defects. A test that was skipped, that was marked as incomplete, or that triggered an issue is not moved to the front. A test suite counts as defective when at least one of the tests it contains counts as defective, so that defects are moved to the front through arbitrarily deeply nested test suites as well.

--order-by defects can also be used on its own, without configuring an order.

Note

Ordering by defects and ordering by duration are based on the test run history. When the recording of the test run history is disabled, for instance using the --do-not-record-test-run-history CLI option or the recordTestRunHistory="false" attribute in the XML configuration file, the tests cannot be ordered this way and PHPUnit emits a test runner warning:

Tests cannot be ordered by defects because recording of the test run history is disabled

The tests are then executed in the order that the remaining stages, if any, lead to.

Resolving dependencies between tests

Dependency resolution is the last stage of the pipeline: it reorders the tests of a test suite so that as many Depends* declarations as possible are satisfied. It is enabled by default and is controlled using the --resolve-dependencies and --ignore-dependencies CLI options, or the resolveDependencies attribute in the XML configuration file.

Because any stage that ran after it would undo the order it establishes, dependency resolution is always applied last.

Aliases

Convenience aliases are available: --random-order (for --order-by random) and --reverse-order (for --order-by reverse).

$ ./tools/phpunit --order-by random --random-order-seed 42
PHPUnit 13.4.0 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.3
Configuration: /path/to/project/phpunit.xml
Random seed:   42

..                                                                  2 / 2 (100%)

Time: 00:00.077, Memory: 10.00 MB

OK (2 tests, 2 assertions)

Deprecated ways of configuring the execution order

The values below are still accepted by --order-by and by the executionOrder attribute in the XML configuration file, but PHPUnit emits a test runner deprecation for them:

depends and no-depends

Whether dependencies between tests are resolved is not an order. Use the --resolve-dependencies and --ignore-dependencies CLI options, or the resolveDependencies attribute in the XML configuration file, instead.

defects written before the order

defects,duration-ascending and duration-ascending,defects currently mean the same thing: the order is applied first and defects are moved to the front afterwards. In PHPUnit 14, the stages will be applied in the order in which they are written, and the two values will therefore mean different things. Write the order first, for instance duration-ascending,defects.

More than one order

For instance duration-ascending,reverse. The order that is written last currently overrides the ones written before it. This will be an error in PHPUnit 14.

duration and size

Use duration-ascending and size-ascending instead.

A value that PHPUnit does not know is rejected with an error when it is used for --order-by. When it is used for the executionOrder attribute then it is currently ignored, which is deprecated and will be an error in PHPUnit 14.

See Execution for the complete reference.