Testing in CI
suggest changeContinuous integration (CI) runs your checks on every push and pull request on a clean machine. For a PHP project that usually means:
- install dependencies with Composer from
composer.lock - check code style (PHP CS Fixer or PHP_CodeSniffer)
- run static analysis (PHPStan or Psalm)
- run the test suite (PHPUnit or Pest) on every supported PHP version
- optionally collect code coverage and test reports
A CI job passes or fails based on the exit code of each command: 0 means success, anything else fails the job. All the tools below follow that rule, so the work is mostly picking the right flags and making the same commands easy to run locally.
The examples use PHPUnit 13 (requires PHP 8.4+), PHPStan 2 and PHP CS Fixer 3. Writing the tests themselves is covered in Unit Testing, PHPUnit Data Providers and Mocks and test doubles.
A project to test
The project layout used in this article:
.
├── composer.json
├── composer.lock
├── phpunit.xml.dist
├── phpstan.dist.neon
├── .php-cs-fixer.dist.php
├── src/
│ └── PriceCalculator.php
└── tests/
├── Unit/
│ └── PriceCalculatorTest.php
└── Integration/
└── OrderRepositoryTest.php
Install the tools as dev dependencies, so CI and every developer use exactly the versions locked in composer.lock:
composer require --dev phpunit/phpunit:^13 phpstan/phpstan:^2 friendsofphp/php-cs-fixer:^3
Commit composer.lock, even for libraries. Without it, every CI run resolves dependencies again and a new release of some package can break the build for reasons unrelated to your change.
One command for everything: Composer scripts
Put the exact commands CI runs into composer.json. Developers then run composer ci before pushing and get the same result as the CI server:
{
"name": "acme/shop",
"type": "project",
"license": "proprietary",
"require": {
"php": ">=8.4"
},
"require-dev": {
"friendsofphp/php-cs-fixer": "^3.95",
"phpstan/phpstan": "^2.3",
"phpunit/phpunit": "^13.4"
},
"autoload": {
"psr-4": { "App\\": "src/" }
},
"autoload-dev": {
"psr-4": { "App\\Tests\\": "tests/" }
},
"scripts": {
"test": "phpunit",
"test:coverage": "phpunit --coverage-clover=build/coverage.xml --coverage-text",
"analyse": "phpstan analyse --no-progress",
"cs": "php-cs-fixer check --diff",
"ci": ["@cs", "@analyse", "@test"]
},
"config": {
"sort-packages": true
}
}
Composer adds vendor/bin to PATH while running scripts, so phpunit means vendor/bin/phpunit. A script with a list of commands stops at the first command that fails and returns its exit code:
$ composer ci
...
Found 3 of 3 files that can be fixed in 0.011 seconds, 18.00 MB memory used
Script php-cs-fixer check --diff handling the cs event returned with error code 8
Script @cs was called via ci
$ echo $?
8
Configuring PHPUnit for CI
Commit the configuration as phpunit.xml.dist. PHPUnit uses phpunit.xml if it exists and falls back to phpunit.xml.dist, so a developer can keep a local, git-ignored override.
<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
bootstrap="vendor/autoload.php"
colors="true"
cacheDirectory=".phpunit.cache"
executionOrder="random"
failOnRisky="true"
failOnWarning="true"
failOnDeprecation="true"
failOnNotice="true"
beStrictAboutOutputDuringTests="true">
<testsuites>
<testsuite name="unit">
<directory>tests/Unit</directory>
</testsuite>
<testsuite name="integration">
<directory>tests/Integration</directory>
</testsuite>
</testsuites>
<source>
<include>
<directory>src</directory>
</include>
</source>
<php>
<ini name="error_reporting" value="-1"/>
<env name="APP_ENV" value="test" force="true"/>
<env name="DATABASE_URL" value="sqlite::memory:"/>
</php>
</phpunit>
What matters for CI:
failOn*attributes. By default PHPUnit reports deprecations, notices and warnings but still exits with0. In CI you want these to fail the build, otherwise a new PHP version's deprecations pile up unnoticed.failOnAllIssues="true"turns on all of them at once.executionOrder="random"finds tests that depend on each other (shared static state, leftover database rows). The seed is printed at the top of the output; reproduce a failing order locally with--random-order-seed.<source>tells PHPUnit which files are "yours". It is required for code coverage and also decides which deprecations count as first-party.<env>sets defaults. Withoutforce="true", an environment variable that the CI job already sets wins, which is how the same file works with SQLite locally and with a real database in CI.
A run with a failing test shows the data set name and exits with 1:
$ vendor/bin/phpunit --testsuite unit
PHPUnit 13.4.3 by Sebastian Bergmann and contributors.
Runtime: PHP 8.5.9
Configuration: /home/kjk/shop/phpunit.xml.dist
Random Seed: 1792257264
F... 4 / 4 (100%)
Time: 00:00.001, Memory: 20.00 MB
There was 1 failure:
1) App\Tests\Unit\PriceCalculatorTest::testTotal@with tax with data (1000, 2, 0.23, 2461)
Failed asserting that 2460 is identical to 2461.
/home/kjk/shop/tests/Unit/PriceCalculatorTest.php:16
FAILURES!
Tests: 4, Assertions: 4, Failures: 1.
$ echo $?
1
Command-line options that are useful in CI:
| Option | Purpose |
|---|---|
--testsuite unit |
run one suite, e.g. fast unit tests in a separate job |
--group db / --exclude-group db |
select tests by #[Group('db')] attribute |
--log-junit build/junit.xml |
JUnit XML report that CI systems display as a test summary |
--coverage-clover <file> |
coverage in Clover XML (Codecov, Coveralls, SonarQube) |
--coverage-cobertura <file> |
coverage in Cobertura XML (GitLab merge request diff coverage) |
--coverage-text |
coverage summary printed to the job log |
--stop-on-defect |
stop at the first problem for a faster red build |
--fail-on-empty-test-suite |
fail if a filter or a wrong path selected no tests |
--teamcity |
TeamCity service messages instead of the normal output |
--colors=never |
plain output for log parsers |
Static analysis and code style
PHPStan reads its configuration from phpstan.dist.neon:
parameters:
level: 8
paths:
- src
- tests
When PHPStan detects it is running in GitHub Actions (the GITHUB_ACTIONS environment variable is set), it prints GitHub annotations in addition to the usual table, so errors show up inline in the pull request diff:
$ vendor/bin/phpstan analyse --no-progress
::error file=tests/Integration/OrderRepositoryTest.php,line=27,col=0::Cannot call method fetchColumn() on PDOStatement|false.
------ ---------------------------------------------------------
Line tests/Integration/OrderRepositoryTest.php
------ ---------------------------------------------------------
27 Cannot call method fetchColumn() on PDOStatement|false.
🪪 method.nonObject
------ ---------------------------------------------------------
[ERROR] Found 1 error
Use --error-format=github to force that format, or --error-format=gitlab for GitLab's code quality report.
When adding PHPStan to an existing project, start with a baseline instead of lowering the level. vendor/bin/phpstan analyse --generate-baseline writes the current errors to phpstan-baseline.neon; include it in the configuration and CI only fails on new errors:
includes:
- phpstan-baseline.neon
parameters:
level: 8
paths:
- src
- tests
PHP CS Fixer is configured in .php-cs-fixer.dist.php. check (an alias for fix --dry-run) only reports; in CI it exits with a non-zero code when any file needs fixing:
<?php
$finder = (new PhpCsFixer\Finder())->in([__DIR__ . '/src', __DIR__ . '/tests']);
return (new PhpCsFixer\Config())
->setRules(['@PER-CS' => true])
->setFinder($finder);
Run the style check on the lowest PHP version you support. PHP CS Fixer warns when the runtime is newer than the php constraint in composer.json, because a fixer could rewrite code using syntax the older version doesn't have.
GitHub Actions
shivammathur/setup-php installs any PHP version with the extensions, ini settings, coverage driver and tools you ask for. A complete workflow in .github/workflows/ci.yml:
name: CI
on:
push:
branches: [master]
pull_request:
# cancel an older run of the same branch when a new commit is pushed
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
quality:
name: Code style and static analysis
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: shivammathur/setup-php@v2
with:
php-version: '8.4' # lowest supported version
coverage: none
tools: cs2pr
- uses: ramsey/composer-install@v3 # composer install + dependency cache
- name: Validate composer.json
run: composer validate --strict --no-check-publish
- name: Security audit
run: composer audit
- name: PHP CS Fixer
run: vendor/bin/php-cs-fixer check --format=checkstyle | cs2pr
- name: PHPStan
run: vendor/bin/phpstan analyse --no-progress
tests:
name: PHP ${{ matrix.php }} (${{ matrix.dependencies }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
php: ['8.4', '8.5']
dependencies: [locked]
include:
- php: '8.4'
dependencies: lowest
steps:
- uses: actions/checkout@v5
- uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php }}
extensions: intl, pdo_sqlite
ini-values: error_reporting=-1, display_errors=On, zend.assertions=1
coverage: none
- uses: ramsey/composer-install@v3
with:
dependency-versions: ${{ matrix.dependencies }}
- name: Run tests
run: vendor/bin/phpunit --log-junit build/junit.xml
- name: Upload test report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v4
with:
name: junit-php${{ matrix.php }}-${{ matrix.dependencies }}
path: build/junit.xml
Notes:
fail-fast: falselets every matrix job finish, so you see whether a failure happens only on PHP 8.5 or everywhere.dependency-versions:locked(the default) runscomposer installfromcomposer.lock;lowestrunscomposer update --prefer-lowestand checks that the minimum versions in your constraints (^2.3) really work;highestupdates everything to the newest allowed versions. Libraries should testlowestandhighest; applications usually only needlocked.zend.assertions=1enablesassert(). Productionphp.inisets it to-1; setting it explicitly means the tests don't depend on the runner's defaults.vendor/bin/phpunit --check-php-configurationprints which recommended settings (error_reporting=-1,zend.assertions=1,memory_limit=-1, ...) are not in effect.cs2prconverts the checkstyle XML of PHP CS Fixer (or PHP_CodeSniffer) into annotations.if: ${{ !cancelled() }}uploads the report even when the tests step failed, which is when you need it.
Without ramsey/composer-install, cache Composer's download directory yourself:
- name: Get Composer cache directory
id: composer-cache
run: echo "dir=$(composer config cache-files-dir)" >> "$GITHUB_OUTPUT"
- uses: actions/cache@v4
with:
path: ${{ steps.composer-cache.outputs.dir }}
key: composer-${{ runner.os }}-${{ matrix.php }}-${{ hashFiles('composer.lock') }}
restore-keys: composer-${{ runner.os }}-${{ matrix.php }}-
- run: composer install --no-interaction --no-progress --prefer-dist
Cache the Composer download cache, not vendor/. Restoring vendor/ built for a different PHP version or OS causes confusing failures.
Testing the next PHP version
To find problems before a new PHP release, add a job for the development version and allow it to fail:
tests-nightly:
runs-on: ubuntu-latest
continue-on-error: true
steps:
- uses: actions/checkout@v5
- uses: shivammathur/setup-php@v2
with:
php-version: nightly
coverage: none
- uses: ramsey/composer-install@v3
with:
composer-options: --ignore-platform-req=php+
- run: vendor/bin/phpunit
--ignore-platform-req=php+ lets Composer install packages whose php constraint doesn't allow the new version yet.
Integration tests against a real database
Service containers start databases next to the job. The test reads its connection from environment variables, with SQLite as the local default:
<?php
declare(strict_types=1);
namespace App\Tests\Integration;
use PHPUnit\Framework\Attributes\Group;
use PHPUnit\Framework\TestCase;
#[Group('db')]
final class OrderRepositoryTest extends TestCase
{
private \PDO $pdo;
protected function setUp(): void
{
$dsn = getenv('DATABASE_URL') ?: 'sqlite::memory:';
$this->pdo = new \PDO($dsn, getenv('DB_USER') ?: null, getenv('DB_PASSWORD') ?: null, [
\PDO::ATTR_ERRMODE => \PDO::ERRMODE_EXCEPTION,
]);
$this->pdo->exec('CREATE TEMPORARY TABLE orders (id INT PRIMARY KEY, total INT NOT NULL)');
}
public function testInsertAndRead(): void
{
$this->pdo->exec('INSERT INTO orders (id, total) VALUES (1, 2460)');
$stmt = $this->pdo->query('SELECT total FROM orders WHERE id = 1');
$this->assertNotFalse($stmt);
$this->assertSame(2460, (int) $stmt->fetchColumn());
}
}
The job runs the same test against PostgreSQL. The options health check makes the job wait until the server accepts connections:
integration:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:17
env:
POSTGRES_USER: app
POSTGRES_PASSWORD: secret
POSTGRES_DB: app_test
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U app"
--health-interval 5s
--health-timeout 5s
--health-retries 10
env:
DATABASE_URL: pgsql:host=127.0.0.1;port=5432;dbname=app_test
DB_USER: app
DB_PASSWORD: secret
steps:
- uses: actions/checkout@v5
- uses: shivammathur/setup-php@v2
with:
php-version: '8.5'
extensions: pdo_pgsql
coverage: none
- uses: ramsey/composer-install@v3
- run: vendor/bin/phpunit --testsuite integration
For MySQL use the mysql:8.4 image with MYSQL_ROOT_PASSWORD / MYSQL_DATABASE, the health command mysqladmin ping -h 127.0.0.1, the pdo_mysql extension and a DSN like mysql:host=127.0.0.1;port=3306;dbname=app_test. Use 127.0.0.1, not localhost: with localhost the MySQL client tries a Unix socket that doesn't exist on the runner.
Code coverage
Coverage needs a driver. Locally, PHPUnit without one prints a warning and runs no tests:
$ vendor/bin/phpunit --coverage-text
...
There was 1 PHPUnit test runner warning:
1) No code coverage driver available that supports line coverage
No tests executed!
In CI use PCOV, which is much faster than Xdebug and only does coverage. Run coverage in one matrix job only, because it slows the tests down:
coverage:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: shivammathur/setup-php@v2
with:
php-version: '8.5'
coverage: pcov
- uses: ramsey/composer-install@v3
- run: vendor/bin/phpunit --coverage-clover build/coverage.xml --coverage-text
- run: php bin/coverage-check.php build/coverage.xml 80
- uses: codecov/codecov-action@v5
with:
files: build/coverage.xml
token: ${{ secrets.CODECOV_TOKEN }}
PHPUnit has no option for a minimum coverage percentage. A short script that reads the Clover report fails the build below a threshold:
<?php
declare(strict_types=1);
// Usage: php bin/coverage-check.php build/coverage.xml 80
[, $file, $min] = $argv + [null, null, null];
if ($file === null || $min === null) {
fwrite(STDERR, "usage: coverage-check.php <clover.xml> <min-percent>\n");
exit(2);
}
$xml = simplexml_load_file($file) ?: throw new RuntimeException("Cannot read $file");
$metrics = $xml->project->metrics;
$total = (int) $metrics['statements'];
$covered = (int) $metrics['coveredstatements'];
$percent = $total === 0 ? 100.0 : $covered / $total * 100;
printf("Line coverage: %.2f%% (%d/%d statements), required: %d%%\n", $percent, $covered, $total, $min);
exit($percent >= (float) $min ? 0 : 1);
$ php bin/coverage-check.php build/coverage.xml 80
Line coverage: 75.00% (3/4 statements), required: 80%
$ echo $?
1
Treat the number as a guard against coverage dropping, not as a goal. Set the threshold a little below the current value and raise it over time.
GitLab CI
The same commands work in any CI system. A .gitlab-ci.yml using the official PHP image:
default:
image: php:8.5-cli
before_script:
- apt-get update && apt-get install -y git unzip
- curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
- composer install --no-interaction --no-progress --prefer-dist
cache:
key:
files: [composer.lock]
paths: [.composer-cache/]
variables:
COMPOSER_CACHE_DIR: "$CI_PROJECT_DIR/.composer-cache"
phpstan:
script:
- vendor/bin/phpstan analyse --no-progress --error-format=gitlab > phpstan.json
artifacts:
when: always
reports:
codequality: phpstan.json
phpunit:
script:
- pecl install pcov && docker-php-ext-enable pcov
- vendor/bin/phpunit --colors=never --log-junit junit.xml --coverage-text --coverage-cobertura cobertura.xml
coverage: '/^\s*Lines:\s*\d+\.\d+\%/'
artifacts:
when: always
reports:
junit: junit.xml
coverage_report:
coverage_format: cobertura
path: cobertura.xml
The coverage regular expression extracts the percentage from the Lines: line of --coverage-text, and GitLab shows it on the merge request. --colors=never keeps ANSI codes out of the text the regex has to match. In a real pipeline, build an image with Composer and the extensions already installed instead of installing them on every run (see Docker deployment).
Tips
- Run the same commands locally and in CI. Keep the logic in Composer scripts or a
Makefileand keep the CI file thin. Debugging a script you can't run locally is slow. - Use
composer install, nevercomposer update, in CI (except for the deliberate lowest/highest matrix). See Difference between composer install and composer update. - Lint syntax for the lowest version if you have code that isn't covered by tests or analysis:
find src -name '*.php' -print0 | xargs -0 -n1 -P4 php -l. - Split slow suites. Run unit tests and static analysis on every push; put database, browser or external-API tests in a separate job, or use ParaTest (
vendor/bin/paratest) to run tests in parallel processes. - Don't let tests talk to real third-party services. Replace them with fakes (see Mocks and test doubles). Network calls make CI slow and flaky.
- Never print secrets. Pass credentials as CI secrets through environment variables. Workflows triggered by pull requests from forks don't get secrets on GitHub, so tests must not require them.
- Fix flaky tests immediately. A test that fails randomly teaches everyone to click "re-run" and ignore red builds. With random execution order, rerun with the printed seed to reproduce order-dependent failures.