Testing in CI

suggest change

Continuous integration (CI) runs your checks on every push and pull request on a clean machine. For a PHP project that usually means:

  1. install dependencies with Composer from composer.lock
  2. check code style (PHP CS Fixer or PHP_CodeSniffer)
  3. run static analysis (PHPStan or Psalm)
  4. run the test suite (PHPUnit or Pest) on every supported PHP version
  5. optionally collect code coverage and test reports

A CI job passes or fails based on the exit code of each command: 0 means success, anything else fails the job. All the tools below follow that rule, so the work is mostly picking the right flags and making the same commands easy to run locally.

The examples use PHPUnit 13 (requires PHP 8.4+), PHPStan 2 and PHP CS Fixer 3. Writing the tests themselves is covered in Unit Testing, PHPUnit Data Providers and Mocks and test doubles.

A project to test

The project layout used in this article:

.
├── composer.json
├── composer.lock
├── phpunit.xml.dist
├── phpstan.dist.neon
├── .php-cs-fixer.dist.php
├── src/
│   └── PriceCalculator.php
└── tests/
    ├── Unit/
    │   └── PriceCalculatorTest.php
    └── Integration/
        └── OrderRepositoryTest.php

Install the tools as dev dependencies, so CI and every developer use exactly the versions locked in composer.lock:

composer require --dev phpunit/phpunit:^13 phpstan/phpstan:^2 friendsofphp/php-cs-fixer:^3

Commit composer.lock, even for libraries. Without it, every CI run resolves dependencies again and a new release of some package can break the build for reasons unrelated to your change.

One command for everything: Composer scripts

Put the exact commands CI runs into composer.json. Developers then run composer ci before pushing and get the same result as the CI server:

{
    "name": "acme/shop",
    "type": "project",
    "license": "proprietary",
    "require": {
        "php": ">=8.4"
    },
    "require-dev": {
        "friendsofphp/php-cs-fixer": "^3.95",
        "phpstan/phpstan": "^2.3",
        "phpunit/phpunit": "^13.4"
    },
    "autoload": {
        "psr-4": { "App\\": "src/" }
    },
    "autoload-dev": {
        "psr-4": { "App\\Tests\\": "tests/" }
    },
    "scripts": {
        "test": "phpunit",
        "test:coverage": "phpunit --coverage-clover=build/coverage.xml --coverage-text",
        "analyse": "phpstan analyse --no-progress",
        "cs": "php-cs-fixer check --diff",
        "ci": ["@cs", "@analyse", "@test"]
    },
    "config": {
        "sort-packages": true
    }
}

Composer adds vendor/bin to PATH while running scripts, so phpunit means vendor/bin/phpunit. A script with a list of commands stops at the first command that fails and returns its exit code:

$ composer ci
...
Found 3 of 3 files that can be fixed in 0.011 seconds, 18.00 MB memory used
Script php-cs-fixer check --diff handling the cs event returned with error code 8
Script @cs was called via ci
$ echo $?
8

Configuring PHPUnit for CI

Commit the configuration as phpunit.xml.dist. PHPUnit uses phpunit.xml if it exists and falls back to phpunit.xml.dist, so a developer can keep a local, git-ignored override.

<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
         bootstrap="vendor/autoload.php"
         colors="true"
         cacheDirectory=".phpunit.cache"
         executionOrder="random"
         failOnRisky="true"
         failOnWarning="true"
         failOnDeprecation="true"
         failOnNotice="true"
         beStrictAboutOutputDuringTests="true">
    <testsuites>
        <testsuite name="unit">
            <directory>tests/Unit</directory>
        </testsuite>
        <testsuite name="integration">
            <directory>tests/Integration</directory>
        </testsuite>
    </testsuites>
    <source>
        <include>
            <directory>src</directory>
        </include>
    </source>
    <php>
        <ini name="error_reporting" value="-1"/>
        <env name="APP_ENV" value="test" force="true"/>
        <env name="DATABASE_URL" value="sqlite::memory:"/>
    </php>
</phpunit>

What matters for CI:

A run with a failing test shows the data set name and exits with 1:

$ vendor/bin/phpunit --testsuite unit
PHPUnit 13.4.3 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.5.9
Configuration: /home/kjk/shop/phpunit.xml.dist
Random Seed:   1792257264

F...                                                                4 / 4 (100%)

Time: 00:00.001, Memory: 20.00 MB

There was 1 failure:

1) App\Tests\Unit\PriceCalculatorTest::testTotal@with tax with data (1000, 2, 0.23, 2461)
Failed asserting that 2460 is identical to 2461.

/home/kjk/shop/tests/Unit/PriceCalculatorTest.php:16

FAILURES!
Tests: 4, Assertions: 4, Failures: 1.
$ echo $?
1

Command-line options that are useful in CI:

Option Purpose
--testsuite unit run one suite, e.g. fast unit tests in a separate job
--group db / --exclude-group db select tests by #[Group('db')] attribute
--log-junit build/junit.xml JUnit XML report that CI systems display as a test summary
--coverage-clover <file> coverage in Clover XML (Codecov, Coveralls, SonarQube)
--coverage-cobertura <file> coverage in Cobertura XML (GitLab merge request diff coverage)
--coverage-text coverage summary printed to the job log
--stop-on-defect stop at the first problem for a faster red build
--fail-on-empty-test-suite fail if a filter or a wrong path selected no tests
--teamcity TeamCity service messages instead of the normal output
--colors=never plain output for log parsers

Static analysis and code style

PHPStan reads its configuration from phpstan.dist.neon:

parameters:
    level: 8
    paths:
        - src
        - tests

When PHPStan detects it is running in GitHub Actions (the GITHUB_ACTIONS environment variable is set), it prints GitHub annotations in addition to the usual table, so errors show up inline in the pull request diff:

$ vendor/bin/phpstan analyse --no-progress
::error file=tests/Integration/OrderRepositoryTest.php,line=27,col=0::Cannot call method fetchColumn() on PDOStatement|false.
 ------ ---------------------------------------------------------
  Line   tests/Integration/OrderRepositoryTest.php
 ------ ---------------------------------------------------------
  27     Cannot call method fetchColumn() on PDOStatement|false.
         🪪  method.nonObject
 ------ ---------------------------------------------------------

 [ERROR] Found 1 error

Use --error-format=github to force that format, or --error-format=gitlab for GitLab's code quality report.

When adding PHPStan to an existing project, start with a baseline instead of lowering the level. vendor/bin/phpstan analyse --generate-baseline writes the current errors to phpstan-baseline.neon; include it in the configuration and CI only fails on new errors:

includes:
    - phpstan-baseline.neon

parameters:
    level: 8
    paths:
        - src
        - tests

PHP CS Fixer is configured in .php-cs-fixer.dist.php. check (an alias for fix --dry-run) only reports; in CI it exits with a non-zero code when any file needs fixing:

<?php
$finder = (new PhpCsFixer\Finder())->in([__DIR__ . '/src', __DIR__ . '/tests']);

return (new PhpCsFixer\Config())
    ->setRules(['@PER-CS' => true])
    ->setFinder($finder);

Run the style check on the lowest PHP version you support. PHP CS Fixer warns when the runtime is newer than the php constraint in composer.json, because a fixer could rewrite code using syntax the older version doesn't have.

GitHub Actions

shivammathur/setup-php installs any PHP version with the extensions, ini settings, coverage driver and tools you ask for. A complete workflow in .github/workflows/ci.yml:

name: CI

on:
  push:
    branches: [master]
  pull_request:

# cancel an older run of the same branch when a new commit is pushed
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

permissions:
  contents: read

jobs:
  quality:
    name: Code style and static analysis
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5

      - uses: shivammathur/setup-php@v2
        with:
          php-version: '8.4'      # lowest supported version
          coverage: none
          tools: cs2pr

      - uses: ramsey/composer-install@v3   # composer install + dependency cache

      - name: Validate composer.json
        run: composer validate --strict --no-check-publish

      - name: Security audit
        run: composer audit

      - name: PHP CS Fixer
        run: vendor/bin/php-cs-fixer check --format=checkstyle | cs2pr

      - name: PHPStan
        run: vendor/bin/phpstan analyse --no-progress

  tests:
    name: PHP ${{ matrix.php }} (${{ matrix.dependencies }})
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        php: ['8.4', '8.5']
        dependencies: [locked]
        include:
          - php: '8.4'
            dependencies: lowest
    steps:
      - uses: actions/checkout@v5

      - uses: shivammathur/setup-php@v2
        with:
          php-version: ${{ matrix.php }}
          extensions: intl, pdo_sqlite
          ini-values: error_reporting=-1, display_errors=On, zend.assertions=1
          coverage: none

      - uses: ramsey/composer-install@v3
        with:
          dependency-versions: ${{ matrix.dependencies }}

      - name: Run tests
        run: vendor/bin/phpunit --log-junit build/junit.xml

      - name: Upload test report
        if: ${{ !cancelled() }}
        uses: actions/upload-artifact@v4
        with:
          name: junit-php${{ matrix.php }}-${{ matrix.dependencies }}
          path: build/junit.xml

Notes:

Without ramsey/composer-install, cache Composer's download directory yourself:

      - name: Get Composer cache directory
        id: composer-cache
        run: echo "dir=$(composer config cache-files-dir)" >> "$GITHUB_OUTPUT"

      - uses: actions/cache@v4
        with:
          path: ${{ steps.composer-cache.outputs.dir }}
          key: composer-${{ runner.os }}-${{ matrix.php }}-${{ hashFiles('composer.lock') }}
          restore-keys: composer-${{ runner.os }}-${{ matrix.php }}-

      - run: composer install --no-interaction --no-progress --prefer-dist

Cache the Composer download cache, not vendor/. Restoring vendor/ built for a different PHP version or OS causes confusing failures.

Testing the next PHP version

To find problems before a new PHP release, add a job for the development version and allow it to fail:

  tests-nightly:
    runs-on: ubuntu-latest
    continue-on-error: true
    steps:
      - uses: actions/checkout@v5
      - uses: shivammathur/setup-php@v2
        with:
          php-version: nightly
          coverage: none
      - uses: ramsey/composer-install@v3
        with:
          composer-options: --ignore-platform-req=php+
      - run: vendor/bin/phpunit

--ignore-platform-req=php+ lets Composer install packages whose php constraint doesn't allow the new version yet.

Integration tests against a real database

Service containers start databases next to the job. The test reads its connection from environment variables, with SQLite as the local default:

<?php
declare(strict_types=1);

namespace App\Tests\Integration;

use PHPUnit\Framework\Attributes\Group;
use PHPUnit\Framework\TestCase;

#[Group('db')]
final class OrderRepositoryTest extends TestCase
{
    private \PDO $pdo;

    protected function setUp(): void
    {
        $dsn = getenv('DATABASE_URL') ?: 'sqlite::memory:';
        $this->pdo = new \PDO($dsn, getenv('DB_USER') ?: null, getenv('DB_PASSWORD') ?: null, [
            \PDO::ATTR_ERRMODE => \PDO::ERRMODE_EXCEPTION,
        ]);
        $this->pdo->exec('CREATE TEMPORARY TABLE orders (id INT PRIMARY KEY, total INT NOT NULL)');
    }

    public function testInsertAndRead(): void
    {
        $this->pdo->exec('INSERT INTO orders (id, total) VALUES (1, 2460)');
        $stmt = $this->pdo->query('SELECT total FROM orders WHERE id = 1');
        $this->assertNotFalse($stmt);
        $this->assertSame(2460, (int) $stmt->fetchColumn());
    }
}

The job runs the same test against PostgreSQL. The options health check makes the job wait until the server accepts connections:

  integration:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:17
        env:
          POSTGRES_USER: app
          POSTGRES_PASSWORD: secret
          POSTGRES_DB: app_test
        ports:
          - 5432:5432
        options: >-
          --health-cmd "pg_isready -U app"
          --health-interval 5s
          --health-timeout 5s
          --health-retries 10
    env:
      DATABASE_URL: pgsql:host=127.0.0.1;port=5432;dbname=app_test
      DB_USER: app
      DB_PASSWORD: secret
    steps:
      - uses: actions/checkout@v5
      - uses: shivammathur/setup-php@v2
        with:
          php-version: '8.5'
          extensions: pdo_pgsql
          coverage: none
      - uses: ramsey/composer-install@v3
      - run: vendor/bin/phpunit --testsuite integration

For MySQL use the mysql:8.4 image with MYSQL_ROOT_PASSWORD / MYSQL_DATABASE, the health command mysqladmin ping -h 127.0.0.1, the pdo_mysql extension and a DSN like mysql:host=127.0.0.1;port=3306;dbname=app_test. Use 127.0.0.1, not localhost: with localhost the MySQL client tries a Unix socket that doesn't exist on the runner.

Code coverage

Coverage needs a driver. Locally, PHPUnit without one prints a warning and runs no tests:

$ vendor/bin/phpunit --coverage-text
...
There was 1 PHPUnit test runner warning:

1) No code coverage driver available that supports line coverage

No tests executed!

In CI use PCOV, which is much faster than Xdebug and only does coverage. Run coverage in one matrix job only, because it slows the tests down:

  coverage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: shivammathur/setup-php@v2
        with:
          php-version: '8.5'
          coverage: pcov
      - uses: ramsey/composer-install@v3
      - run: vendor/bin/phpunit --coverage-clover build/coverage.xml --coverage-text
      - run: php bin/coverage-check.php build/coverage.xml 80
      - uses: codecov/codecov-action@v5
        with:
          files: build/coverage.xml
          token: ${{ secrets.CODECOV_TOKEN }}

PHPUnit has no option for a minimum coverage percentage. A short script that reads the Clover report fails the build below a threshold:

<?php
declare(strict_types=1);

// Usage: php bin/coverage-check.php build/coverage.xml 80
[, $file, $min] = $argv + [null, null, null];
if ($file === null || $min === null) {
    fwrite(STDERR, "usage: coverage-check.php <clover.xml> <min-percent>\n");
    exit(2);
}

$xml = simplexml_load_file($file) ?: throw new RuntimeException("Cannot read $file");
$metrics = $xml->project->metrics;
$total = (int) $metrics['statements'];
$covered = (int) $metrics['coveredstatements'];
$percent = $total === 0 ? 100.0 : $covered / $total * 100;

printf("Line coverage: %.2f%% (%d/%d statements), required: %d%%\n", $percent, $covered, $total, $min);
exit($percent >= (float) $min ? 0 : 1);
$ php bin/coverage-check.php build/coverage.xml 80
Line coverage: 75.00% (3/4 statements), required: 80%
$ echo $?
1

Treat the number as a guard against coverage dropping, not as a goal. Set the threshold a little below the current value and raise it over time.

GitLab CI

The same commands work in any CI system. A .gitlab-ci.yml using the official PHP image:

default:
  image: php:8.5-cli
  before_script:
    - apt-get update && apt-get install -y git unzip
    - curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
    - composer install --no-interaction --no-progress --prefer-dist
  cache:
    key:
      files: [composer.lock]
    paths: [.composer-cache/]

variables:
  COMPOSER_CACHE_DIR: "$CI_PROJECT_DIR/.composer-cache"

phpstan:
  script:
    - vendor/bin/phpstan analyse --no-progress --error-format=gitlab > phpstan.json
  artifacts:
    when: always
    reports:
      codequality: phpstan.json

phpunit:
  script:
    - pecl install pcov && docker-php-ext-enable pcov
    - vendor/bin/phpunit --colors=never --log-junit junit.xml --coverage-text --coverage-cobertura cobertura.xml
  coverage: '/^\s*Lines:\s*\d+\.\d+\%/'
  artifacts:
    when: always
    reports:
      junit: junit.xml
      coverage_report:
        coverage_format: cobertura
        path: cobertura.xml

The coverage regular expression extracts the percentage from the Lines: line of --coverage-text, and GitLab shows it on the merge request. --colors=never keeps ANSI codes out of the text the regex has to match. In a real pipeline, build an image with Composer and the extensions already installed instead of installing them on every run (see Docker deployment).

Tips

Feedback about page:

Feedback:
Optional: your email if you want me to get back to you:



Table Of Contents