Code coverage
suggest changeCode coverage tells you which parts of your code ran while the tests ran. A coverage report shows the lines, branches and methods that no test reached. Those are the places where a bug can hide without any test failing.
In Java the standard tool is JaCoCo (Java Code Coverage). It works as a Java agent: when the JVM loads a class, JaCoCo adds probes to the bytecode on the fly. Your source and compiled classes are never modified. When the JVM exits, the agent writes the collected data to a binary file (jacoco.exec). A separate step combines that file with your class files and sources to produce HTML, XML and CSV reports.
Maven, Gradle, IntelliJ IDEA, Eclipse (EclEmma), SonarQube and Codecov all use JaCoCo or read its formats. This article covers JaCoCo with Maven, Gradle and the command line, how to read the report, how to enforce minimum coverage, excluding code, multi-module projects, and the limits of coverage as a measure of test quality.
What JaCoCo measures
JaCoCo reports six counters:
| Counter | What it counts |
|---|---|
INSTRUCTION |
Bytecode instructions executed. The finest measure, independent of source formatting. |
BRANCH |
Outcomes of if, switch, ?:, &&, || and loop conditions. An if has 2 branches. |
LINE |
Source lines with at least one instruction executed. Needs classes compiled with debug info (the default for javac, Maven and Gradle). |
COMPLEXITY |
Cyclomatic complexity: the minimum number of paths that cover every branch. |
METHOD |
Methods (including constructors and lambdas) with at least one instruction executed. |
CLASS |
Classes with at least one method executed. |
Branch coverage is usually the most useful number. Line coverage can be 100% while half of the decisions were never tested both ways. For example, if (x > 0) count++; written on one line is "covered" as soon as it runs once, even if x > 0 was never false.
The example project
We'll use a small class with three decisions:
// src/main/java/com/example/shop/Discounts.java
package com.example.shop;
public final class Discounts {
private Discounts() {
}
public static int percentFor(int quantity, boolean member) {
if (quantity <= 0) {
throw new IllegalArgumentException("quantity must be positive");
}
int percent = 0;
if (quantity >= 10) {
percent = 5;
}
if (member) {
percent += 10;
}
return percent;
}
}
and a single, incomplete test:
// src/test/java/com/example/shop/DiscountsTest.java
package com.example.shop;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.assertEquals;
class DiscountsTest {
@Test
void bulkMemberGetsBothDiscounts() {
assertEquals(15, Discounts.percentFor(12, true));
}
}
The test passes. Without a coverage report you would not know that the exception path and the "not bulk" and "not a member" paths were never run.
JaCoCo with Maven
Add the jacoco-maven-plugin. The prepare-agent goal sets a Maven property named argLine that holds -javaagent:...jacocoagent.jar=destfile=target/jacoco.exec. Surefire reads argLine when it starts the test JVM. The report goal generates the reports after the tests.
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>shop</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<maven.compiler.release>21</maven.compiler.release>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.junit</groupId>
<artifactId>junit-bom</artifactId>
<version>6.0.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.5.4</version>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.15</version>
<executions>
<execution>
<id>prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
By default report is bound to the verify phase. Binding it to test as above means mvn test is enough to get a report:
$ mvn test
...
[INFO] --- jacoco:0.8.15:prepare-agent (prepare-agent) @ shop ---
[INFO] argLine set to -javaagent:/home/alice/.m2/repository/org/jacoco/org.jacoco.agent/0.8.15/org.jacoco.agent-0.8.15-runtime.jar=destfile=/home/alice/shop/target/jacoco.exec
...
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] --- jacoco:0.8.15:report (report) @ shop ---
[INFO] Loading execution data file /home/alice/shop/target/jacoco.exec
[INFO] Analyzed bundle 'shop' with 1 classes
The reports are written to target/site/jacoco/:
target/site/jacoco/
├── index.html <- open this in a browser
├── jacoco.xml <- for CI tools, SonarQube, Codecov
├── jacoco.csv <- one row per class
└── com.example.shop/
├── Discounts.html
└── Discounts.java.html
Pitfall: overriding argLine
If you configure Surefire's argLine yourself, for example to raise the heap or add --add-opens, you replace the value JaCoCo set. The tests still pass, but jacoco.exec is never written and the report step prints Skipping JaCoCo execution due to missing execution data file.
Use Surefire's late property evaluation, @{argLine}, to append to it instead:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.5.4</version>
<configuration>
<argLine>@{argLine} -Xmx1g --add-opens java.base/java.util=ALL-UNNAMED</argLine>
</configuration>
</plugin>
If JaCoCo is sometimes skipped (for example with -Djacoco.skip=true), @{argLine} is left unresolved and the JVM fails to start. Declare an empty default to avoid that:
<properties>
<argLine></argLine>
</properties>
The same problem happens with forkCount=0: the tests run inside the Maven JVM, so the agent never gets attached. Keep the default forkCount=1.
Reading the report
index.html shows a table per package and class:
| Element | Missed Instructions | Cov. | Missed Branches | Cov. | Missed Lines | Lines | Missed Methods | Methods |
|---|---|---|---|---|---|---|---|---|
| Discounts | ... | 73% | 3 of 6 | 50% | 1 | 8 | 0 | 1 |
The private empty constructor does not appear in the method count. JaCoCo filters it out on purpose, because a utility class constructor is never meant to be called.
Click through to Discounts.java.html to see the source, colored line by line:
- Green: every instruction on the line ran, and all its branches were taken.
- Yellow: the line ran, but some of its branches were not taken.
- Red: no instruction on the line ran.
Lines with decisions also get a diamond in the margin. Hovering over it shows a tooltip such as 1 of 2 branches missed.
For our test the source view looks like this (the colors are written in the margin here):
yellow ◆ if (quantity <= 0) { 1 of 2 branches missed.
red throw new IllegalArgumentException("quantity must be positive");
green int percent = 0;
yellow ◆ if (quantity >= 10) { 1 of 2 branches missed.
green percent = 5;
yellow ◆ if (member) { 1 of 2 branches missed.
green percent += 10;
green return percent;
Every if was only ever true or only ever false. That tells you exactly which tests to add:
package com.example.shop;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
class DiscountsTest {
@ParameterizedTest
@CsvSource({
"12, true, 15",
"12, false, 5",
" 3, true, 10",
" 3, false, 0",
})
void discount(int quantity, boolean member, int expected) {
assertEquals(expected, Discounts.percentFor(quantity, member));
}
@Test
void rejectsNonPositiveQuantity() {
assertThrows(IllegalArgumentException.class,
() -> Discounts.percentFor(0, true));
}
}
Run mvn test again and every line turns green with 6 of 6 branches covered. See Parameterized tests for more on @CsvSource.
The CSV report is easy to process in scripts. Its columns are:
GROUP,PACKAGE,CLASS,INSTRUCTION_MISSED,INSTRUCTION_COVERED,BRANCH_MISSED,BRANCH_COVERED,LINE_MISSED,LINE_COVERED,COMPLEXITY_MISSED,COMPLEXITY_COVERED,METHOD_MISSED,METHOD_COVERED
For example, to print branch coverage per class:
$ awk -F, 'NR > 1 && $6 + $7 > 0 { printf "%-30s %5.1f%%\n", $3, 100 * $7 / ($6 + $7) }' target/site/jacoco/jacoco.csv
Discounts 100.0%
Enforcing a minimum with the check goal
The check goal fails the build when coverage falls below the limits you set. Add a third execution:
<execution>
<id>check</id>
<goals>
<goal>check</goal>
</goals>
<configuration>
<rules>
<rule>
<element>BUNDLE</element>
<limits>
<limit>
<counter>LINE</counter>
<value>COVEREDRATIO</value>
<minimum>0.80</minimum>
</limit>
<limit>
<counter>BRANCH</counter>
<value>COVEREDRATIO</value>
<minimum>0.70</minimum>
</limit>
</limits>
</rule>
</rules>
</configuration>
</execution>
check is bound to verify by default, so run mvn verify. With only the original single test, the build fails:
$ mvn verify
...
[INFO] --- jacoco:0.8.15:check (check) @ shop ---
[INFO] Loading execution data file /home/alice/shop/target/jacoco.exec
[INFO] Analyzed bundle 'shop' with 1 classes
[WARNING] Rule violated for bundle shop: branches covered ratio is 0.50, but expected minimum is 0.70
[INFO] BUILD FAILURE
[ERROR] Failed to execute goal org.jacoco:jacoco-maven-plugin:0.8.15:check (check) on project shop: Coverage checks have not been met. See log for details.
A rule has these parts:
element: what each limit is applied to.BUNDLEis the whole module.PACKAGE,CLASS,SOURCEFILEandMETHODapply the limit to every package, class, file or method separately.counter:INSTRUCTION,BRANCH,LINE,COMPLEXITY,METHODorCLASS.value:COVEREDRATIO,MISSEDRATIO,COVEREDCOUNT,MISSEDCOUNTorTOTALCOUNT.minimum/maximum: the bound. Ratios are between 0 and 1.
A rule can also have <includes> and <excludes> with class name patterns. For example, here is a rule that says no class may have more than 10 lines without coverage, except generated code:
<rule>
<element>CLASS</element>
<excludes>
<exclude>com.example.shop.generated.*</exclude>
</excludes>
<limits>
<limit>
<counter>LINE</counter>
<value>MISSEDCOUNT</value>
<maximum>10</maximum>
</limit>
</limits>
</rule>
A practical way to introduce a threshold on an existing project is to set it at the current coverage, then raise it over time. This "ratchet" stops coverage from dropping without demanding a big effort up front.
JaCoCo with Gradle
Gradle ships a jacoco plugin. It attaches the agent to every Test task and adds two tasks: jacocoTestReport and jacocoTestCoverageVerification. Here is a build.gradle.kts:
plugins {
java
jacoco
}
repositories {
mavenCentral()
}
dependencies {
testImplementation(platform("org.junit:junit-bom:6.0.0"))
testImplementation("org.junit.jupiter:junit-jupiter")
testRuntimeOnly("org.junit.platform:junit-platform-launcher")
}
java {
toolchain {
languageVersion = JavaLanguageVersion.of(21)
}
}
jacoco {
toolVersion = "0.8.15"
}
tasks.test {
useJUnitPlatform()
finalizedBy(tasks.jacocoTestReport) // always produce a report after tests
}
tasks.jacocoTestReport {
dependsOn(tasks.test)
reports {
xml.required = true
html.required = true
csv.required = false
}
}
tasks.jacocoTestCoverageVerification {
violationRules {
rule {
limit {
counter = "BRANCH"
value = "COVEREDRATIO"
minimum = "0.70".toBigDecimal()
}
}
rule {
element = "CLASS"
excludes = listOf("com.example.shop.generated.*")
limit {
counter = "LINE"
value = "MISSEDCOUNT"
maximum = "10".toBigDecimal()
}
}
}
}
tasks.check {
dependsOn(tasks.jacocoTestCoverageVerification)
}
Always set toolVersion. The JaCoCo version bundled with Gradle may be older than your JDK, and an agent that doesn't understand the class file version fails with Unsupported class file major version.
$ ./gradlew check
> Task :jacocoTestCoverageVerification FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':jacocoTestCoverageVerification'.
> Rule violated for bundle shop: branches covered ratio is 0.50, but expected minimum is 0.70
The data file goes to build/jacoco/test.exec. Reports go to build/reports/jacoco/test/html/index.html and build/reports/jacoco/test/jacocoTestReport.xml.
Excluding code from coverage
Some code is not worth measuring: generated sources, main methods that only wire things together, DTOs produced by a code generator. There are three ways to keep it out of the numbers.
1. Exclude classes in the build. In Maven, set excludes on the plugin. The patterns match class file paths:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.15</version>
<configuration>
<excludes>
<exclude>com/example/shop/generated/**</exclude>
<exclude>**/*Application.class</exclude>
</excludes>
</configuration>
<!-- executions as above -->
</plugin>
Putting <excludes> at the plugin level applies it to prepare-agent (the classes are not instrumented), report and check.
In Gradle, filter the class directories of the report task, and do the same for jacocoTestCoverageVerification if you use it:
tasks.jacocoTestReport {
classDirectories.setFrom(
files(classDirectories.files.map {
fileTree(it) {
exclude("com/example/shop/generated/**", "**/*Application.class")
}
})
)
}
2. Annotate with @Generated. JaCoCo skips any class or method annotated with an annotation whose simple name contains Generated and whose retention is CLASS or RUNTIME. The annotation can come from any package, so you can define your own:
package com.example.shop;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
@Retention(RetentionPolicy.CLASS)
@Target({ElementType.TYPE, ElementType.METHOD, ElementType.CONSTRUCTOR})
public @interface ExcludeFromCoverageGenerated {
}
public class Main {
@ExcludeFromCoverageGenerated
public static void main(String[] args) {
new Server(Config.fromEnv()).start();
}
}
Note that javax.annotation.processing.Generated (the one many code generators use) has SOURCE retention, so it is gone from the class file and JaCoCo cannot see it. Lombok users can add lombok.addLombokGeneratedAnnotation = true to lombok.config, which makes Lombok put @lombok.Generated (class retention) on every method it generates.
Don't use this annotation to hide hand-written code you just don't want to test. That defeats the point of measuring.
3. Rely on built-in filters. JaCoCo already ignores a lot of compiler-generated bytecode that has no corresponding source you could test: synthetic methods, the values() and valueOf() methods of enums, private empty no-argument constructors, the hidden branches javac generates for try-with-resources, switch on strings and enums, pattern matching, and the equals, hashCode and toString methods the compiler generates for records. That is one reason to keep JaCoCo up to date: each release adds filters for newer language features.
Multi-module projects
Each Maven module gets its own report, which only counts tests in that module. If your service module's tests exercise classes in the core module, that coverage is missing from core's report.
The report-aggregate goal merges execution data from all modules a project depends on. The usual setup is a dedicated module that depends on every module you want in the report:
<!-- coverage/pom.xml -->
<artifactId>coverage</artifactId>
<packaging>pom</packaging>
<dependencies>
<dependency>
<groupId>com.example</groupId>
<artifactId>core</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.example</groupId>
<artifactId>service</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.15</version>
<executions>
<execution>
<id>report-aggregate</id>
<phase>verify</phase>
<goals>
<goal>report-aggregate</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
Every other module still needs prepare-agent, usually configured once in the parent POM. After mvn verify the combined report is in coverage/target/site/jacoco-aggregate/.
In Gradle 8 and later, apply the jacoco-report-aggregation plugin to one project and list the other projects as dependencies:
// coverage/build.gradle.kts
plugins {
base
id("jacoco-report-aggregation")
}
dependencies {
jacocoAggregation(project(":core"))
jacocoAggregation(project(":service"))
}
reporting {
reports {
val testCodeCoverageReport by creating(JacocoCoverageReport::class) {
testSuiteName = "test"
}
}
}
Run ./gradlew :coverage:testCodeCoverageReport.
Integration tests
Tests run by the Failsafe plugin (*IT.java) execute in their own JVM. Use the prepare-agent-integration and report-integration goals, which write to jacoco-it.exec and site/jacoco-it/. To get one combined number, use the merge goal to merge jacoco.exec and jacoco-it.exec, then run report on the merged file with <dataFile>.
If the integration tests start your application as a separate process (a Spring Boot jar, a server in a container), the build tool cannot attach the agent for you. Start that process with the agent yourself, as shown in the next section.
Using the agent directly
You don't need a build tool. Download the JaCoCo distribution zip from jacoco.org. It contains lib/jacocoagent.jar and lib/jacococli.jar. Attach the agent to any JVM with -javaagent:
$ java -javaagent:jacocoagent.jar=destfile=jacoco.exec,includes=com.example.* \
-jar app.jar
The most useful agent options are:
| Option | Meaning |
|---|---|
destfile |
Where to write execution data (default jacoco.exec). |
append |
true (default) adds to an existing file, false overwrites it. |
includes / excludes |
Class name patterns to instrument or skip, separated by :. * and ? are wildcards. |
output |
file (default), tcpserver or tcpclient, for collecting data from a running server. |
jmx |
true exposes an MBean so you can dump data without stopping the JVM. |
The data is written when the JVM shuts down normally. A process killed with SIGKILL (kill -9, or a container stopped too hard) loses it. For long-running servers use output=tcpserver and fetch the data on demand:
$ java -javaagent:jacocoagent.jar=output=tcpserver,address=*,port=6300 -jar app.jar &
# ... run your end-to-end tests against the server ...
$ java -jar jacococli.jar dump --address localhost --port 6300 --destfile e2e.exec
Then generate a report from the data, your compiled classes and the sources. Use the exact class files that ran: JaCoCo matches classes by a checksum, so recompiled classes produce Execution data for class ... does not match warnings and show up as not covered.
$ java -jar jacococli.jar report e2e.exec \
--classfiles target/classes \
--sourcefiles src/main/java \
--html coverage-html \
--xml coverage.xml \
--name "End-to-end tests"
[INFO] Loading execution data file /home/alice/shop/e2e.exec.
[INFO] Analyzing 1 classes.
jacococli.jar merge a.exec b.exec --destfile all.exec combines data from several runs, for example unit tests plus end-to-end tests.
Coverage in CI
Most CI services and code review tools read jacoco.xml. A typical GitHub Actions workflow runs the build and uploads the report:
name: build
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-java@v4
with:
distribution: temurin
java-version: 21
cache: maven
- run: mvn -B verify
- uses: actions/upload-artifact@v4
with:
name: coverage-report
path: target/site/jacoco/
Services like Codecov and SonarQube can also comment on pull requests with the coverage of changed lines. That is often a better gate than a global percentage: it asks new code to be tested without forcing anyone to backfill tests for old code.
Coverage in the IDE
IntelliJ IDEA has Run with Coverage on every test run configuration. It can use its own coverage engine or JaCoCo, and it colors the editor gutter green, yellow and red. In Eclipse, EclEmma (built in to most Eclipse packages) does the same with JaCoCo. This is the fastest feedback loop while you're writing tests. Use the build tool's report for CI and for numbers you want to compare over time.
What coverage does not tell you
Coverage measures what ran, not what was checked. This test reaches 100% line and branch coverage of Discounts and verifies nothing:
@Test
void runsEverything() {
Discounts.percentFor(12, true);
Discounts.percentFor(3, false);
try {
Discounts.percentFor(0, false);
} catch (IllegalArgumentException expected) {
}
}
Keep this in mind when reading the numbers:
- Low coverage is a clear signal; high coverage is a weak one. Red code is definitely untested. Green code may still be untested in any useful sense.
- Branch coverage is not path coverage. Three independent
ifs have 6 branches but 8 paths. Two tests can cover all branches while missing a bug that only happens for one combination. - Missing code is invisible. If you forgot to handle
nullor an overflow, there is no line to turn red. - 100% is rarely worth chasing. The last few percent are often defensive code and error handling that cost a lot to reach. A target around 70-85% for branches, combined with reviewing what is not covered, works well for most projects.
Mutation testing
Mutation testing checks whether your tests actually notice changes in behavior. A tool makes small changes ("mutants") to your bytecode, such as turning >= into > or += 10 into -= 10. Then it runs the tests against each mutant. If the tests still pass, the mutant "survived" and you have a test that runs the code without checking it.
PIT (pitest) is the standard mutation testing tool for Java:
<plugin>
<groupId>org.pitest</groupId>
<artifactId>pitest-maven</artifactId>
<version>1.30.1</version>
<dependencies>
<dependency>
<groupId>org.pitest</groupId>
<artifactId>pitest-junit5-plugin</artifactId>
<version>1.2.3</version>
</dependency>
</dependencies>
<configuration>
<targetClasses>
<param>com.example.shop.*</param>
</targetClasses>
</configuration>
</plugin>
$ mvn test-compile org.pitest:pitest-maven:mutationCoverage
...
================================================================================
- Statistics
================================================================================
>> Line Coverage (for mutated classes only): 8/10 (80%)
>> 2 tests examined
>> Generated 7 mutations Killed 1 (14%)
>> Mutations with no coverage 0. Test strength 14%
PIT counts the private constructor's lines, which JaCoCo filters out, hence 8/10. With the assertion-free test above, only one of the seven mutants is killed: negating quantity <= 0 makes the call with 12 throw. The other six survive. With the parameterized tests from earlier, all mutants but one are killed. The survivor changes quantity >= 10 to quantity > 10: no test uses exactly 10, so the boundary is untested even though branch coverage is 100%. Adding the rows "10, false, 5" and "9, false, 0" kills it. The HTML report in target/pit-reports/ shows each surviving mutant on its line. Mutation testing is much slower than plain coverage, so it is usually run on demand or nightly rather than on every build.
Summary
- JaCoCo is the standard Java coverage tool. It instruments classes at load time through a Java agent and writes
jacoco.exec. - In Maven use
prepare-agent,reportandcheck. If you set Surefire'sargLine, write@{argLine}to keep the agent. - In Gradle apply the
jacocoplugin, settoolVersion, and wirejacocoTestReportandjacocoTestCoverageVerificationinto the build. - Look at branch coverage and at the yellow and red lines in the HTML report, not just the headline percentage.
- Exclude generated code with build excludes or a
@Generatedannotation with class retention. - Use
report-aggregate(Maven) orjacoco-report-aggregation(Gradle) to combine multi-module coverage. - Coverage shows what ran, not what was verified. Use assertions that matter, and consider mutation testing with PIT.
See Unit Testing for writing the tests themselves.