Coverage
Code coverage measures which lines, branches, or statements your tests actually run. It’s an imperfect proxy for test quality — aim high but don’t worship the number. Use it to find untested code, enforce baselines in CI, and ratchet quality up over time.
Tools, thresholds, ratcheting, pitfalls
EXAMPLE
# 1) JavaScript / TypeScript — Vitest + V8 coverage
# vitest.config.ts
import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
coverage: {
provider: 'v8', # or 'istanbul' for older code
reporter: ['text', 'html', 'lcov', 'json-summary'],
include: ['src/**/*.{ts,tsx}'],
exclude: ['**/*.test.ts', '**/*.spec.ts', '**/__mocks__/**'],
thresholds: {
lines: 80,
functions: 80,
branches: 75,
statements: 80,
// perFile: true, # apply per-file (stricter)
},
},
},
});
npx vitest --coverage
# 2) Python — pytest + coverage.py
# pyproject.toml
[tool.coverage.run]
branch = true
source = ["src"]
omit = ["tests/*", "*/migrations/*"]
[tool.coverage.report]
fail_under = 80
show_missing = true
# pytest --cov=src --cov-report=term-missing --cov-fail-under=80
# 3) Go — built-in
go test ./... -coverprofile=coverage.out -covermode=count
go tool cover -func=coverage.out # per-function
go tool cover -html=coverage.out -o coverage.html # browsable
# Threshold check (CI):
go tool cover -func=coverage.out | tail -1 | awk '{print $3}' | sed 's/%//' | awk '{ if ($1 < 80) exit 1 }'
# 4) Rust — cargo-llvm-cov or tarpaulin
cargo install cargo-llvm-cov
cargo llvm-cov --html --output-dir target/cov
cargo llvm-cov --fail-under-lines 80
# 5) Java — JaCoCo via Maven / Gradle
# Maven
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.11</version>
<executions>
<execution>
<goals><goal>prepare-agent</goal></goals>
</execution>
<execution>
<id>report</id>
<phase>test</phase>
<goals><goal>report</goal></goals>
</execution>
<execution>
<id>jacoco-check</id>
<goals><goal>check</goal></goals>
<configuration>
<rules>
<rule>
<element>BUNDLE</element>
<limits>
<limit><counter>LINE</counter><value>COVEREDRATIO</value><minimum>0.80</minimum></limit>
<limit><counter>BRANCH</counter><value>COVEREDRATIO</value><minimum>0.70</minimum></limit>
</limits>
</rule>
</rules>
</configuration>
</execution>
</executions>
</plugin>
# 6) C# — coverlet + ReportGenerator
dotnet test --collect:"XPlat Code Coverage" --results-directory ./coverage
dotnet reportgenerator -reports:coverage/**/coverage.cobertura.xml -targetdir:coverage/html -reporttypes:HtmlInline_AzurePipelines
# Threshold via xunit.runner.json or coverlet.runsettings
# 7) GitHub Actions — typical flow
name: tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: '20', cache: 'npm' }
- run: npm ci
- run: npx vitest --coverage --reporter=verbose
- uses: actions/upload-artifact@v4
with: { name: coverage, path: coverage/ }
- uses: codecov/codecov-action@v4
with:
files: coverage/lcov.info
token: ${{ secrets.CODECOV_TOKEN }}
- name: Comment PR coverage
uses: davelosert/vitest-coverage-report-action@v2
if: github.event_name == 'pull_request'
# 8) Codecov / Coveralls — pick one
# • Codecov — feature-rich; comments PRs with diff coverage, charts; free for OSS
# • Coveralls — straightforward, reliable
# • SonarCloud — coverage + smells + complexity + security
# All accept lcov / cobertura / jacoco reports.
# 9) Diff coverage — the right baseline gate
# Don't gate on TOTAL coverage in a legacy codebase — you'll never get there.
# Gate on DIFF coverage: changed lines must hit X%.
# • Codecov: 'coverage > X% on the patch'
# • SonarCloud: 'new code' coverage condition
# Real-world targets: 90% on new code, 75-80% on changed lines overall.
# 10) Ratcheting up
# Each PR sets the floor slightly higher.
# • If current line coverage = 62%, set CI threshold to 62%
# • Each PR is allowed to keep it the same or raise it
# • Cannot LOWER it — fails CI
# • Over months, the floor climbs as devs naturally add tests with features
# 11) What coverage misses
# • Mutation testing (Stryker, mutmut) — does the assertion actually CATCH bugs?
# • Integration paths — unit coverage is great, but bugs live at seams
# • Performance regressions
# • Race conditions / concurrency
# 100% coverage with weak assertions is worse than 70% with sharp ones.
# 12) Excluding generated / framework code
# • Generated protobufs, ORMs, migrations
# • Boilerplate Hubris (constructors, getters with no logic)
# • Third-party stubs / mocks
# Configure via coverage tool's exclude option. Don't game the metric, but don't reward
# 'I tested my generated file' either.
# 13) Branch coverage > statement coverage
# Branch coverage catches missing 'else' paths. Most tools default to statements; flip to branch.
# Vitest: coverage.branches: 80
# pytest: branch = true
# JaCoCo: counter BRANCH
# 14) Local feedback loop
# • watch mode: npx vitest --coverage --watch
# • LCOV file → VS Code 'Coverage Gutters' extension shows uncovered lines inline
# • IntelliJ / WebStorm: 'Run with Coverage' overlay
# Make 'see uncovered' a one-keystroke action so devs use it daily.
# 15) Coverage in monorepos
# • Per-package thresholds (each workspace declares its own minimum)
# • Aggregate report at the repo root (merge lcov files)
# • Per-team ownership: 'platform' code at 90%, 'experimental' at 60%
# 16) Common bugs
# • Including tests in source pattern → fake-high coverage (tests test themselves)
# • Excluding business-logic 'because it's hard to test' → masks risk
# • Coverage flake on parallel test runners → use a final merge step
# • Threshold gate on a fresh repo at 80% then nothing changes → ratchet upward
# • Time-based flakes (Date.now) → freeze the clock; otherwise low branch coverage
# • Async paths uncovered — testing the happy path only; add rejected promise paths
# • Reporting only line coverage — branch tells the real story for conditionals
# • Coverage tool slows tests dramatically → use V8 native coverage (faster than Istanbul)
Why it matters
Treat coverage as a guardrail, not a target. Run branch coverage in CI, gate diff coverage on PRs at 80–90%, ratchet the baseline upward over time, and pair the metric with mutation testing or integration tests so 100% line coverage can’t mask weak assertions.
Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.
Example
Example
# Many runners enforce thresholds.
jest --coverage --coverage-threshold='{"global":{"branches":80,"lines":85}}'
Try it Yourself »
Discussion
Loading…