CI lanes
Run oxmutate in CI the way the work is shaped: only the lines a pull request touched, a run split across a matrix, verdicts reused between runs, and a gate that fails a change which weakened its own tests.
A whole-project run is the slowest way to use a mutation tester and rarely the one CI needs.
Each lane below is a flag on oxmutate run, and they compose.
The pull-request lane: --since
npx oxmutate run --since origin/main --reporter clear-text,json,sarifOnly mutants on lines changed since the ref are scored; the rest of the project is not mutated.
--reporter sarif writes reports/mutation/mutation.sarif, which GitHub's code scanning shows
as an annotation on each surviving line of the diff:
- run: npx oxmutate run --since origin/${{ github.base_ref }} --reporter clear-text,json,sarif
- uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: reports/mutation/mutation.sarifCheck out with enough history for the ref to exist (fetch-depth: 0, or a fetch of the base
branch).
A gate on weaker tests: --frozen-oracle
npx oxmutate run --since origin/main --frozen-oracle origin/mainThe mutants are scored twice: against the change's tests, and against the base ref's versions of the test files the change edited. A mutant the base's tests killed that the change's tests let survive means the change removed or loosened an assertion. The run then exits 1 and names those mutants, whatever the score.
A matrix: --shard i/n
strategy:
matrix:
shard: [1, 2, 3, 4]
steps:
- run: npx oxmutate run --shard ${{ matrix.shard }}/4 --report-dir reports/shard-${{ matrix.shard }}Shards partition the mutants exactly: every mutant is in one shard and no mutant is in two, and the split is deterministic, so a re-run of shard 2 scores the same mutants.
Reuse between runs: --incremental
npx oxmutate run --incrementalA verdict is reused when neither its source file nor any of its covering tests changed since the
run that recorded it (reports/oxmutate-incremental.json, or incrementalFile). Cache that file
between CI runs to make a nightly full run pay only for what moved.
Fail on a score: --break
npx oxmutate run --break 80Exit 1 below a mutation score of 80, the same as thresholds.break in the config. The flag wins.
Exit codes
| Code | Meaning |
|---|---|
| 0 | The run completed, and passed its gates |
| 1 | It ran, and a gate failed: the score is under --break, or the change weakened its tests |
| 2 | A usage error: an unknown flag or a malformed value; rewrite the command |
| 3 | The config or the suite did not load: the dry run failed, or an instrumented file did not parse |
Where the published numbers will come from
oxmutate-bench.yml runs oxmutate and Stryker against the same burgee packages on a runner
nothing else shares, and its numbers are the ones to quote. It is added in M1 and has not run
yet; the figures measured so far are from a shared laptop, and
m1-results.md
says how far they can be trusted.
Configuration
Every key oxmutate reads: Stryker's keys, read from stryker.config.* as they stand, the keys only oxmutate reads, and the Stryker keys it does not use, each with the reason.
For agents
What a coding agent reads from oxmutate: one survivor record per mutant with a fix line, the --json envelope, an MCP server with run and survivors as tools, the schema, and exit codes that say whether to rewrite the command.