SIGNALS Documentation
API Reference

Mutation Testing

Local-only playbook for Pest mutation testing — how to run it against a target path, read survivors, and when it is worth the cost.

Binding

Mutation testing is a local tool. It never runs in CI.

A full mutation run re-executes the covering tests once per mutant. On a suite this size that is hours, not minutes, and the score moves for reasons that have nothing to do with the change under review (a refactor that merges two branches changes the mutant count). Gating a pull request on it would buy noise at a very high price. The blocking gates stay where they are: the four test lanes, the 95% line-coverage floor, Pint, and PHPStan.

What mutation testing is good for is a targeted question, asked deliberately, about one class you have just written: do my tests actually check this logic, or do they merely execute it? Line coverage cannot tell you the difference. Mutation testing can.

Not to be confused with the mutation manifest

The repository contains a ProductionMutationManifest (tests/Support/ProductionMutationManifest.php, pinned data under framework-plans/remediation-tooling/). Despite the shared word, it is a completely different tool and nothing in this page applies to it.

Pest mutation testing ProductionMutationManifest
What "mutation" means An artificial edit to production source, made to see whether a test notices A write to production data — an INSERT/UPDATE/DELETE crossing a service-layer boundary
Purpose Measure test strength Review gate: every write boundary must be enumerated and reviewed
When it runs Manually, locally, on a chosen path In the suite, as a ratchet
Failure means Your tests are weak An unreviewed write boundary appeared

If you changed something under app/ and a manifest test failed, you need to re-pin the manifest — not read this page.

Running it

Pest ships mutation testing behind --mutate. Always scope it: an unscoped run is not useful and will not finish in a reasonable time.

# One class — the normal case.
vendor/bin/pest --mutate --covered-only \
    --path=app/Services/Pricing/DiscountResolver.php \
    tests/Feature/Pricing

# One directory, when you have just built a whole slice.
vendor/bin/pest --mutate --covered-only \
    --path=app/Actions/Invoices \
    tests/Feature/Actions/Invoices

Notes on the flags:

  • --path= scopes which production code gets mutated. Without it Pest mutates everything under the <source> block in phpunit.xml (all of app/).
  • The trailing argument scopes which tests run against each mutant. Pointing it at the directory that actually covers the target is the single biggest factor in run time.
  • --covered-only skips mutants on lines no test touches. Those are already reported by the coverage floor; mutation testing has nothing to add there.
  • --parallel works and is worth using. Do not combine it with --tia: Tia's cached replay defeats the entire point, because a mutant is a source change Tia has no dependency graph for.
  • The Pgsql and Browser lanes are excluded automatically by scoping the test path; do not aim a mutation run at tests/Pgsql.

Reading the output

Every mutant is one of:

  • Killed — a test failed when the code was altered. This is the good case: the behaviour is genuinely pinned.
  • Survived — the code was altered and the whole suite still passed. This is the finding. Some assertion you believe exists does not exist, or asserts something weaker than you think.
  • Uncovered — no test executes the line (suppressed by --covered-only).
  • Timed out / not executable — treat as noise, not as signal.

Only survivors are worth your time. For each one, read the mutant diff Pest prints and ask what behaviour would have to be true for both the original and the mutant to pass? The answer is the assertion you are missing.

Typical survivors and what they usually mean:

Mutation Usual cause
> → >= on a boundary The test only uses values far from the boundary. Add the exact-boundary case.
&& → `
Return value replaced with null/[] The caller's result is never asserted, only that it did not throw.
An if body removed entirely A side effect (event, audit log, cache flush) is never asserted. Event::fake() and assert it.
Arithmetic operator swapped A money or quantity calculation is asserted with a value where the operators coincide (0, 1, or equal operands).

Do not chase a score. A percentage target would push people towards assertions written to kill mutants rather than to describe behaviour, which is worse than no mutation testing at all. The output is a list of questions, and some of them have the legitimate answer "that mutant is equivalent" — an edit that genuinely cannot change observable behaviour. Those are not bugs in your tests and there is nothing to fix.

When to bother

Worth a run:

  • New invariant-bearing code: pricing and discount pipelines, tax resolution, availability arithmetic, rate calculation, state-machine guards, permission and cost-visibility checks. Anywhere a silently wrong number or a silently permissive branch is the failure mode.
  • A class you are about to declare finished that has 100% line coverage and suspiciously few assertions.
  • After fixing a bug that the suite should have caught but did not — mutate the class and find out what else it is not catching.

Not worth a run:

  • DTOs, enums, factories, migrations, seeders, and config.
  • Blade views and Livewire render paths.
  • Straight-line delegation — a controller that builds a DTO and calls an action.
  • Anything you are still actively changing. Run it once the design settles.

Workflow

  1. Finish the code and its tests. Get the relevant lane green.
  2. Run the scoped mutation command above.
  3. Triage survivors. Add the missing assertions to the existing tests — strengthening a test beats appending a new one written to kill a mutant.
  4. Re-run the scoped command to confirm the survivors are gone.
  5. Re-run the affected lane normally, then continue through the standard pre-commit gate. Nothing about the mutation run is committed — there is no baseline file, no score to pin, and no CI job to update.