app: coverage floor (86) + framework-glue exclusions
Measured before choosing: 86.17%, identical via `mix test --cover` and via the exported sample + `mix test.coverage`, stable across seeds. Floor is the integer below that, with the rule written next to it: a FLOOR, ratcheted up as behavior-test yaks land, never lowered (a drop means the change or an exclusion is wrong). Exclusions are the modules the suite cannot or should not exercise, each justified in the config comment: `use`-generated glue (Repo, Mailer, Telemetry, ErrorHTML/ErrorJSON), the `use FirehoseWeb` macro, production release tasks (Release), the lib_dev credo check (NoConnShadowing), and test scaffolding (DataCase, ConnCase, AccountsFixtures, Test.FakeBlog). Everything else counts, CoreComponents/controllers/LiveViews included. Wiring matches blogex and the fast/push split: `test.coverage` runs the suite once with `--cover --export-coverage default` (deleting stale exports first, since `mix test.coverage` imports every .coverdata it finds and would pool them), then the `coverage` sensor enforces the floor from that export. The sensor guards on the export existing because `mix test.coverage` exits 0 when it finds nothing — a floor check that did not run must not pass.
This commit is contained in:
+26
-8
@@ -3,7 +3,7 @@
|
||||
MISE_BIN ?= mise
|
||||
MISE_EXEC = $(MISE_BIN) exec --
|
||||
|
||||
.PHONY: check precommit deps compile test format credo verify
|
||||
.PHONY: check precommit deps compile test test.coverage coverage format credo verify
|
||||
|
||||
# Run all static analysis checks (no database required)
|
||||
check: credo format
|
||||
@@ -24,6 +24,20 @@ compile:
|
||||
test: deps compile
|
||||
$(MISE_EXEC) mix test
|
||||
|
||||
# Run the suite once with coverage exported, so the coverage floor check (and,
|
||||
# later, CRAP) reads the export instead of running the suite a second time.
|
||||
# Stale exports are removed first: the floor must never report an older run.
|
||||
test.coverage: deps compile
|
||||
rm -f cover/*.coverdata
|
||||
$(MISE_EXEC) mix test --cover --export-coverage default
|
||||
|
||||
# Enforce the coverage floor (mix.exs -> test_coverage) from that export. No
|
||||
# test run. `mix test.coverage` exits 0 when it finds no export at all, so the
|
||||
# guard below is load-bearing: a floor check that did not run must not pass.
|
||||
coverage:
|
||||
@test -f cover/default.coverdata || { echo "coverage: no exported coverage - run 'make test.coverage' first" >&2; exit 1; }
|
||||
$(MISE_EXEC) mix test.coverage
|
||||
|
||||
# Format code
|
||||
format:
|
||||
$(MISE_EXEC) mix format
|
||||
@@ -106,15 +120,19 @@ credo:
|
||||
# target reports as wired.
|
||||
# - sobelow (yak: sobelow in the Lens pass)
|
||||
# - coverage (yak: coverage floors + framework-glue exclusions policy)
|
||||
# WIRED below (test.coverage + coverage).
|
||||
# - crap (yak: CRAP scores with ex_crap (both apps))
|
||||
# - mutation (yak: mutation testing with muex — a sensor, not a gate)
|
||||
#
|
||||
# Runtime note: the coverage yak should make this export coverage from a single
|
||||
# suite run (test --cover --export-coverage) that CRAP reuses, instead of
|
||||
# running the suite twice.
|
||||
VERIFY_SENSORS :=
|
||||
# Runtime note: the coverage sensor exports from a single suite run
|
||||
# (`test.coverage` -> `mix test --cover --export-coverage default`) that CRAP is
|
||||
# meant to reuse, instead of running the suite twice.
|
||||
#
|
||||
# The suite runs FIRST, because it is what produces the coverage export the
|
||||
# sensors read; sensors still run cheapest-first among themselves.
|
||||
VERIFY_SENSORS := coverage
|
||||
|
||||
verify:
|
||||
@echo "==> verify(app): sensors [$(if $(VERIFY_SENSORS),$(VERIFY_SENSORS),none wired yet)]"
|
||||
@for sensor in $(VERIFY_SENSORS); do $(MAKE) $$sensor || exit 1; done
|
||||
@$(MAKE) test
|
||||
@echo "==> verify(app): suite (coverage exported), then sensors [$(VERIFY_SENSORS)]"
|
||||
@$(MAKE) test.coverage
|
||||
@for sensor in $(VERIFY_SENSORS); do echo "==> verify(app): sensor $$sensor"; $(MAKE) $$sensor || exit 1; done
|
||||
|
||||
Reference in New Issue
Block a user