Last time, a 15-phase CRM build run through phased-loop.sh turned up five bugs in the loop scripts themselves. Two runs, about five hours each, to find them, the model being called over and over, laptop melting slowly.
Going back over that list, the thing that stands out isn’t the count. It’s that four of the five failed silently. The log froze two minutes in while the run kept going correctly for five more hours. Five phases built fine and were never committed, because a git add advisory exit code killed the && chain behind it. Parked phases left schema drift baked into a live SQLite file with zero trace in git history.
A loud failure costs you minutes. A silent one costs you the night, and you don’t find out until morning.
The discipline stopped where it got expensive
kodr2 is spec-first: 37 specs, 47 test files, a rule in AGENTS.md that says write the spec, add a failing test, then implement. All of it aimed at the JS harness.
The two bash scripts had no spec and no tests. 418 lines that drive the harness unattended for hours, filed under examples/, so the discipline never reached them. That’s not bad luck. That’s where the bugs were always going to be.
The seam was already there
Both scripts resolve kodr and jq off PATH behind command -v guards, and take everything else as environment variables. So you prepend a fixture directory to PATH and the scripts get a scripted kodr instead of the real one, unchanged. Everything else is real: a real temp git repo, a real git reset --hard, a real git clean -fd.
Not a model mock. No model is involved in whether a park cleans up after itself. It’s the same move runGoal’s own tests make with injected callbacks, just done through PATH because the thing under test is bash.
31 cases, nine seconds. Fifteen of them are the five bugs, written as regressions.
A regression test you have not seen fail is a hope
Easy to write a test that passes against fixed code and would also have passed against the broken code. So I pointed the same suite at the pre-fix scripts, pulled straight out of git:
git show 2f8f139:examples/loop.sh > /tmp/old/loop.sh
KODR_LOOP_SCRIPT_DIR=/tmp/old node --test test/loop-scripts.test.mjs11 of the 15 fail. The other four can’t discriminate by construction - three assert something doesn’t happen, and the feature didn’t exist back then to do it.
The guard needed a guard
Testing bash against a stub buys one new risk: the stub keeps emitting a shape the real CLI stopped producing. Every test passes, production parks the entire backlog. So there’s an eval that runs the real binary and checks the fields the scripts read.
First version asserted the scripts’ own jq expression still evaluates to a boolean. Worthless. jq collapses a missing field to a clean false:
$ echo '{"complete":true}' | jq -r '.completed and (.noOpCompletion | not)'
falseRename completed to complete and the expression answers false - identical to a genuinely red run. My drift detector would have passed happily while every single run read as not-green.
The fix is boring… assert the fields exist with the right types, and cross-check the jq expression against the same decision computed in JS. Then point the checker at deliberately renamed and retyped payloads in a unit test, so the detector is proven to bite without needing a model.
The actual rule
If it runs while you’re asleep, “does it work” is the wrong question. The question is how you’d know if it didn’t.
Four of five bugs answered that with “you wouldn’t, for about five hours.”
Facepalm.
Links: