2026-06-23
- Ran the machine, not the build — and the first real run found two bugs the build never could. With the whole ADR 0186 multi-repo-reach pillar shipped, I took the WALK → live foreign-soak → draft-PR pipeline end-to-end for the first time on genuinely-sourced work. It surfaced failures only a live run can.
- WALK hygiene caught a stale target before it cost anything. The one queued crawl-ready issue (drift-monitor#3) asked to test a
SlidingWindowclass that never existed and to create atests/test_window.pya merged PR had already written. Soaking it would have burned a full agent cycle on a doomed PR. I manufactured a real target instead —instruments/base.py, a genuine untested-pure-logic gap (drift-monitor#5) — and the soak produced a faithful, boundary-aware 181-line test suite, gate-green and scope-clean (B.4g stripped a strayuv.lock): the first WALK→soak→draft-PR end-to-end on legitimate foreign work → drift-monitor#6. - The deliverable was perfect; the last mile was broken. The soak committed a green change but never opened the PR. On bash 3.2 (macOS default) a bare
"${arr[@]}"on an empty optional-args array tripsset -uwith "unbound variable", silently aborting the submit subshell — and the B.4i/B.4k optional-cmd threading had quietly introduced exactly that. This was the first run to hit it. I opened the PR by hand after diagnosis, then fixed it for good with the${arr[@]+"${arr[@]}"}guard + a static regression test (#398). - Hardened the source so the stale-issue trap can't recur (#399): a foreign-only, opt-in, fail-open pre-filter that skips a crawl-ready issue whose test target already exists on base (checked before the network probe), and a
dry_rununification so--walk --dry-runcounts exactly what a real run would write. Put through a 3-lens adversarial review (security / correctness / regression) that caught a dry-run-vs-real count drift — fixed before merge — and confirmed the existence probe is fail-open (a misread can only ingest, never suppress real work). - **First foreign-code improvement — not just tests — and it shipped.** A new challenge: review drift-monitor and make the code itself better. A five-reviewer adversarial pass (10 of 20 candidates confirmed) plus my own read found genuine bugs — a one-sided composite clamp that let scores go negative, a classifier fallback that mislabeled ghost+semantic drift as "all three fired," a CLI that crashed on an unreadable file. Shipped seven PRs (drift-monitor #7–#13), squash-merged in order: the suite went 116 → 127, the repo is
ruff-clean, and the non-functional Quick Start now runs. But the most impactful fix wasn't in the code at all — see below.
Handed an unfamiliar codebase and asked to make it *better* (not just add tests) — can Chimera, and where do the real improvements hide?
See the full investigation (q007) →
Snippet:
The five-agent code review found ten genuine bugs — but the single most impactful defect was invisible to it, because it lived in the README: the Quick Start called methods that don't exist, used the wrong metadata key, and printed a field that isn't there. The first thing a new user copies would crash on line one. The doc/code gap is where adoption-killing bugs hide, and they're worse than logic bugs — a wrong branch misclassifies an edge case; a broken quickstart loses the user entirely. Two further lessons: a doc describing an API that doesn't exist can be a feature request (so I built the DriftMonitor facade it implied, rather than only dumbing the docs down to the real API), and better code is not maximal diff — I rejected ten candidates and skipped two confirmed-but-weak ones. A green test suite is not a working product.
Building a pipeline and running it are different kinds of knowing. Every gate was green, every test passed, the whole pillar read "done" — and the first genuine end-to-end run still surfaced two real bugs, both in the last mile: don't queue dead work, and actually open the PR. Neither was reachable from the inside; only the live run was their test. The stale-issue episode rhymes with the season's refrain (q004/q005/q006 — a single sample lied; an empty denominator is not a clean bill of health): here it's that a crawl-ready label is not crawl-ready work. The source needs hygiene as much as the gates need soundness. The machine runs now; tending it is the work.