CARD: scaffold-d-differential-oracle — Differential / Characterization Oracle (Go)
01Discipline v0.2 · BETA · T2 · Go
02Reference instance of the AI-Native Pattern Card format, Go projection.
03Demonstrates all three bands, especially the operational Band 3.
04This card is itself BETA (its
conform checker is specified; the pilot instance is the research/go-demo fuzz
differential).
Band 1 — Identity & Recognition
05Classification: layer = E (Verification coupling); mechanism = scaffold class D.
06Intent: When code is replaced or refactored, pin its observable behavior with a runnable check that compares the new implementation against the old one (differential) or against a captured baseline (characterization), so that a reader — especially a weak one — can change code freely and receive a pass/fail signal on whether behavior moved. Go's projection stands on native fuzzing: the language ships the input generator, the corpus store, and the minimizer in the standard toolchain.
07Also Known As: golden test; characterization test (Feathers); approval test;
back-to-back test; differential fuzzing; FuzzXxx target; testdata/ corpus.
08Applicability / Recognition: Apply when ANY of these signals are present —
- 09a cell is being replaced or its internals rewritten while its contract is meant to stay fixed (the replacement protocol, R-040, guide §10);
- legacy behavior exists that nobody fully understands but must be preserved;
- a refactor spans multiple files and the reader cannot prove by inspection that behavior is unchanged;
- a weak agent is assigned a modification task and needs a safety net it cannot derive.
10Detector seed: a diff that modifies the body of an item carrying
//spec:implements … without a corresponding oracle artifact (a Fuzz/differential
test or a golden) in the cell's test files → recognition fires.
Band 2 — Justification & Tradeoffs
11Motivation: A weak agent is asked to optimize the naive planner cell. It rewrites
the diff loop. By inspection, neither the agent nor a fast reviewer can be sure the
change preserved behavior across edge states. With a differential fuzz target —
FuzzPlannersAgree feeding identical generated (desired, actual) state pairs to
naiveplanner and batchplanner and asserting equal action sets — the agent gets an
immediate mechanical verdict: behavior held, or here is a minimized counterexample the
fuzzer already shrank. The expensive cognition ("what are all the edge states?") was
materialized once as a harness plus a committed seed corpus; the weak agent consumes
the verdict.
12Structure & Participants:
- 13Subject-old — the prior implementation (kept temporarily, or captured as goldens in
testdata/). - Subject-new — the replacement.
- Input source — a
FuzzXxxtarget's generated inputs + the committedtestdata/seed corpus (runs deterministically in plaingo test;-fuzzexplores locally). - Comparator — the equality/equivalence predicate (deep-equal, or a documented divergence list).
- Oracle harness — the fuzz/differential test in the cell's
_test.go, run with-race.
14Collaborations: Pairs with Class B (defined types shrink the input space the oracle must cover) and Class C (contracts define what "equivalent" means). Consumes Class E (the per-package loop runs the seeds). Emits Class F diagnostics (a failure cites the violated REQ + the minimized counterexample). In a raid, this card is the differential-safety gate every behavior-changing card application must pass.
15Goals / Non-Goals:
- 16Goals: detect unintended behavior change during replacement/refactor; give weak readers a modification safety net; make "behavior preserved" a machine fact.
- Non-Goals: NOT a correctness proof (new-vs-old agreement inherits the old bugs); NOT a substitute for the spec; NOT for greenfield code with no prior behavior; NOT open-ended CI fuzzing (CI runs the committed seeds — deterministic; exploration is a local/scheduled activity).
17Consequences:
- 18(+) Aggressive refactoring becomes safe; drift is caught mechanically, with shrunk counterexamples for free (the toolchain minimizes).
- (+) "Change the implementation" and "preserve the contract" vary independently.
- (−) Authoring the input encoding for fuzz args (fuzzing takes primitive-typed args; structured states need a decode step) and the comparator costs effort.
- (−) Characterization goldens enshrine current behavior including bugs — pair with a
spec edge marking intentional vs incidental; goldens must fail loudly, never
auto-update (the
-updateflag never runs in CI).
19Alternatives:
- 20Formal proof: Go has no mainstream Kani/Creusot analogue — the differential/property oracle carries proportionally more of the modification-safety load here (the same honest asymmetry the TS card records).
- Manual review: fails exactly where needed (large multi-file edits, weak readers).
- Fresh unit tests: test what the author thought to test; the differential oracle tests behavior the author never enumerated.
21Risks & Assumptions:
- 22Assumes the old implementation is available or capturable.
- Assumes the fuzz encoding reaches representative states; a weak encoding gives false confidence — seed the corpus with the known hard cases.
- Sunset: if generation-time tooling plus contracts ever make behavior-preservation statically provable for a class of cells, the oracle retires for that class.
- Transfer risk: executable-scaffold value for modification is [E-mid]; this card is
a prime pilot validation target on
research/go-demo.
23Evidence & Transfer-strength: R-040 (replacement protocol, production), R2C-008 (executable scaffolds transformative, benchmark), Feathers characterization (production). Class: production + benchmark. Tag: [E-mid].
Band 3 — Operation
24Trigger: WHEN a diff modifies the body of an item bearing //spec:implements …,
OR a cell is marked for replacement (replaces= in its manifest directive), OR a
refactor touches > 1 file in a cell whose contract is unchanged — THEN apply before
merge.
25Mode: gate.
26Routine (≤7 steps, each verifiable):
- 27Identify the behavioral surface to preserve (the seam's methods).
- Keep
oldreachable (the replaced cell stays in the tree until the oracle is green), or capture goldens from it intotestdata/. - Write/extend a
FuzzXxxdifferential target decoding fuzz bytes into representative seam inputs; seedtestdata/with the known hard cases. - Bind
oldvsnewunder the comparator (documented divergence list otherwise). - Run seeds in the per-package loop (
go test -race); on a counterexample, fixnew(NOT the oracle) until green; let-fuzzexplore locally before landing. - Once green, remove
old(or commit the goldens) and leave the oracle + corpus in place. - Tag the oracle
//spec:verifies <uri> r=<N>.
28Checker: conform rule replacement-has-oracle — flags a modified
//spec:implements item whose cell lacks a differential/characterization test
referencing it; backed by go test ./<cell>/ -race running the seeds. (Status:
specified; pilot task.)
29Raid role: layer = behavior-preserving (wraps every behavior-changing card in any raid); batch = per-cell.
30Budget: gate-time, does not crowd the edit-time active set; first-signal = one per-package seed run (target < 60s; corpus size tuned to stay in budget).