The shadow model judged by the old model’s evidence
BrambleGrid is a fictional grid-asset inspection company. Regional electric utilities hire it to combine drone imagery, asset records, and certified field inspection so scarce crews visit pole-top equipment that most needs a closer look.
A drone could photograph thousands of connector assemblies in a morning. A physical visit was different. It needed access coordination, a qualified crew, and enough time to perform the same close-range inspection protocol on every asset. BrambleGrid had capacity for 240 flexible visits per week. Twenty always remained reserved for manual safety reports that did not come from a model.
That made field inspection both an operating resource and an evidence resource.
BrambleGrid's ML-assisted product was called Spanwatch. For one recent connector evidence snapshot, Spanwatch estimated the probability that a protocol-C3 inspection within seven days would find an actionable connector condition. It did not predict an outage, and it could not authorize a repair.
Dalia Moravec (the ML reliability engineer responsible for calibration, evaluation cohorts, and launch recommendations) owned Spanwatch's model evidence.
Jon Ibarra (the field-inspection planning lead responsible for the weekly 240-visit capacity ledger) owned which proposed visits entered the field workflow.
Mara Venn (the lead asset-integrity engineer responsible for protocol-C3 adjudication and label revision) owned the final condition record. A utility duty engineer—not Spanwatch, Dalia, Jon, or Mara—separately decided whether to restrict or repair an asset.
The boundaries looked fussy until the week BrambleGrid tried to replace its model.
How one connector becomes evidence
The journey matters because its handoffs prevent a fluent model score from quietly becoming an operational fact. Admission, physical visit state, condition adjudication, and utility action remain separate authorities; the evaluation can make a bounded claim only by retaining those handoffs.
Three things in that path were easy to collapse in conversation.
The score estimated what a C3 inspection would find. The condition label recorded what a completed C3 inspection did find. Between them sat a : a versioned rule that combined the score, thresholds, capacity, and current context to choose inspect, review, or defer.
The threshold was not part of the model. A score did not spend a visit. A visit did not authorize repair. Repair did not prove that a connector would otherwise have failed.
Those distinctions were already reflected in BrambleGrid's implementation. Large images lived in restricted object storage, while small events carried evidence references. Point-in-time features fed two PyTorch models behind a scoring service. A Kotlin policy service owned thresholds. A PostgreSQL-backed FieldOps service and capacity ledger owned admission and visit state. A separate Condition Registry owned C3 observations and append-only revisions. Iceberg tables, dbt jobs, and MLflow identities supported evaluation.
The tool names were replaceable. The authority boundaries were not.
Concepts in this story5 concepts
A versioned rule that maps model evidence, thresholds, current context, capacity, and constraints into an action such as automate, inspect, review, defer, or abstain. The model supplies evidence; the policy owns the action boundary.
Running a candidate model beside the live system without letting it control production actions. This limits exposure, but observed outcomes still reflect the incumbent policy unless the evaluation creates an independent evidence path.
Labels observed for a non-random subset because a prior decision policy determines which cases receive the action, inspection, approval, review, or follow-up that can reveal an outcome.
The explicit register of eligible units available for selection into a sample. It is distinct from the target population, the design that chooses units from the frame, and the observed cohort that actually produces usable labels.
A bounded live policy that deliberately acquires evidence where an incumbent and candidate would choose different actions. It can estimate value in that disagreement region, but it does not establish whole-population model quality.
The safe way to test S5
Spanwatch's production model was S4.
Each week, policy P17.4 divided the 240 flexible visits like this:
| Route | Visits per week |
|---|---|
| S4 score at least 0.86: urgent field admission | 120 |
| S4 score at least 0.58 and below 0.86: deterministic desk-review route | 100 |
| Manual and safety reserve | 20 |
| Total | 240 |
The threshold branch was deterministic: every connector satisfying 0.58 <= S4 < 0.86 entered the review band. The review desk could examine 500 review-band cases each week, but it could escalate at most 100 into the field budget. A reviewed case that did not receive a visit remained unobserved for the C3 condition target. It did not become CLEAR.
Dalia's candidate model, S5, detected a condition pattern on newer composite cross-arms that S4 often scored below 0.58. On BrambleGrid's older test data, S5 looked promising.
The team did not let it control field work immediately. For four weeks, S4 and S5 scored the same evidence snapshots, but only S4 entered P17.4.
That was a : the candidate produced predictions beside the live model without controlling the live action.
It was a competent safety choice. S5 could not unexpectedly consume field capacity. It could not delay S4's urgent inspections. It could not change a utility decision while the team was still evaluating it.
The team made one assumption that felt equally safe:
Because S5 cannot change production, the condition labels created during the shadow are neutral evidence for comparing S4 and S5.
That assumption survived the launch checklist.
The report that rejected S5
Across four stable weeks, the locked eligibility register—the sampling frame—contained 40,000 unique eligible connector assemblies. In the team's matrix, screen positive meant only that a model scored the case at or above 0.58 and would send it toward the evidence-acquisition path. It did not mean the connector was faulty.
| Cell | Reader-facing meaning | S4 screen | S5 screen | Connectors |
|---|---|---|---|---|
| A | Both would inspect or review | positive | positive | 1,600 |
| B | S4 only | positive | negative | 400 |
| C | S5 only | negative | positive | 800 |
| D | Neither | negative | negative | 37,200 |
| Total | 40,000 |
S4's policy created 880 protocol-C3 observations through the model-selected route:
| Cell | C3 visits selected by S4 | Final actionable | Final clear | No model-route C3 label |
|---|---|---|---|---|
| A: both screen positive | 720 | 360 | 360 | 880 |
| B: S4 only | 160 | 112 | 48 | 240 |
| C: S5 only | 0 | — | — | 800 unknown |
| D: neither | 0 | — | — | 37,200 unknown |
The launch report performed a technically correct join between final C3 labels and the two score receipts.
S4 had selected all 880 visited cases in A and B. Of those, 472 were actionable:
S4 observed yield = 472 / 880 = 53.6%
Among the 720 visited cases in A that S5 also screened positive, 360 were actionable:
S5-positive observed yield = 360 / 720 = 50.0%
Fifty-three point six beat fifty.
The report concluded that S5 found no extra field value. The launch council rejected it and proposed raising S4's admission threshold, which would spend still fewer visits near the uncertain boundary.
Nothing in the arithmetic was fabricated. Every one of the 880 condition labels could be correct.
The problem was where the labels existed.
The sequence exposes the asymmetry the report hid: correct arithmetic over the S4-created cohort said nothing about the candidate-only region. Cell C's 800 connectors remained unknown, and later routine-route evidence could reopen that question without filling the original cohort.
The incumbent policy controlled label observability
The deployment had created : outcomes were observed for a non-random subset chosen by the existing score and policy.
S4 did not change whether a physical connector condition existed. It changed which conditions BrambleGrid paid to observe under protocol C3.
That distinction bounded the report.
Among connectors S4 selected for inspection, the S5-positive subset had lower observed actionable-condition yield. The result could describe those visited cases.
It could not compare an S4-controlled policy with an S5-controlled policy across 40,000 eligible connectors. S5's claimed value was in C—the exact cell where shadow mode prevented it from causing observations.
The was the locked eligibility register of 40,000 connectors. Their point-in-time S4/S5 score receipts assigned them to A, B, C, or D; those attributes did not turn the observed subset into a different frame. P17.4 was the selection and observation design applied to the frame: it determined which connectors had a non-zero path to a model-route inspection. The 880 completed visits in A and B with final binary C3 labels were the observed cohort.
That vocabulary bounded the inference precisely. The 53.6% and 50.0% yields described subsets of the S4-selected observed cohort. They did not compare an S4-controlled policy with an S5-controlled policy over the 40,000-member sampling frame, because P17.4 created no model-route observations in C or D.
This was deeper than replacing missing values with zero. BrambleGrid had not done that. Its dataset correctly marked unvisited assets as unknown. The team had still overextended a valid narrow result into a release decision about a different action policy.
The proposed threshold increase made the mistake self-reinforcing. S4 would send fewer boundary cases into FieldOps. Those cases would produce fewer future C3 labels. A later candidate would be judged in an evidence world shaped even more strongly by S4.
What the annual route did—and did not—show
BrambleGrid also operated a routine annual inspection program. It had its own selection policy, AR-2026.3, based on inspection age, route geography, access windows, and planned utility work. Many not-yet-due assets had no chance of inclusion. The annual route was selected, versioned evidence—not a random control group.
About two months after the launch review, visits under a successor route plan found actionable conditions on composite-arm connectors in the score region S5 had elevated and S4 had left alone.
That discovery mattered because it falsified the council's confidence that the S4-selected report had closed the question.
It did not retroactively label cell C.
Those later visits observed connector condition at a later time, under a different route version and outside the original seven-day target window. They did not prove that S5 was globally more accurate. They did not prove that any repaired connector would have failed. They told Dalia only that the omitted region deserved a valid evidence path.
The council withdrew its threshold proposal and asked a better question:
Which decision are we trying to make, and what observation policy would let us make that claim?
Dalia answered by reconstructing two exact worked counterfactuals. Each design would have had to be assigned at the original four-week frame lock, before the target labels were visible. The tables below are worked alternative outcomes for those predeclared designs, not measurements BrambleGrid collected after the annual-route discovery. Neither alternative actually ran during the incident, they could not both have shared one capacity ledger, and the old score receipts and cohort gained no labels from this reconstruction.
Worked counterfactual A: a sentinel frame
If BrambleGrid had wanted to compare the S4-positive and S5-positive score regions across the declared four-week frame, it would have needed non-zero observation probability in every cell from the original frame lock.
The first worked design would have reserved 60 of the 240 weekly visits for a predeclared sentinel sample:
| Weekly route | Visits |
|---|---|
| S4 urgent | 120 |
| S4 lower-priority review-band escalations | 40 |
| Sentinel A / B / C / D | 10 / 10 / 20 / 20 |
| Manual and safety reserve | 20 |
| Total | 240 |
The sampling service would have created a keyed random order independently within A, B, C, and D before condition evidence was visible. Within each cell, every eligible connector would have had one equal inclusion probability. Geography and asset attributes could balance route execution only after that keyed order was fixed; they were operational attributes, not statistical sub-strata with separate inclusion probabilities or weights. A cancellation would have remained missing unless the system used the next pre-ranked case from the same cell before the seven-day window closed.
After four weeks, the exact worked counterfactual would have contained 240 completed sentinel visits:
| Cell | Sentinel visits | Actionable | Observed sample yield |
|---|---|---|---|
| A | 40 | 20 | 50.0% |
| B | 40 | 12 | 30.0% |
| C | 80 | 56 | 70.0% |
| D | 80 | 1 | 1.25% |
Because the cell-level inclusion probabilities would have been known and non-zero, the evaluation could have accounted for the unequal sampling. For this declared frame, the worked estimate was:
S4-positive region yield = 46.0%
S5-positive region yield = 56.7%
difference = +10.7 percentage points
approximate 95% range = +6.0 to +15.3 points
This result would have changed the release discussion. It would not have proved that S5 was globally safer or would prevent more service events. The both-low D estimate remained especially imprecise: one worked actionable observation carried a large weight. The range described uncertainty under the locked sampling assumptions, not certainty about the future.
The design would have cost 60 lower-priority S4 review-band visits each week for four weeks—25% of all flexible capacity. Urgent inspections and the 20 manual/safety slots would have remained protected, but some incumbent boundary cases would have waited in the normal cycle. BrambleGrid would also have paid for more travel to apparently routine assets and waited through the final 48-hour adjudication window.
The sentinel would have answered a broad score-region comparison by buying broader evidence at the original frame lock.
Worked counterfactual B: expose only the disagreement
In the reconstruction, Jon posed a narrower question:
Before comparing both models over the whole frame, can we learn whether cell C is valuable enough to keep investigating?
For that question, the second worked alternative was a : a bounded live policy that would have acquired evidence where incumbent and candidate actions differed, had it been assigned at the original frame lock.
S4 would have remained the policy everywhere except cell C. Inside C, the system would have randomly selected 40 of roughly 200 eligible connectors each week for field inspection. Every selection would have carried its P17-CANARY.1 policy version and assignment probability.
| Weekly route | Visits |
|---|---|
| S4 urgent | 120 |
| S4 lower-priority review-band escalations | 60 |
| Randomly assigned cell-C disagreements | 40 |
| Manual and safety reserve | 20 |
| Total | 240 |
After two weeks and the final 48-hour adjudication window, the exact worked counterfactual had 56 of 80 inspected C connectors as actionable:
local C yield = 56 / 80 = 70.0%
compatible 95% range = 59.2% to 78.9%
That result could have supported continuing, expanding, or stopping a bounded C-region rule. It could have rejected the statement that S5's exclusive region had produced no measured gain.
It could not estimate S5 in A or D. It could not detect a blind spot shared by both models. It could not establish whole-population precision, recall, calibration, safety, or future utility.
The canary would have cost 40 lower-priority S4 review-band visits each week. For two weeks, BrambleGrid would have operated mixed policies. If the canary had paused, new C admissions would have stopped, but already completed visits and labels would have kept their original canary receipts. Rollback could have restored future admission to P17.4; it could not have rewritten the evidence already created.
The counterfactual canary would have answered a local disagreement question faster by buying evidence only in C.
The two designs were alternatives, not layers of a universal fix, and neither was retroactively executed. If the original question had been “compare score-positive regions across the declared frame,” the sentinel would have fit. If it had been “is C worth bounded exposure before a broader comparison,” the canary would have fit. If field operations could not have accepted either capacity cost, “not enough evidence yet” would have remained an honest release decision.
The join had to remember who selected the label
The old evaluation table joined two scores to a final condition. That was not enough. It lost the route that made the condition observable.
Dalia reduced the production receipt to a teaching shape:
type DecisionEvidence = {
evidenceSnapshotId: string;
s4: ScoreReceipt;
s5: ScoreReceipt;
policyId: string;
capacitySnapshotId: string;
selectionSource: 'S4' | 'manual' | 'annual' | 'sentinel' | 'canary';
samplingFrameId?: string;
assignmentProbability?: number;
fieldVisit:
| 'completed'
| 'expired-unobserved'
| 'cancelled'
| 'not-selected';
inspectionProtocolId?: string;
conditionLabelVersion?: string;
};
The real receipt also retained connector lineage, evidence and feature versions, both calibrations, threshold branch, FieldOps case state, raw observation digest, adjudication, repair reference, delayed outcome contract, cohort version, and release decision.
The compact type carried the central rule: the evaluation needed the policy, frame, and assignment process that made a label observable.
An inner join from score rows to final labels could be technically correct and still erase every connector that never had a chance to produce a label. The corrected cohort began with all 40,000 frame members, then made completed, indeterminate, expired, cancelled, and not-selected visible before choosing any binary denominator.
Where each responsibility lives
The incident sequence showed when the inference escaped the observed cohort and was generalized to its sampling frame. The production topology shows where the corrected responsibilities would live and lets the reader compare two counterfactual evidence designs anchored to the original frame lock.
Switching the visual changes the hypothetical observation policy, capacity receipt, and claim—not the systems that own visit state, condition evidence, or utility action. The Sentinel view spends 60 weekly slots across A–D for four weeks; the Canary view spends 40 weekly slots only in C for two weeks. They are alternative what-if deployments at the original frame lock, never simultaneous layers and never a way to write later discoveries into old labels.
The question Dalia used next
The next model review was not about grid connectors. A warranty team wanted to evaluate a candidate that flagged battery packs the incumbent rarely sent to diagnostic teardown.
Dalia did not begin with the candidate's headline precision. She drew two score columns and asked who controlled access to the authoritative diagnostic label.
The domain changed. The review question did not:
Your candidate raises cases the incumbent always leaves untouched. In shadow mode, which outcomes can the candidate never create, and what evidence would you need before changing the live threshold?
A useful answer has five parts:
- Name the incumbent policy that selects observations.
- Mark the candidate-only disagreement region.
- State which target remains unknown there.
- Choose a bounded evidence-acquisition design and pay its capacity cost.
- Limit the conclusion to the population and decision that design can support.
A shadow model can leave production unchanged while being graded on outcomes selected by the incumbent.
The safest release decision is not always “ship” or “reject.” Sometimes it is:
We have a result for the old policy's evidence. We do not yet have evidence for the policy we are being asked to approve.
Concepts in this story5 concepts
A versioned rule that maps model evidence, thresholds, current context, capacity, and constraints into an action such as automate, inspect, review, defer, or abstain. The model supplies evidence; the policy owns the action boundary.
Running a candidate model beside the live system without letting it control production actions. This limits exposure, but observed outcomes still reflect the incumbent policy unless the evaluation creates an independent evidence path.
Labels observed for a non-random subset because a prior decision policy determines which cases receive the action, inspection, approval, review, or follow-up that can reveal an outcome.
The explicit register of eligible units available for selection into a sample. It is distinct from the target population, the design that chooses units from the frame, and the observed cohort that actually produces usable labels.
A bounded live policy that deliberately acquires evidence where an incumbent and candidate would choose different actions. It can estimate value in that disagreement region, but it does not establish whole-population model quality.
BrambleGrid, Spanwatch, its people, incident, protocol names, data, and metrics are fictional. Technical grounding: the 2026 study “Discovery of an Artificial Intelligence Label Feedback Loop” shows that labels influenced by a deployed AI workflow can reverse the apparent comparison of a later model; the associated confidence-gated production workflow documents selective automatic action and human review; “The Selective Labels Problem” establishes why a prior decision policy can determine which outcomes become observable; and “Performative Prediction” provides the broader foundation for deployed predictions changing later evidence. These sources establish mechanisms and limits. They do not describe BrambleGrid or prove a grid-asset incident, queue collapse, outage, injury, or the superiority of S5.