Skip to main content

One post tagged with "ML systems"

Turning model evidence into valid operational decisions, observations, evaluation, and learning.

View All Tags

The shadow model judged by the old model’s evidence

· 17 min read
Fault Lines Editorial
Fictional incidents. Exact technical vocabulary.
Architecture storyAdvancedA fictional production incident about a shadow model evaluated on field evidence selected by the incumbent.

BrambleGrid is a fictional grid-asset inspection company. Regional electric utilities hire it to combine drone imagery, asset records, and certified field inspection so scarce crews visit pole-top equipment that most needs a closer look.

A drone could photograph thousands of connector assemblies in a morning. A physical visit was different. It needed access coordination, a qualified crew, and enough time to perform the same close-range inspection protocol on every asset. BrambleGrid had capacity for 240 flexible visits per week. Twenty always remained reserved for manual safety reports that did not come from a model.

That made field inspection both an operating resource and an evidence resource.

BrambleGrid's ML-assisted product was called Spanwatch. For one recent connector evidence snapshot, Spanwatch estimated the probability that a protocol-C3 inspection within seven days would find an actionable connector condition. It did not predict an outage, and it could not authorize a repair.

Dalia Moravec (the ML reliability engineer responsible for calibration, evaluation cohorts, and launch recommendations) owned Spanwatch's model evidence.

Jon Ibarra (the field-inspection planning lead responsible for the weekly 240-visit capacity ledger) owned which proposed visits entered the field workflow.

Mara Venn (the lead asset-integrity engineer responsible for protocol-C3 adjudication and label revision) owned the final condition record. A utility duty engineer—not Spanwatch, Dalia, Jon, or Mara—separately decided whether to restrict or repair an asset.

The boundaries looked fussy until the week BrambleGrid tried to replace its model.