11.9No story yet
Production monitoring, drift, quality objectives, and diagnosis
Detect loss of decision validity and identify whether the cause is population, data, feature, label, model, calibration, policy, threshold, reviewer, intervention, or outcome change.
No story has been commissioned for this area.
11.10No story yet
Experiments, shadowing, canaries, and safe intervention rollout
Compare decision systems without exposing users or operations to uncontrolled interventions or evaluating a shadow system on outcomes it did not cause.
No story has been commissioned for this area.
11.11No story yet
Retraining, reproducibility, rollback, and model-system inventory
Reproduce and change the complete decision system across data, features, labels, preprocessing, model, calibration, policy, threshold, reviewer workflow, and environment versions.
No story has been commissioned for this area.
11.12No story yet
Generative-model signals in decision systems
Use LLM or multimodal outputs as versioned, evaluated decision evidence without treating structured generation, explanation, self-confidence, or a model judge as authoritative truth.
No story has been commissioned for this area.
11.13No story yet
Professional capstone — mixed-version decision loops
Defend a production decision system whose population, labels, features, models, policies, reviewers, interventions, and governance controls change on different clocks.
No story has been commissioned for this area.