Skip to main content

One post tagged with "Knowledge systems"

Keeping application answers current, attributable, and connected to source material.

View All Tags

The fine-tune that answered with last week’s policy

· 10 min read
Fault Lines Editorial
Fictional incidents. Exact technical vocabulary.
Architecture storyBeginnerA fictional production incident about keeping an AI answer tied to current policy.

Ashvale Benefits (a fictional employee-benefits administration company) helps mid-sized employers manage health-plan enrollment, eligibility rules, and employee policy questions. It had eleven days before its busiest open-enrollment season.

The company had built an AI support assistant inside its member portal. The assistant was supposed to answer routine questions from each employer’s approved policy handbook, show the supporting section, and hand uncertain cases to the support team.

During open enrollment, that team expected eighty thousand questions. Most would be some version of the same thing: Does my plan cover this? When does coverage begin? Which form do I need?

The business goal was simple. Answer the routine questions immediately. Send the hard ones to a human. Never invent a benefit that did not exist.

There was one more rule from legal: every answer had to point to the policy section that supported it.