4 posts tagged with "LLM engineering"

Harborlight People is a fictional workforce-management SaaS company. Its employer customers use the platform for payroll, scheduling, employee relations, support, and access administration. They can also configure a workflow through which current and former workers request copies of their personal data.

The employer customer defines the request policy, the source categories in scope, and who may make the final release decision. Harborlight operates the software and a managed privacy-operations team under that customer-defined policy. The process in this story is Harborlight's fictional design, not a universal legal requirement.

Harborlight handled about 3,800 worker-data requests per month. Most looked simple from the request portal: prove who you are, describe the employment period, wait while the records are assembled, and collect a package from an authenticated portal.

The work behind that path was not a single database query.

A worker might have a legal name, a preferred name, a former surname, several email addresses, and more than one worker identifier after a rehire or contractor conversion. A support ticket might mention a person without being about that person. An attachment might be linked to a ticket whose requester and uploader were neither its subject. A fixed join could recover the obvious records. It could not reliably resolve every free-text reference, copied attachment, or derived record.

That was why Harborlight built Lumen, a bounded AI privacy worker. Lumen read case-scoped records through narrow adapters, resolved aliases and free-text references, proposed which records concerned the requester, explained the evidence behind each proposal, and suggested redactions. Lumen could produce candidates. It did not own a person's stable identity, record-subject truth, case policy, approval, delivery credential, or the release of bytes.

Mara Chen, the Senior Privacy Operations Specialist responsible for defining request scope and resolving ambiguous record matches, supervised the managed workflow. Under each employer's policy, she could approve an exact release.

The distinction sounded conservative enough: Lumen proposed; Harborlight decided.

Northstar Ledger (a fictional B2B commerce-infrastructure company) routes checkout and inventory updates for regional grocery chains. Its customers do not run one Northstar checkout page. They use Northstar behind their own mobile apps, self-checkout kiosks, and delivery sites.

At the Saturday peak, those systems send roughly 48,000 order operations per minute. A short outage creates queues. An incorrect recovery can create something worse: orders accepted through a path that the company no longer believes is authoritative.

Northstar stored checkout state in an Amazon Aurora PostgreSQL global database. The primary Region served writes. A secondary Region stayed ready for disaster recovery. The operating rule was strict: Northstar could advertise only the writer confirmed by Aurora's current topology.

Imani (the Staff Reliability Engineer responsible for Northstar's database-recovery control plane) had spent the previous six months reducing the time between an incident page and a safe recovery decision.

Her team built Relay, a bounded AI incident worker. Relay read approved telemetry, compared an incident with reviewed runbooks, and proposed a typed recovery plan. It could not promote a database or change traffic. A deterministic policy service checked every proposal. A human incident commander then approved the exact target and the maximum tolerated data loss before any effect was allowed.

This mattered because Relay was useful precisely where incidents were messy. It could gather replication lag, recent deployments, health probes, and runbook constraints in seconds. It could explain why one recovery target fitted the evidence better than another.

It could not make an ambiguous cloud operation unambiguous by thinking harder.

Ashvale Benefits (a fictional employee-benefits administration company) helps mid-sized employers manage health-plan enrollment, eligibility rules, and employee policy questions. It had eleven days before its busiest open-enrollment season.

The company had built an AI support assistant inside its member portal. The assistant was supposed to answer routine questions from each employer’s approved policy handbook, show the supporting section, and hand uncertain cases to the support team.

During open enrollment, that team expected eighty thousand questions. Most would be some version of the same thing: Does my plan cover this? When does coverage begin? Which form do I need?

The business goal was simple. Answer the routine questions immediately. Send the hard ones to a human. Never invent a benefit that did not exist.

There was one more rule from legal: every answer had to point to the policy section that supported it.

Kiteframe Pay (a fictional payment-risk infrastructure company) helps online marketplaces decide whether a card order should be approved, declined, or reviewed by a human. Merchants call Kiteframe during checkout, before they capture money or promise inventory to a shopper.

Most orders never need generative AI. Deterministic rules settle obvious cases in under 80 milliseconds. The difficult eight percent—new devices, unusual delivery patterns, sparse account histories—enter Aster, Kiteframe's AI-assisted risk analyst.

Aster was not a chatbot and could not approve a payment. It gathered approved evidence, used a language model to produce a typed risk recommendation with cited signals, and passed that recommendation to a deterministic merchant-policy engine. Without Aster, those ambiguous orders went to a manual-review queue. During a large sale, that queue could grow faster than Kiteframe's analysts could empty it.

At 02:13 on Tuesday, model latency rose sharply. Seven minutes later, Kiteframe's entire checkout success rate had fallen from 99.4% to 71%.

The model provider was recovering.

Kiteframe's retries would not let it.