One post tagged with "Realtime multimodal"

The interruption that stopped the sentence and not the payment

August 10, 2026 · 40 min read

Fault Lines Editorial

Fictional incidents. Exact technical vocabulary.

Architecture storyProfessionalA fictional production incident about barge-in, separate cancellation domains, and why a spoken assistant must never describe an effect it does not own.

Before the incident

What you'll learn from this story

Separate the five cancellation domains a barge-in touches—media playout, model generation, turn and command intent, the tool request, and the external effect—and name which system owns each one and what its acknowledgement proves.
Explain why a committed turn is a probabilistic decision about when a person finished speaking, and why an irreversible effect must not depend on one.
Reconcile conversation history to the audio a person actually heard, so a resumed session cannot claim a confirmation that was generated and never played.
Distinguish suppressing a late asynchronous result from the conversation from discarding it from the authoritative effect record.
Defend a choice between holding an effect before its irreversible commit point and narrating only what the effect ledger reports, given the different residual risk each leaves.

Ferrowick is a fictional construction-operations platform used by mid-sized building contractors. Customers run whole projects on it: purchase orders, delivery receipting, retention tracking, subcontractor valuations, and the weekly supplier payment run. Ferrowick holds no customer money; payments are initiated through Norwell Pay, a fictional licensed payment provider that submits into a bank scheme.

Kelter is Ferrowick's realtime site assistant, and it exists because of one physical fact. The person who knows what actually arrived is standing in a delivery bay in gloves and hearing protection, beside a reversing lorry, with a clipboard in one hand. She is not going to open a laptop. Kelter runs on a site handset or a paired headset, connects over WebRTC to a regional media edge, listens, and speaks.

A payment passes three decisions, made by different people at different times. On Monday the project manager approves the week's run in the web application, deciding what may be paid at all. During the week the site operations lead receipts deliveries, deciding what arrived. When retention conditions are met she releases the matching payment by voice, deciding what is sent now and in what order.

That third decision matters, because a project's available balance is finite and sequenced. Releasing early does not move money to the wrong week; it consumes the headroom a later payment depends on. A supplier who is not paid does not deliver, and a crew that arrives to an empty bay stands idle at full cost.

Priya Raghavan is Site Operations Lead on the fictional Halden Row project; she receipts deliveries and releases approved payments. Tomas Idrissi is a Staff Engineer who owns Ferrowick's realtime interaction plane — media session, connection epoch, turn state, playout, interruption handling. Dela Okonkwo is a Principal Engineer who owns the payment-effect plane — the gateway, the operation ledger, the Norwell Pay integration. Ruth Vance is Head of Payment Operations, and owns what a customer is told when the state of a payment is uncertain.

Tomas and Dela both built correct systems. The sentence that caused this incident lived in Tomas's plane and was about Dela's, and nobody had ever decided which of them owned it.