How a restaurant voice agent keeps the order coherent when the customer keeps changing it

Why this is the one thing worth testing on any voice AI vendor

Customers do not order in straight lines. They add something, remember a second item, ask if a topping exists, change their mind, ask for a different bread. A voice agent that can take a single clean order is table stakes. A voice agent that can keep the cart coherent across five or six customer turns of additions, swaps, and remaps is the actual product.

The failure mode that costs restaurants money is not the obviously broken call (no one’s ordering through a system that hangs up). It’s the call that completes, gets posted to the POS, and the kitchen prints something different from what the customer heard on the phone. The receipt says “soft drink” but the caller ordered “Coke.” The kitchen ticket says “scrambled” but the caller asked for “over easy.” The cheesecake comes out plain because the agent forgot the strawberry add at the end.

The way to prevent that is not to be clever about state tracking. It is to make state drift impossible by construction: one cart object, full read-back on every change, POS names not caller words, refuse to invent IDs. The next section walks through how each of those rules shows up in the demo.

The four rules, applied

Same call, viewed as a state machine instead of a transcript. Each rule shows up in at least one of the four cart mutations below.

The cart object, mutation by mutation

This is the cart as it exists in the agent’s memory at each turn of the public demo. The diff column shows what changed since the previous snapshot. The bottom of each card explains why the change happened that way and not another way.

Timestamps from src/components/voice-activity-data.ts, generated from public/audio/dennys-order.mp3 via the build script referenced in the file header. Audio plays in the public demo at aiphoneordering.com.

The two state changes most vendor pages quietly skip

Adding the cheesecake at 63.10 seconds is the easy case. Two harder cases live inside the same call, and most vendor write-ups skip both because acknowledging them invites questions the vendor would rather not answer.

1. The brand-to-SKU remap at 23.10 seconds

The caller asked for a Coke. The POS does not have a Coke SKU; it has a generic soft-drink line item. A naive agent quietly rewrites “Coke” to “soft drink” and moves on. The receipt at pickup says “soft drink” and the caller is mildly confused: did they get what they asked for? PieLine’s agent surfaces the remap aloud at 23.10 seconds: “For your Coke, I’ll add a soft drink to your order.”That one sentence converts a downstream confusion-at-pickup into an upstream confirmation-during-call. If the caller wanted to push back (“do you have Pepsi?”) this is the moment.

2. The free-form-modifier resolution at 71.34 seconds

“Can you add strawberries, if that’s an option?” is the kind of turn where most demos cut to a fade. A guess-it agent guesses, the kitchen does not have strawberries, the customer gets a topping-less cheesecake and a surprise line item on the bill. PieLine’s agent does the actual lookup: enumerate the modifier groups attached to the New York style cheesecake, search for any name that resolves to “strawberry,” find mod_topping_strawberry, confirm back in the POS’s display name: “One slice of New York style cheesecake with strawberry topping.” If the lookup had failed, the agent would have said so out loud and offered the closest real option, not invented one.

Both of these are state mutations the caller cannot see, and both are surfaced in the spoken turn so the cart the caller hears matches the cart that gets posted.

What you should run on a demo before signing a contract

If you are evaluating any voice AI vendor for your restaurant, run this four-turn script on their demo line. It exercises the exact state-mutation paths above. Vendors that handle it cleanly are doing the work. Vendors that read back a delta instead of the whole cart, or that quietly remap items without saying so, are leaving you exposed.

1. Order two items where one of them is a brand name the chain probably does not stock verbatim (Coke, Sprite, Heinz ketchup). Listen for the remap-aloud at the next agent turn. If the agent does not surface the new name, the receipt will surprise you.
2. Order an item with at least one required modifier (a sandwich with a bread choice, a pizza with a size). After the agent asks, answer both required questions in one breath in reverse order. Confirm the read-back attaches them to the right groups.
3. Mid-call, add a third item with a free-form modifier that may or may not exist (“can you add jalapeños to the burger?”). Listen for whether the agent does the actual catalog lookup or whether it cheerfully agrees and moves on.
4. Before confirming, change your mind about one of the original items (“actually, scratch the soft drink”). The next agent turn should read back the whole remaining cart, not just say “okay, removed.” If you only hear an acknowledgement, ask the agent to read your order back. If it cannot, the cart and the POS may already be out of sync.