Mid-order clarification on a restaurant phone call: when the AI asks, and when it refuses

The numbers that matter

Four measurements from the 102-second call. Every value is reproducible from the transcription file linked above.

The 102-second call, with the clarification moments marked

This is the full demo transcript at second-level resolution. The annotated turns are the ones doing clarification work: the bridge pause, the batched required-modifier ask, the brand-name remap, the free-form add-on, and the final read-back.

Timestamps from src/components/voice-activity-data.ts (Deepgram multichannel transcription of public/audio/dennys-order.mp3). Play the same audio inside the demo widget at aiphoneordering.com.

The five rules behind the choreography

The behavior above is not negotiated in prompt land. Each rule maps to a property of the POS catalog the agent ingested during onboarding, or to a constraint baked into the tool surface the agent is allowed to call.

1. 1. Clarify only when the POS demands it.

   For each item the caller orders, look at the modifier groups. If a group has min_selections ≥ 1, the cart cannot be valid until an answer is collected. That is the only thing that triggers a question. On the demo the Lumberjack Slam has two such groups (eggs prep, bread choice) and the Coke has zero. The agent asks two questions about the slam and zero about the Coke. Optional modifiers are not surfaced unless the caller brings them up.

2. 2. Batch every required question for one item into one breath.

   The agent does not ask eggs, wait, then ask bread. It asks both together at 15.98 seconds: “how would you like your eggs cooked, and what kind of bread would you like?”A real caller can hold two choices in their head and reply with both. The agent listens for either word in the caller’s response and slots whichever appears, in whatever order they say it. This roughly halves the clarification turns per item.

3. 3. Enumerate every legal answer aloud.

   For the bread group the agent lists every option: “white, brown, multigrain, or sourdough?” A caller who has never eaten at this restaurant should not have to guess what the kitchen carries. The option list comes verbatim from the POS modifier group, so the names the caller hears are the names the kitchen ticket will print.

4. 4. Resolve free-form requests against the existing modifier list. Never invent.

   At 65.98 seconds the caller asks “can you add strawberries, if that’s an option?” The agent does not guess a price or fabricate a topping ID. It checks whether the cheesecake item has a strawberry modifier in any of its groups, finds mod_topping_strawberry, and confirms at 71.34 seconds using the POS’s own display name. If strawberries had not existed in the catalog, the agent would have said so and offered the closest real option from the same group. The tool surface refuses unmapped strings.

5. 5. Read the full order back before placing it.

   At 75.42 seconds the agent reads every item and every modifier in the order they will appear on the receipt: “a Lumberjack Slam with scrambled eggs and sourdough bread, a soft drink, and a New York style cheesecake with strawberry topping. Is that correct?”The caller confirms at 85.78 seconds, and the cart posts at 89.12 seconds. The 2.4 seconds between “Placing your order now” and “Done. Your total is $34.11”is the POS round-trip. The total is the POS’s number, not the agent’s.

The decision tree, in one diagram

Every turn the agent takes follows the same skeleton. The branches that matter are the required-group check (rule 1) and the modifier-list resolution (rule 4).

Why a human cashier’s clarifications get worse during rush

A line cook who steps to the phone with three tables waiting and a printer chattering behind them is under acute time pressure. Two predictable failure modes follow.

They skip the clarification entirely. The order goes in with a default (scrambled eggs, white bread) because asking would cost 15 seconds and the line at the counter would notice. The kitchen makes the default. The caller gets the wrong eggs at pickup, comps the meal, and may or may not come back. National Restaurant Association data suggests roughly a third of customers do not return after a wrong order.

They guess what the menu allows.Asked “can you add strawberries to the cheesecake?”, the cashier who is not standing at the prep station may say yes when the answer is no, or say no when the answer is yes. The customer either receives a bill for a topping the kitchen does not have, or misses out on something that was always available.

The agent does not have either failure mode because it does not feel time pressure and it can only call the modifier IDs the POS gave it. PieLine production deployments run at 95%+ order accuracy versus roughly 75% for staff handling phone orders during the dinner rush; the gap is almost entirely about clarification consistency.

Where the agent deliberately stays quiet

Bad mid-order clarification often shows up not as a missed question but as a question asked when none was needed. Common false positives the agent is configured to skip:

• Single-SKU drinks.A Coke does not need clarification; it is one item with no modifiers. The agent says “I’ll add a soft drink” to surface the brand-to-SKU remap and moves on.
• Optional add-ons the caller did not mention.“Would you like cheese on that?” is dead time if the caller did not bring it up. Optional modifiers are surfaced only when the caller asks or when the upsell hook is enabled for that item.
• Cross-item assumptions.Two of the same item ordered (“two Lumberjack Slams”) do not generate one question per slam if the caller has already answered both. The agent reuses the answer set across identical items and only re-asks if the caller signals they want them different.
• Confirmation theater.The agent does not re-confirm every line as it is added. There is one mid-call check (“Anything else for your order?”) and one full read-back at the end. That is the budget. More than that and the call drags.