← Biscuit Builder

How Biscuit Builder works

A technical walkthrough of the app: how a headset click turns into a spoken answer, how the menu is kept as data, how the model is kept from guessing, and how the quiz decides what to ask next.

1The loop, end to end

Biscuit Builder is a native iPhone app written in Swift and SwiftUI, targeting iOS 18. The Ask tab runs one loop: listen, ask, speak. Speech recognition and speech output both happen on the phone. The only network call is a single text request to Claude.

1. On phone

   Headset clickThe play/pause button on the earbuds triggers the same action as the big on-screen button.
2. ↓
3. On phone

   Speech → textApple's SFSpeechRecognizer streams a transcript, biased toward menu names. Stops on "over", a 1.3 s pause, or 10 s.
4. ↓
5. Network

   Text → ClaudeOne request to the Anthropic Messages API (claude-haiku-5-5) with the full menu prompt plus the last exchange for follow-ups.
6. ↓
7. On phone

   Parse the replySplit off a hidden ITEM: line. The rest is the spoken answer.
8. ↓
9. On phone

   Speak + showThe answer is read aloud with AVSpeechSynthesizer. The build card for the tagged item is drawn from the app's own menu data.

The whole loop lives in Assistant.swift as a small state machine with four phases: idle, listening, thinking, speaking. One button does whatever makes sense for the current phase:

Assistant.swift

func buttonPressed() {
    switch phase {
    case .idle: Task { await startListening() }
    case .listening: listener.finish()          // tap again = "I'm done talking"
    case .speaking: speaker.stop()              // tap while talking = stop
    case .thinking: break
    }
}

2The headset button

The goal was to keep the phone in a pocket. Earbuds have a middle button that sends a "play/pause" remote command, the same one music apps receive. The app listens for it through MPRemoteCommandCenter.

The catch: iOS only delivers remote commands to the app that's currently playing audio. So the app generates one second of silent 16-bit PCM audio in memory, wraps it in a WAV header, and loops it forever with AVAudioPlayer. That keeps Biscuit Builder as the "now playing" app, so the click comes to it.

HeadsetButton.swift

player = try? AVAudioPlayer(data: wav)
player?.numberOfLoops = -1    // one second of silence, looped forever
player?.play()

A second catch shows up with Bluetooth headsets. To use a Bluetooth mic, iOS switches the headset into hands-free (call) mode, and during a "call" the system keeps the button for itself. So the audio session only switches to .playAndRecord with .allowBluetoothHFP while the app is actually listening. The moment listening ends, it switches back to plain .playback and the button belongs to the app again. A 0.4-second debounce ignores the double signal some headsets send for a single click.

3Listening

Speech recognition uses Apple's on-device framework (SFSpeechRecognizer, en-US). Audio from the mic is tapped off AVAudioEngine in 1024-frame buffers and appended to a streaming recognition request. Partial results are turned on, so the "ear" line on screen updates word by word as the cook talks.

Menu names like "Tootie" and "Peppi" aren't normal English, so the request includes contextualStrings: every item name plus kitchen words like "chunkies", "tator rounds", "button four", and "over". That biases the recognizer toward words it should expect.

There are four ways a question ends:

• Saying "over." If the last word of the partial transcript is "over", listening stops immediately and the word is stripped before the question is sent. This is the fastest way to end a question.
• A 1.3-second pause. A timer restarts on every new partial result. If nothing new arrives for 1.3 s, the question is done.
• Clicking again. A second click means "I'm done talking."
• A 10-second cap. A safety net in case the cook walks away mid-question.

SpeechListener.swift

nonisolated static func endsWithStopWord(_ text: String) -> Bool {
    words(in: text).last == "over"
}

4The menu as data

Every item lives in one array in MenuData.swift. Each item has a name, a category, and an ordered list of layers. For biscuits and flapjacks, layers go bottom to top, like the build board in the kitchen. Each layer can carry a count.

MenuData.swift

MenuItem(name: "Mickey", layers: [Layer("Canadian bacon", 3), Layer("egg"), Layer("melted cheese")]),
MenuItem(name: "Miner",  layers: [Layer("bacon", 3), Layer("potato"), Layer("melted cheese")]),

Three things are generated from that one array:

1. The menu section of the prompt. Each item becomes a line like Mickey: three Canadian bacon, egg, melted cheese. Counts are written as words because the answer will be spoken.
2. An ingredient lookup. A reverse index, one line per ingredient: potato: Thundering Herd, Duke, Mountaineer, …. This was added after testing (see section 10).
3. The on-screen build cards and the quiz questions.

Because the prompt and the cards come from the same data, fixing a build in one place fixes what the model says, what the card shows, and what the quiz asks.

5The prompt

The system prompt is assembled in Menu.swift:

// Menu.swift
static let systemPrompt = rules + "\n\nMenu:\n" + MenuData.promptLines
    + "\n\n" + lookup + "\n" + MenuData.ingredientLines
    + "\n\n" + sides + "\n\n" + fryer + "\n\n" + examples

Grounding

The model is told to use only the menu in the prompt and never guess. If it can't tell which item the cook means, it says "Which item?" If something isn't on the menu, it says "That's not on my menu." If asked for a fryer cook time, which hasn't been confirmed yet, it says "I don't have cook times yet, just the buttons."

Answer style for a kitchen

The answer is heard, not read, while the cook's hands are busy. So the rules push for very short answers:

• Answer only what was asked. "What's on the Duke?" gets the full build. "How many bacon on the Duke?" gets "Duke, three bacon."
• Say the item's name first, so the cook can catch a mishearing. Don't repeat it on follow-ups about the same item.
• "What kind of cheese" means melted vs. slice, not a cheese variety. "What kind of meat" means bacon, sausage, and so on, not a brand or cut.
• When names collide (Bacon Biscuit, Bacon Flapjack, Bacon Wrap), ask which one.
• Known sound-alikes are listed: "dictator" → Big Tator, "minor" → Miner, "button for" → button four.

Examples do more than rules

The prompt ends with about two dozen worked question/answer pairs, including multi-turn follow-ups. In testing, the model copied the examples more reliably than it followed written rules, so any rule change gets a matching example.

Question: what's on the tootie
Answer: Tootie, one country ham, cheese slice, then egg.
ITEM: Tootie
Follow-up: there's no bacon on that
Answer: Right, no bacon.
ITEM: Tootie
Follow-up: and the mountaineer
Answer: Mountaineer, one country ham, potato, cheese slice, then egg.
ITEM: Mountaineer

6The API request

The app calls the Anthropic Messages API directly with URLSession. No SDK, no server in between.

ClaudeClient.swift

let body: [String: Any] = [
    "model": "claude-haiku-5-5",
    "max_tokens": 150,
    "thinking": ["type": "disabled"],
    "system": system,
    "messages": messages
]

• Haiku because it's the fastest and cheapest model, and the job is lookup and phrasing, not reasoning.
• Thinking disabled. The model thinks by default. Turned off, answers come back faster, and the thinking doesn't eat into the 150-token answer limit.
• 150 tokens is plenty for a one- or two-sentence answer plus the tag, and caps any runaway reply.
• 15-second timeout. On failure the app says "Sorry, I couldn't get an answer" instead of hanging.
• The reply is read by picking the content block whose type is "text".

Follow-ups without server memory

The API keeps no memory between requests, so the app sends context itself: the full system prompt every time, plus the last two turns (the previous question and answer). That's what lets "And the Miner?" work. History is cleared after 90 seconds of no questions, so an old order doesn't bleed into a new one.

Assistant.swift

// Forget the previous item after 90 seconds so old orders don't confuse it.
if Date().timeIntervalSince(lastAnswerAt) > 90 { history = [] }
let messages = history + [["role": "user", "content": question]]
// ...after the reply:
history = Array((messages + [["role": "assistant", "content": reply]]).suffix(4))

7The ITEM tag and build cards

Every reply ends with one extra line naming the item the answer is about, or none:

Mickey, melted cheese.
ITEM: Mickey

The app strips that line before speaking and uses it to look up the item in MenuData. The lookup forgives small differences like "Politician" vs. "The Politician".

Cards follow the category: biscuits and flapjacks read top to bottom like the board ("Top of Biscuit ↑" … "Bottom of Biscuit ↓"), wraps are numbered in the order things go on the tortilla, the Big and Little Tator are numbered because they're built in a set order, and everything else is an "Includes" list.

8Speaking the answer

Answers are spoken with Apple's on-device AVSpeechSynthesizer, at 1.05× the default rate. A voice picker lets the cook choose from the installed voices. (Siri's voices aren't available to third-party apps.)

An OpenAI cloud voice (gpt-4o-mini-tts) is also built in, but it's switched off with one flag. When it's on, Claude still writes every answer; OpenAI only reads it aloud. The implementation:

• Streams raw 24 kHz PCM and queues each chunk on an AVAudioPlayerNode as it arrives, so playback starts about 0.05 s into the download instead of waiting for the whole clip.
• Opens the connection while the cook is still talking (a warm-up request), to save the handshake time later.
• Caches finished clips on the phone, so a repeated answer plays instantly at no cost.
• Falls back to the Apple voice if there's no key, an error, or no audio within 5 seconds.

In testing it sounded more natural but was noticeably slower and less consistent than the on-device voice, so the app uses the Apple voice for now.

9The quiz

The Quiz tab uses the same MenuData. Modes include Build It (stack an item from the bottom up by tapping ingredient pills), Name It (name an item from its ingredients), Picture (name a biscuit from its photo), and Facts (single-detail questions). A category picker narrows the pool to one part of the menu.

Choosing what comes next

Each pool gets a shuffled deck. Every item is dealt once before anything repeats, like going through flash cards. Missed items are saved between sessions and get one or two extra cards in the next deck, so they come back sooner without taking over. A correct answer pays a miss back down.

Quiz.swift

if deck.isEmpty {
    deck = names
    for name in names where (missed[name] ?? 0) > 0 {
        deck += Array(repeating: name, count: min(missed[name]!, 2))
    }
    deck.shuffle()
}
let index = deck.firstIndex { $0 != lastKey } ?? 0   // never the same item twice in a row

Making Build It hard enough

Build It always shows an even number of pills, at least six, even for simple items. Decoys are filled in by priority: special amounts first (like odd gravy scoop counts for Biscuits and Gravy), then ingredients from look-alike items, then anything else on the menu. Facts skips questions the item's name gives away, like "What meat?" on the Sausage Flapjack.

10Testing and fixes

The app was tested on a real phone with real questions. The "ear" line on screen shows exactly what speech recognition heard, which separates two kinds of failure: a mishearing (fixed with speech hints or a sound-alike rule) and a bad answer (fixed in the rules or examples). Some of the fixes:

11Cost, limits, and what's next

Cost

About $0.0004 per question on Haiku. Most of that is the system prompt, which is sent with every request.

Known limits

• The API key lives in the app. Fine for a personal build on one phone. A version for other people would route requests through a small server so the key never ships on a device.
• The phone has to be unlocked with the app open. Using the headset button with the phone locked in a pocket is untested.
• Needs a connection. Speech recognition and speech output are on-device, but the answer comes from the API.
• No cook times. Left out on purpose until they're confirmed from the fryer itself.
• Free signing. Installed from Xcode with a free Apple account, so it has to be reinstalled every 7 days.

Ideas for later

• Prompt caching, once the instructions grow, to cut cost and latency on the repeated system prompt.
• A voice quiz: the app asks out loud, the cook answers by talking, and the model grades it.
• An iPad mounted at the prep table showing the item photo while answering.
• Custom speech vocabulary training for menu names.
• A single realtime voice model for speech in and out, with Claude still answering. Faster and more natural, but a rebuild.