Building Deterministic On-Device Dictation

Users call a dictation app fast when it feels predictable. They call it slow when it surprises them, even if the median benchmark looks good.

That is why deterministic behavior is central to dictation engineering.

Determinism in product terms

Determinism means the same input and environment produce similar completion behavior. For users, this shows up as confidence: they know when text will be ready.

In writing tools, confidence is a performance feature.

Designing the critical path

Our guiding principle is to keep the post-speech path local and short:

• Capture audio locally.
• Run transcription and cleanup locally.
• Insert final text directly at cursor.

Each external dependency added to this path increases variance risk.

Latency budget thinking

Instead of one total number, break latency into budget slices:

• Trigger overhead.
• Audio finalization.
• Transcription final pass.
• Cleanup and punctuation.
• Text insertion.

This makes bottlenecks visible and prevents local improvements from being hidden by downstream delays.

Tail latency is the real enemy

Average speed can look fine while p90 and p95 feel bad. Tail spikes are what cause users to abandon voice workflows and return to typing.

Engineering for tails often means removing conditional branches and network dependencies in finalization steps.

Failure modes to plan for

• Resource contention on local machine.
• Long-running cleanup paths for noisy speech.
• Insertion timing conflicts in complex editors.

A deterministic architecture does not eliminate failures. It narrows and simplifies them.

Why this matters for teams

When behavior is predictable, onboarding is easier and support load drops. Teams can write clearer guidance because the tool behaves consistently across normal usage.

Further reading

For product-level context, see On-Device Speech to Text for Mac and our public speed benchmark.