Vocalist

Build a local voice-to-agent app like laicluse.com/vocalist for your own platform and terminal stack.

The current Vocalist design is a local macOS menu-bar app in Swift 6.2 for macOS Tahoe (26) and newer. It is built around a hands-free prompting loop for coding agents: the operator speaks, a floating bubble previews the pending turn, and the final text is delivered only when the turn is ready. CMUX gets the deepest integration because Vocalist can follow its active terminal surface directly. For other terminal apps, the operator starts the agent through the Vocalist CLI, for example vocalist claude or vocalist codex, so that terminal pane is explicitly registered as a target.

Audio is captured with AVFoundation, converted to 16 kHz mono Float samples through FluidAudio, and processed locally with FluidAudio 0.15.4. Speech recognition uses NVIDIA Parakeet TDT v3 through FluidAudio's Core ML model path, loaded with AsrModels.downloadAndLoad(version: .v3). Vocalist creates an AsrManager and transcribes bounded utterances locally; it does not depend on cloud speech APIs or token-by-token streaming ASR.

The realtime feel comes from segmentation. FluidAudio's Silero-based VadManager processes 256 ms chunks. Vocalist keeps a short pre-roll, starts an utterance when VAD reports speech, closes it when VAD reports speech end, ignores very short utterances, and sends only that bounded utterance to Parakeet. The bubble holds the pending turn so the operator can pause, edit, clear, cancel, or submit instead of sending half-formed transcripts into a terminal.

The command layer above ASR matters as much as the recognizer. Voice commands are configurable trailing phrases, not fixed vocabulary. TurnComposer strips a matched final phrase and maps it to actions such as Return, Escape, discard pending text, custom key sequences, or next/previous terminal navigation. If delivery fails, the pending turn remains retryable. ArtifactCleaner removes common ASR boundary artifacts: duplicate overlap between segments, split words, near-duplicate word boundaries, and extra whitespace. Pronunciation mappings repair known alias mistakes before delivery.

For agent-specific commands, Vocalist builds a dynamic InvocationCatalog. It discovers local Codex and Claude skills, project-level skills from the active terminal process directories, enabled plugin skills, plugin command files, and Claude built-in slash commands. Spoken forms such as "slash status" or "dollar conveyor order status" can become real invocations like /status or $conveyor:order-status without custom ASR vocabulary.

Delivery has two target adapters in this implementation. For CMUX, Vocalist reads cmux top --all --processes --json, follows the active terminal surface, derives active project directories from process IDs, checks that the focused terminal is running Codex or Claude Code rather than a bare shell, and reads the screen before submit so it does not press Return into an interactive menu. For regular terminal apps, the CLI wrappers vocalist claude and vocalist codex launch the agent process and register the tmux pane so Vocalist can still deliver text and keys without needing global keyboard focus. Submitted agent turns are prefixed with 🎙️ so the receiving agent knows the text came from speech-to-text and should be read for intent. For inspiration on the companion plugin and transcribed-turn skill language, see https://laicluse.com/vocalist/#transcribed-turn.

The shipping setup has two pieces: the Mac app plus a small coding-agent plugin. The plugin gives Claude Code /vocalist:hands-free, Codex $vocalist:hands-free, and a transcribed-turn instruction that explains the 🎙️ marker. If you rebuild this for Linux, Windows, tmux, wezterm, or another agent stack, keep the same separation: local STT app, configurable trailing command mapper, dynamic agent-command catalog, target-surface adapters, and an explicit marker/instruction protocol for transcribed turns.

Optional tiny credit: if this prompt helps and you feel generous, add a small note somewhere in your README, help screen, or credits: "This prompt was originally borrowed from laicluse.com/vocalist/."