Voice

APX has a unified voice layer: a TTS (text-to-speech) facade for speaking text and a Whisper-based STT (speech-to-text) sidecar for transcribing mic audio. The apx voice CLI exposes both. The Desktop window and the Deck use the same underlying engines.

Commands

# Synthesize and play
apx voice say "Hello, world!"
apx voice say "Hello" --provider piper
apx voice say "Hello" --provider gemini --voice Aoede
apx voice say "Hello" --no-play          # generate file, skip playback

# Listen (mic → STT → super-agent → TTS reply)
apx voice listen                          # stop on silence (sox required)
apx voice listen --seconds 5             # fixed-duration capture
apx voice listen --seconds 5 --no-play  # transcribe + agent, no audio playback

# List configured providers
apx voice providers

--provider overrides the configured default for that call only. --voice overrides the voice ID within a provider (supported by OpenAI, ElevenLabs, and Gemini). --no-play writes the audio file to ~/.apx/tmp/tts/ and prints the path without playing it.

apx

$ apx voice providers
  TTS providers                          default: piper

✓ piper        local    es_AR-daniela-high.onnx
✓ elevenlabs   cloud    eleven_multilingual_v2   key ••••a91f
✓ openai       cloud    tts-1                     key ••••7c2d
⚠ gemini       cloud    no key — run apx config set voice.tts.gemini.api_key
• mock         local    silent WAV (tests only)

STT
✓ whisper      local    model base   lang es

apx voice providers — live status of each engine

Providers

ID	Local?	Key needed	Quality	Notes
piper	yes	no	Good	Default recommendation. Requires piper binary + .onnx model.
elevenlabs	no	yes	Excellent	Free tier: 10 k chars/month. Model eleven_multilingual_v2.
openai	no	yes	Good	Reuses engines.openai.api_key. Model tts-1.
gemini	no	yes	Good	Returns raw L16 PCM — APX wraps in WAV automatically. Supports emotion tags.
custom:<slug>	depends	optional	Depends	Any OpenAI-compatible speech server (e.g. a local QVox/Qwen3-TTS instance). Supports emotion tags.
mock	yes	no	Silent	Silent WAV placeholder. Only useful for tests.

The auto provider probes in order: piper → elevenlabs → openai → gemini → mock. Custom providers are only used when you add them to the chain order. On a fresh install with no provider configured, auto falls through to mock (silence). Configure at least one real provider before expecting audio.

Tip

Piper is the recommended default for local, offline, key-free voice. See decision spec/decisions/004-piper-default-local-tts.md. The Argentine Spanish voice es_AR-daniela-high has the highest published quality for es-AR.

Configuration

Provider settings live in ~/.apx/config.json under voice.tts:

{
  "voice": {
    "tts": {
      "provider": "piper",
      "piper": {
        "bin": "piper",
        "model": "/home/you/.apx/voices/es_AR-daniela-high.onnx"
      },
      "elevenlabs": {
        "api_key": "...",
        "model": "eleven_multilingual_v2",
        "voice_id": "..."
      },
      "openai": {
        "api_key": "...",
        "model": "tts-1",
        "voice": "alloy",
        "format": "mp3"
      },
      "gemini": {
        "api_key": "...",
        "model": "gemini-2.5-flash-preview-tts",
        "voice": "Aoede"
      }
    }
  }
}

Switch the active provider with:

apx config set voice.tts.provider piper

You can also manage providers from the Web panel under Voices (/m/voice).

Setup paths

Piper (local, no internet)

Piper runs fully offline. You need the piper binary and one .onnx voice model.

1. Install the Piper binary:

   macOS (arm64)

   curl -L https://github.com/rhasspy/piper/releases/latest/download/piper_macos_aarch64.tar.gz \
     -o /tmp/piper.tar.gz
   sudo tar xzf /tmp/piper.tar.gz -C /usr/local/bin --strip-components=1

   Linux (x86_64)

   curl -L https://github.com/rhasspy/piper/releases/latest/download/piper_linux_x86_64.tar.gz \
     -o /tmp/piper.tar.gz
   sudo tar xzf /tmp/piper.tar.gz -C /usr/local/bin --strip-components=1

2. Download a voice model. For Argentine Spanish (recommended):

   mkdir -p ~/.apx/voices
   cd ~/.apx/voices
   curl -LO https://huggingface.co/rhasspy/piper-voices/resolve/main/es/es_AR/daniela/high/es_AR-daniela-high.onnx
   curl -LO https://huggingface.co/rhasspy/piper-voices/resolve/main/es/es_AR/daniela/high/es_AR-daniela-high.onnx.json

3. Point APX at it:

   apx config set voice.tts.provider piper
   apx config set voice.tts.piper.model "$HOME/.apx/voices/es_AR-daniela-high.onnx"

4. Test:

   apx voice say "hola mundo" --provider piper

Gemini cloud (quickest with an existing key)

apx config set voice.tts.provider gemini
apx config set voice.tts.gemini.api_key '<YOUR_GEMINI_KEY>'
apx voice say "hola mundo" --provider gemini

Note

Gemini TTS returns raw L16 PCM, not MP3. APX wraps it in a WAV header automatically. Output files are .wav; if you need MP3 use ffmpeg to convert.

ElevenLabs

apx config set voice.tts.provider elevenlabs
apx config set voice.tts.elevenlabs.api_key '<YOUR_11L_KEY>'
apx config set voice.tts.elevenlabs.voice_id '<VOICE_ID>'
apx voice say "hola mundo" --provider elevenlabs

Custom OpenAI-compatible providers (QVox / Qwen3-TTS)

Beyond the built-in providers, you can point APX at any OpenAI-compatible speech server — for example a local QVox / Qwen3-TTS instance. Custom providers live under voice.tts.custom.<slug> and are all served through the same openai adapter; each one surfaces in the chain with engine id custom:<slug>.

{
  "voice": {
    "tts": {
      "custom": {
        "qvox": {
          "base_url": "http://127.0.0.1:5111/v1",
          "api_key": "",
          "model": "qwen3-tts",
          "voice": "default",
          "label": "QVox (local)"
        }
      }
    }
  }
}

base_url is config-only — it is never hardcoded, so custom:<slug> works with any OpenAI-compatible server, not just QVox. Key handling and requests differ from stock OpenAI when base_url is set:

• The engine uses only its own api_key (often empty for a local/open server) — it never falls back to engines.openai.api_key or OPENAI_API_KEY, so your OpenAI key never leaks to a third-party endpoint.
• Extra fields the stock OpenAI API doesn’t accept — instruct (the base voice/style), language, and temperature — are forwarded to custom endpoints only.
• The default response format for custom endpoints is wav (stock OpenAI defaults to mp3).

Add and manage custom providers from the Web panel under Voices (/m/voice) — Add/Remove a custom provider, edit base_url and the advanced model/voice fields there instead of hand-editing JSON.

Emotion tags (per-engine capability)

Some TTS backends — today the custom:<slug> providers and gemini — can parse inline [tag] markers in the text and change the spoken emotion per-segment (e.g. [excited] ¡Listo! [calm] Lo dejé anotado.). This is a per-engine, opt-in capability, not a hardcoded QVox feature: enable it with voice.tts.<id>.emotions.enabled (or voice.tts.custom.<slug>.emotions.enabled for custom providers).

{
  "voice": {
    "tts": {
      "custom": {
        "qvox": {
          "base_url": "http://127.0.0.1:5111/v1",
          "emotions": {
            "enabled": true,
            "tags": ["happy", "sad", "excited", "angry", "calm", "whisper", "shout", "laugh", "cry", "narrator", "neutral"]
          }
        }
      }
    }
  }
}

tags is optional — it falls back to the canonical set shown above (mirrors QVox’s default tag set) when omitted.

When emotions are enabled for the engine that will actually speak the reply, APX injects a short guide into the voice-mode system prompt so the agent knows the [tag] syntax and uses it sparingly. The guide always matches the first enabled engine in the configured chain — not just any tag-capable engine — so the agent never emits tags a different (non-tag) engine ends up speaking with.

Tags are a TTS-only signal:

• They are kept in the text handed to the TTS engine, so a tag-aware engine can act on them.
• They are stripped from everything the user reads — the chat bubble, message history, and the RAG index — via stripEmotionTags(). On an engine that doesn’t support tags, this same function scrubs any stray markers before synthesis so they’re never read aloud literally.

From the Web panel Voices module, each tag-capable provider row shows a compact Emotions toggle so you can turn the capability on/off per engine without opening the full configure dialog.

The unified voice turn endpoint

POST /voice/turn is a single bidirectional round-trip:

1. STT — transcribes incoming audio (or accepts text directly, skipping STT).
2. Agent — runs the super-agent on the transcribed text.
3. TTS — synthesizes the reply and returns an audio file path.

# Drive from curl with pre-transcribed text
curl -X POST http://127.0.0.1:7430/voice/turn \
  -H "Authorization: Bearer $(cat ~/.apx/daemon.token)" \
  -H "Content-Type: application/json" \
  -d '{"text": "What tasks are open?", "channel": "voice"}'

Response:

{
  "user_text": "What tasks are open?",
  "reply_text": "You have 3 open tasks…",
  "reply_audio_path": "/home/you/.apx/tmp/tts/reply-abc123.wav",
  "reply_duration_s": 4.1,
  "reply_mime": "audio/wav",
  "provider": "piper"
}

The Desktop window and the Deck both use this endpoint internally. The channel field (voice, deck, desktop, telegram) controls response formatting: voice/deck channels get short, spoken-friendly replies; telegram gets Markdown-formatted text.

Speech-to-text (STT)

The local STT engine is Whisper-based and adapts to your hardware instead of forcing you to pick between CTranslate2/MLX/whisper.cpp yourself:

Hardware	Recommended backend	Model
Apple Silicon (Metal)	mlx (mlx-whisper, GPU/Neural Engine)	mlx-community/whisper-large-v3-turbo
NVIDIA (CUDA)	faster (faster-whisper, CUDA)	large-v3
AMD / Radeon	faster (CPU — CTranslate2 has no ROCm support)	small
CPU only	faster (CPU)	small

transcription.local.backend can be auto (default, uses the table above), mlx, or faster. From the Web panel Voices module, the STT card shows a Detected hardware block (acceleration badge — Metal/CUDA/Vulkan-ROCm/CPU — plus GPU name and total memory) and an Acceleration/Engine selector so you can pick the backend explicitly; the model list adapts to whichever backend is selected and shows on-disk download status per model.

Dedicated venv (isolates mlx from your system Python)

mlx-whisper pulls in torch/scipy/numba, which can pollute or clash with your other Python projects if installed into the system or user-site interpreter. APX instead owns a dedicated virtualenv at ~/.apx/runtime/whisper-venv: it creates the venv, installs faster-whisper/mlx-whisper into it, and the whisper server process spawns using that venv’s interpreter. If the venv doesn’t exist yet (e.g. an older install), APX falls back to the system python3 — the legacy path that installs Whisper packages into the user-site.

Language is forced from config, not auto-detected

STT resolves the effective transcription language with this priority: an explicit transcription.local.language, else config.user.language, else "auto" (Whisper auto-detects). The Desktop capture path used to always send an auto language header to /transcribe/chunk, which overrode your configured language and hurt accuracy on short clips and proper nouns. /transcribe/chunk now only overrides the resolved language when the caller pins a real one (not "auto") — so your configured language always stands unless you explicitly ask for auto-detection.

Playback

apx voice say uses the first available system player: afplay (macOS) → paplay → aplay → play (sox) → ffplay.

If none is found the file is written and the path is printed — no error, but no sound. Install any of the above to get playback.

Troubleshooting

apx voice providers shows only mock

No real provider is configured. Follow one of the setup paths above.

File is generated but no sound

No system audio player is on your PATH. On macOS afplay is always present. On Linux install sox (apt install sox) or pulseaudio-utils.

Gemini output sounds distorted or won’t play

Older APX versions didn’t wrap the raw L16 PCM in a WAV header. Update APX or convert with ffmpeg -f s16le -ar 24000 -ac 1 -i raw.pcm out.wav.

apx voice listen returns empty text

sox is required for silence-detection recording. Install it or use --seconds N to specify a fixed duration instead.

apx

$ apx voice say "hello" --provider piper
✓ provider  piper (local)
✓ voice     en_US-amy-medium
• synthesizing 1 sentence … 0.4s
♪ playing 1.2s of audio
✓ done

apx voice say — synthesizing and playing a test phrase

Related

• Desktop — floating window that uses TTS/STT
• Deck — companion app voice mode
• Web panel → Voices module — UI for provider config