Phones — iPhone & Android

Talk to Watari from your phone with no app and no web page — just the built-in voice assistant calling the brain's /talk endpoint over your Tailnet. Below: the full iPhone Siri Shortcut (the one most people want), the Android equivalents, and the gotchas we hit so you don't.

Before you start

  1. Both the phone and the brain host are on the same Tailscale tailnet.
  2. The brain is reachable on its HTTP sidecar (port 8766):
    # on the brain host (VPS or laptop):
JARVIS_BRAIN_HOST=0.0.0.0  uv run python -m jarvis.brain.server
# -> HTTP sidecar on http://0.0.0.0:8766  (POST /talk, GET /healthz)
  3. You have your JARVIS_API_AUTH_TOKEN (any non-empty token enables auth; required for non-loopback access).
  4. Note the brain's tailnet IP (e.g. 100.x.y.z) — use it, not the LAN IP, so it works anywhere.

iPhone — the Siri Shortcut (step by step)

Open the Shortcuts app → + (new shortcut) → add these actions in order:

  1. Dictate Text — captures your speech. (Russian iOS: it's “Продиктовать текст”. Set its language to the one you'll speak.)
  2. Get Contents of URL — the request to the brain:
    • URL: http://<brain-tailnet-ip>:8766/talk?token=<YOUR_TOKEN>
    • Method: POST
    • Request Body: JSON
    • Add one field: key text, value = the Dictated Text variable (tap to insert the magic variable — don't type the words).
    • Headers: none needed — the token rides in the URL query string. (You can instead send the body as {"text": "...", "token": "..."} if you prefer.)
  3. Play the response — the endpoint returns Watari's voice as audio/mpeg, so “Play” speaks it in his ElevenLabs voice.
  4. Name the shortcut, then trigger it with “Hey Siri, <name>”.
Siri-name gotcha (important). Siri matches the shortcut name phonetically in your phone's language. On a Russian iPhone, “Watari” transcribes to Cyrillic and never matches a Latin name — so Siri won't launch it. Fix: name the shortcut something your Siri reliably hears in its own language (e.g. Cyrillic Джарвис, or a simple word/number). The spoken name and the assistant's identity are independent — the shortcut name is just the Siri trigger.

Simpler all-iOS variant (Siri's own voice, no audio decode)

Add &format=text to the URL and use Speak Text on the JSON reply field instead of “Play”. This uses the Siri voice and works on any iOS — handy if audio playback is ever fiddly.

http://<brain-tailnet-ip>:8766/talk?token=<YOUR_TOKEN>&format=text

With AirPods

Connect AirPods to the iPhone and run the same shortcut — iOS routes the reply into the AirPods automatically, and dictation uses their mic. Nothing extra to set.

Android — two ways

  • No-app voice (mirror of the iPhone shortcut): use Tasker, HTTP Shortcuts, or a Google Assistant routine to POST your dictated text to http://<brain-tailnet-ip>:8766/talk?token=… and play the audio reply. Declare the listening endpoint with the android (speaker) or android-headphones device hint so barge-in/routing behave.
  • Full mic stream (Termux edge-lite): install Termux + Termux:Boot, run the Python edge there, and it streams mic → brain and plays TTS just like a laptop — wake word and barge-in included. Disable battery optimisation so it survives in the background.

With earbuds on the phone, the android-headphones hint makes the session private (barge-in on) and the reply plays in the earbuds.

Troubleshooting

SymptomCause & fix
Siri won't launch the shortcutName mismatch in your language — rename to something Siri hears reliably (see the gotcha above).
“Couldn't understand / repeat”The brain returned empty — usually an LLM provider hiccup. The failover chain + empty-response retry fix this; make sure the brain is current and a fast primary (e.g. Groq) is set.
“Time limited” / shortcut times outThe turn took too long. Use a fast primary model and keep the brain on the always-on host; typical /talk is ~3s.
401 UnauthorizedThe ?token= doesn't match JARVIS_API_AUTH_TOKEN on the brain.
Can't connect at allBrain not bound to 0.0.0.0, phone not on the tailnet, or a firewall blocks 8765/8766. Prefer the tailnet IP; on Windows allow those ports inbound.
File-path requests failUse forward slashes in paths (C:/tmp/x.txt) — backslashes break the JSON body.
Backup channel. If dictation is ever flaky, the same brain answers over Telegram as text — and can reply with a voice note.