#UNOQDreamLab: Offline Voice Companion on the Arduino UNO Q

161

License: Apache License, Version 2.0 UNO Q Dream Lab Arduino

Overview

This is a voice assistant that runs entirely on one Arduino UNO Q 4GB: wake word, speech recognition, language model, and speech synthesis all run on the board, with no cloud and no network at runtime.

You say "Hey Jarvis", wait for a chime, and ask a question. The spoken answer arrives about 7 to 10 seconds after you stop talking. Replies are capped at one or two short sentences and written for a child.

Tuning cut the time for a short question from 16.7 s to 8.7 s, and for a long answer from 37.8 s to 12.8 s. It is a conversation companion, not a fact engine (see Limits).

How it works

A conversation turn runs seven stages on the board, and the wake-word listener stays off from detection until the reply has finished playing.

Voice assistant conversation flow

The USB microphone can be opened by only one process at a time, so the listener lets go of it before recording starts. Stopping it also means the assistant cannot trigger on its own voice.

The speech synthesis voice and the language model are loaded once at startup, so each reply takes seconds instead of the 10 s load.

Hardware

Arduino UNO Q 4GB — runs every stage: Qualcomm QRB2210 Linux side, 3.6 GB RAM usable
Arduino USB-C Hub (8 in 1) — the board has one USB-C port; the hub carries power, mic, and speaker
Arduino USB-C Power Supply (45 W) — powers the hub and the board
SuziePi USB microphone (2-pack, one used) — speech input, seen by Linux as a standard sound card
Waveshare USB sound card with 8 Ω 5 W speakers — speech output through a standard USB audio device

Software and setup

Everything is built from source or installed with pip on the board's stock Debian 13 image (Python 3.13.5), and the finished assistant starts with one command.

Boot the board through the official hub, join WiFi in Arduino App Lab, and connect over SSH.
Build whisper.cpp v1.7.6 with CMake, without the SDL2 example, and download the tiny.en model (74 MB).
Build llama-server from llama.cpp and download Qwen3.5-0.8B at Q4_0 quantization (563 MB) from the ggml-org repository.
Install Piper with pip install piper-tts and download the en_US-hfc_female-medium voice.
Install openWakeWord 0.4.0, which ships ONNX models and needs no TensorFlow Lite runtime.
Copy the project folder to the board and run the start script.

bash run.sh

The script starts the language model server if it is not already running, waits until it accepts connections, and launches the assistant. bash run.sh stop stops the server.

The code is split into small single-purpose modules: wake word, chime, recording, transcription, language model, speech, device lookup, and one config.py that holds every tunable and the benchmark notes behind it. The microphone and speaker are found by their description, not their card number, because both USB devices register under the same short name and their numbers can swap between boots.

Performance

Most turns finish 6 to 10 seconds after you stop talking, down from 17 seconds on the first full run; the longest replies take 13 to 19 seconds.

Transcribe — 6.6 s → 2.9 s — audio window cut from 30 s to 15 s
Answer — 6.7 s → 4.1 s — prompt asks for one or two short sentences
Speak — 3.3 s → 1.7 s — shorter reply, less to synthesize
Total after you stop talking — 16.7 s → 8.7 s

Audio window tuning results

Whisper pads every clip to a 30 s window, so a 5 s question costs the same as a 30 s one. Shrinking the window removes most of that cost, but below about 500 the model loops. We chose 750 over the faster 500 to stay clear of that cliff; a window of 400 also looped.

Other measurements that shaped the build:

Model size — tiny.en transcribed an 11 s clip in 6.3 s against 14.1 s for base.en, and base.en heard the same words on two clips where tiny.en missed.
Prompt cache — the language model took 10.2 s on its first request and 2.9 s on repeats, because the fixed system prompt is processed once.
Voice loading — Piper takes 10.1 s to load and about 1 to 1.5 s to speak a short reply, so it loads once at startup.
Wake word — openWakeWord needs 30 ms per 80 ms audio chunk, so it keeps up with the microphone in real time.

What went wrong

Six surprises cost the most time, and each one has a fix in the code.

The language model kept talking — Ran past 247 tokens on "What is your favorite color?", although the llama.cpp docs said thinking is off by default — Send enable_thinking: false with every request; cap replies at 60 tokens
A bigger model was no better — Qwen3.5 2B gave a confident wrong date for dinosaurs; the 0.8B model, unsteered, answered a color question romantically — Stay at 0.8B; the system prompt fixes tone and length; the assistant is scoped as a companion, not a fact engine
The obvious audio window was the worst — A window equal to the 5 s clip (250) produced 16.6 s of garbled text — Use 750, tested on five varied clips against the 30 s default
Quiet speech was misheard — "Tallest animal" came out as "all this animal" at every window size and on two models; mic gain was already at 100%, and a 7x software boost changed nothing — Speak close to the mic; the chime marks when recording starts
Silence became text — Whisper returns the tag [BLANK_AUDIO], which would have gone to the language model as a question — Drop bare non-speech tags before the model sees them
USB devices swapped card numbers — Mic and speaker both register as "Device" — Find each card by its description

Limits

This is a companion for short chats, not a source of facts, and these are the limits we measured or saw.

Facts are unreliable. Asked about World War II, the 0.8B model gave a confident answer with wrong claims. The prompt tells it to admit uncertainty, and it does not always obey.
No memory between turns. Each question is answered on its own, so it said it has no feelings and, one turn later, that it feels "a little warm".
Goodbyes are weak. "Take care" got "I'm busy here, but I'm happy to hear it."
Not conversational speed. Most replies take 6 to 10 seconds, and the longest took 19.
Fixed 5 second recording. A long question is cut off, and a short one still waits the full five seconds.
Close-range speech only. It was tested with the speaker near the microphone, not across a room.
False triggers happen. During filming, a throat-clear alone triggered a turn with no wake word spoken.
English only, because both the speech model and the voice are English models.

What's next

Four changes are planned, in this order.

Raise the wake-word threshold and test more broadly against background noise and conversation, since a throat-clear alone was enough to trigger a turn once.
Replace the fixed 5 second recording with voice-activity detection, so recording stops when the speaker does. whisper.cpp has a built-in option to try first.
Decide how to handle factual questions. One option is a prompt rule to say it is not sure and suggest asking a grown-up.
Add an animated face on a small I2C OLED, as a second phase, because HDMI output on the UNO Q has thin documentation.