Xiaozhi Voice Assistant Example Walkthrough#

Overview#

This example implements the “Xiaozhi” voice assistant client on CanMV K230, communicating with the cloud-based Xiaozhi service via WebSocket to complete voice collection, upload, TTS playback, interface status display, and supports face detection/registration wake-up and MCP / IoT device control capabilities.

Example location:

Development board firmware path: /sdcard/examples/26-xiaozhi/
SDK source code path: src/canmv/resources/examples/26-xiaozhi/

Environment Preparation#

Before running this example, please confirm:

Hardware: CanMV K230 development board, camera module, on-board microphone and speaker/headphone; the default conversation button is GPIO21 (active low), and the status LED is GPIO52. These can be modified in PROJECT_CONFIG at the top of run.py.
Network: The development board is connected to the internet via LAN or WLAN and can access the Xiaozhi OTA / WebSocket service (default api.tenclass.net).
Storage: The SD card contains this example directory and related resources (fonts, emoji images, etc.).
Configuration: All user configurations are uniformly located in PROJECT_CONFIG at the top of run.py. On the first run, /data/xiaozhi.cfg will be generated; this file only saves the auto-generated UUID and MAC, and does not need to be manually modified.

How to Run#

First-Time Activation#

The first time you start, you need to complete device activation; otherwise, you cannot connect to the Xiaozhi service normally.

Open the Xiaozhi official website in your browser: https://xiaozhi.me/, and follow the page instructions to complete device binding/activation.
The activation code will be printed in the serial port log on the first startup, please pay attention to check, for example:
```
收到激活码: xxxxxx
设备需要激活，激活码: xxxxxx
```
Enter the activation code from the log into the corresponding location on the official website to complete activation. After successful activation, run the example again to establish a connection normally; subsequent starts generally do not require repeated activation.

Startup#

Open and run in CanMV IDE:

/sdcard/examples/26-xiaozhi/run.py

Before running, you only need to modify PROJECT_CONFIG at the top of run.py; other project files do not need to be modified:

PROJECT_CONFIG = {
    "hardware": {
        "key_pin": 21,
        "led_pin": 52,
    },
    "network": {
        "type": "wifi_sta",       # "default"、"lan" 或 "wifi_sta"
        "wlan_device": "auto",    # "auto"、"usb"、"sdio" 或 "spi"
        "timeout": 20,
        "ssid": "canaan",
        "password": "Canaan314",
    },
    "xiaozhi": {
        "ota_url": "https://api.tenclass.net/xiaozhi/ota/",
        "websocket_url": "wss://api.tenclass.net:443/xiaozhi/v1/",
    },
    "webtts": {
        "app_id": "",
        "api_key": "",
        "api_secret": "",
        "sample_rate": 16000,
        "voice_name": "xiaoyan",
        "speed": 50,
    },
}

When network.type is set to "default", it will reuse the default interface that is already connected to the network; when set to "lan", LAN is used; when set to "wifi_sta", the SSID and password above are used to connect to Wi-Fi.

The program will complete in sequence: UI / camera initialization → device initialization → OTA activation → connecting to the Xiaozhi WebSocket service.

After successful startup, the serial port and interface status will prompt something like:

小智语音助手已启动，按Ctrl+C退出

The interface status will eventually enter “Waiting for Button Wake-up”.

Important Reminder: Wait for “Waiting for Button Wake-up” Before Pressing the Button to Talk#

Please be sure to wait until the interface/status shows “Waiting for Button Wake-up” before pressing the button to start a conversation. If you press the button before network connection, device activation, or session establishment is complete, you may see a prompt like “Network connecting, please wait”; in this case, you need to wait for the connection to complete before trying again.

Recommended interaction flow:

Wait for the status to change to “Waiting for Button Wake-up”.
Press and hold the on-board conversation button (GPIO21): recording starts, the status changes to “Waiting for your command”, and the LED (GPIO52) lights up.
Speak into the microphone.
Release the button: recording ends and is uploaded, waiting for the cloud response; the status is “Speaking” during TTS playback.
After playback ends, return to “Waiting for Button Wake-up” and you can proceed with the next round of conversation.

Additional notes:

This example uses manual button interaction mode by default (hold to speak, release to end).
If a face has been registered, recognition of a registered user can also trigger face wake-up (sending wake-up text) without pressing the button.
You can stop the script in the IDE or use Ctrl+C to exit; on exit, face detection and WebSocket / audio threads will be stopped first.

Code Architecture#

The routine is split into multiple layers by responsibility, with the entry point being run.py:

run.py                 # Startup entry: UI + Client assembly, main loop, exit cleanup
├── xiaozhiclient.py   # Business hub: activation, session, button/face wakeup, message dispatch
├── websocket_client.py# WebSocket send/receive (including Opus audio frames)
├── http_client.py     # OTA / activation HTTP requests
├── audio_manager.py   # Recording / Opus codec / TTS playback queue
├── config.py          # Configuration loading, UUID/MAC, WS address and hello
├── state.py           # Protocol message types, session state, message construction/parsing
├── lvgl_ai.py         # LVGL UI + camera + face detection thread
├── face_reg_rec.py    # Face recognition / registration
└── iot/               # MCP / IoT device abstraction
    ├── thing.py
    ├── thing_manager.py
    └── things/speaker.py   # Example device: speaker volume control

Module Description#

Module	Function
`run.py`	Creates `XiaoZhi_UI` and `XiaoZhiClient`, registers callbacks, refreshes LVGL and UI update queue in the main loop
`xiaozhiclient.py`	Device lifecycle, button recording, face wakeup, handles hello / STT / TTS / LLM / IoT / MCP messages
`websocket_client.py`	Maintains WSS connection, heartbeat, text/binary frame send/receive
`audio_manager.py`	AI/AO capture and playback, Opus encoding for upload and decoding for playback (pre-buffer queue used on playback side to reduce stuttering)
`lvgl_ai.py`	UI state, expressions, subtitles; face detection overlay; cross-thread UI updates via queue flushed in main loop
`iot/`	Abstracts board-side capabilities as Things, reports descriptors / states, executes control commands issued from the cloud

Data Flow (Schematic)#

Button pressed
  → AudioManager recording → Opus encoding
  → WebSocket binary upload
  → Cloud ASR / LLM / TTS
  → WebSocket sends Opus + text message
  → AudioManager playback + UI updates state/subtitle/expression

Cloud IoT / MCP control
  → ThingManager.invoke(...)
  → e.g. Speaker.setvolume to adjust volume
  → Send back states

Startup Flow (Core Logic)#

xiaozhi_client = XiaoZhiClient.get_instance(PROJECT_CONFIG)
xiaozhi_gui = XiaoZhi_UI()
xiaozhi_gui.user_gui_init()

xiaozhi_client.init_device(audio_download_callback, tts_state_callback, ws_state_callback)
xiaozhi_client.active_device()
xiaozhi_client.connect_to_server()
xiaozhi_client.register_update_callback(text_callback, llm_callback, status_callback)
xiaozhi_client.start_key_trigle_thread()
xiaozhi_client.start_face_wakeup_thread()

while True:
    xiaozhi_gui.flush_ui_updates()
    time.sleep_ms(lv.task_handler())
    # ... face result sync, reconnect on demand, etc.

MCP and IoT Features#

This example declares MCP support in the hello message of the connection handshake:

"features": {
    "mcp": True
}

Key implementation points:

Capability declaration: Set features.mcp = True in state.MessageBuilder.build_hello_message() to inform the cloud client that MCP-related capabilities are supported.
IoT device model: Register board-side devices (the current example is Speaker) through ThingManager. After the session is established, report descriptors (device and method descriptions, such as the volume property and the setvolume method) as well as states (current device state).

Command execution: After receiving a type: "iot" message, it is dispatched and executed by ThingManager.invoke(), for example:

{
  "type": "iot",
  "commands": [
    {
      "name": "Speaker",
      "method": "setvolume",
      "parameters": { "volume": 50 }
    }
  ],
  "session_id": "..."
}

If the state changes after execution, states will be reported again.

MCP message channel: The protocol side already supports parsing and dispatching entry points for type: "mcp" (_hanle_mcp_message), which can be extended for MCP tool invocation; the current example uses IoT Things (such as speaker volume) to demonstrate side-controllable capabilities.

When extending new devices, refer to iot/things/speaker.py:

Inherit from Thing and define properties and methods;
Register with add_thing(...) in ThingManager.initialize_iot_devices().

UI and Status Prompts#

The meanings of common UI/status texts are as follows:

Status Text	Meaning
网络连接中，请稍后。。。。	WebSocket is not yet ready, please wait before pressing the button
等待按键唤醒	Ready, press and hold the button to start speaking
等待您的指令	Recording, please speak
获取指令结束	Button released, ending this round of recording
说话中 / 说话结束	TTS playback start / end
网络已连接 / 网络连接中断	WebSocket connection status change

Notes#

Wait for “等待按键唤醒” before pressing the button to talk, to avoid mistakenly triggering recording during activation or connection establishment.
For the first launch, please complete activation on the official website first (https://xiaozhi.me/); the activation code can be found in the serial port log.
A stable network is required; the first activation depends on the OTA interface, please confirm that the firewall/proxy will not block HTTPS / WSS.
Face detection and TTS playback will simultaneously occupy computing power and memory. For long-running sessions, please ensure stable cooling and power supply.
When stopping the script from the IDE, soft reboot related logs may appear (such as Display unbind, ISP stop), which are part of the resource recycling process and generally do not affect subsequent runs.

For specific multimedia and network related interfaces, please refer to the documentation: audio, display, sensor, lvgl, network, socket.

Xiaozhi Voice Assistant Example Walkthrough

Contents

Xiaozhi Voice Assistant Example Walkthrough#

Overview#

Environment Preparation#

How to Run#

First-Time Activation#

Startup#

Important Reminder: Wait for “Waiting for Button Wake-up” Before Pressing the Button to Talk#

Code Architecture#

Module Description#

Data Flow (Schematic)#

Startup Flow (Core Logic)#

MCP and IoT Features#

UI and Status Prompts#

Notes#