Note

This is the documentation for the latest development branch and may refer to features that are not available in released versions. If you are looking for the documentation for a specific release, use the drop-down menu on the left and select the desired version.

K230 SDK CanMV Board Demo User Guide

Overview

Demo Introduction

GPU

The source code path is buildroot-overlay/package/vg_lite, which includes 5 example programs

• tiger: Render a 640x480 image of a tiger and save it as tiger.png

• linearGrad: Render a gradient pattern and save it as linearGrad.png

• imgIndex: Render four images using color lookup tables and save them as png files

• vglite_drm: Display patterns drawn using GPU on the screen

• vglite_cube: Display a cube border drawn using GPU on the screen

Face Detection

The face detection demo takes input from the camera and displays the results on the screen. This demo is only used to demonstrate AI-related features of KPU and nncase.

The source code is located at buildroot-overlay\package\face_detect, and the executable program is placed at output/k230_canmv_defconfig/target/app/face_detect. Enter /app/face_detct on the board and execute ./face_detect.elf face_detection_320.kmodel to run it. It will draw boxes around the faces on the screen.

Camera Capture and Display

v4l2-drm captures images from the camera and displays them on the screen. The source code path is buildroot-overlay/package/vvcam/v4l2-drm

Use the command line v4l2-drm -d 1 -w 480 -h 320 to run it

-d specifies the video device

-w specifies the image width

-h specifies the image height

-f specifies the image format, supports NV12/NV16/BGR3/BG3P, currently only NV12 and BGR3 support display

-s disables display (only capture), press q to exit after the program runs, press d to save an image. Multiple devices can be opened simultaneously, for example v4l2-drm -d 1 -w 480 -h 320 -d 2 -w 1920 -h 1080 -f BGR3 -s.

Camera Capture and Streaming

You can use ffmpeg to open the /dev/video1 device for encoding and streaming to PC. First, install ffmpeg on the PC, then create a test.sdp file and fill in the following content

SDP:
v=0
o=- 0 0 IN IP4 127.0.0.1
s=No Name
c=IN IP4 10.100.228.227
t=0 0
a=tool:libavformat 58.76.100
m=video 1233 RTP/AVP 96
b=AS:20000
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1

Where 10.100.228.227 should be changed to the PC’s IP address, then run on the PC

ffplay -protocol_whitelist "file,udp,rtp,tcp" -i test.sdp -fflags nobuffer -analyzeduration 1000000 -flags low_delay

Then run on the board

ffmpeg -f v4l2 -i /dev/video1 -video_size 1920x1080 -vsync pass
through -vcodec h264_v4l2m2m -b:v 10M -f rtp rtp://10.100.228.227:1233

Note that 10.100.228.227 should be changed to the PC’s IP address.

Audio

Using audio_demo for Audio Recording and Playback

audio_demo is an example program running on the K230 development board. It implements audio recording and playback functionality by calling the ALSA (Advanced Linux Sound Architecture) API. This example program not only demonstrates how to perform audio processing on the K230 development board, but also provides reference code for developers to implement audio functionality using the ALSA API interface.

Recording WAV File

Use the following command to record an audio file in WAV format:

audio_demo -type 2 -filename test.wav -channels 2 -samplerate 16000

Parameter description:

• -type 2: Specifies the operation type as recording.

• -filename: Sets the output file name.

• -channels: Number of recording channels, 2 for stereo.

• -samplerate: Sets the sample rate, in Hz.

The process will automatically exit after recording for 15s (do not kill the process during recording, otherwise it will cause the recorded file to be abnormal). The audio file is saved as test.wav in the current directory.

Playing WAV File

Use the following command to play the recorded WAV file:

audio_demo -type 0 -filename test.wav

Parameter description:

• -type 0: Specifies the operation type as playing a WAV file.

• -filename: The name of the file to play.

Playing MP3 File

audio_demo also supports playing audio files in MP3 format:

audio_demo -type 1 -filename example.mp3

Parameter description:

• -type 1: Specifies the operation type as playing an MP3 file.

• -filename: The name of the MP3 file to play.

Using FFmpeg for Audio Recording and Playback

Audio Recording

Use the following command to call the ALSA interface through FFmpeg to record sound and save it as a WAV file:

ffmpeg -f alsa -i hw:0 -t 15 -ac 2 -ar 44100 -y test.wav

Where:

• -f alsa: Specifies the audio input format as ALSA.

• -i hw:0: Specifies the audio input device, hw:0 represents the default audio device.

• -t 15: Sets the recording duration to 15 seconds.

• -ac 2: Sets the number of recording channels to 2 (stereo).

• -ar 44100: Sets the sample rate to 44100Hz.

• -y test.wav: The output file name is test.wav.

After executing the above command, the recorded sound will be saved as a WAV format audio file test.wav.

Audio Playback

Use ffmpeg to play the audio file:

ffmpeg -i test.wav -f alsa hw:0,0

Using arecord and aplay to Implement Audio Recording and Playback

Record Audio

arecord -Dhw:0,0 -d 10 -f cd -r 44100 -c 2 -t wav test.wav

Play Audio

aplay -Dhw:0,0 -v test.wav

Volume Control

Use the amixer command for volume adjustment and mute control:

Volume Control

• View the control list:

  amixer controls
  

• Get the current volume:

  amixer cget numid=1
  

• Set the volume to 12:

  amixer cset numid=1 12
  

Mute Control

• Mute:

  amixer cset numid=2 0
  

• Unmute:

  amixer cset numid=2 1