Audio Demo#
Overview#
This sample calls MPI APIs to implement audio input, output, encode/decode, and Audio 3A processing end to end. The sample covers the following test cases:
Audio input test: capture environmental sound and save it as a WAV file to verify input accuracy and stability.
Audio output test: play a specified WAV file and evaluate the output quality through headphones.
Simultaneous audio input/output test: capture and play audio in real time to verify low-latency processing.
Audio codec test: use the built-in
G711a/uandOpus16-bitcodecs to validate encoding and decoding reliability.
Functional Description#
Audio Input#
The audio-input path captures environmental sound and saves it to a file for later analysis. In this sample, the system records 15 seconds of audio data and saves it in standard WAV format. The generated WAV file can be played directly in VLC for quick verification.
Audio Output#
The audio-output path plays a WAV file so the user can evaluate the playback result through headphones or speakers. Users can upload different WAV files to verify compatibility and playback behavior.
Audio Input and Output#
The simultaneous input/output path mainly validates the I2S module. During the test, the I2S input captures live environmental sound and the I2S output plays it back immediately. With headphones connected, the user can directly hear the real-time loopback result.
Audio Codec#
The system includes built-in G711a/u 16-bit audio codecs and also supports registering additional external codecs when needed.
Audio 3A#
Audio 3A (Automatic Acoustic Adjustment) includes:
ANS (Automatic Noise Suppression): reduces environmental noise and improves signal cleanliness.
AGC (Automatic Gain Control): automatically adjusts gain so the signal stays within a reasonable range.
AEC (Acoustic Echo Cancellation): removes echo during audio transmission to improve communication clarity.
These functions can be enabled through the corresponding bit flags. For the exact API usage and parameter settings, refer to the API documentation.
Usage#
Source Location#
Demo source path:
/src/rtsmart/examples/mpp/sample_audio
Assuming the demo has already been built correctly, boot the board and enter /sdcard/app/examples/mpp. The test executable is sample_audio.elf.
Parameter Description#
After boot, enter /sdcard/app/examples/mpp and run ./sample_audio.elf -help to view the usage.
Parameter |
Description |
Default |
|---|---|---|
|
test case selector for different modules, |
- |
|
input/output sample rate, |
|
|
whether to enable the built-in codec, |
|
|
kernel log level, |
- |
|
sample precision, |
|
|
WAV or G711 file name to load or save |
- |
|
channel count, mono or stereo, |
|
|
mono input selection, |
|
|
Audio 3A enable flags: |
|
Representative help output:
msh /sharefs/app>./sample_audio.elf
Please input:
-type: test audio function[0-12]
type 0:sample ai i2s module
type 1:sample ai pdm module
type 2:sample ao i2s module
type 3:sample ai(i2s) to ao (api) module
type 4:sample ai(i2s) to ao (sysbind) module
type 5:sample ai(pdm) to ao (api) module
type 6:sample ai(pdm) bind ao (sysbind) module
type 7:sample aenc(ai->aenc->file) (sysbind) module
type 8:sample adec(file->adec->ao) (sysbind) module
type 9:sample aenc(ai->aenc->file) (api) module
type 10:sample adec(file->adec->ao) (api) module
type 11:sample overall test (ai->aenc->file file->adec->ao) module
type 12:sample overall test (ai->aenc adec->ao loopback ) module
type 13:sample overall test (opus ai->aenc adec->ao loopback ) module
-samplerate: set audio sample(8000 ~ 192000)
-enablecodec: enable audio codec(0,1)
-loglevel: show kernel log level[0,7]
-bitwidth: set audio bit width(16,24,32)
-channels: channel count
-monochannel:0:mic input 1:headphone input
-filename: load or save file name
-audio3a: enable audio3a:enable_ans:0x01,enable_agc:0x02,enable_aec:0x04
Examples#
I2S Audio Input Test#
Run the following command to capture 15 seconds of PCM audio data and save it as a WAV file:
./sample_audio.elf -type 0 -enablecodec 1 -bitwidth 16 -filename test.wav -audio3a 1
Representative output:
./sample_audio.elf -type 0 -enablecodec 1 -bitwidth 16 -filename test.wav -audio3a 1
audio type:0,sample rate:44100,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:7.46 MB
vb_set_config ok
sample ai i2s module
audio i2s set clk freq is 2822400(2822400),ret:1
audio codec adc clk freq is 11289600(11289600)
ans_enable
========ans_enable:1,agc_enable:0,aec_enable:0
audio_save_init get vb block size:2646044
======kd_mpi_sys_mmap total size:2646044
[0s] timestamp 0 us,curpts:1505976917
[1s] timestamp 1000000 us,curpts:1506976917
[2s] timestamp 2000000 us,curpts:1507976917
[3s] timestamp 3000000 us,curpts:1508976917
[4s] timestamp 4000000 us,curpts:1509976917
[5s] timestamp 5000000 us,curpts:1510976917
[6s] timestamp 6000000 us,curpts:1511976917
[7s] timestamp 7000000 us,curpts:1512976917
[8s] timestamp 8000000 us,curpts:1513976917
[9s] timestamp 9000000 us,curpts:1514976917
[10s] timestamp 10000000 us,curpts:1515976917
[11s] timestamp 11000000 us,curpts:1516976917
[12s] timestamp 12000000 us,curpts:1517976917
[13s] timestamp 13000000 us,curpts:1518976917
dump binary memory test1.wav 0x10225000 0x104ab01c
[14s] timestamp 14000000 us,curpts:1519976917
destroy vb block
sample done
The audio data is saved to the specified WAV file.
I2S Audio Output Test#
This test loops a WAV file to verify the audio-output path. Press any key to exit.
./sample_audio.elf -type 2 -filename test.wav -enablecodec 1
Representative output:
./sample_audio.elf -type 2 -filename test.wav -enablecodec 1
audio type:2,sample rate:44100,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:7.46 MB
vb_set_config ok
enter q key to exit
sample ao i2s module
========read_wav_header:headerlen:44,channel:2,samplerate:44100,bitpersample:16
open file:test.wav ok,file size:2646044,data size:2646000,wav header size:44
=======_get_audio_frame virt_addr:0x1002aa000
audio i2s set clk freq is 2822400(2822400),ret:1
audio init codec dac clk freq is 11289600
audio set codec dac clk freq is 11289600(11289600)
q
diable ao audio
destroy vb block
sample done
Audio is played through the headphone or speaker path until the user exits.
I2S Audio Input/Output API Test#
Use the API path to test real-time input and output:
./sample_audio.elf -type 3 -bitwidth 16 -enablecodec 1 -samplerate 8000 -audio3a 1
Representative output:
./sample_audio.elf -type 3 -bitwidth 16 -enablecodec 1 -samplerate 8000 -audio3a 1
audio type:3,sample rate:8000,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:1.35 MB
vb_set_config ok
enter q key to exit
sample ai(i2s) to ao module
audio i2s set clk freq is 512000(512000),ret:1
audio codec adc clk freq is 2048000(2048000)
ans_enable
========ans_enable:1,agc_enable:0,aec_enable:0
audio i2s set clk freq is 512000(512000),ret:1
audio init codec dac clk freq is 2048000
audio set codec dac clk freq is 2048000(2048000)
[0s] timestamp 0 us,curpts:2017301433
[1s] timestamp 1000000 us,curpts:2018301433
...
q
diable ao module
diable ai module
release vb block
destroy vb block
sample done
I2S Audio Input and Output System Bind Test#
Use AI/AO binding through kd_mpi_sys_bind to test real-time audio loopback:
./sample_audio.elf -type 4 -bitwidth 16 -enablecodec 1 -samplerate 8000 -audio3a 1
Representative output:
./sample_audio.elf -type 4 -bitwidth 16 -enablecodec 1 -samplerate 8000 -audio3a 1
audio type:4,sample rate:8000,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:1.35 MB
vb_set_config ok
enter q key to exit
sample ai(i2s) bind ao module
audio i2s set clk freq is 512000(512000),ret:1
audio codec adc clk freq is 2048000(2048000)
ans_enable
========ans_enable:1,agc_enable:0,aec_enable:0
audio i2s set clk freq is 512000(512000),ret:1
audio init codec dac clk freq is 2048000
audio set codec dac clk freq is 2048000(2048000)
q
diable ao module
diable ai module
release vb block
destroy vb block
sample done
Encode Test#
Use the following commands to get audio from AI, encode it, and save it to a file. The codec path supports G711a/u/lpcm, 16-bit.
System-bind mode:
./sample_audio.elf -type 7 -bitwidth 16 -enablecodec 1 -filename /sharefs/i2s_codec.g711a -audio3a 1
Representative output:
./sample_audio.elf -type 7 -bitwidth 16 -enablecodec 1 -filename /sharefs/i2s_codec.g711a -audio3a 1
audio type:7,sample rate:44100,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:7.46 MB
vb_set_config ok
enter q key to exit
sample aenc module (sysbind)
audio i2s set clk freq is 2822400(2822400),ret:1
audio codec adc clk freq is 11289600(11289600)
ans_enable
========ans_enable:1,agc_enable:0,aec_enable:0
q
destroy vb block
sample done
API mode:
./sample_audio.elf -type 9 -bitwidth 16 -enablecodec 1 -filename /sharefs/i2s_codec.g711a -audio3a 1
Representative output:
./sample_audio.elf -type 9 -bitwidth 16 -enablecodec 1 -filename /sharefs/i2s_codec.g711a -audio3a 1
audio type:9,sample rate:44100,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:7.46 MB
vb_set_config ok
enter q key to exit
sample aenc module (api)
audio i2s set clk freq is 2822400(2822400),ret:1
audio codec adc clk freq is 11289600(11289600)
ans_enable
========ans_enable:1,agc_enable:0,aec_enable:0
q
destroy vb block
sample done
Decode Test#
Use the following commands to read a file, decode it, and play it back. The codec path supports G711a/u/lpcm, 16-bit.
System-bind mode:
./sample_audio.elf -type 8 -filename /sharefs/i2s_codec.g711a -enablecodec 1 -bitwidth 16
Representative output:
./sample_audio.elf -type 8 -filename /sharefs/i2s_codec.g711a -enablecodec 1 -bitwidth 16
audio type:8,sample rate:44100,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:7.46 MB
vb_set_config ok
enter q key to exit
sample adec module (sysbind)
audio i2s set clk freq is 2822400(2822400),ret:1
audio init codec dac clk freq is 11289600
audio set codec dac clk freq is 11289600(11289600)
adec_bind_call_back dev_id:0 chn_id:0
read file again
q
adec_bind_call_back dev_id:0 chn_id:0
destroy vb block
sample done
API mode:
./sample_audio.elf -type 10 -filename /sharefs/i2s_codec.g711a -enablecodec 1 -bitwidth 16
Representative output:
./sample_audio.elf -type 10 -filename /sharefs/i2s_codec.g711a -enablecodec 1 -bitwidth 16
audio type:10,sample rate:44100,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:7.46 MB
vb_set_config ok
enter q key to exit
sample adec module (api)
audio i2s set clk freq is 2822400(2822400),ret:1
audio init codec dac clk freq is 11289600
audio set codec dac clk freq is 11289600(11289600)
read file again
q
destroy vb block
sample done
Full Audio Pipeline Test 1#
This full audio pipeline test includes two bound paths: AI -> AENC and ADEC -> AO. It uses the built-in codec with 16-bit precision, enables AEC-related audio cleanup, and verifies timestamps in the encoded G711 stream.
./sample_audio.elf -type 12 -samplerate 48000 -enablecodec 1 -audio3a 1 &
Representative output:
./sample_audio.elf -type 12 -samplerate 48000 -enablecodec 1 -audio3a 1 &
audio type:12,sample rate:48000,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:8.12 MB
vb_set_config ok
enter q key to exit
sample ai->aenc adec->ao module (loopback)
Force the sampling accuracy to be set to 16,use inner cocdec
audio i2s set clk freq is 3072000(3072000),ret:1
audio codec adc clk freq is 12288000(12288000)
audio i2s set clk freq is 3072000(3072000),ret:1
audio init codec dac clk freq is 11289600
audio set codec dac clk freq is 12288000(12288000)
adec_bind_call_back dev_id:0 chn_id:0
[0s] g711 stream timestamp 0 us,curpts:341326051
[1s] g711 stream timestamp 1000000 us,curpts:342326051
...
q
adec_bind_call_back dev_id:0 chn_id:0
destroy vb block
sample done
You can inspect module bindings with:
cat /proc/umap/sysbind
Representative binding output:
-----BIND RELATION TABLE--------------------------------------------------------
FirMod FirDev FirChn SecMod SecDev SecChn TirMod TirDev TirChn SendCnt rstCnt
ai 0 0 aenc 0 0 null 0 0 310 0
adec 0 0 ao 0 0 null 0 0 310 0
Full Audio Pipeline Test 2#
This test uses the same loopback structure as the previous one, but verifies timestamps on the Opus encoded stream.
./sample_audio.elf -type 13 -samplerate 48000 -enablecodec 1 -audio3a 1 &
Representative output:
./sample_audio.elf -type 13 -samplerate 48000 -enablecodec 1 -audio3a 1 &
audio type:13,sample rate:48000,bit width:16,channels:2,enablecodec:1,monochannel:0
vb_set_config ok
audio_data_vb_create_pool poolid 0
enter q key to exit
sample ai->aenc adec->ao module (opus,loopback)
Force the sampling accuracy to be set to 16,use inner cocdec
_opus_open_encoder enc_inst:0x79da80, samplerate:8000,channels:1
_opus_open_encoder ok
_opus_open_decoder dec_inst:0x0, samplerate:8000, channels:1 before
_opus_open_decoder ok
audio i2s set clk freq is 512000(512000),ret:1
audio codec adc clk freq is 2048000(2048000)
audio i2s set clk freq is 512000(512000),ret:1
audio init codec dac clk freq is 11289600
audio set codec dac clk freq is 2048000(2048000)
adec_bind_call_back dev_id:0 chn_id:0
...
q
adec_bind_call_back dev_id:0 chn_id:0
destroy vb block
sample done
You can also inspect module bindings with:
cat /proc/umap/sysbind
Representative binding output:
-----BIND RELATION TABLE--------------------------------------------------------
FirMod FirDev FirChn SecMod SecDev SecChn TirMod TirDev TirChn SendCnt rstCnt
ai 0 0 aenc 0 0 null 0 0 310 0
adec 0 0 ao 0 0 null 0 0 310 0
Tip
For detailed interfaces of the audio module, refer to the API documentation.