# Audio Demo

## Overview

This sample calls MPI APIs to implement audio input, output, encode/decode, and Audio 3A processing end to end. The sample covers the following test cases:

- **Audio input test**: capture environmental sound and save it as a WAV file to verify input accuracy and stability.
- **Audio output test**: play a specified WAV file and evaluate the output quality through headphones.
- **Simultaneous audio input/output test**: capture and play audio in real time to verify low-latency processing.
- **Audio codec test**: use the built-in `G711a/u` and `Opus` `16-bit` codecs to validate encoding and decoding reliability.

## Functional Description

### Audio Input

The audio-input path captures environmental sound and saves it to a file for later analysis. In this sample, the system records `15` seconds of audio data and saves it in standard WAV format. The generated WAV file can be played directly in VLC for quick verification.

### Audio Output

The audio-output path plays a WAV file so the user can evaluate the playback result through headphones or speakers. Users can upload different WAV files to verify compatibility and playback behavior.

### Audio Input and Output

The simultaneous input/output path mainly validates the I2S module. During the test, the I2S input captures live environmental sound and the I2S output plays it back immediately. With headphones connected, the user can directly hear the real-time loopback result.

### Audio Codec

The system includes built-in `G711a/u` `16-bit` audio codecs and also supports registering additional external codecs when needed.

### Audio 3A

Audio 3A (Automatic Acoustic Adjustment) includes:

- **ANS (Automatic Noise Suppression)**: reduces environmental noise and improves signal cleanliness.
- **AGC (Automatic Gain Control)**: automatically adjusts gain so the signal stays within a reasonable range.
- **AEC (Acoustic Echo Cancellation)**: removes echo during audio transmission to improve communication clarity.

These functions can be enabled through the corresponding bit flags. For the exact API usage and parameter settings, refer to the API documentation.

## Usage

### Source Location

Demo source path:

```bash
/src/rtsmart/examples/mpp/sample_audio
```

Assuming the demo has already been built correctly, boot the board and enter `/sdcard/app/examples/mpp`. The test executable is `sample_audio.elf`.

### Parameter Description

After boot, enter `/sdcard/app/examples/mpp` and run `./sample_audio.elf -help` to view the usage.

| Parameter | Description | Default |
| --- | --- | --- |
| `type` | test case selector for different modules, `[0, 13]` | - |
| `samplerate` | input/output sample rate, `8k` to `192k` | `44100` |
| `enablecodec` | whether to enable the built-in codec, `[0, 1]`; `1` means enabled | `0` |
| `loglevel` | kernel log level, `[0, 7]` | - |
| `bitwidth` | sample precision, `[16, 24, 32]` | `16` |
| `filename` | WAV or G711 file name to load or save | - |
| `channels` | channel count, mono or stereo, `[1, 2]` | `2` |
| `monochannel` | mono input selection, `[0, 1]`; `0`: onboard mic, `1`: headphone input | `0` |
| `audio3a` | Audio 3A enable flags: `enable_ans: 0x01`, `enable_agc: 0x02`, `enable_aec: 0x04`; multiple bits can be combined | `0` |

Representative help output:

```text
msh /sharefs/app>./sample_audio.elf
Please input:
-type: test audio function[0-12]
  type 0:sample ai i2s module
  type 1:sample ai pdm module
  type 2:sample ao i2s module
  type 3:sample ai(i2s) to ao (api) module
  type 4:sample ai(i2s) to ao (sysbind) module
  type 5:sample ai(pdm) to ao (api) module
  type 6:sample ai(pdm) bind ao (sysbind) module
  type 7:sample aenc(ai->aenc->file) (sysbind) module
  type 8:sample adec(file->adec->ao) (sysbind) module
  type 9:sample aenc(ai->aenc->file) (api) module
  type 10:sample adec(file->adec->ao) (api) module
  type 11:sample overall test (ai->aenc->file file->adec->ao) module
  type 12:sample overall test (ai->aenc  adec->ao loopback ) module
  type 13:sample overall test (opus ai->aenc  adec->ao loopback ) module
-samplerate: set audio sample(8000 ~ 192000)
-enablecodec: enable audio codec(0,1)
-loglevel: show kernel log level[0,7]
-bitwidth: set audio bit width(16,24,32)
-channels: channel count
-monochannel:0:mic input 1:headphone input
-filename: load or save file name
-audio3a: enable audio3a:enable_ans:0x01,enable_agc:0x02,enable_aec:0x04
```

## Examples

### I2S Audio Input Test

Run the following command to capture `15` seconds of PCM audio data and save it as a WAV file:

```bash
./sample_audio.elf -type 0 -enablecodec 1 -bitwidth 16 -filename test.wav -audio3a 1
```

Representative output:

```text
./sample_audio.elf -type 0 -enablecodec 1 -bitwidth 16 -filename test.wav -audio3a 1
audio type:0,sample rate:44100,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:7.46 MB
vb_set_config ok
sample ai i2s module
audio i2s set clk freq is 2822400(2822400),ret:1
audio codec adc clk freq is 11289600(11289600)
ans_enable
========ans_enable:1,agc_enable:0,aec_enable:0
audio_save_init get vb block size:2646044
======kd_mpi_sys_mmap total size:2646044
[0s] timestamp 0 us,curpts:1505976917
[1s] timestamp 1000000 us,curpts:1506976917
[2s] timestamp 2000000 us,curpts:1507976917
[3s] timestamp 3000000 us,curpts:1508976917
[4s] timestamp 4000000 us,curpts:1509976917
[5s] timestamp 5000000 us,curpts:1510976917
[6s] timestamp 6000000 us,curpts:1511976917
[7s] timestamp 7000000 us,curpts:1512976917
[8s] timestamp 8000000 us,curpts:1513976917
[9s] timestamp 9000000 us,curpts:1514976917
[10s] timestamp 10000000 us,curpts:1515976917
[11s] timestamp 11000000 us,curpts:1516976917
[12s] timestamp 12000000 us,curpts:1517976917
[13s] timestamp 13000000 us,curpts:1518976917
dump binary memory test1.wav 0x10225000 0x104ab01c
[14s] timestamp 14000000 us,curpts:1519976917
destroy vb block
sample done
```

The audio data is saved to the specified WAV file.

### I2S Audio Output Test

This test loops a WAV file to verify the audio-output path. Press any key to exit.

```bash
./sample_audio.elf -type 2 -filename test.wav -enablecodec 1
```

Representative output:

```text
./sample_audio.elf -type 2 -filename test.wav -enablecodec 1
audio type:2,sample rate:44100,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:7.46 MB
vb_set_config ok
enter q key to exit
sample ao i2s module
========read_wav_header:headerlen:44,channel:2,samplerate:44100,bitpersample:16
open file:test.wav ok,file size:2646044,data size:2646000,wav header size:44
=======_get_audio_frame virt_addr:0x1002aa000
audio i2s set clk freq is 2822400(2822400),ret:1
audio init codec dac clk freq is 11289600
audio set codec dac clk freq is 11289600(11289600)
q
diable ao audio
destroy vb block
sample done
```

Audio is played through the headphone or speaker path until the user exits.

### I2S Audio Input/Output API Test

Use the API path to test real-time input and output:

```bash
./sample_audio.elf -type 3 -bitwidth 16 -enablecodec 1 -samplerate 8000 -audio3a 1
```

Representative output:

```text
./sample_audio.elf -type 3 -bitwidth 16 -enablecodec 1 -samplerate 8000 -audio3a 1
audio type:3,sample rate:8000,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:1.35 MB
vb_set_config ok
enter q key to exit
sample ai(i2s) to ao module
audio i2s set clk freq is 512000(512000),ret:1
audio codec adc clk freq is 2048000(2048000)
ans_enable
========ans_enable:1,agc_enable:0,aec_enable:0
audio i2s set clk freq is 512000(512000),ret:1
audio init codec dac clk freq is 2048000
audio set codec dac clk freq is 2048000(2048000)
[0s] timestamp 0 us,curpts:2017301433
[1s] timestamp 1000000 us,curpts:2018301433
...
q
diable ao module
diable ai module
release vb block
destroy vb block
sample done
```

### I2S Audio Input and Output System Bind Test

Use AI/AO binding through `kd_mpi_sys_bind` to test real-time audio loopback:

```bash
./sample_audio.elf -type 4 -bitwidth 16 -enablecodec 1 -samplerate 8000 -audio3a 1
```

Representative output:

```text
./sample_audio.elf -type 4 -bitwidth 16 -enablecodec 1 -samplerate 8000 -audio3a 1
audio type:4,sample rate:8000,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:1.35 MB
vb_set_config ok
enter q key to exit
sample ai(i2s) bind ao module
audio i2s set clk freq is 512000(512000),ret:1
audio codec adc clk freq is 2048000(2048000)
ans_enable
========ans_enable:1,agc_enable:0,aec_enable:0
audio i2s set clk freq is 512000(512000),ret:1
audio init codec dac clk freq is 2048000
audio set codec dac clk freq is 2048000(2048000)
q
diable ao module
diable ai module
release vb block
destroy vb block
sample done
```

### Encode Test

Use the following commands to get audio from `AI`, encode it, and save it to a file. The codec path supports `G711a/u/lpcm`, `16-bit`.

System-bind mode:

```bash
./sample_audio.elf -type 7 -bitwidth 16 -enablecodec 1 -filename /sharefs/i2s_codec.g711a -audio3a 1
```

Representative output:

```text
./sample_audio.elf -type 7 -bitwidth 16 -enablecodec 1 -filename /sharefs/i2s_codec.g711a -audio3a 1
audio type:7,sample rate:44100,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:7.46 MB
vb_set_config ok
enter q key to exit
sample aenc module (sysbind)
audio i2s set clk freq is 2822400(2822400),ret:1
audio codec adc clk freq is 11289600(11289600)
ans_enable
========ans_enable:1,agc_enable:0,aec_enable:0
q
destroy vb block
sample done
```

API mode:

```bash
./sample_audio.elf -type 9 -bitwidth 16 -enablecodec 1 -filename /sharefs/i2s_codec.g711a -audio3a 1
```

Representative output:

```text
./sample_audio.elf -type 9 -bitwidth 16 -enablecodec 1 -filename /sharefs/i2s_codec.g711a -audio3a 1
audio type:9,sample rate:44100,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:7.46 MB
vb_set_config ok
enter q key to exit
sample aenc module (api)
audio i2s set clk freq is 2822400(2822400),ret:1
audio codec adc clk freq is 11289600(11289600)
ans_enable
========ans_enable:1,agc_enable:0,aec_enable:0
q
destroy vb block
sample done
```

### Decode Test

Use the following commands to read a file, decode it, and play it back. The codec path supports `G711a/u/lpcm`, `16-bit`.

System-bind mode:

```bash
./sample_audio.elf -type 8 -filename /sharefs/i2s_codec.g711a -enablecodec 1 -bitwidth 16
```

Representative output:

```text
./sample_audio.elf -type 8 -filename /sharefs/i2s_codec.g711a -enablecodec 1 -bitwidth 16
audio type:8,sample rate:44100,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:7.46 MB
vb_set_config ok
enter q key to exit
sample adec module (sysbind)
audio i2s set clk freq is 2822400(2822400),ret:1
audio init codec dac clk freq is 11289600
audio set codec dac clk freq is 11289600(11289600)
adec_bind_call_back dev_id:0 chn_id:0
read file again
q
adec_bind_call_back dev_id:0 chn_id:0
destroy vb block
sample done
```

API mode:

```bash
./sample_audio.elf -type 10 -filename /sharefs/i2s_codec.g711a -enablecodec 1 -bitwidth 16
```

Representative output:

```text
./sample_audio.elf -type 10 -filename /sharefs/i2s_codec.g711a -enablecodec 1 -bitwidth 16
audio type:10,sample rate:44100,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:7.46 MB
vb_set_config ok
enter q key to exit
sample adec module (api)
audio i2s set clk freq is 2822400(2822400),ret:1
audio init codec dac clk freq is 11289600
audio set codec dac clk freq is 11289600(11289600)
read file again
q
destroy vb block
sample done
```

### Full Audio Pipeline Test 1

This full audio pipeline test includes two bound paths: `AI -> AENC` and `ADEC -> AO`. It uses the built-in codec with `16-bit` precision, enables AEC-related audio cleanup, and verifies timestamps in the encoded `G711` stream.

```bash
./sample_audio.elf -type 12 -samplerate 48000 -enablecodec 1 -audio3a 1 &
```

Representative output:

```text
./sample_audio.elf -type 12 -samplerate 48000 -enablecodec 1 -audio3a 1 &
audio type:12,sample rate:48000,bit width:16,channels:2,enablecodec:1,monochannel:0
mmz blk total size:8.12 MB
vb_set_config ok
enter q key to exit
sample ai->aenc  adec->ao module (loopback)
Force the sampling accuracy to be set to 16,use inner cocdec
audio i2s set clk freq is 3072000(3072000),ret:1
audio codec adc clk freq is 12288000(12288000)
audio i2s set clk freq is 3072000(3072000),ret:1
audio init codec dac clk freq is 11289600
audio set codec dac clk freq is 12288000(12288000)
adec_bind_call_back dev_id:0 chn_id:0
[0s] g711 stream timestamp 0 us,curpts:341326051
[1s] g711 stream timestamp 1000000 us,curpts:342326051
...
q
adec_bind_call_back dev_id:0 chn_id:0
destroy vb block
sample done
```

You can inspect module bindings with:

```bash
cat /proc/umap/sysbind
```

Representative binding output:

```text
-----BIND RELATION TABLE--------------------------------------------------------
  FirMod  FirDev  FirChn  SecMod  SecDev  SecChn  TirMod  TirDev  TirChn    SendCnt     rstCnt
      ai       0       0    aenc       0       0    null       0       0        310          0
    adec       0       0      ao       0       0    null       0       0        310          0
```

### Full Audio Pipeline Test 2

This test uses the same loopback structure as the previous one, but verifies timestamps on the `Opus` encoded stream.

```bash
./sample_audio.elf -type 13 -samplerate 48000 -enablecodec 1 -audio3a 1 &
```

Representative output:

```text
./sample_audio.elf -type 13 -samplerate 48000 -enablecodec 1 -audio3a 1 &
audio type:13,sample rate:48000,bit width:16,channels:2,enablecodec:1,monochannel:0
vb_set_config ok
audio_data_vb_create_pool poolid 0
enter q key to exit
sample ai->aenc  adec->ao module (opus,loopback)
Force the sampling accuracy to be set to 16,use inner cocdec
_opus_open_encoder enc_inst:0x79da80, samplerate:8000,channels:1
_opus_open_encoder ok
_opus_open_decoder dec_inst:0x0, samplerate:8000, channels:1 before
_opus_open_decoder ok
audio i2s set clk freq is 512000(512000),ret:1
audio codec adc clk freq is 2048000(2048000)
audio i2s set clk freq is 512000(512000),ret:1
audio init codec dac clk freq is 11289600
audio set codec dac clk freq is 2048000(2048000)
adec_bind_call_back dev_id:0 chn_id:0
...
q
adec_bind_call_back dev_id:0 chn_id:0
destroy vb block
sample done
```

You can also inspect module bindings with:

```bash
cat /proc/umap/sysbind
```

Representative binding output:

```text
-----BIND RELATION TABLE--------------------------------------------------------
  FirMod  FirDev  FirChn  SecMod  SecDev  SecChn  TirMod  TirDev  TirChn    SendCnt     rstCnt
      ai       0       0    aenc       0       0    null       0       0        310          0
    adec       0       0      ao       0       0    null       0       0        310          0
```

```{admonition} Tip
For detailed interfaces of the audio module, refer to the [API documentation](../../api_reference/mpp/audio.md).
```