# Multimedia Middleware API Manual

## Overview

### Overview

This document aims to provide developers with detailed information about the K230 multimedia middleware, including the API interfaces, header files and usage instructions of each module. This middleware covers multiple functional modules such as RTSP server, RTSP client, RTSP streamer, media player, and MP4 format encapsulation and decapsulation, ogg format encapsulation and decapsulation, etc., which helps developers gain an in-depth understanding of its application scenarios and working principles. Please note that this guide may be updated from time to time, and developers are advised to always refer to the latest version of the documentation.

The middleware code is located in the system path `src/rtsmart/mpp/middleware`, where the `src/rtsmart/mpp/middleware/src` directory contains multimedia encapsulated API interfaces, and the `src/rtsmart/examples/mpp` directory contains sample codes for using these API interfaces.

This document provides a detailed description and examples of the API interface of each module to help users better understand and use the APIs of different modules.

### Functional Description

#### rtsp server

rtsp-server supports sending audio and video data on the board to the rtsp client using the rtsp server protocol.

Common usage scenarios include:

- Real-time audio and video transmission: Real-time transmission of audio and video data on the board to rtsp client through rtsp server.
- Multimedia streaming service: Build a rtsp server to provide audio and video streaming services for clients to play and access.
- Remote monitoring: Send the audio and video data on the board to the remote client using the rtsp server protocol to realize the remote monitoring function.

#### rtsp client

rtsp-client supports the board to use the rtsp client protocol to obtain audio and video data from the rtsp server.

Common usage scenarios include:

- Real-time monitoring system: obtain real-time audio and video data from the rtsp server through rtsp-client for monitoring and recording.
- Multimedia player: Use rtsp-client to obtain audio and video data from the rtsp server for playing multimedia content.
- Video conferencing system: Obtain audio and video data from the rtsp server through rtsp-client to implement the video conferencing function.

#### rtsp pusher

The video data on the board is pushed to a third-party streaming media server using the RTSP protocol. The client can obtain the video data pushed by the board through the third-party streaming media server.

Common usage scenarios include:

- Video surveillance system: Push the video data on the board to the streaming media server for real-time viewing by the surveillance client.
- Video live broadcast: Push the video data on the board to the streaming media server for viewers to watch the live broadcast through the streaming media server.
- Video recording: Push the video data on the board to the streaming media server to realize the video recording function.
- Video distribution: Push the video data on the board to the streaming media server for multiple clients to obtain video data at the same time.

#### player

Realize mp4 file playback. Video supports h264, h265, and audio supports g711a/u.

#### MP4 format encapsulation and decapsulation

Realize the encapsulation and decapsulation between audio and video and mp4 format.

#### Other

This document contains the API reference of the K230 middleware, including the integration of live555 and ffmpeg open source multimedia libraries. Users can utilize the powerful functions provided by these libraries to implement multimedia processing and transmission according to their own needs.

## API Reference

### rtsp-server

KdRtspServer provides the following APIs:

- [Init](#kdrtspserverinit): initialization.
- [DeInit](#kdrtspserverdeinit): Deinitialization.
- [CreateSession](#kdrtspservercreatesession): Create rtsp session.
- [DestroySession](#kdrtspserverdestroysession): Destroy rtsp session.
- [Start](#kdrtspserverstart): Start the rtsp-server service.
- [Stop](#kdrtspserverstop): Stop the rtsp-server service.
- [SendVideoData](#kdrtspserversendvideodata): Write video stream data.
- [SendAudioData](#kdrtspserversendaudiodata): Write audio stream data.

#### KdRtspServer::Init

Description

rtsp-server initialization.

Syntax

int Init(Port port = 8554, IOnBackChannel *back_channel = nullptr);

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| port | rtsp service port. | Input |
| back_channel | Audio data callback pointer from the peer. | Input |

Return Value

| Return Value | Description |
|---|---|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: rtsp_server.h
- Library File: librtsp_server.a

Notes

None.

Example

None.

Related Topics

None.

#### KdRtspServer::DeInit

Description

Deinitialize.

Syntax

void DeInit();

Parameters

None.

Return Value

None.

Requirements

- Header File: rtsp_server.h
- Library File: librtsp_server.a

Example

None.

#### KdRtspServer::CreateSession

Description

Create RtspSession.

Syntax

int CreateSession(const std::string &session_name, const SessionAttr &session_attr);

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| session_name | Stream URL. | Input |
| session_attr | session configuration parameters. | Input |

Return Value

| Return Value | Description |
|--------|-------------------------------|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: rtsp_server.h
- Library File: librtsp_server.a

Notes

Example

None.

#### KdRtspServer::DestroySession

Description

Destroy rtsp session.

Syntax

int DestroySession(const std::string &session_name);

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| session_name | Stream URL. | Input |

Return Value

| Return Value | Description |
|--------|-------------------------------|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: rtsp_server.h
- Library File: librtsp_server.a

Notes

Example

None.

Related Topics

#### KdRtspServer::Start

Description

Start the rtsp server service.

Syntax

void Start();

Parameters

None.

Return Value

None.

Requirements

- Header File: rtsp-server.h
- Library File: librtsp_server.a

Notes
None

Example

None.

Related Topics

#### KdRtspServer::Stop

Description

Stop the rtsp-server service.

Syntax

void Stop();

Parameters

None

Return Value

None

Requirements

- Header File: rtsp_server.h
- Library File: librtsp_server.a

Notes

Example

None.

#### KdRtspServer::SendVideoData

Description

Write video stream data.

Syntax

int SendVideoData(const std::string &session_name, const uint8_t *data, size_t size, uint64_t timestamp);

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| session_name | stream url | Input |
| data | Video stream address. | Input |
| size | Video stream size. | Input |
| timestamp | Stream timestamp (milliseconds) | Input |

Return Value

| Return Value | Description |
|---|---|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: rtsp_server.h
- Library File: librtsp_server.a

Example

None.

#### KdRtspServer::SendAudioData

Description

Write audio stream data.

Syntax

int SendAudioData(const std::string &session_name, const uint8_t *data, size_t size, uint64_t timestamp);

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| session_name | stream url | Input |
| data | Audio stream address. | Input |
| size | Audio stream size. | Input |
| timestamp | Stream timestamp (milliseconds) | Input |

Return Value

| Return Value | Description |
|---|---|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: rtsp_server.h
- Library File: librtsp_server.a

Example

None.

### rtsp-client

The KdRtspClient module provides the following APIs:

- [Init](#kdrtspclientinit): initialization.
- [DeInit](#kdrtspclientdeinit): Deinitialization.
- [Open](#kdrtspclientopen): Open and run rtspclient connection.
- [Close](#kdrtspclientclose): Close the rtspclient connection.
- [SendAudioData](#kdrtspclientsendaudiodata): Write backchannel audio stream data.

#### KdRtspClient::Init

Description

initialization

Syntax

int Init(const RtspClientInitParam &param);

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| param | rtspclient initialization parameters | Input |

```c
    class IOnAudioData {
    public:
        virtual ~IOnAudioData() {}
        virtual void OnAudioData(const uint8_t *data, size_t size, uint64_t timestamp) = 0;
    };

    class IOnVideoData {
    public:
        enum VideoType {VideoTypeInvalid, VideoTypeH264, VideoTypeH265};
        virtual ~IOnVideoData() {}
        virtual void OnVideoType(VideoType type, uint8_t *extra_data, size_t extra_data_size) = 0;
        virtual void OnVideoData(const uint8_t *data, size_t size, uint64_t timestamp, bool keyframe) = 0;
    };

    class IRtspClientEvent {
    public:
        virtual ~IRtspClientEvent() {}
        virtual void OnRtspClientEvent(int event) = 0; // event 0: shutdown
    };

    struct RtspClientInitParam {
        IOnVideoData *on_video_data{nullptr}; // Callback for video stream frames received from server side
        IOnAudioData *on_audio_data{nullptr}; // Callback for audio stream frames received from server side
        IRtspClientEvent *on_event{nullptr};  // RtspClient event callback
        bool enableBackchanel{false};         // Whether to enable audio backchannel
    };
```

Return Value

| Return Value | Description |
|--------|-------------------------------|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: rtsp_client.h
- Library File: librtsp_client.a

Notes

None.

Example

None.

Related Topics

None.

#### KdRtspClient::Deinit

Description

Deinitialize.

Syntax

void DeInit();

Parameters

None.

Return Value

None.

Requirements

- Header File: rtsp_client.h
- Library File: librtsp_client.a

Notes

None.

Example

None.

Related Topics

None.

#### KdRtspClient::Open

Description

Open and run rtspclient connection

Syntax

int Open(const char *url);

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| url | RTSP URL. | Input |

Return Value

| Return Value | Description |
|--------|-------------------------------|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: rtsp_client.h
- Library file: librtsp_client.a

Notes

None.

Example

None.

Related Topics

None.

#### KdRtspClient::Close

Description

Close rtsp client.

Syntax

void Close();

Parameters

Return Value

Requirements

- Header File: rtsp_client.h
- Library file: librtsp_client.a

Notes

None.

Example

None.

Related Topics

None.

#### KdRtspClient::SendAudioData

Description

Write audio back channel stream data.

Syntax

int SendAudioData(const uint8_t *data, size_t size, uint64_t timestamp);

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| data | Audio stream data address | Input |
| size | Audio stream data size | Output |
| timestamp | Audio stream data timestamp (milliseconds) | Output |

Return Value

| Return Value | Description |
|--------|-------------------------------|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: rtsp_client.h
- Library File: librtsp_client.a

Notes

None.

Example

None.

Related Topics

None.

### Player package

The KdPlayer module provides the following APIs:

- [kd_player_init](#kd_player_init): initialization.
- [kd_player_deinit](#kd_player_deinit): Deinitialization.
- [kd_player_setdatasource](#kd_player_setdatasource): Set media playback files.
- [kd_player_regcallback](#kd_player_regcallback): Register event callback.
- [kd_player_start](#kd_player_start): Start playing.
- [kd_player_stop](#kd_player_stop): Stop playing.

#### kd_player_init

Description

Player initialization.

Syntax

k_s32 kd_player_init();

Parameters

None

Return Value

| Return Value | Description |
|--------|-------------------------------|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: kplayer.h
- Library File: libkplayer.a

Notes

None.

Example

None.

Related Topics

None.

#### kd_player_deinit

Description

Deinitialize.

Syntax

k_s32 kd_player_deinit();

Parameters

None

Return Value

| Return Value | Description |
|--------|-------------------------------|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: kplayer.h
- Library File: libkplayer.a

Notes

None.

Example

None.

Related Topics

None.

#### kd_player_setdatasource

Description

Deinitialize.

Syntax

k_s32 kd_player_setdatasource(const k_char* filePath);

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| filePath | media file path | Input |

Return Value

| Return Value | Description |
|--------|-------------------------------|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: kplayer.h
- Library File: libkplayer.a

Notes

None.

Example

None.

Related Topics

None.

#### kd_player_regcallback

Description

Register player event callbacks.

Syntax

k_s32 kd_player_regcallback( K_PLAYER_EVENT_FN pfnCallback,void* pData);

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| pfnCallback | callback function pointer | Input |
| pData | callback data pointer | Input |

Return Value

| Return Value | Description |
|--------|-------------------------------|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: kplayer.h
- Library File: libkplayer.a

Notes

None.

Example

None.

Related Topics

None.

#### kd_player_start

Description

Start playing.

Syntax

k_s32 kd_player_start();

Parameters

None

Return Value

| Return Value | Description |
|--------|-------------------------------|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: kplayer.h
- Library File: libkplayer.a

Notes

None.

Example

None.

Related Topics

None.

#### kd_player_stop

Description

Stop playing.

Syntax

k_s32 kd_player_stop();

Parameters

None

Return Value

| Return Value | Description |
|--------|-------------------------------|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: kplayer.h
- Library File: libkplayer.a

Notes

None.

Example

None.

Related Topics

None.

### MP4 format encapsulation and decapsulation

MP4 format encapsulation and decapsulation provides the following API:

- [kd_mp4_create](#kd_mp4_create): MP4 instance creation
- [kd_mp4_destroy](#kd_mp4_destroy):MP4 instance destroyed
- [kd_mp4_create_track](#kd_mp4_create_track): Create track for MP4
- [kd_mp4_destroy_tracks](#kd_mp4_destroy_tracks): Destroy all tracks for MP4
- [kd_mp4_write_frame](#kd_mp4_write_frame): Write frame data in MP4.
- [kd_mp4_get_file_info](#kd_mp4_get_file_info): Get MP4 file information.
- [kd_mp4_get_track_by_index](#kd_mp4_get_track_by_index): Get track information based on the subscript.
- [kd_mp4_get_frame](#kd_mp4_get_frame): Get track code stream information.

#### kd_mp4_create

Description

MP4 instance creation

Syntax

int kd_mp4_create(KD_HANDLE *mp4_handle, k_mp4_config_s \*mp4_cfg);

Parameters

| parameter | Description | Input/Output |
|------------|------------|--------------|
| mp4_handle | MP4 instance handle | Output |
| mp4_cfg | Parameter configuration information | Input |

Return Value

| Return Value | Description |
|----------|--------|
| 0 | Success |
| non-0 | fail |

Requirements

- Header File: mp4_format.h
- Library File: libmp4.a

Notes

The currently created MP4 instance can be specified as a muxer instance or a demuxer instance through the mp4_cfg configuration information.

Example

Refer to mp4_muxer and mp4_demuxer under samples

Related Topics

None

#### kd_mp4_destroy

Description

MP4 instance destruction

Syntax

int kd_mp4_destroy(KD_HANDLE mp4_handle);

Parameters

| parameter | Description | Input/Output |
|------------|------------|----------------|
| mp4_handle | MP4 instance handle | Input |

Return Value

| Return Value | Description |
|--------------|-----------|
| 0 | Success |
| non-0 | fail |

Requirements

- Header File: mp4_format.h
- Library File: libmp4.a

Notes

None

Example

Refer to mp4_muxer and mp4_demuxer under samples

Related Topics

None

#### kd_mp4_create_track

Description

Create track for MP4

Syntax

int kd_mp4_create_track(KD_HANDLE mp4_handle, KD_HANDLE \*track_handle, k_mp4_track_info_s \*mp4_track_info);

Parameters

| parameter | Description | Input/Output |
|------------|------------|----------------|
| mp4_handle | MP4 instance handle | Input |
| track_handle | track handle | Output |
| mp4_track_info | track configuration | Input |

Return Value

| Return Value | Description |
|-------------|------------|
| 0 | Success |
| non-0 | fail |

Requirements

- Header File: mp4_format.h
- Library File: libmp4.a

Notes

- This API is a muxer API and is used when the created MP4 instance is a muxer instance.
- Currently, each MP4 supports creating up to 3 tracks.

Example

Refer to mp4_muxer under samples

Related Topics

None

#### kd_mp4_destroy_tracks

Description

Destroy all tracks for MP4

Syntax

int kd_mp4_destroy_tracks(KD_HANDLE mp4_handle);

Parameters

| parameter | Description | Input/Output |
|------------|------------|----------------|
| mp4_handle | MP4 example | Input |

Return Value

| Return Value | Description |
|--------------|-----------|
| 0 | Success |
| non-0 | fail |

Requirements

- Header File: mp4_format.h
- Library File: libmp4.a

Notes

- This API is a muxer API and is used when the created MP4 instance is a muxer instance.
- Please call this interface after creating the MP4 instance and track.

Example

Refer to mp4_muxer under samples

Related Topics

None

#### kd_mp4_write_frame

Description

Writing frame data to MP4

Syntax

int kd_mp4_write_frame(KD_HANDLE mp4_handle, KD_HANDLE track_handle, k_mp4_frame_data_s *frame_data);

Parameters

| parameter | Description | Input/Output |
|------------|------------|----------------|
| mp4_handle | MP4 instance handle | Input |
| track_handle | track handle | Input |
| frame_data | Frame data information | Input |

Return Value

| Return Value | Description |
|--------------|-----------|
| 0 | Success |
| non-0 | fail |

Requirements

- Header File: mp4_format.h
- Library File: libmp4.a

Notes

- This API is a muxer API and is used when the created MP4 instance is a muxer instance.
- Please call this interface after creating the MP4 instance and track.

Example

Refer to mp4_muxer under samples

Related Topics

None

#### kd_mp4_get_file_info

Description

Get MP4 file information

Syntax

int kd_mp4_get_file_info(KD_HANDLE mp4_handle, k_mp4_file_info_s *file_info);

Parameters

| parameter | Description | Input/Output |
|------------|------------|----------------|
| mp4_handle | MP4 instance handle | Input |
| file_info | MP4 file information | Output |

Return Value

| Return Value | Description |
|--------------|-----------|
| 0 | Success |
| non-0 | fail |

Requirements

- Header File: mp4_format.h
- Library File: libmp4.a

Notes

- This API is the demuxer API, which is used when the created MP4 instance is a demuxer instance.

Example

Refer to mp4_demuxer under samples

Related Topics

None

#### kd_mp4_get_track_by_index

Description

Get track information based on subscript

Syntax

int kd_mp4_get_track_by_index(KD_HANDLE mp4_handle, uint32_t index, k_mp4_track_info_s *mp4_track_info);

Parameters

| parameter | Description | Input/Output |
|------------|------------|----------------|
| mp4_handle | MP4 instance handle | Input |
| index | subscript | Input |
| mp4_track_info | track information | Output |

Return Value

| Return Value | Description |
|--------------|------------|
| 0 | Success |
| non-0 | fail |

Requirements

- Header File: mp4_format.h
- Library File: libmp4.a

Notes

- This API is the demuxer API, which is used when the created MP4 instance is a demuxer instance.

Example

Refer to mp4_demuxer under samples

Related Topics

None

#### kd_mp4_get_frame

Description

Get track code stream information

Syntax

int kd_mp4_get_frame(KD_HANDLE mp4_handle, k_mp4_frame_data_s *frame_data);

Parameters

| parameter | Description | Input/Output |
|------------|------------|----------------|
| mp4_handle | MP4 instance handle | Input |
| frame_data | Code stream information | Output |

Return Value

| Return Value | Description |
|--------------|-----------|
| 0 | Success |
| non-0 | fail |

Requirements

- Header File: mp4_format.h
- Library File: libmp4.a

Notes

- This API is the demuxer API, which is used when the created MP4 instance is a demuxer instance.

Example

Refer to mp4_demuxer under samples

Related Topics

None

### rtsp pusher

RTSP push streaming provides the following API:

- [Init](#kdrtsppusherinit): initialization.
- [DeInit](#kdrtsppusherdeinit): Deinitialization.
- [Open](#kdrtsppusheropen): Establish an rtsp push connection with the streaming media server.
- [Close](#kdrtsppusherclose): Close the streaming server connection.
- [PushVideoData](#kdrtsppusherpushvideodata): Push video data to streaming media.

#### KdRtspPusher::Init

Description

initialization

Syntax

int Init(const RtspPusherInitParam &param);

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| param | rtsppusher initialization parameters | Input |

```c
class IRtspPusherEvent {
  public:
    virtual ~IRtspPusherEvent() {}
    virtual void OnRtspPushEvent(int event) = 0; // event 0: connect  ok; event 1:disconnet ;   event 2:reconnect ok
};

struct RtspPusherInitParam {
    int video_width;
    int video_height;
    char sRtspUrl[256];
    IRtspPusherEvent *on_event{nullptr};
};
```

Return Value

| Return Value | Description |
|--------|-------------------------------|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: rtsp_pusher.h
- Library File: librtsp_pusher.a

Notes

None.

Example

None.

Related Topics

None.

#### KdRtspPusher::Deinit

Description

Deinitialize.

Syntax

void DeInit();

Parameters

None.

Return Value

None.

Requirements

- Header File: rtsp_pusher.h
- Library File: librtsp_pusher.a

Notes

None.

Example

None.

Related Topics

None.

#### KdRtspPusher::Open

Description

Establish an RTSP push connection with the streaming media server.

Syntax

int Open();

Parameters

None.

Return Value

| Return Value | Description |
|--------|-------------------------------|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: rtsp_pusher.h
- Library File: librtsp_pusher.a

Notes

None.

Example

None.

Related Topics

None.

#### KdRtspPusher::Close

Description

Close the streaming server connection.

Syntax

void Close();

Parameters

Return Value

Requirements

- Header File: rtsp_pusher.h
- Library File: librtsp_pusher.a

Notes

None.

Example

None.
Related Topics

None.

#### KdRtspPusher::PushVideoData

Description

Push video data to streaming media.

Syntax

int  PushVideoData(const uint8_t *data, size_t size, bool key_frame,uint64_t timestamp);

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| data | Video stream data address | Input |
| size | Video stream data size | Input |
| key_frame | Is it a keyframe? | Input |
| timestamp | Video stream data timestamp (microseconds) | Input |

Return Value

| Return Value | Description |
|--------|-------------------------------|
| 0 | success. |
| non-0 | fail. |

Requirements

- Header File: rtsp_pusher.h
- Library File: librtsp_pusher.a

Notes

The current version only supports 264 encoding push streaming.

Example

None.

Related Topics

None.

### ogg

This module provides support for the Ogg container format and can be used for audio data encapsulation (muxing) and decapsulation (demuxing). Suitable for scenarios where raw audio frames need to be packed into Ogg files/streams, or audio frames extracted from Ogg data.
Ogg Muxer (wrapper)
[kd_ogg_muxer_init](#kd_ogg_muxer_init): Initialize Ogg wrapper instance.
[kd_ogg_write_frame](#kd_ogg_write_frame): Writes a frame of audio data to the Ogg wrapper.
[kd_ogg_write_frame_ex](#kd_ogg_write_frame_ex): Extended version, supports obtaining generated Ogg page data.
[kd_ogg_muxer_destroy](#kd_ogg_muxer_destroy): Destroy the Ogg wrapper instance and release resources.
Ogg Demuxer (decapsulator)
[kd_ogg_demuxer_init](#kd_ogg_demuxer_init): Initialize Ogg dewrapper instance.
[kd_ogg_demuxer_feed_page](#kd_ogg_demuxer_feed_page): Input a complete Ogg page data to the depackager (for streaming mode).
[kd_ogg_demuxer_feed_page_ex](#kd_ogg_demuxer_feed_page_ex): Extended version, supports extracting raw frame data from the page to the specified buffer.
[kd_ogg_demuxer_destroy](#kd_ogg_demuxer_destroy): Destroy Ogg dewrapper instance.

#### Functional Description

- **Ogg Muxer (encapsulator)**: Encapsulates original audio frames (such as Opus encoding format) into Ogg pages (pages) according to the Ogg container specification, and supports writing to files or outputting to memory/network streams through callbacks.
- **Ogg Demuxer**: Parses the audio frame data from the Ogg file or stream, and notifies the upper-layer application through a callback.

The current implementation does not bind specific audio encodings (such as Opus) and only handles the Ogg container layer. Audio encoding/decoding is the responsibility of the upper layer.

#### Usage scenarios

- Audio is recorded and saved as Ogg format file;
- Transmit Ogg-encapsulated audio streams via RTSP or custom protocols;
- Extract raw audio frames from Ogg files for playback or analysis;
- Integrate with WebRTC, VoIP and other systems to process Ogg encapsulated audio data.
- The Ogg and Opus encoding format is used to carry the storage and streaming of voice ASR input data and the encapsulation and distribution of TTS output data, adapting to real-time voice interaction scenarios.

#### API Reference

##### type definition

```c
typedef void kd_ogg_muxer;
typedef void kd_ogg_demuxer;
```

##### callback function type

- **kd_ogg_write_callback**: used to stream write Ogg pages.

```c
typedef int (*kd_ogg_write_callback)(const void *ptr, size_t size, void *user_data);
```

- **kd_ogg_frame_callback**: used to receive decapsulated audio frames.

```c
typedef void (*kd_ogg_frame_callback)(const uint8_t *data, size_t len, void *user_data);
```

##### Ogg Muxer API

###### kd_ogg_muxer_init

Description

Initialize the Ogg wrapper instance.

Syntax

```c
int kd_ogg_muxer_init(kd_ogg_muxer **ogg_muxer, kd_ogg_muxer_params *params);
```

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| ogg_muxer | output wrapper handle | Output |
| params | Initialization parameters (see table below) | Input |

`kd_ogg_muxer_params` structure members:

| member | Description |
|---|---|
| filename[128] | If it is not empty, write to the file; if it is empty, use write_cb to stream output. |
| sample_rate | Audio sample rate (e.g. 48000) |
| channels | Number of channels (e.g. 1 or 2) |
| serial_no | Ogg stream sequence number (set to 0 to automatically generate) |
| write_cb | Write callback in streaming mode (required when filename is empty) |
| user_data | The user data pointer passed to the callback |

Return Value

| Return Value | Description |
|---|---|
| 0 | Success |
| non-0 | fail |

Requirements

- Header File: libogg.h
- Library File: libogg.a

Notes

None.

Related Topics

None.

###### kd_ogg_write_frame

Description

Writes a frame of audio data to the Ogg wrapper.

Syntax

```c
int kd_ogg_write_frame(kd_ogg_muxer *ogg_muxer, kd_ogg_frame_params *frame);
```

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| ogg_muxer | Initialized wrapper handle | Input |
| frame | Frame information containing data, len, frame_samples | Input |

Return Value

| Return Value | Description |
|---|---|
| 0 | Success |
| non-0 | fail |

Requirements

- Header File: libogg.h
- Library File: libogg.a

Notes

None.

Related Topics

None.

###### kd_ogg_write_frame_ex

Description

The extended version supports obtaining the generated Ogg page data (suitable for scenarios that require manual page processing).

Syntax

```c
int kd_ogg_write_frame_ex(kd_ogg_muxer *ogg_muxer, kd_ogg_frame_params_ex *frame);
```

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| ogg_muxer | Initialized wrapper handle | Input |
| frame | Extended frame information containing input frame data and output page buffer | Input/Output |

Return Value

| Return Value | Description |
|---|---|
| 0 | Success |
| non-0 | fail |

Requirements

- Header File: libogg.h
- Library File: libogg.a

Notes

The caller needs to ensure that the out_page buffer is large enough.

Related Topics

None.

###### kd_ogg_muxer_destroy

Description

Destroy the Ogg wrapper instance and release resources.

Syntax

```c
int kd_ogg_muxer_destroy(kd_ogg_muxer *ogg_muxer);
```

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| ogg_muxer | Initialized wrapper handle | Input |

Return Value

| Return Value | Description |
|---|---|
| 0 | Success |
| non-0 | fail |

Requirements

- Header File: libogg.h
- Library File: libogg.a

Notes

None.

Related Topics

None.

##### Ogg Demuxer API

###### kd_ogg_demuxer_init

Description

Initialize the Ogg dewrapper instance.

Syntax

```c
int kd_ogg_demuxer_init(kd_ogg_demuxer **ogg_demuxer, kd_ogg_demuxer_params *params);
```

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| ogg_demuxer | Output dewrapper handle | Output |
| params | Initialization parameters (see table below) | Input |

`kd_ogg_demuxer_params` structure members:

| member | Description |
|---|---|
| filename[128] | If it is not empty, read from the file; if it is empty, you need to enter data through feed_page |
| frame_cb | Callback for receiving decapsulated audio frames (required) |
| user_data | User data passed to callback |
| sample_rate | The actual value is filled in by demuxer after initialization |
| channels | The actual value is filled in by demuxer after initialization |

Return Value

| Return Value | Description |
|---|---|
| 0 | Success |
| non-0 | fail |

Requirements

- Header File: libogg.h
- Library File: libogg.a

Notes

None.

Related Topics

None.

###### kd_ogg_demuxer_feed_page

Description

Input a complete Ogg page data to the depackager (for streaming mode).

Syntax

```c
int kd_ogg_demuxer_feed_page(kd_ogg_demuxer *ogg_demuxer, const uint8_t *page_data, size_t page_size);
```

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| ogg_demuxer | Initialized dewrapper handle | Input |
| page_data | Ogg page data address | Input |
| page_size | Ogg page data size | Input |

Return Value

| Return Value | Description |
|---|---|
| 0 | Success |
| non-0 | fail |

Requirements

- Header File: libogg.h
- Library File: libogg.a

Notes

None.

Related Topics

None.

###### kd_ogg_demuxer_feed_page_ex

Description

The extended version supports extracting raw frame data from the page to the specified buffer.

Syntax

```c
int kd_ogg_demuxer_feed_page_ex(kd_ogg_demuxer *ogg_demuxer, kd_ogg_page_params_ex *page);
```

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| ogg_demuxer | Initialized dewrapper handle | Input |
| page | Extended page information containing input page data and output framebuffer | Input/Output |

Return Value

| Return Value | Description |
|---|---|
| 0 | Success |
| non-0 | fail |

Requirements

- Header File: libogg.h
- Library File: libogg.a

Notes

None.

Related Topics

None.

###### kd_ogg_demuxer_destroy

Description

Destroy the Ogg dewrapper instance.

Syntax

```c
int kd_ogg_demuxer_destroy(kd_ogg_demuxer *ogg_demuxer);
```

Parameters

| Parameter Name | Description | Input/Output |
|---|---|---|
| ogg_demuxer | Initialized dewrapper handle | Input |

Return Value

| Return Value | Description |
|---|---|
| 0 | Success |
| non-0 | fail |

Requirements

- Header File: libogg.h
- Library File: libogg.a

Notes

None.

Related Topics

None.

##### Things to note

1. Ogg protocol details such as page segmentation, sequence numbers, checksums, etc. are automatically handled internally by the Ogg wrapper.
1. Timestamps are managed by the upper layer, and the Ogg container itself does not store absolute timestamps, but only records sample offsets.
1. Currently only single audio tracks are supported.
1. If using stream mode, make sure write_cb or feed_page is called correctly to maintain data flow continuity.