Note

This is the documentation for the latest development branch and may refer to features that are not available in released versions. If you are looking for the documentation for a specific release, use the drop-down menu on the left and select the desired version.

Multimedia Middleware API Manual

Overview

Overview

This document aims to provide developers with detailed information about the K230 multimedia middleware, including the API interfaces, header files, and usage instructions for each module. This middleware covers multiple functional modules such as RTSP server, RTSP client, RTSP pusher, media player, MP4 format muxer/demuxer, OGG format muxer/demuxer, and WebRTC real-time communication, helping developers gain an in-depth understanding of their application scenarios and working principles. Please note that this guide may be updated periodically, and it is recommended that developers always refer to the latest version of the documentation.

The middleware code is located in the system path src/rtsmart/mpp/middleware, where the src/rtsmart/mpp/middleware/src directory contains the API interfaces for multimedia muxer/demuxer, and the src/rtsmart/examples/mpp directory contains example code that uses these API interfaces.

This document provides detailed descriptions and examples of the API interfaces for each module, helping users better understand and use the APIs of different modules.

Module List

Module	Description
rtsp-server	RTSP server, sends audio/video data from the board to clients via RTSP protocol
rtsp-client	RTSP client, obtains audio/video data from an RTSP server
rtsp-pusher	RTSP pusher, pushes video data from the board to a third-party streaming media server
Player	MP4 file playback, video supports H264/H265, audio supports G711a/u/OPUS
MP4 Muxer/Demuxer	Muxing and demuxing between audio/video and MP4 format
Ogg Muxer/Demuxer	Muxing and demuxing between audio and Ogg format
WebRTC	Peer-to-peer real-time audio/video and data channel communication based on WebRTC protocol

Feature Description

rtsp server

rtsp-server supports sending audio and video data on the board to rtsp client clients using the rtsp server protocol.

Common usage scenarios include:

• Real-time audio and video transmission: Real-time transmission of audio and video data on the board to rtsp client clients through the rtsp server.

• Multimedia streaming service: Set up an rtsp server to provide audio and video streaming services for clients to play and access.

• Remote monitoring: Send the audio and video data on the board to remote clients using the rtsp server protocol to implement remote monitoring functionality.

rtsp client

rtsp-client supports the board using the rtsp client protocol to obtain audio and video data from the rtsp server.

Common usage scenarios include:

• Real-time monitoring system: Obtain real-time audio and video data from the rtsp server through rtsp-client for monitoring and recording.

• Multimedia player: Use rtsp-client to obtain audio and video data from the rtsp server for playing multimedia content.

• Video conferencing system: Obtain audio and video data from the rtsp server through rtsp-client to implement video conferencing functionality.

rtsp pusher

Implements pushing the video data on the board to a third-party streaming media server using the RTSP protocol, and clients can obtain the video data pushed by the board through the third-party streaming media server.

Common usage scenarios include:

• Video monitoring system: Push the video data on the board to the streaming media server for monitoring clients to view in real time.

• Video live streaming: Push the video data on the board to the streaming media server for viewers to watch live streaming through the streaming media server.

• Video recording: Push the video data on the board to the streaming media server to implement video recording functionality.

• Video distribution: Push the video data on the board to the streaming media server for multiple clients to obtain video data simultaneously.

Player

Implements mp4 file playback. The video supports h264, h265, and the audio supports g711a/u/opus.

MP4 Format Muxer/Demuxer

Implements muxer and demuxer between audio/video and mp4 format.

WebRTC

The WebRTC (Web Real-Time Communication) module provides peer-to-peer real-time audio/video communication and data channel transmission capabilities based on the WebRTC protocol. This module is implemented on top of the libpeer library, supporting ICE connection establishment, SDP negotiation, audio/video codec transmission, DataChannel communication, and built-in signaling service connection functionality.

Common usage scenarios include:

• Real-time audio/video calls: Implement peer-to-peer audio/video calls between the board and a browser or other WebRTC endpoints via the WebRTC protocol.

• Remote monitoring: Transmit video data captured by the board’s camera to a remote browser client in real time via WebRTC.

• Data channel communication: Use WebRTC DataChannel to transmit custom data (e.g., sensor data, control commands) between the board and the remote peer.

• Low-latency streaming: Compared to protocols like RTSP, WebRTC provides lower end-to-end latency, suitable for scenarios with high real-time requirements.

Others

This document includes the API reference for the K230 middleware, which includes the integration of the live555 and ffmpeg open-source multimedia libraries. Users can leverage the powerful features provided by these libraries to implement multimedia processing and transmission according to their own needs.

API Reference

rtsp-server

KdRtspServer provides the following APIs:

• Init：Initialize.

• DeInit：Deinitialize.

• CreateSession：Create rtsp session.

• DestroySession：Destroy rtsp session.

• Start：Start rtsp-server service.

• Stop：Stop rtsp-server service.

• SendVideoData：Write video stream data.

• SendAudioData：Write audio stream data.

KdRtspServer::Init

【Description】

Initialize rtsp-server.

【Syntax】

int Init(Port port = 8554, IOnBackChannel *back_channel = nullptr);

【Parameters】

Parameter	Description	Input/Output
port	rtsp service port.	Input
back_channel	Callback pointer for audio data from the peer.	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: rtsp_server.h

• Library file: librtsp_server.a

【Note】

None.

【Example】

None.

【Related Topics】

None.

KdRtspServer::DeInit

【Description】

Deinitialize.

【Syntax】

void DeInit();

【Parameters】

None.

【Return Value】

None.

【Requirements】

• Header file: rtsp_server.h

• Library file: librtsp_server.a

【Example】

None.

KdRtspServer::CreateSession

【Description】

Create RtspSession.

【Syntax】

int CreateSession(const std::string &session_name, const SessionAttr &session_attr);

【Parameters】

Parameter	Description	Input/Output
session_name	stream url.	Input
session_attr	session configuration parameters.	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: rtsp_server.h

• Library file: librtsp_server.a

【Note】

【Example】

None.

KdRtspServer::DestroySession

【Description】

Destroy rtsp session.

【Syntax】

int DestroySession(const std::string &session_name);

【Parameters】

Parameter	Description	Input/Output
session_name	stream url.	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: rtsp_server.h

• Library file: librtsp_server.a

【Note】

【Example】

None.

【Related Topics】

KdRtspServer::Start

【Description】

Start rtsp server service.

【Syntax】

void Start();

【Parameters】

None.

【Return Value】

None.

【Requirements】

• Header file: rtsp-server.h

• Library file: librtsp_server.a

【Note】 None

【Example】

None.

【Related Topics】

KdRtspServer::Stop

【Description】

Stop rtsp-server service.

【Syntax】

void Stop();

【Parameters】

None

【Return Value】

None

【Requirements】

• Header file: rtsp_server.h

• Library file: librtsp_server.a

【Note】

【Example】

None.

KdRtspServer::SendVideoData

【Description】

Write video stream data.

【Syntax】

int SendVideoData(const std::string &session_name, const uint8_t *data, size_t size, uint64_t timestamp);

【Parameters】

Parameter	Description	Input/Output
session_name	stream url	Input
data	Video stream address.	Input
size	Video stream size.	Input
timestamp	Stream timestamp (milliseconds)	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: rtsp_server.h

• Library file: librtsp_server.a

【Example】

None.

KdRtspServer::SendAudioData

【Description】

Write audio stream data.

【Syntax】

int SendAudioData(const std::string &session_name, const uint8_t *data, size_t size, uint64_t timestamp);

【Parameters】

Parameter	Description	Input/Output
session_name	stream url	Input
data	Audio stream address.	Input
size	Audio stream size.	Input
timestamp	Stream timestamp (milliseconds)	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: rtsp_server.h

• Library file: librtsp_server.a

【Example】

None.

rtsp-client

KdRtspClient module provides the following APIs:

• Init: Initialization.

• DeInit: Deinitialization.

• Open: Open and run rtspclient connection.

• Close: Close rtspclient connection.

• SendAudioData: Write backchannel audio stream data.

KdRtspClient::Init

【Description】

Initialization.

【Syntax】

int Init(const RtspClientInitParam &param);

【Parameters】

Parameter Name	Description	Input/Output
param	rtspclient initialization parameters	Input

    class IOnAudioData {
    public:
        virtual ~IOnAudioData() {}
        virtual void OnAudioData(const uint8_t *data, size_t size, uint64_t timestamp) = 0;
    };

    class IOnVideoData {
    public:
        enum VideoType {VideoTypeInvalid, VideoTypeH264, VideoTypeH265};
        virtual ~IOnVideoData() {}
        virtual void OnVideoType(VideoType type, uint8_t *extra_data, size_t extra_data_size) = 0;
        virtual void OnVideoData(const uint8_t *data, size_t size, uint64_t timestamp, bool keyframe) = 0;
    };

    class IRtspClientEvent {
    public:
        virtual ~IRtspClientEvent() {}
        virtual void OnRtspClientEvent(int event) = 0; // event 0: shutdown
    };

    struct RtspClientInitParam {
        IOnVideoData *on_video_data{nullptr}; // Callback for video stream frames received from the server side
        IOnAudioData *on_audio_data{nullptr}; // Callback for audio stream frames received from the server side
        IRtspClientEvent *on_event{nullptr};  // rtspclient event callback
        bool enableBackchanel{false};         // Whether to enable audio backchannel
    };

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: rtsp_client.h

• Library file: librtsp_client.a

【Note】

None.

【Example】

None.

【Related Topics】

None.

KdRtspClient::Deinit

【Description】

Deinitialization.

【Syntax】

void DeInit();

【Parameters】

None.

【Return Value】

None.

【Requirements】

• Header file: rtsp_client.h

• Library file: librtsp_client.a

【Note】

None.

【Example】

None.

【Related Topics】

None.

KdRtspClient::Open

【Description】

Open and run rtspclient connection.

【Syntax】

int Open(const char *url);

【Parameters】

Parameter Name	Description	Input/Output
url	rtsp url.	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: rtsp_client.h

• Library file: librtsp_client.a

【Note】

None.

【Example】

None.

【Related Topics】

None.

KdRtspClient::Close

【Description】

Close rtsp client.

【Syntax】

void Close();

【Parameters】

【Return Value】

【Requirements】

• Header file: rtsp_client.h

• Library file: librtsp_client.a

【Note】

None.

【Example】

None.

【Related Topics】

None.

KdRtspClient::SendAudioData

【Description】

Write audio backchannel stream data.

【Syntax】

int SendAudioData(const uint8_t *data, size_t size, uint64_t timestamp);

【Parameters】

Parameter Name	Description	Input/Output
data	Audio stream data address	Input
size	Audio stream data size	Output
timestamp	Audio stream data timestamp (milliseconds)	Output

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: rtsp_client.h

• Library file: librtsp_client.a

【Note】

None.

【Example】

None.

【Related Topics】

None.

Player Wrapper

The KdPlayer module provides the following APIs:

• kd_player_init: Initialize.

• kd_player_deinit: Deinitialize.

• kd_player_setdatasource: Set the media playback file.

• kd_player_regcallback: Register event callback.

• kd_player_start: Start playback.

• kd_player_stop: Stop playback.

kd_player_init

【Description】

Initialize the player.

【Syntax】

k_s32 kd_player_init();

【Parameters】

None

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: kplayer.h

• Source: src/rtsmart/examples/mpp/sample_player (compiled directly into the sample_player example; no longer packaged as a standalone library)

【Note】

None.

【Example】

None.

【Related Topics】

None.

kd_player_deinit

【Description】

Deinitialize.

【Syntax】

k_s32 kd_player_deinit();

【Parameters】

None

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: kplayer.h

• Source: src/rtsmart/examples/mpp/sample_player (compiled directly into the sample_player example; no longer packaged as a standalone library)

【Note】

None.

【Example】

None.

【Related Topics】

None.

kd_player_setdatasource

【Description】

Deinitialize.

【Syntax】

k_s32 kd_player_setdatasource(const k_char* filePath);

【Parameters】

Parameter Name	Description	Input/Output
filePath	Media file path	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: kplayer.h

• Source: src/rtsmart/examples/mpp/sample_player (compiled directly into the sample_player example; no longer packaged as a standalone library)

【Note】

None.

【Example】

None.

【Related Topics】

None.

kd_player_regcallback

【Description】

Register the player event callback.

【Syntax】

k_s32 kd_player_regcallback( K_PLAYER_EVENT_FN pfnCallback,void* pData);

【Parameters】

Parameter Name	Description	Input/Output
pfnCallback	Callback function pointer	Input
pData	Callback data pointer	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: kplayer.h

• Source: src/rtsmart/examples/mpp/sample_player (compiled directly into the sample_player example; no longer packaged as a standalone library)

【Note】

None.

【Example】

None.

【Related Topics】

None.

kd_player_start

【Description】

Start playback.

【Syntax】

k_s32 kd_player_start();

【Parameters】

None

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: kplayer.h

• Source: src/rtsmart/examples/mpp/sample_player (compiled directly into the sample_player example; no longer packaged as a standalone library)

【Note】

None.

【Example】

None.

【Related Topics】

None.

kd_player_stop

【Description】

Stop playback.

【Syntax】

k_s32 kd_player_stop();

【Parameters】

None

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: kplayer.h

• Source: src/rtsmart/examples/mpp/sample_player (compiled directly into the sample_player example; no longer packaged as a standalone library)

【Note】

None.

【Example】

None.

【Related Topics】

None.

MP4 Format Muxing and Demuxing

MP4 format muxing and demuxing provides the following APIs:

• kd_mp4_create: Create MP4 instance

• kd_mp4_destroy: Destroy MP4 instance

• kd_mp4_create_track: Create track for MP4

• kd_mp4_destroy_tracks: Destroy all tracks for MP4

• kd_mp4_write_frame: Write frame data to MP4.

• kd_mp4_get_file_info: Get MP4 file information.

• kd_mp4_get_track_by_index: Get track information by index.

• kd_mp4_get_frame: Get track stream information.

kd_mp4_create

【Description】

Create MP4 instance

【Syntax】

int kd_mp4_create(KD_HANDLE *mp4_handle, k_mp4_config_s *mp4_cfg);

【Parameters】

Parameter	Description	Input/Output
mp4_handle	MP4 instance handle	Output
mp4_cfg	Parameter configuration information	Input

【Return Value】

Return Value	Description
0	Success
Non-0	Failure

【Requirements】

• Header file: mp4_format.h

• Library file: libmp4_format.a

【Note】

Through mp4_cfg configuration information, you can specify whether the currently created MP4 instance is a muxer instance or a demuxer instance

【Example】

Refer to mp4_muxer and mp4_demuxer under samples

【Related Topics】

None

kd_mp4_destroy

【Description】

Destroy MP4 instance

【Syntax】

int kd_mp4_destroy(KD_HANDLE mp4_handle);

【Parameters】

Parameter	Description	Input/Output
mp4_handle	MP4 instance handle	Input

【Return Value】

Return Value	Description
0	Success
Non-0	Failure

【Requirements】

• Header file: mp4_format.h

• Library file: libmp4_format.a

【Note】

None

【Example】

Refer to mp4_muxer and mp4_demuxer under samples

【Related Topics】

None

kd_mp4_create_track

【Description】

Create track for MP4

【Syntax】

int kd_mp4_create_track(KD_HANDLE mp4_handle, KD_HANDLE *track_handle, k_mp4_track_info_s *mp4_track_info);

【Parameters】

Parameter	Description	Input/Output
mp4_handle	MP4 instance handle	Input
track_handle	track handle	Output
mp4_track_info	track configuration	Input

【Return Value】

Return Value	Description
0	Success
Non-0	Failure

【Requirements】

• Header file: mp4_format.h

• Library file: libmp4_format.a

【Note】

• This API is a muxer API, used when the created MP4 instance is a muxer instance

• Currently each MP4 supports creating up to 3 tracks

【Example】

Refer to mp4_muxer under samples

【Related Topics】

None

kd_mp4_destroy_tracks

【Description】

Destroy all tracks for MP4

【Syntax】

int kd_mp4_destroy_tracks(KD_HANDLE mp4_handle);

【Parameters】

Parameter	Description	Input/Output
mp4_handle	MP4 instance	Input

【Return Value】

Return Value	Description
0	Success
Non-0	Failure

【Requirements】

• Header file: mp4_format.h

• Library file: libmp4_format.a

【Note】

• This API is a muxer API, used when the created MP4 instance is a muxer instance

• Please call this interface after creating the MP4 instance and tracks

【Example】

Refer to mp4_muxer under samples

【Related Topics】

None

kd_mp4_write_frame

【Description】

Write frame data to MP4

【Syntax】

int kd_mp4_write_frame(KD_HANDLE mp4_handle, KD_HANDLE track_handle, k_mp4_frame_data_s *frame_data);

【Parameters】

Parameter	Description	Input/Output
mp4_handle	MP4 instance handle	Input
track_handle	track handle	Input
frame_data	Frame data information	Input

【Return Value】

Return Value	Description
0	Success
Non-0	Failure

【Requirements】

• Header file: mp4_format.h

• Library file: libmp4_format.a

【Note】

• This API is a muxer API, used when the created MP4 instance is a muxer instance

• Please call this interface after creating the MP4 instance and tracks

• frame_data->time_stamp is in microseconds

【Example】

Refer to mp4_muxer under samples

【Related Topics】

None

kd_mp4_get_file_info

【Description】

Get MP4 file information

【Syntax】

int kd_mp4_get_file_info(KD_HANDLE mp4_handle, k_mp4_file_info_s *file_info);

【Parameters】

Parameter	Description	Input/Output
mp4_handle	MP4 instance handle	Input
file_info	MP4 file information	Output

【Return Value】

Return Value	Description
0	Success
Non-0	Failure

【Requirements】

• Header file: mp4_format.h

• Library file: libmp4_format.a

【Note】

• This API is a demuxer API, used when the created MP4 instance is a demuxer instance

【Example】

Refer to mp4_demuxer under samples

【Related Topics】

None

kd_mp4_get_track_by_index

【Description】

Get track information by index

【Syntax】

int kd_mp4_get_track_by_index(KD_HANDLE mp4_handle, uint32_t index, k_mp4_track_info_s *mp4_track_info);

【Parameters】

Parameter	Description	Input/Output
mp4_handle	MP4 instance handle	Input
index	Index	Input
mp4_track_info	track information	Output

【Return Value】

Return Value	Description
0	Success
Non-0	Failure

【Requirements】

• Header file: mp4_format.h

• Library file: libmp4_format.a

【Note】

• This API is a demuxer API, used when the created MP4 instance is a demuxer instance

【Example】

Refer to mp4_demuxer under samples

【Related Topics】

None

kd_mp4_get_frame

【Description】

Get track stream information

【Syntax】

int kd_mp4_get_frame(KD_HANDLE mp4_handle, k_mp4_frame_data_s *frame_data);

【Parameters】

Parameter	Description	Input/Output
mp4_handle	MP4 instance handle	Input
frame_data	Stream information	Output

【Return Value】

Return Value	Description
0	Success
Non-0	Failure

【Requirements】

• Header file: mp4_format.h

• Library file: libmp4_format.a

【Note】

• This API is a demuxer API, used when the created MP4 instance is a demuxer instance

• frame_data->time_stamp is in microseconds

【Example】

Refer to mp4_demuxer under samples

【Related Topics】

None

rtsp pusher

rtsp push streaming provides the following APIs:

• Init：Initialize.

• DeInit：Deinitialize.

• Open：Establish an rtsp push streaming connection with the streaming media server.

• Close：Close the streaming media server connection.

• PushVideoData：Push video data to the streaming media.

KdRtspPusher::Init

【Description】

Initialize

【Syntax】

int Init(const RtspPusherInitParam &param);

【Parameters】

Parameter Name	Description	Input/Output
param	rtsppusher initialization parameters	Input

class IRtspPusherEvent {
  public:
    virtual ~IRtspPusherEvent() {}
    virtual void OnRtspPushEvent(int event) = 0; // event 0: connect  ok; event 1:disconnet ;   event 2:reconnect ok
};

struct RtspPusherInitParam {
    int video_width;
    int video_height;
    char sRtspUrl[256];
    IRtspPusherEvent *on_event{nullptr};
};

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: rtsp_pusher.h

• Library file: librtsp_pusher.a

【Note】

None.

【Example】

None.

【Related Topics】

None.

KdRtspPusher::Deinit

【Description】

Deinitialize.

【Syntax】

void DeInit();

【Parameters】

None.

【Return Value】

None.

【Requirements】

• Header file: rtsp_pusher.h

• Library file: librtsp_pusher.a

【Note】

None.

【Example】

None.

【Related Topics】

None.

KdRtspPusher::Open

【Description】

Establish an rtsp push streaming connection with the streaming media server.

【Syntax】

int Open();

【Parameters】

None.

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: rtsp_pusher.h

• Library file: librtsp_pusher.a

【Note】

None.

【Example】

None.

【Related Topics】

None.

KdRtspPusher::Close

【Description】

Close the streaming media server connection.

【Syntax】

void Close();

【Parameters】

【Return Value】

【Requirements】

• Header file: rtsp_pusher.h

• Library file: librtsp_pusher.a

【Note】

None.

【Example】

None. 【Related Topics】

None.

KdRtspPusher::PushVideoData

【Description】

Push video data to the streaming media.

【Syntax】

int PushVideoData(const uint8_t *data, size_t size, bool key_frame,uint64_t timestamp);

【Parameters】

Parameter Name	Description	Input/Output
data	Video stream data address	Input
size	Video stream data size	Input
key_frame	Whether it is a key frame	Input
timestamp	Video stream data timestamp (microseconds)	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: rtsp_pusher.h

• Library file: librtsp_pusher.a

【Note】

The current version only supports 264 encoding for push streaming.

【Example】

None.

【Related Topics】

None.

ogg

This module provides support for the Ogg container format, which can be used for muxing and demuxing audio data. It is suitable for scenarios where raw audio frames need to be packaged into Ogg files/streams, or audio frames need to be extracted from Ogg data.

Ogg Muxer kd_ogg_muxer_init: Initializes an Ogg muxer instance. kd_ogg_write_frame: Writes one frame of audio data to the Ogg muxer. kd_ogg_write_frame_ex: Extended version that supports retrieving the generated Ogg page data. kd_ogg_muxer_destroy: Destroys the Ogg muxer instance and releases resources.

Ogg Demuxer kd_ogg_demuxer_init: Initializes an Ogg demuxer instance. kd_ogg_demuxer_feed_page: Feeds one complete Ogg page to the demuxer (used in stream mode). kd_ogg_demuxer_feed_page_ex: Extended version that supports extracting raw frame data from a page into a specified buffer. kd_ogg_demuxer_destroy: Destroys the Ogg demuxer instance.

Function Description

• Ogg Muxer: Encapsulates raw audio frames (e.g., Opus encoded format) into Ogg pages according to the Ogg container specification, supporting writing to files or outputting to memory/network streams via callbacks.

• Ogg Demuxer: Parses audio frame data from Ogg files or streams, and notifies the upper-layer application via callbacks.

The current implementation is not bound to a specific audio codec (e.g., Opus) and only handles the Ogg container layer. Audio encoding/decoding must be handled by the upper layer.

Use Cases

• Audio recording and saving as Ogg format files;

• Transmitting Ogg-encapsulated audio streams via RTSP or custom protocols;

• Extracting raw audio frames from Ogg files for playback or analysis;

• Integration with WebRTC, VoIP, and other systems to handle Ogg-encapsulated audio data.

• Using the Ogg + Opus encoding format to carry storage and stream transmission of voice ASR input data, as well as encapsulation and distribution of TTS output data, suitable for real-time voice interaction scenarios.

API Reference

Type Definitions

typedef void kd_ogg_muxer;
typedef void kd_ogg_demuxer;

Callback Function Types

• kd_ogg_write_callback: Used for stream-mode writing of Ogg pages.

typedef int (*kd_ogg_write_callback)(const void *ptr, size_t size, void *user_data);

• kd_ogg_frame_callback: Used to receive demuxed audio frames.

typedef void (*kd_ogg_frame_callback)(const uint8_t *data, size_t len, void *user_data);

Ogg Muxer API

kd_ogg_muxer_init

【Description】

Initializes an Ogg muxer instance.

【Syntax】

int kd_ogg_muxer_init(kd_ogg_muxer **ogg_muxer, kd_ogg_muxer_params *params);

【Parameters】

Parameter Name	Description	Input/Output
ogg_muxer	Output muxer handle	Output
params	Initialization parameters (see table below)	Input

kd_ogg_muxer_params structure members:

Member	Description
filename[128]	If non-empty, write to file; if empty, use write_cb for stream output
sample_rate	Audio sample rate (e.g., 48000)
channels	Number of channels (e.g., 1 or 2)
serial_no	Ogg stream serial number (set to 0 for auto-generation)
write_cb	Write callback in stream mode (required when filename is empty)
user_data	User data pointer passed to the callback

【Return Value】

Return Value	Description
0	Success
Non-zero	Failure

【Requirements】

• Header file: libogg.h

• Library file: libogg.a

【Notes】

None.

【Related Topics】

None.

kd_ogg_write_frame

【Description】

Writes one frame of audio data to the Ogg muxer.

【Syntax】

int kd_ogg_write_frame(kd_ogg_muxer *ogg_muxer, kd_ogg_frame_params *frame);

【Parameters】

Parameter Name	Description	Input/Output
ogg_muxer	Initialized muxer handle	Input
frame	Frame information containing data, len, frame_samples	Input

【Return Value】

Return Value	Description
0	Success
Non-zero	Failure

【Requirements】

• Header file: libogg.h

• Library file: libogg.a

【Notes】

None.

【Related Topics】

None.

kd_ogg_write_frame_ex

【Description】

Extended version that supports retrieving the generated Ogg page data (suitable for scenarios requiring manual page handling).

【Syntax】

int kd_ogg_write_frame_ex(kd_ogg_muxer *ogg_muxer, kd_ogg_frame_params_ex *frame);

【Parameters】

Parameter Name	Description	Input/Output
ogg_muxer	Initialized muxer handle	Input
frame	Extended frame information containing input frame data and output page buffer	Input/Output

【Return Value】

Return Value	Description
0	Success
Non-zero	Failure

【Requirements】

• Header file: libogg.h

• Library file: libogg.a

【Notes】

The caller must ensure that the out_page buffer is large enough.

【Related Topics】

None.

kd_ogg_muxer_destroy

【Description】

Destroys the Ogg muxer instance and releases resources.

【Syntax】

int kd_ogg_muxer_destroy(kd_ogg_muxer *ogg_muxer);

【Parameters】

Parameter Name	Description	Input/Output
ogg_muxer	Initialized muxer handle	Input

【Return Value】

Return Value	Description
0	Success
Non-zero	Failure

【Requirements】

• Header file: libogg.h

• Library file: libogg.a

【Notes】

None.

【Related Topics】

None.

Ogg Demuxer API

kd_ogg_demuxer_init

【Description】

Initialize the Ogg demuxer instance.

【Syntax】

int kd_ogg_demuxer_init(kd_ogg_demuxer **ogg_demuxer, kd_ogg_demuxer_params *params);

【Parameters】

Parameter Name	Description	Input/Output
ogg_demuxer	Output demuxer handle	Output
params	Initialization parameters (see table below)	Input

kd_ogg_demuxer_params structure members:

Member	Description
filename[128]	If non-empty, read from file; if empty, data must be input via feed_page
frame_cb	Callback for receiving demuxed audio frames (must be provided)
user_data	User data passed to the callback
sample_rate	Actual value filled by demuxer after initialization
channels	Actual value filled by demuxer after initialization

【Return Value】

Return Value	Description
0	Success
Non-zero	Failure

【Requirements】

• Header file: libogg.h

• Library file: libogg.a

【Notes】

None.

【Related Topics】

None.

kd_ogg_demuxer_feed_page

【Description】

Input a complete Ogg page data to the demuxer (for streaming mode).

【Syntax】

int kd_ogg_demuxer_feed_page(kd_ogg_demuxer *ogg_demuxer, const uint8_t *page_data, size_t page_size);

【Parameters】

Parameter Name	Description	Input/Output
ogg_demuxer	Initialized demuxer handle	Input
page_data	Ogg page data address	Input
page_size	Ogg page data size	Input

【Return Value】

Return Value	Description
0	Success
Non-zero	Failure

【Requirements】

• Header file: libogg.h

• Library file: libogg.a

【Notes】

None.

【Related Topics】

None.

kd_ogg_demuxer_feed_page_ex

【Description】

Extended version, supports extracting raw frame data from pages to a specified buffer.

【Syntax】

int kd_ogg_demuxer_feed_page_ex(kd_ogg_demuxer *ogg_demuxer, kd_ogg_page_params_ex *page);

【Parameters】

Parameter Name	Description	Input/Output
ogg_demuxer	Initialized demuxer handle	Input
page	Extended page information containing input page data and output frame buffer	Input/Output

【Return Value】

Return Value	Description
0	Success
Non-zero	Failure

【Requirements】

• Header file: libogg.h

• Library file: libogg.a

【Notes】

None.

【Related Topics】

None.

kd_ogg_demuxer_destroy

【Description】

Destroy the Ogg demuxer instance.

【Syntax】

int kd_ogg_demuxer_destroy(kd_ogg_demuxer *ogg_demuxer);

【Parameters】

Parameter Name	Description	Input/Output
ogg_demuxer	Initialized demuxer handle	Input

【Return Value】

Return Value	Description
0	Success
Non-zero	Failure

【Requirements】

• Header file: libogg.h

• Library file: libogg.a

【Notes】

None.

【Related Topics】

None.

Notes

1. The Ogg container internally handles Ogg protocol details such as page segmentation, sequence numbers, and checksums.

2. Timestamps are managed by the upper layer; the Ogg container itself does not store absolute timestamps, but only records sample offsets.

3. Currently, only a single audio track is supported.

4. If stream mode is used, ensure that write_cb or feed_page is called correctly to maintain data stream continuity.

WebRTC

The WebRTC module is based on the libpeer library and provides the following three groups of APIs:

• Peer: Global initialization and deinitialization.

• PeerConnection: WebRTC connection management, including SDP negotiation, ICE candidates, audio/video transmission, DataChannel, etc.

• PeerSignaling: Signaling service connection and communication.

Peer

The Peer module provides the following APIs:

• peer_init: WebRTC global initialization.

• peer_deinit: WebRTC global deinitialization.

peer_init

【Description】

WebRTC global initialization. Must be called before using any other WebRTC APIs.

【Syntax】

int peer_init();

【Parameters】

None.

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: peer.h

• Library file: libpeer.a

【Note】

Must be called before creating a PeerConnection or connecting to the signaling service.

【Example】

None.

【Related Topics】

None.

peer_deinit

【Description】

WebRTC global deinitialization, releases global resources. Call when WebRTC functionality is no longer needed.

【Syntax】

void peer_deinit();

【Parameters】

None.

【Return Value】

None.

【Requirements】

• Header file: peer.h

• Library file: libpeer.a

【Note】

Should be called after destroying all PeerConnection instances and disconnecting the signaling connection.

【Example】

None.

【Related Topics】

None.

PeerConnection

The PeerConnection module provides the following APIs:

• peer_connection_create: Create a PeerConnection instance.

• peer_connection_destroy: Destroy a PeerConnection instance.

• peer_connection_close: Close the PeerConnection connection.

• peer_connection_loop: Drive the PeerConnection event loop.

• peer_connection_get_state: Get the connection state.

• peer_connection_state_to_string: Convert the connection state enum to a string.

• peer_connection_create_offer: Create an SDP Offer.

• peer_connection_create_answer: Create an SDP Answer.

• peer_connection_set_local_description: Set the local SDP description.

• peer_connection_set_remote_description: Set the remote SDP description.

• peer_connection_add_ice_candidate: Add a remote ICE candidate.

• peer_connection_onicecandidate: Register an ICE candidate callback.

• peer_connection_oniceconnectionstatechange: Register an ICE connection state change callback.

• peer_connection_ondatachannel: Register DataChannel event callbacks.

• peer_connection_create_datachannel: Create a DataChannel.

• peer_connection_create_datachannel_sid: Create a DataChannel with a specified Stream ID.

• peer_connection_datachannel_send: Send a text message via DataChannel.

• peer_connection_datachannel_send_sid: Send a text message via DataChannel with a specified Stream ID.

• peer_connection_datachannel_send_binary: Send binary data via DataChannel.

• peer_connection_datachannel_send_binary_sid: Send binary data via DataChannel with a specified Stream ID.

• peer_connection_lookup_sid: Look up the Stream ID of a DataChannel by label.

• peer_connection_lookup_sid_label: Look up the label of a DataChannel by Stream ID.

• peer_connection_send_audio: Send audio data.

• peer_connection_send_video: Send video data.

• peer_connection_on_receiver_packet_loss: Register a receiver packet loss callback.

• peer_connection_get_sctp: Get the SCTP instance pointer.

Data Types

The PeerConnection module uses the following data types and enums:

SdpType

SDP type enum, used to distinguish between Offer and Answer.

Enum Value	Description
SDP_TYPE_OFFER	SDP Offer.
SDP_TYPE_ANSWER	SDP Answer.

PeerConnectionState

PeerConnection connection state enum.

Enum Value	Description
PEER_CONNECTION_CLOSED	Connection is closed.
PEER_CONNECTION_NEW	New connection, ICE negotiation has not started yet.
PEER_CONNECTION_CHECKING	ICE connectivity checks are in progress.
PEER_CONNECTION_CONNECTED	At least one ICE candidate pair has succeeded.
PEER_CONNECTION_COMPLETED	All ICE candidate checks have completed, connection established.
PEER_CONNECTION_FAILED	ICE connectivity check has failed.
PEER_CONNECTION_DISCONNECTED	Connection has been disconnected.

DataChannelType

DataChannel data type enum.

Enum Value	Description
DATA_CHANNEL_NONE	No DataChannel.
DATA_CHANNEL_STRING	String type DataChannel.
DATA_CHANNEL_BINARY	Binary type DataChannel.

DecpChannelType

DataChannel transmission reliability type enum, used to specify transmission characteristics when creating a DataChannel.

Enum Value	Description
DATA_CHANNEL_RELIABLE	Reliable ordered transmission.
DATA_CHANNEL_RELIABLE_UNORDERED	Reliable unordered transmission.
DATA_CHANNEL_PARTIAL_RELIABLE_REXMIT	Partially reliable, retransmission guarantee.
DATA_CHANNEL_PARTIAL_RELIABLE_REXMIT_UNORDERED	Partially reliable unordered, retransmission guarantee.
DATA_CHANNEL_PARTIAL_RELIABLE_TIMED	Partially reliable, timeout guarantee.
DATA_CHANNEL_PARTIAL_RELIABLE_TIMED_UNORDERED	Partially reliable unordered, timeout guarantee.

MediaCodec

Media codec type enum.

Enum Value	Description
CODEC_NONE	No codec.
CODEC_H264	H.264 video codec.
CODEC_H265	H.265 video codec.
CODEC_VP8	VP8 video codec (not implemented yet).
CODEC_MJPEG	MJPEG video codec (not implemented yet).
CODEC_OPUS	Opus audio codec.
CODEC_PCMA	G.711A audio codec.
CODEC_PCMU	G.711U audio codec.

IceServer

ICE server configuration structure.

typedef struct IceServer {
  const char* urls;       // ICE server URL (e.g., "stun:stun.l.google.com:19302")
  const char* username;   // Username (required for TURN servers)
  const char* credential; // Credential (required for TURN servers)
} IceServer;

PeerConfiguration

PeerConnection configuration structure, passed in when creating a PeerConnection.

typedef struct PeerConfiguration {
  IceServer ice_servers[5];           // ICE server list, up to 5
  MediaCodec audio_codec;             // Audio codec type
  uint32_t audio_sample_rate;         // Audio sample rate in Hz (e.g., 8000). Used for Opus fmtp to constrain browser encoder
  MediaCodec video_codec;             // Video codec type
  DataChannelType datachannel;        // DataChannel data type
  void (*onaudiotrack)(uint8_t* data, size_t size, void* userdata);   // Received audio frame callback
  void (*onvideotrack)(uint8_t* data, size_t size, void* userdata);   // Received video frame callback
  void (*on_request_keyframe)(void* userdata);                        // Key frame request callback
  void* user_data;                    // User data pointer passed to callback functions
} PeerConfiguration;

peer_connection_create

【Description】

Create a PeerConnection instance.

【Syntax】

PeerConnection* peer_connection_create(PeerConfiguration* config);

【Parameters】

Parameter	Description	Input/Output
config	PeerConnection configuration parameters.	Input

【Return Value】

Return Value	Description
Non-NULL	Success, returns a PeerConnection instance pointer.
NULL	Failure.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

peer_init() must be called for global initialization before calling this function.

【Example】

None.

【Related Topics】

None.

peer_connection_destroy

【Description】

Destroy a PeerConnection instance and release resources.

【Syntax】

void peer_connection_destroy(PeerConnection* pc);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input

【Return Value】

None.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

peer_connection_close() should be called to close the connection before destroying.

【Example】

None.

【Related Topics】

None.

peer_connection_close

【Description】

Close the PeerConnection connection.

【Syntax】

void peer_connection_close(PeerConnection* pc);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input

【Return Value】

None.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_connection_loop

【Description】

Drive the PeerConnection event loop, handling network I/O and state updates. Must be called periodically in the main loop.

【Syntax】

int peer_connection_loop(PeerConnection* pc);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

This function should be called continuously after the connection is established to maintain normal operation.

【Example】

None.

【Related Topics】

None.

peer_connection_get_state

【Description】

Get the current connection state of the PeerConnection.

【Syntax】

PeerConnectionState peer_connection_get_state(PeerConnection* pc);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input

【Return Value】

Return Value	Description
PeerConnectionState enum value	Current connection state.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_connection_state_to_string

【Description】

Convert a PeerConnectionState enum value to a readable string.

【Syntax】

const char* peer_connection_state_to_string(PeerConnectionState state);

【Parameters】

Parameter	Description	Input/Output
state	PeerConnection connection state enum value.	Input

【Return Value】

Return Value	Description
Non-NULL	Success, returns the string corresponding to the state.
NULL	Failure.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_connection_create_offer

【Description】

Create an SDP Offer for initiating WebRTC connection negotiation. After calling, ICE candidate information can be obtained through the peer_connection_onicecandidate callback.

【Syntax】

const char* peer_connection_create_offer(PeerConnection* pc);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input

【Return Value】

Return Value	Description
Non-NULL	Success, returns the SDP Offer string.
NULL	Failure.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

The returned SDP string is valid until the next call to this function. Copy it if long-term storage is needed.

【Related Topics】

None.

peer_connection_create_answer

【Description】

Create an SDP Answer in response to a remote SDP Offer. Must be called after setting the remote Offer via peer_connection_set_remote_description.

【Syntax】

const char* peer_connection_create_answer(PeerConnection* pc);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input

【Return Value】

Return Value	Description
Non-NULL	Success, returns the SDP Answer string.
NULL	Failure.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

The returned SDP string is valid until the next call to this function. Copy it if long-term storage is needed.

【Related Topics】

None.

peer_connection_set_local_description

【Description】

Set the local SDP description.

【Syntax】

void peer_connection_set_local_description(PeerConnection* pc, const char* sdp, SdpType sdp_type);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
sdp	SDP description string.	Input
sdp_type	SDP type (Offer or Answer).	Input

【Return Value】

None.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_connection_set_remote_description

【Description】

Set the remote SDP description.

【Syntax】

void peer_connection_set_remote_description(PeerConnection* pc, const char* sdp, SdpType sdp_type);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
sdp	Remote SDP description string.	Input
sdp_type	SDP type (Offer or Answer).	Input

【Return Value】

None.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_connection_add_ice_candidate

【Description】

Add a remote ICE candidate.

【Syntax】

int peer_connection_add_ice_candidate(PeerConnection* pc, char* ice_candidate);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
ice_candidate	ICE candidate string.	Input

【Return Value】

Return Value	Description
0	Success.
-1	Failure.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_connection_onicecandidate

【Description】

Register an ICE candidate callback function, triggered when a new ICE candidate is discovered.

【Syntax】

void peer_connection_onicecandidate(PeerConnection* pc, void (*onicecandidate)(char* sdp_text, void* userdata));

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
onicecandidate	ICE candidate callback function. Callback parameters: sdp_text is the ICE candidate SDP text, userdata is the user data.	Input

【Return Value】

None.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

ICE candidates obtained in the callback must be sent to the remote peer through the signaling channel.

【Related Topics】

None.

peer_connection_oniceconnectionstatechange

【Description】

Register an ICE connection state change callback function, triggered when the ICE connection state changes.

【Syntax】

void peer_connection_oniceconnectionstatechange(PeerConnection* pc,
    void (*oniceconnectionstatechange)(PeerConnectionState state, void* userdata));

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
oniceconnectionstatechange	ICE connection state change callback function. Callback parameters: state is the current connection state, userdata is the user data.	Input

【Return Value】

None.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_connection_ondatachannel

【Description】

Register DataChannel event callback functions, including message reception, channel open, and channel close events.

【Syntax】

void peer_connection_ondatachannel(PeerConnection* pc,
    void (*onmessage)(char* msg, size_t len, void* userdata, uint16_t sid),
    void (*onopen)(void* userdata),
    void (*onclose)(void* userdata));

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
onmessage	Message reception callback. Callback parameters: msg is the message content, len is the message length, userdata is the user data, sid is the Stream ID.	Input
onopen	DataChannel open callback. Callback parameters: userdata is the user data.	Input
onclose	DataChannel close callback. Callback parameters: userdata is the user data.	Input

【Return Value】

None.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_connection_create_datachannel

【Description】

Create a DataChannel.

【Syntax】

int peer_connection_create_datachannel(PeerConnection* pc, DecpChannelType channel_type,
    uint16_t priority, uint32_t reliability_parameter, char* label, char* protocol);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
channel_type	DataChannel transmission reliability type.	Input
priority	DataChannel priority.	Input
reliability_parameter	Reliability parameter (retransmission count or timeout, depending on channel_type).	Input
label	DataChannel label name.	Input
protocol	DataChannel protocol name.	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_connection_create_datachannel_sid

【Description】

Create a DataChannel with a specified Stream ID.

【Syntax】

int peer_connection_create_datachannel_sid(PeerConnection* pc, DecpChannelType channel_type,
    uint16_t priority, uint32_t reliability_parameter, char* label, char* protocol, uint16_t sid);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
channel_type	DataChannel transmission reliability type.	Input
priority	DataChannel priority.	Input
reliability_parameter	Reliability parameter.	Input
label	DataChannel label name.	Input
protocol	DataChannel protocol name.	Input
sid	Specified Stream ID.	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_connection_datachannel_send

【Description】

Send a text message via DataChannel.

【Syntax】

int peer_connection_datachannel_send(PeerConnection* pc, char* message, size_t len);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
message	Message buffer.	Input
len	Message length.	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_connection_datachannel_send_sid

【Description】

Send a text message via DataChannel with a specified Stream ID.

【Syntax】

int peer_connection_datachannel_send_sid(PeerConnection* pc, char* message, size_t len, uint16_t sid);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
message	Message buffer.	Input
len	Message length.	Input
sid	Stream ID.	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_connection_datachannel_send_binary

【Description】

Send binary data via DataChannel (PPID=53).

【Syntax】

int peer_connection_datachannel_send_binary(PeerConnection* pc, const char* data, size_t len);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
data	Data buffer.	Input
len	Data length.	Input

【Return Value】

Return Value	Description
0	Success.
-1	Failure.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_connection_datachannel_send_binary_sid

【Description】

Send binary data via DataChannel with a specified Stream ID (PPID=53).

【Syntax】

int peer_connection_datachannel_send_binary_sid(PeerConnection* pc, const char* data, size_t len, uint16_t sid);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
data	Data buffer.	Input
len	Data length.	Input
sid	Stream ID.	Input

【Return Value】

Return Value	Description
0	Success.
-1	Failure.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_connection_lookup_sid

【Description】

Look up the Stream ID of a DataChannel by its label.

【Syntax】

int peer_connection_lookup_sid(PeerConnection* pc, const char* label, uint16_t* sid);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
label	DataChannel label name.	Input
sid	Output Stream ID.	Output

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure (corresponding DataChannel not found).

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_connection_lookup_sid_label

【Description】

Look up the label of a DataChannel by its Stream ID.

【Syntax】

char* peer_connection_lookup_sid_label(PeerConnection* pc, uint16_t sid);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
sid	Stream ID.	Input

【Return Value】

Return Value	Description
Non-NULL	Success, returns the label string.
NULL	Failure (corresponding DataChannel not found).

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_connection_send_audio

【Description】

Send audio data.

【Syntax】

int peer_connection_send_audio(PeerConnection* pc, const uint8_t* packet, size_t bytes, uint64_t timestamp_us);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
packet	Audio data buffer.	Input
bytes	Audio data size in bytes.	Input
timestamp_us	Timestamp in microseconds.	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

The audio data format must be consistent with the audio_codec configured in PeerConfiguration.

【Related Topics】

None.

peer_connection_send_video

【Description】

Send video data.

【Syntax】

int peer_connection_send_video(PeerConnection* pc, const uint8_t* packet, size_t bytes, uint64_t timestamp_us);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
packet	Video data buffer.	Input
bytes	Video data size in bytes.	Input
timestamp_us	Timestamp in microseconds.	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

The video data format must be consistent with the video_codec configured in PeerConfiguration.

【Related Topics】

None.

peer_connection_on_receiver_packet_loss

【Description】

Register a receiver packet loss callback function, triggered when an RTCP receiver report is received.

【Syntax】

void peer_connection_on_receiver_packet_loss(PeerConnection* pc,
    void (*on_receiver_packet_loss)(float fraction_loss, uint32_t total_loss, void* userdata));

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input
on_receiver_packet_loss	Packet loss callback function. Callback parameters: fraction_loss is the packet loss rate (0.0~1.0), total_loss is the total number of lost packets, userdata is the user data.	Input

【Return Value】

None.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

Encoding parameters can be dynamically adjusted or key frames requested based on the packet loss rate.

【Related Topics】

None.

peer_connection_get_sctp

【Description】

Get the SCTP instance pointer for low-level SCTP protocol operations.

【Syntax】

void* peer_connection_get_sctp(PeerConnection* pc);

【Parameters】

Parameter	Description	Input/Output
pc	PeerConnection instance pointer.	Input

【Return Value】

Return Value	Description
Non-NULL	Success, returns the SCTP instance pointer.
NULL	Failure.

【Requirements】

• Header file: peer_connection.h

• Library file: libpeer.a

【Note】

This is a low-level interface and is generally not needed by applications.

【Related Topics】

None.

PeerSignaling

The PeerSignaling module provides signaling service connection and communication functionality, supporting interaction with a signaling server via HTTP or MQTT protocols to complete SDP exchange and ICE candidate delivery for WebRTC connections. This module can be disabled at compile time by defining the DISABLE_PEER_SIGNALING macro.

The PeerSignaling module provides the following APIs:

• peer_signaling_connect: Connect to the signaling server.

• peer_signaling_disconnect: Disconnect from the signaling server.

• peer_signaling_loop: Drive the signaling event loop.

• peer_signaling_set_custom_rpc_handler: Register a custom RPC method handler callback.

• peer_signaling_publish: Publish a message via the signaling layer’s MQTT connection.

peer_signaling_connect

【Description】

Connect to the signaling server and establish a communication channel with the signaling service.

【Syntax】

int peer_signaling_connect(const char* url, const char* token, PeerConnection* pc);

【Parameters】

Parameter	Description	Input/Output
url	Signaling server URL.	Input
token	Authentication token.	Input
pc	PeerConnection instance pointer.	Input

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: peer_signaling.h

• Library file: libpeer.a

【Note】

A PeerConnection instance must be created before calling this function.

【Example】

None.

【Related Topics】

None.

peer_signaling_disconnect

【Description】

Disconnect from the signaling server.

【Syntax】

void peer_signaling_disconnect();

【Parameters】

None.

【Return Value】

None.

【Requirements】

• Header file: peer_signaling.h

• Library file: libpeer.a

【Note】

None.

【Related Topics】

None.

peer_signaling_loop

【Description】

Drive the signaling event loop, handling signaling message transmission and reception. Must be called periodically in the main loop.

【Syntax】

int peer_signaling_loop();

【Parameters】

None.

【Return Value】

Return Value	Description
0	Success.
Non-0	Failure.

【Requirements】

• Header file: peer_signaling.h

• Library file: libpeer.a

【Note】

This function should be called continuously after the signaling connection is established to maintain normal signaling communication.

【Related Topics】

None.

peer_signaling_set_custom_rpc_handler

【Description】

Register a custom RPC method handler callback for handling application-specific RPC methods not recognized by the built-in signaling layer. Must be called after peer_signaling_connect(). Only one handler can be registered at a time; subsequent calls replace the previous one. Pass NULL to unregister.

【Syntax】

void peer_signaling_set_custom_rpc_handler(peer_signaling_custom_rpc_cb cb, void* userdata);

【Parameters】

Parameter	Description	Input/Output
cb	Custom RPC method handler callback function.	Input
userdata	User data pointer passed to the callback function.	Input

Callback function type definition:

typedef int (*peer_signaling_custom_rpc_cb)(const char* method, cJSON* params, int id,
    cJSON** result, cJSON** error, void* userdata);

Callback parameter descriptions:

Parameter	Description	Input/Output
method	RPC method name (e.g., “list_recordings”).	Input
params	The “params” field as a cJSON object (may be NULL).	Input
id	JSON-RPC request ID.	Input
result	Output: set to a cJSON result object (caller takes ownership).	Output
error	Output: set to a cJSON error object (caller takes ownership).	Output
userdata	User data pointer.	Input

Callback return value:

Return Value	Description
0	Method was handled (*result or *error must be set).
-1	Method not handled (signaling layer will return METHOD_NOT_FOUND).

【Return Value】

None.

【Requirements】

• Header file: peer_signaling.h

• Library file: libpeer.a

【Note】

The callback function is called from the signaling thread (peer_signaling_loop). Implementations must be thread-safe with respect to application state.

【Related Topics】

None.

peer_signaling_publish

【Description】

Publish a message to an arbitrary MQTT topic via the signaling layer’s existing MQTT connection, reusing the connection established by peer_signaling_connect() without the need for a separate MQTT client. Only available in MQTT mode (proto == 0).

【Syntax】

int peer_signaling_publish(const char* topic, const char* message);

【Parameters】

Parameter	Description	Input/Output
topic	MQTT topic (e.g., “/devices/heartbeat”).	Input
message	Message payload (JSON string).	Input

【Return Value】

Return Value	Description
0	Success.
-1	Failure (HTTP mode or MQTT connection not established).

【Requirements】

• Header file: peer_signaling.h

• Library file: libpeer.a

【Note】

• Only available in MQTT signaling mode; calling in HTTP mode will return -1.

• Must be called after peer_signaling_connect() has successfully established an MQTT connection.

【Related Topics】

None.

WebRTC Typical Usage Flow

1. Call peer_init() for global initialization.

2. Configure the PeerConfiguration structure, setting ICE servers, audio/video codec types, callback functions, etc.

3. Call peer_connection_create() to create a PeerConnection instance.

4. Register callback functions: peer_connection_onicecandidate(), peer_connection_oniceconnectionstatechange(), peer_connection_ondatachannel(), etc.

5. Call peer_signaling_connect() to connect to the signaling server.

6. Continuously call peer_signaling_loop() and peer_connection_loop() in the main loop.

7. The signaling layer automatically completes SDP exchange and ICE candidate delivery. Once the connection is established, audio/video data can be sent via peer_connection_send_audio()/peer_connection_send_video(), or data can be sent/received via DataChannel.

8. After communication ends, call peer_signaling_disconnect() to disconnect the signaling connection, call peer_connection_close() to close the PeerConnection, and call peer_connection_destroy() to destroy the instance.

9. Call peer_deinit() for global deinitialization.