FFmpeg#

Overview#

FFmpeg is a cross-platform multimedia framework for recording, conversion, and streaming. K230 RTOS SDK integrates FFmpeg so it can complement the MPP-based media pipeline.

Feature Summary#

FFmpeg Components#

FFmpeg includes the following major components:

libavcodec: audio and video codec library
libavformat: media container muxing and demuxing library
libavutil: utility functions
libswscale: image scaling and color-space conversion
libswresample: audio resampling
libavfilter: audio and video filter library

Supported Codecs#

FFmpeg supports a large number of codecs, including:

Video codecs: H.264, H.265, MPEG-2, MPEG-4, VP8, VP9, and more
Audio codecs: AAC, MP3, Opus, Vorbis, FLAC, and more
Image codecs: JPEG, PNG, GIF, BMP, and more

Supported Formats#

FFmpeg supports many container formats and streaming protocols:

Video formats: MP4, MKV, AVI, MOV, FLV, and more
Audio formats: MP3, AAC, FLAC, OGG, and more
Streaming protocols: RTSP, RTMP, HTTP, HLS, DASH, and more

Application Scenarios#

FFmpeg is feature-rich, but many of its features run on the CPU. On K230, core media capabilities such as audio/video encoding, decoding, and format conversion are primarily implemented through MPP APIs. FFmpeg is mainly used for functionality that MPP does not directly cover, such as MP4 mux/demux and RTSP push/pull workflows.

Build Notes#

FFmpeg Library Integration#

K230 RTOS SDK already provides prebuilt FFmpeg libraries in the SDK library directory, so applications can link against them directly.

Link Libraries#

Example Makefile flags:

LDFLAGS += -lavcodec -lavformat -lavutil -lswscale -lm

Example CMake usage:

target_link_libraries(your_target
    avcodec
    avformat
    avutil
    swscale
    m
)

Usage#

Basic Workflow#

Register codecs and formats#

#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>

av_register_all();

Open an input file#

AVFormatContext *pFormatCtx = avformat_alloc_context();
avformat_open_input(&pFormatCtx, filename, NULL, NULL);
avformat_find_stream_info(pFormatCtx, NULL);

Find the video stream#

int video_index = -1;
for (int i = 0; i < pFormatCtx->nb_streams; i++) {
    if (pFormatCtx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) {
        video_index = i;
        break;
    }
}

Create the decoder context#

AVCodecParameters *codecPar = pFormatCtx->streams[video_index]->codecpar;
AVCodec *pCodec = avcodec_find_decoder(codecPar->codec_id);
AVCodecContext *pCodecCtx = avcodec_alloc_context3(pCodec);
avcodec_parameters_to_context(pCodecCtx, codecPar);
avcodec_open2(pCodecCtx, pCodec, NULL);

Read and decode frames#

AVPacket *packet = av_packet_alloc();
AVFrame *frame = av_frame_alloc();

while (av_read_frame(pFormatCtx, packet) >= 0) {
    if (packet->stream_index == video_index) {
        avcodec_send_packet(pCodecCtx, packet);
        while (avcodec_receive_frame(pCodecCtx, frame) == 0) {
            // Process the decoded frame
        }
    }
    av_packet_unref(packet);
}

Release resources#

av_frame_free(&frame);
av_packet_free(&packet);
avcodec_free_context(&pCodecCtx);
avformat_close_input(&pFormatCtx);

Tip

FFmpeg is a powerful multimedia framework, but its APIs are relatively complex. It is recommended to read the official FFmpeg documentation and example code first. For full API details, see the FFmpeg official documentation.

Tip

When using FFmpeg for video decoding on K230, consider combining it with the K230 hardware codecs (VDEC/VENC) to improve performance.