# FFmpeg

## Overview

FFmpeg is a cross-platform multimedia framework for recording, conversion, and streaming. K230 RTOS SDK integrates FFmpeg so it can complement the MPP-based media pipeline.

## Feature Summary

### FFmpeg Components

FFmpeg includes the following major components:

- `libavcodec`: audio and video codec library
- `libavformat`: media container muxing and demuxing library
- `libavutil`: utility functions
- `libswscale`: image scaling and color-space conversion
- `libswresample`: audio resampling
- `libavfilter`: audio and video filter library

### Supported Codecs

FFmpeg supports a large number of codecs, including:

- Video codecs: H.264, H.265, MPEG-2, MPEG-4, VP8, VP9, and more
- Audio codecs: AAC, MP3, Opus, Vorbis, FLAC, and more
- Image codecs: JPEG, PNG, GIF, BMP, and more

### Supported Formats

FFmpeg supports many container formats and streaming protocols:

- Video formats: MP4, MKV, AVI, MOV, FLV, and more
- Audio formats: MP3, AAC, FLAC, OGG, and more
- Streaming protocols: RTSP, RTMP, HTTP, HLS, DASH, and more

## Application Scenarios

FFmpeg is feature-rich, but many of its features run on the CPU. On K230, core media capabilities such as audio/video encoding, decoding, and format conversion are primarily implemented through MPP APIs. FFmpeg is mainly used for functionality that MPP does not directly cover, such as MP4 mux/demux and RTSP push/pull workflows.

## Related K230 Paths

- RTSP pusher wrapper: `src/rtsmart/mpp/middleware/src/rtsp_pusher`
- Demo: `src/rtsmart/examples/mpp/sample_rtsppusher`

## Build Notes

### FFmpeg Library Integration

K230 RTOS SDK already provides prebuilt FFmpeg libraries in the SDK library directory, so applications can link against them directly.

### Link Libraries

Example Makefile flags:

```makefile
LDFLAGS += -lavcodec -lavformat -lavutil -lswscale -lm
```

Example CMake usage:

```cmake
target_link_libraries(your_target
    avcodec
    avformat
    avutil
    swscale
    m
)
```

## Usage

### Basic Workflow

#### Register codecs and formats

```c
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>

av_register_all();
```

#### Open an input file

```c
AVFormatContext *pFormatCtx = avformat_alloc_context();
avformat_open_input(&pFormatCtx, filename, NULL, NULL);
avformat_find_stream_info(pFormatCtx, NULL);
```

#### Find the video stream

```c
int video_index = -1;
for (int i = 0; i < pFormatCtx->nb_streams; i++) {
    if (pFormatCtx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) {
        video_index = i;
        break;
    }
}
```

#### Create the decoder context

```c
AVCodecParameters *codecPar = pFormatCtx->streams[video_index]->codecpar;
AVCodec *pCodec = avcodec_find_decoder(codecPar->codec_id);
AVCodecContext *pCodecCtx = avcodec_alloc_context3(pCodec);
avcodec_parameters_to_context(pCodecCtx, codecPar);
avcodec_open2(pCodecCtx, pCodec, NULL);
```

#### Read and decode frames

```c
AVPacket *packet = av_packet_alloc();
AVFrame *frame = av_frame_alloc();

while (av_read_frame(pFormatCtx, packet) >= 0) {
    if (packet->stream_index == video_index) {
        avcodec_send_packet(pCodecCtx, packet);
        while (avcodec_receive_frame(pCodecCtx, frame) == 0) {
            // Process the decoded frame
        }
    }
    av_packet_unref(packet);
}
```

#### Release resources

```c
av_frame_free(&frame);
av_packet_free(&packet);
avcodec_free_context(&pCodecCtx);
avformat_close_input(&pFormatCtx);
```

```{admonition} Tip
FFmpeg is a powerful multimedia framework, but its APIs are relatively complex. It is recommended to read the official FFmpeg documentation and example code first. For full API details, see the [FFmpeg official documentation](https://ffmpeg.org/documentation.html).
```

```{admonition} Tip
When using FFmpeg for video decoding on K230, consider combining it with the K230 hardware codecs (`VDEC`/`VENC`) to improve performance.
```