# K230 Integrated Demo: Single-Channel Smart IPC

## Overview

This integrated demo combines network streaming, local rendering, and face detection into a single smart IPC application. With one camera connected, the system can:

- Display the live image on HDMI or LCD and overlay face-detection results
- Stream video over the network through RTSP
- Choose whether the streamed picture is the raw camera frame or the composited frame with AI overlays

## Usage

Source path: `/src/rtsmart/examples/integrated_poc/smart_ipc`

Enable the build in `make menuconfig`:

- `RT-Smart UserSpace Examples Configuration`
- `Enable build integrated examples`
- `Smart IPC examples`

After a successful build, the executable is deployed to:

`/sdcard/app/examples/integrated_poc/smart_ipc/smart_ipc.elf`

Run on target:

`/sdcard/app/examples/integrated_poc/smart_ipc/smart_ipc.elf [options]`

Examples:

- HDMI output:

```sh
./smart_ipc.elf -C 0
```

- LCD output:

```sh
./smart_ipc.elf -C 1
```

Show help:

```sh
./smart_ipc.elf -H
```

Detailed options:

| Option | Description | Default |
| --- | --- | --- |
| `-H` | show help information | - |
| `-S` | Sensor type, auto-detected by default | `SENSOR_TYPE_MAX` |
| `-V` | Enable audio capture | `1` |
| `-a` | Audio sample rate | `8000` |
| `-c` | Audio channel count | `1` |
| `-t` | Video codec type: `h264` or `h265` | `h264` |
| `-w` / `-h` | Video encode width and height | `1280x720` |
| `-b` | Bitrate in kbps | `2000` |
| `-D` | Encode raw sensor output or VO WBC composited output | `0` |
| `-C` | Connector type: `0` HDMI, `1` LCD | HDMI |
| `-A` / `-I` | AI input width and height | `1280x720` |
| `-K` | `kmodel` path | `face_detection_320.kmodel` |
| `-T` / `-N` | Face detection threshold and NMS threshold | `0.6` / `0.4` |
| `-E` | Enable rendering | `1` |
| `-F` | Enable AI analysis | `1` |
| `-G` | Enable encoded network streaming | `1` |

## Code Structure

Important files in the project include:

- `face_detection`: face-detection module
- `media.cpp`: media implementation; wraps MPI initialization and startup for camera, display, and codecs
- `media.h`: media interface declarations
- `smart_ipc.cpp`: main application flow
- `smart_ipc.h`: `MySmartIPC` class and interfaces

`MySmartIPC` encapsulates all integrated smart-camera logic. In `main`, the app creates the object and calls `init` and `start`.

Inside `MySmartIPC`, wrapped MPI interfaces in `KdMedia` initialize VB buffers, camera/display, and codecs. The following callbacks are then used:

- encoded video callback: `OnVEncData`
- encoded audio callback: `OnAEncData`
- AI input frame callback: `OnAIFrameData`

`MySmartIPC` also uses `KdRtspServer` and `FaceDetection` for RTSP streaming and face detection. In `OnAIFrameData`, detection results are drawn on the VO OSD layer. In `OnVEncData` and `OnAEncData`, encoded streams are pushed to network clients.

## Data Pipeline

The pipeline uses three camera channels for local display, AI input, and encoded streaming:

1. camera input captures real-time video and splits it into three paths
1. one path is displayed through HDMI/LCD
1. one path is sent to AI for face detection and overlay
1. one path is encoded and streamed:
    - raw sensor frame path: encode directly from sensor channel (no AI overlays)
    - composited frame path: get VO WBC-composited frame with AI overlays, then encode and stream

This pipeline supports local display, AI analysis, and network streaming simultaneously.

![pipeline](https://kendryte-download.canaan-creative.com/developer/pictures/pipeline_1.png)

## Demo Walkthrough

To view the live stream from a PC, make sure the board has a valid network IP.

1. run `ifconfig` on the board to confirm IP address
1. run `ping <PC_IP>` on the board to verify connectivity

For HDMI output, connect HDMI to a monitor. For LCD output, connect an LCD panel.

```sh
cd /sdcard/app/examples/integrated_poc/smart_ipc
./smart_ipc.elf -C 0 -D 0 # HDMI output, RTSP streams raw frame
./smart_ipc.elf -C 1 -D 1 # LCD output, RTSP streams composited AI-overlay frame
```

The serial log shows a prompt like: `Play this stream using the URL rtsp://<IP>:8554/test`.
Open this URL in VLC on your PC to view the stream.

![smart_ipc](https://kendryte-download.canaan-creative.com/developer/pictures/smart_ipc.png)