# v4l2_drm Python API Description

## Overview

`k230_v4l2_drm` is a Python binding library for V4L2 video capture and DRM display on the Canaan Kendryte K230 platform. It integrates Linux V4L2 (Video4Linux2) camera capture and DRM (Direct Rendering Manager) display output into a unified pipeline, supporting:

- Multi-channel camera capture: capture multiple video streams simultaneously through multiple contexts
- Zero-copy DRM display: V4L2 buffers sent directly to DRM display, no extra copy required
- OSD overlay: overlay ARGB8888/RGB888 layers (such as detection boxes, text) on video frames
- Rotation/Flip: hardware-level rotation (0°/90°/180°/270°) and mirror flipping
- Crop/Offset: set display offset and crop region
- Background display thread: `display_start()` continuously refreshes the display in a background thread

> Usage reference examples: [py_demo/ai/yolo_v4l2_drm.py](https://github.com/kendryte/k230_linux_sdk/blob/dev/buildroot-overlay/package/python-k230/py_demo/ai/yolo_v4l2_drm.py), [py_demo/ai/face_detect.py](https://github.com/kendryte/k230_linux_sdk/blob/dev/buildroot-overlay/package/python-k230/py_demo/ai/face_detect.py)
>

## Complete API Quick Reference

| Method/Property | Signature | Description |
| ------------- | ------------------------------------------------------------ | ---------------------- |
| **Construction** | `V4l2Drm(context_num=1, osd=False)` | Create instance |
| **DRM Initialization** | `drm_init(drm_id=0) → (w, h)` | Initialize display, return resolution |
| **Configuration** | `set_context(index, device, width, height, format, display)` | Configure camera parameters |
|  | `set_offset(index, x, y)` | Set display offset |
|  | `set_crop(index, w, h, ox, oy)` | Set crop region |
|  | `set_rotation(index, rotation)` | Set rotation |
|  | `set_flip(index, hflip, vflip)` | Set flip |
|  | `set_buffer_num(index, num)` | Set buffer count |
| **Pipeline** | `setup() → bool` | Initialize pipeline |
|  | `display_start()` | Start background display thread |
|  | `display_stop()` | Stop display thread |
|  | `dump_start(index) → bool` | Start capture |
|  | `dump_stop(index) → bool` | Stop capture |
|  | `dump_frame(index, timeout_ms) → bool` | Capture one frame |
|  | `dump_release(index) → bool` | Release frame buffer |
| **Data** | `get_buffer_data(index) → ndarray` | Get raw frame data |
|  | `get_buffer_array(index) → ndarray` | Get frame array (with shape) |
| **Properties** | `get_context_count() → int` | Context count |
|  | `get_width(index) → int` | Capture width |
|  | `get_height(index) → int` | Capture height |
|  | `get_device(index) → int` | V4L2 device number |
|  | `get_video_format(index) → int` | Format FourCC |
|  | `get_video_format_str(index) → str` | Format string |
|  | `get_offset_x(index) → int` | X offset |
|  | `get_offset_y(index) → int` | Y offset |
|  | `get_rotation(index) → int` | Rotation value |
|  | `get_hflip(index) → int` | Horizontal flip |
|  | `get_vflip(index) → int` | Vertical flip |
|  | `get_frame_count(index) → int` | Frame count |
|  | `set_frame_count(index, count)` | Set frame count |
|  | `get_video_fd(index) → int` | V4L2 file descriptor |
|  | `display_frame_count` (property) | Display frame count (read/write) |
| **OSD** | `set_osd_format(fourcc)` | Set OSD format |
|  | `osd_update(img)` | Update OSD image |

## Module Constants

### Rotation Enumeration (`Rotation`)

| Constant | Value | Description |
| -------------- | --- | ---------------------- |
| `ROTATION_0`   | 0   | No rotation |
| `ROTATION_90`  | 1   | Rotate 90° clockwise |
| `ROTATION_180` | 2   | Rotate 180° |
| `ROTATION_270` | 3   | Rotate 270° clockwise |
| `REFLECT_X`    | 4   | Horizontal mirror (flip along X axis) |
| `REFLECT_Y`    | 5   | Vertical mirror (flip along Y axis) |

### Pixel Format (FourCC String)

Used for the `format` parameter of `set_context()`:

| Constant | Value | Description |
| -------------- | --------- | --------------------------------- |
| `FORMAT_NV12`  | `b"NV12"` | YUV 4:2:0 semi-planar format (Y + interleaved UV) |
| `FORMAT_NV21`  | `b"NV21"` | YUV 4:2:0 semi-planar format (Y + interleaved VU) |
| `FORMAT_NV16`  | `b"NV16"` | YUV 4:2:2 semi-planar format |
| `FORMAT_NV61`  | `b"NV61"` | YUV 4:2:2 semi-planar format |
| `FORMAT_BGR24` | `b"BGR3"` | BGR 24-bit format |
| `FORMAT_RGB24` | `b"RGB3"` | RGB 24-bit format |
| `FORMAT_YUYV`  | `b"YUYV"` | YUYV 4:2:2 packed format |

> **Note**: The `format` parameter accepts a string (e.g., `"NV12"`), which is automatically converted to FourCC internally. K230 also supports the `"BG3P"` format (BGR planar format, used for AI inference input).

### V4L2 Pixel Format Constants (Integer)

| Constant | Description |
| -------------------- | ----------------------- |
| `V4L2_PIX_FMT_NV12`  | V4L2 NV12 format four-character code |
| `V4L2_PIX_FMT_NV21`  | V4L2 NV21 format four-character code |
| `V4L2_PIX_FMT_NV16`  | V4L2 NV16 format four-character code |
| `V4L2_PIX_FMT_NV61`  | V4L2 NV61 format four-character code |
| `V4L2_PIX_FMT_BGR24` | V4L2 BGR24 format four-character code |
| `V4L2_PIX_FMT_RGB24` | V4L2 RGB24 format four-character code |
| `V4L2_PIX_FMT_YUYV`  | V4L2 YUYV format four-character code |

### DRM Format Constants (Integer)

| Constant | Description |
| --------------------- | ---------------------------------------- |
| `DRM_FORMAT_NV12`     | DRM NV12 format |
| `DRM_FORMAT_NV21`     | DRM NV21 format |
| `DRM_FORMAT_NV16`     | DRM NV16 format |
| `DRM_FORMAT_NV61`     | DRM NV61 format |
| `DRM_FORMAT_BGR888`   | DRM BGR888 format |
| `DRM_FORMAT_RGB888`   | DRM RGB888 format |
| `DRM_FORMAT_YUYV`     | DRM YUYV format |
| `DRM_FORMAT_ARGB8888` | DRM ARGB8888 format (used for OSD) |
| `DRM_FORMAT_XRGB8888` | DRM XRGB8888 format (used for OSD, no alpha channel) |

---

## Class: `V4l2Drm`

### Constructor

```python
V4l2Drm(context_num=1, osd=False)
```

| Parameter     | Type   | Default | Description                                                |
| ------------- | ------ | ------- | ---------------------------------------------------------- |
| `context_num` | `int`  | `1`     | Number of video capture contexts (corresponding to the number of camera channels) |
| `osd`         | `bool` | `False` | Whether to enable OSD overlay layer                        |

**Example:**

```python
# Single camera, no OSD
v4l2 = V4l2Drm(context_num=1)

# Dual cameras (e.g., one for display + one for AI inference), enable OSD
v4l2 = V4l2Drm(context_num=2, osd=True)
```

---

### Initialization and Configuration

#### `drm_init(drm_id=0)`

Initialize the DRM display device.

| Parameter | Type  | Default | Description                                          |
| --------- | ----- | ------- | ---------------------------------------------------- |
| `drm_id`  | `int` | `0`     | DRM device number (corresponding to `/dev/dri/cardN`) |

**Return value**: `tuple(int, int)` — Returns `(width, height)` screen resolution on success; returns `(-1, -1)` on failure

```python
width, height = v4l2.drm_init(drm_id=0)
if width == -1:
    raise RuntimeError("DRM init failed")
```

#### `set_context(index, device, width, height, format, display)`

Configure the camera parameters for the specified context.

| Parameter | Type   | Default | Description                                       |
| --------- | ------ | ------- | ------------------------------------------------- |
| `index`   | `int`  | `0`     | Context index (0 ~ context_num-1)                 |
| `device`  | `int`  | `1`     | V4L2 device number (corresponding to `/dev/videoN`) |
| `width`   | `int`  | `640`   | Capture resolution width                          |
| `height`  | `int`  | `480`   | Capture resolution height                         |
| `format`  | `str`  | `"NV12"`| Pixel format (e.g., `"NV12"`, `"BG3P"`, `"YUYV"`) |
| `display` | `bool` | `True`  | Whether to output this video stream to the screen for display |

```python
# Context 0: Camera 1, 1080p, NV12 format, for display
v4l2.set_context(index=0, device=1, width=1920, height=1080, format="NV12", display=True)

# Context 1: Camera 2, 720p, BG3P format, for AI inference only, not displayed
v4l2.set_context(index=1, device=2, width=1280, height=720, format="BG3P", display=False)
```

#### `set_offset(index, x, y)`

Set the display offset for the specified context on the screen.

| Parameter | Type  | Default | Description       |
| --------- | ----- | ------- | ----------------- |
| `index`   | `int` | `0`     | Context index     |
| `x`       | `int` | `0`     | X offset (pixels) |
| `y`       | `int` | `0`     | Y offset (pixels) |

```python
# Offset the image of context 0 to position (100, 50)
v4l2.set_offset(index=0, x=100, y=50)
```

#### `set_crop(index, w, h, ox, oy)`

Set the crop region for the specified context.

| Parameter | Type  | Default | Description                       |
| --------- | ----- | ------- | --------------------------------- |
| `index`   | `int` | `0`     | Context index                     |
| `w`       | `int` | `0`     | Crop width (0 = no crop)          |
| `h`       | `int` | `0`     | Crop height (0 = no crop)         |
| `ox`      | `int` | `0`     | Crop region X offset              |
| `oy`      | `int` | `0`     | Crop region Y offset              |

```python
# Crop a 640x480 region with starting offset (100, 100)
v4l2.set_crop(index=0, w=640, h=480, ox=100, oy=100)
```

#### `set_rotation(index, rotation)`

Set the rotation mode for the specified context.

| Parameter  | Type  | Default | Description                                    |
| ---------- | ----- | ------- | ---------------------------------------------- |
| `index`    | `int` | `0`     | Context index                                  |
| `rotation` | `int` | `0`     | Rotation value, using `Rotation` enum constants |

```python
from k230_v4l2_drm import V4l2Drm, ROTATION_90, ROTATION_0

# Automatically select rotation based on screen orientation
if width > height:
    v4l2.set_rotation(0, ROTATION_0)   # Landscape, no rotation
else:
    v4l2.set_rotation(0, ROTATION_90)  # Portrait, rotate 90 degrees
```

#### `set_flip(index, hflip, vflip)`

Set the mirror flip for the specified context.

| Parameter | Type  | Default | Description                       |
| --------- | ----- | ------- | --------------------------------- |
| `index`   | `int` | `0`     | Context index                     |
| `hflip`   | `int` | `0`     | Horizontal flip (0=off, 1=on)     |
| `vflip`   | `int` | `0`     | Vertical flip (0=off, 1=on)       |

```python
# Horizontal mirror (left-right flip)
v4l2.set_flip(index=0, hflip=1, vflip=0)
```

#### `set_buffer_num(index, num)`

Set the number of V4L2 buffers for the specified context.

| Parameter | Type  | Default | Description                              |
| --------- | ----- | ------- | ---------------------------------------- |
| `index`   | `int` | `0`     | Context index                            |
| `num`     | `int` | `4`     | Number of buffers (minimum 3, recommended 4) |

```python
v4l2.set_buffer_num(index=0, num=4)
```

---

### Pipeline Control

#### `setup()`

Initialize and start the V4L2-DRM pipeline. Call after all `set_context` / `set_rotation` and other configurations are complete.

**Return value**: `bool` — returns `True` on success, `False` on failure

```python
if not v4l2.setup():
    raise RuntimeError("V4L2-DRM setup failed!")
```

#### `display_start()`

Start the display refresh loop in a background thread. After calling, the display thread runs continuously, submitting V4L2 buffers to the DRM display.

```python
v4l2.display_start()
```

#### `display_stop()`

Stop the display refresh loop and wait for the background thread to end.

```python
v4l2.display_stop()
```

#### `dump_start(index)`

Start video capture for the specified context (begin V4L2 streaming capture).

| Parameter | Type  | Default | Description        |
| --------- | ----- | ------- | ------------------ |
| `index`   | `int` | `0`     | Context number |

**Return value**: `bool`

```python
v4l2.dump_start(index=0)
```

#### `dump_stop(index)`

Stop video capture for the specified context.

| Parameter | Type  | Default | Description        |
| --------- | ----- | ------- | ------------------ |
| `index`   | `int` | `0`     | Context number |

**Return value**: `bool`

```python
v4l2.dump_stop(index=0)
```

#### `dump_frame(index, timeout_ms)`

Capture one frame from the specified context. Blocks until a frame is available or timeout occurs.

| Parameter    | Type  | Default | Description                |
| ------------ | ----- | ------- | -------------------------- |
| `index`      | `int` | `0`     | Context number             |
| `timeout_ms` | `int` | `1000`  | Timeout duration (milliseconds) |

**Return value**: `bool` — returns `True` on successful capture, `False` on timeout or failure

```python
if v4l2.dump_frame(index=0, timeout_ms=1000):
    # Successfully obtained one frame
    frame_data = v4l2.get_buffer_data(index=0)
    # ... Process frame data ...
    v4l2.dump_release(index=0)  # Must release!
else:
    print("Frame capture timeout")
```

> **Important**: After a successful `dump_frame`, you must call `dump_release` to release the buffer, otherwise subsequent frames cannot be enqueued.

#### `dump_release(index)`

Release the captured frame buffer for the specified context, making it available for V4L2 to reuse.

| Parameter | Type  | Default | Description        |
| --------- | ----- | ------- | ------------------ |
| `index`   | `int` | `0`     | Context number |

**Return value**: `bool`

---

### Data Access

#### `get_buffer_data(index)`

Get the raw data of the current frame for the specified context, returns a numpy array.

| Parameter | Type  | Default | Description            |
| --------- | ----- | ------- | ---------------------- |
| `index`   | `int` | `0`     | Context index |

**Return value**: `numpy.ndarray` — A one-dimensional uint8 array containing the raw pixel data of the frame

```python
raw_data = v4l2.get_buffer_data(index=0)
# raw_data.shape = (frame_size,), dtype = uint8
```

#### `get_buffer_array(index)`

Get the current frame data of the specified context, returns a numpy array with shape information.

| Parameter | Type  | Default | Description            |
| --------- | ----- | ------- | ---------------------- |
| `index`   | `int` | `0`     | Context index |

**Return value**: `numpy.ndarray` — Numpy array with shape

- For **NV12** format: shape = `(height * 3 // 2, width)` or similar
- For **BG3P** format (BGR planar): shape = `(3, height, width)` i.e., NCHW layout
- For **YUYV** format: shape = `(height, width, 2)`

```python
frame = v4l2.get_buffer_array(index=1)
# For BG3P format: frame.shape = (3, 720, 1280), can be directly used for AI inference
```

---

### Property Query

| Method                              | Return Type | Description                                                  |
| ----------------------------------- | ----------- | ------------------------------------------------------------ |
| `get_context_count()`               | `int`       | Total number of contexts                                     |
| `get_width(index=0)`                | `int`       | Capture width of the specified context                       |
| `get_height(index=0)`               | `int`       | Capture height of the specified context                      |
| `get_device(index=0)`               | `int`       | V4L2 device number of the specified context                  |
| `get_video_format(index=0)`         | `int`       | V4L2 format of the specified context (integer FourCC)        |
| `get_video_format_str(index=0)`     | `str`       | Format string of the specified context (e.g., `"NV12"`)      |
| `get_offset_x(index=0)`             | `int`       | X offset of the specified context                            |
| `get_offset_y(index=0)`             | `int`       | Y offset of the specified context                            |
| `get_rotation(index=0)`             | `int`       | Rotation value of the specified context                      |
| `get_hflip(index=0)`                | `int`       | Horizontal flip status of the specified context              |
| `get_vflip(index=0)`                | `int`       | Vertical flip status of the specified context                |
| `get_frame_count(index=0)`          | `int`       | Frame count of the specified context                         |
| `set_frame_count(index=0, count=0)` | `void`      | Set the frame count of the specified context                 |
| `get_video_fd(index=0)`             | `int`       | V4L2 file descriptor of the specified context (can be used with `select`/`poll`) |

---

### Display Frame Count Property

#### `display_frame_count` (Property)

Current display frame counter, readable and writable.

```python
# Read
count = v4l2.display_frame_count

# Write
v4l2.display_frame_count = 0
```

---

### OSD (On-Screen Display) Overlay

The OSD feature allows overlaying a transparent layer on top of the video image, commonly used for drawing AI detection boxes, text, etc.

> **Prerequisite**: When constructing `V4l2Drm`, `osd=True` must be set.

#### `set_osd_format(fourcc)`

Set the OSD layer format.

| Parameter | Type  | Description                                                                 |
| --------- | ----- | --------------------------------------------------------------------------- |
| `fourcc`  | `int` | DRM format constant, usually `DRM_FORMAT_ARGB8888` or `DRM_FORMAT_XRGB8888` |

```python
from k230_v4l2_drm import V4l2Drm, DRM_FORMAT_ARGB8888

v4l2 = V4l2Drm(context_num=1, osd=True)
v4l2.drm_init()
# ...
v4l2.set_osd_format(DRM_FORMAT_ARGB8888)
```

#### `osd_update(img)`

Update the OSD layer content. Writes a numpy array image data into the OSD plane.

| Parameter | Type            | Description    |
| --------- | --------------- | -------------- |
| `img`     | `numpy.ndarray` | OSD image data |

**Image Requirements:**

- When format is ARGB8888: shape = `(height, width, 4)`, dtype = `uint8`
- When format is RGB888: shape = `(height, width, 3)`, dtype = `uint8`
- Size should match the display resolution
- Alpha channel: 0 = fully transparent, 255 = fully opaque

```python
import numpy as np

# Create a fully transparent OSD image (same size as display)
osd_img = np.zeros((display_height, display_width, 4), dtype=np.uint8)

# Draw a red rectangle box (ARGB format: B, G, R, A)
osd_img[100:200, 100:300] = [0, 0, 255, 255]  # Red, opaque

# Draw a green text region
osd_img[10:50, 10:200] = [0, 255, 0, 255]  # Green, opaque

# Update OSD
v4l2.osd_update(osd_img)
```

> **Note**: Each call to `osd_update` will completely replace the OSD layer content. If multiple elements need to be overlaid, draw them on the numpy array first and then submit all at once.

---

---