Note

This is the documentation for the latest development branch and may refer to features that are not available in released versions. If you are looking for the documentation for a specific release, use the drop-down menu on the left and select the desired version.

v4l2_drm Python API Description

v4l2_drm Python API Description#

Overview#

k230_v4l2_drm is a Python binding library for V4L2 video capture and DRM display on the Canaan Kendryte K230 platform. It integrates Linux V4L2 (Video4Linux2) camera capture and DRM (Direct Rendering Manager) display output into a unified pipeline, supporting:

Multi-channel camera capture: capture multiple video streams simultaneously through multiple contexts
Zero-copy DRM display: V4L2 buffers sent directly to DRM display, no extra copy required
OSD overlay: overlay ARGB8888/RGB888 layers (such as detection boxes, text) on video frames
Rotation/Flip: hardware-level rotation (0°/90°/180°/270°) and mirror flipping
Crop/Offset: set display offset and crop region
Background display thread: display_start() continuously refreshes the display in a background thread

Usage reference examples: py_demo/ai/yolo_v4l2_drm.py, py_demo/ai/face_detect.py

Complete API Quick Reference#

Method/Property	Signature	Description
Construction	`V4l2Drm(context_num=1, osd=False)`	Create instance
DRM Initialization	`drm_init(drm_id=0) → (w, h)`	Initialize display, return resolution
Configuration	`set_context(index, device, width, height, format, display)`	Configure camera parameters
	`set_offset(index, x, y)`	Set display offset
	`set_crop(index, w, h, ox, oy)`	Set crop region
	`set_rotation(index, rotation)`	Set rotation
	`set_flip(index, hflip, vflip)`	Set flip
	`set_buffer_num(index, num)`	Set buffer count
Pipeline	`setup() → bool`	Initialize pipeline
	`display_start()`	Start background display thread
	`display_stop()`	Stop display thread
	`dump_start(index) → bool`	Start capture
	`dump_stop(index) → bool`	Stop capture
	`dump_frame(index, timeout_ms) → bool`	Capture one frame
	`dump_release(index) → bool`	Release frame buffer
Data	`get_buffer_data(index) → ndarray`	Get raw frame data
	`get_buffer_array(index) → ndarray`	Get frame array (with shape)
Properties	`get_context_count() → int`	Context count
	`get_width(index) → int`	Capture width
	`get_height(index) → int`	Capture height
	`get_device(index) → int`	V4L2 device number
	`get_video_format(index) → int`	Format FourCC
	`get_video_format_str(index) → str`	Format string
	`get_offset_x(index) → int`	X offset
	`get_offset_y(index) → int`	Y offset
	`get_rotation(index) → int`	Rotation value
	`get_hflip(index) → int`	Horizontal flip
	`get_vflip(index) → int`	Vertical flip
	`get_frame_count(index) → int`	Frame count
	`set_frame_count(index, count)`	Set frame count
	`get_video_fd(index) → int`	V4L2 file descriptor
	`display_frame_count` (property)	Display frame count (read/write)
OSD	`set_osd_format(fourcc)`	Set OSD format
	`osd_update(img)`	Update OSD image

Module Constants#

Rotation Enumeration (`Rotation`)#

Constant	Value	Description
`ROTATION_0`	0	No rotation
`ROTATION_90`	1	Rotate 90° clockwise
`ROTATION_180`	2	Rotate 180°
`ROTATION_270`	3	Rotate 270° clockwise
`REFLECT_X`	4	Horizontal mirror (flip along X axis)
`REFLECT_Y`	5	Vertical mirror (flip along Y axis)

Pixel Format (FourCC String)#

Used for the format parameter of set_context():

Constant	Value	Description
`FORMAT_NV12`	`b"NV12"`	YUV 4:2:0 semi-planar format (Y + interleaved UV)
`FORMAT_NV21`	`b"NV21"`	YUV 4:2:0 semi-planar format (Y + interleaved VU)
`FORMAT_NV16`	`b"NV16"`	YUV 4:2:2 semi-planar format
`FORMAT_NV61`	`b"NV61"`	YUV 4:2:2 semi-planar format
`FORMAT_BGR24`	`b"BGR3"`	BGR 24-bit format
`FORMAT_RGB24`	`b"RGB3"`	RGB 24-bit format
`FORMAT_YUYV`	`b"YUYV"`	YUYV 4:2:2 packed format

Note: The format parameter accepts a string (e.g., "NV12"), which is automatically converted to FourCC internally. K230 also supports the "BG3P" format (BGR planar format, used for AI inference input).

V4L2 Pixel Format Constants (Integer)#

Constant	Description
`V4L2_PIX_FMT_NV12`	V4L2 NV12 format four-character code
`V4L2_PIX_FMT_NV21`	V4L2 NV21 format four-character code
`V4L2_PIX_FMT_NV16`	V4L2 NV16 format four-character code
`V4L2_PIX_FMT_NV61`	V4L2 NV61 format four-character code
`V4L2_PIX_FMT_BGR24`	V4L2 BGR24 format four-character code
`V4L2_PIX_FMT_RGB24`	V4L2 RGB24 format four-character code
`V4L2_PIX_FMT_YUYV`	V4L2 YUYV format four-character code

DRM Format Constants (Integer)#

Constant	Description
`DRM_FORMAT_NV12`	DRM NV12 format
`DRM_FORMAT_NV21`	DRM NV21 format
`DRM_FORMAT_NV16`	DRM NV16 format
`DRM_FORMAT_NV61`	DRM NV61 format
`DRM_FORMAT_BGR888`	DRM BGR888 format
`DRM_FORMAT_RGB888`	DRM RGB888 format
`DRM_FORMAT_YUYV`	DRM YUYV format
`DRM_FORMAT_ARGB8888`	DRM ARGB8888 format (used for OSD)
`DRM_FORMAT_XRGB8888`	DRM XRGB8888 format (used for OSD, no alpha channel)

Class: `V4l2Drm`#

Constructor#

V4l2Drm(context_num=1, osd=False)

Parameter	Type	Default	Description
`context_num`	`int`	`1`	Number of video capture contexts (corresponding to the number of camera channels)
`osd`	`bool`	`False`	Whether to enable OSD overlay layer

Example:

# Single camera, no OSD
v4l2 = V4l2Drm(context_num=1)

# Dual cameras (e.g., one for display + one for AI inference), enable OSD
v4l2 = V4l2Drm(context_num=2, osd=True)

Initialization and Configuration#

`drm_init(drm_id=0)`#

Initialize the DRM display device.

Parameter	Type	Default	Description
`drm_id`	`int`	`0`	DRM device number (corresponding to `/dev/dri/cardN`)

Return value: tuple(int, int) — Returns (width, height) screen resolution on success; returns (-1, -1) on failure

width, height = v4l2.drm_init(drm_id=0)
if width == -1:
    raise RuntimeError("DRM init failed")

`set_context(index, device, width, height, format, display)`#

Configure the camera parameters for the specified context.

Parameter	Type	Default	Description
`index`	`int`	`0`	Context index (0 ~ context_num-1)
`device`	`int`	`1`	V4L2 device number (corresponding to `/dev/videoN`)
`width`	`int`	`640`	Capture resolution width
`height`	`int`	`480`	Capture resolution height
`format`	`str`	`"NV12"`	Pixel format (e.g., `"NV12"`, `"BG3P"`, `"YUYV"`)
`display`	`bool`	`True`	Whether to output this video stream to the screen for display

# Context 0: Camera 1, 1080p, NV12 format, for display
v4l2.set_context(index=0, device=1, width=1920, height=1080, format="NV12", display=True)

# Context 1: Camera 2, 720p, BG3P format, for AI inference only, not displayed
v4l2.set_context(index=1, device=2, width=1280, height=720, format="BG3P", display=False)

`set_offset(index, x, y)`#

Set the display offset for the specified context on the screen.

Parameter	Type	Default	Description
`index`	`int`	`0`	Context index
`x`	`int`	`0`	X offset (pixels)
`y`	`int`	`0`	Y offset (pixels)

# Offset the image of context 0 to position (100, 50)
v4l2.set_offset(index=0, x=100, y=50)

`set_crop(index, w, h, ox, oy)`#

Set the crop region for the specified context.

Parameter	Type	Default	Description
`index`	`int`	`0`	Context index
`w`	`int`	`0`	Crop width (0 = no crop)
`h`	`int`	`0`	Crop height (0 = no crop)
`ox`	`int`	`0`	Crop region X offset
`oy`	`int`	`0`	Crop region Y offset

# Crop a 640x480 region with starting offset (100, 100)
v4l2.set_crop(index=0, w=640, h=480, ox=100, oy=100)

`set_rotation(index, rotation)`#

Set the rotation mode for the specified context.

Parameter	Type	Default	Description
`index`	`int`	`0`	Context index
`rotation`	`int`	`0`	Rotation value, using `Rotation` enum constants

from k230_v4l2_drm import V4l2Drm, ROTATION_90, ROTATION_0

# Automatically select rotation based on screen orientation
if width > height:
    v4l2.set_rotation(0, ROTATION_0)   # Landscape, no rotation
else:
    v4l2.set_rotation(0, ROTATION_90)  # Portrait, rotate 90 degrees

`set_flip(index, hflip, vflip)`#

Set the mirror flip for the specified context.

Parameter	Type	Default	Description
`index`	`int`	`0`	Context index
`hflip`	`int`	`0`	Horizontal flip (0=off, 1=on)
`vflip`	`int`	`0`	Vertical flip (0=off, 1=on)

# Horizontal mirror (left-right flip)
v4l2.set_flip(index=0, hflip=1, vflip=0)

`set_buffer_num(index, num)`#

Set the number of V4L2 buffers for the specified context.

Parameter	Type	Default	Description
`index`	`int`	`0`	Context index
`num`	`int`	`4`	Number of buffers (minimum 3, recommended 4)

v4l2.set_buffer_num(index=0, num=4)

Pipeline Control#

`setup()`#

Initialize and start the V4L2-DRM pipeline. Call after all set_context / set_rotation and other configurations are complete.

Return value: bool — returns True on success, False on failure

if not v4l2.setup():
    raise RuntimeError("V4L2-DRM setup failed!")

`display_start()`#

Start the display refresh loop in a background thread. After calling, the display thread runs continuously, submitting V4L2 buffers to the DRM display.

v4l2.display_start()

`display_stop()`#

Stop the display refresh loop and wait for the background thread to end.

v4l2.display_stop()

`dump_start(index)`#

Start video capture for the specified context (begin V4L2 streaming capture).

Parameter	Type	Default	Description
`index`	`int`	`0`	Context number

Return value: bool

v4l2.dump_start(index=0)

`dump_stop(index)`#

Stop video capture for the specified context.

Parameter	Type	Default	Description
`index`	`int`	`0`	Context number

Return value: bool

v4l2.dump_stop(index=0)

`dump_frame(index, timeout_ms)`#

Capture one frame from the specified context. Blocks until a frame is available or timeout occurs.

Parameter	Type	Default	Description
`index`	`int`	`0`	Context number
`timeout_ms`	`int`	`1000`	Timeout duration (milliseconds)

Return value: bool — returns True on successful capture, False on timeout or failure

if v4l2.dump_frame(index=0, timeout_ms=1000):
    # Successfully obtained one frame
    frame_data = v4l2.get_buffer_data(index=0)
    # ... Process frame data ...
    v4l2.dump_release(index=0)  # Must release!
else:
    print("Frame capture timeout")

Important: After a successful dump_frame, you must call dump_release to release the buffer, otherwise subsequent frames cannot be enqueued.

`dump_release(index)`#

Release the captured frame buffer for the specified context, making it available for V4L2 to reuse.

Parameter	Type	Default	Description
`index`	`int`	`0`	Context number

Return value: bool

Data Access#

`get_buffer_data(index)`#

Get the raw data of the current frame for the specified context, returns a numpy array.

Parameter	Type	Default	Description
`index`	`int`	`0`	Context index

Return value: numpy.ndarray — A one-dimensional uint8 array containing the raw pixel data of the frame

raw_data = v4l2.get_buffer_data(index=0)
# raw_data.shape = (frame_size,), dtype = uint8

`get_buffer_array(index)`#

Get the current frame data of the specified context, returns a numpy array with shape information.

Parameter	Type	Default	Description
`index`	`int`	`0`	Context index

Return value: numpy.ndarray — Numpy array with shape

For NV12 format: shape = (height * 3 // 2, width) or similar
For BG3P format (BGR planar): shape = (3, height, width) i.e., NCHW layout
For YUYV format: shape = (height, width, 2)

frame = v4l2.get_buffer_array(index=1)
# For BG3P format: frame.shape = (3, 720, 1280), can be directly used for AI inference

Property Query#

Method	Return Type	Description
`get_context_count()`	`int`	Total number of contexts
`get_width(index=0)`	`int`	Capture width of the specified context
`get_height(index=0)`	`int`	Capture height of the specified context
`get_device(index=0)`	`int`	V4L2 device number of the specified context
`get_video_format(index=0)`	`int`	V4L2 format of the specified context (integer FourCC)
`get_video_format_str(index=0)`	`str`	Format string of the specified context (e.g., `"NV12"`)
`get_offset_x(index=0)`	`int`	X offset of the specified context
`get_offset_y(index=0)`	`int`	Y offset of the specified context
`get_rotation(index=0)`	`int`	Rotation value of the specified context
`get_hflip(index=0)`	`int`	Horizontal flip status of the specified context
`get_vflip(index=0)`	`int`	Vertical flip status of the specified context
`get_frame_count(index=0)`	`int`	Frame count of the specified context
`set_frame_count(index=0, count=0)`	`void`	Set the frame count of the specified context
`get_video_fd(index=0)`	`int`	V4L2 file descriptor of the specified context (can be used with `select`/`poll`)

Display Frame Count Property#

`display_frame_count` (Property)#

Current display frame counter, readable and writable.

# Read
count = v4l2.display_frame_count

# Write
v4l2.display_frame_count = 0

OSD (On-Screen Display) Overlay#

The OSD feature allows overlaying a transparent layer on top of the video image, commonly used for drawing AI detection boxes, text, etc.

Prerequisite: When constructing V4l2Drm, osd=True must be set.

`set_osd_format(fourcc)`#

Set the OSD layer format.

Parameter	Type	Description
`fourcc`	`int`	DRM format constant, usually `DRM_FORMAT_ARGB8888` or `DRM_FORMAT_XRGB8888`

from k230_v4l2_drm import V4l2Drm, DRM_FORMAT_ARGB8888

v4l2 = V4l2Drm(context_num=1, osd=True)
v4l2.drm_init()
# ...
v4l2.set_osd_format(DRM_FORMAT_ARGB8888)

`osd_update(img)`#

Update the OSD layer content. Writes a numpy array image data into the OSD plane.

Parameter	Type	Description
`img`	`numpy.ndarray`	OSD image data

Image Requirements:

When format is ARGB8888: shape = (height, width, 4), dtype = uint8
When format is RGB888: shape = (height, width, 3), dtype = uint8
Size should match the display resolution
Alpha channel: 0 = fully transparent, 255 = fully opaque

import numpy as np

# Create a fully transparent OSD image (same size as display)
osd_img = np.zeros((display_height, display_width, 4), dtype=np.uint8)

# Draw a red rectangle box (ARGB format: B, G, R, A)
osd_img[100:200, 100:300] = [0, 0, 255, 255]  # Red, opaque

# Draw a green text region
osd_img[10:50, 10:200] = [0, 255, 0, 255]  # Green, opaque

# Update OSD
v4l2.osd_update(osd_img)

Note: Each call to osd_update will completely replace the OSD layer content. If multiple elements need to be overlaid, draw them on the numpy array first and then submit all at once.

v4l2_drm Python API Description

Contents

v4l2_drm Python API Description#

Overview#

Complete API Quick Reference#

Module Constants#

Rotation Enumeration (Rotation)#

Pixel Format (FourCC String)#

V4L2 Pixel Format Constants (Integer)#

DRM Format Constants (Integer)#

Class: V4l2Drm#

Constructor#

Initialization and Configuration#

drm_init(drm_id=0)#

set_context(index, device, width, height, format, display)#

set_offset(index, x, y)#

set_crop(index, w, h, ox, oy)#

set_rotation(index, rotation)#

set_flip(index, hflip, vflip)#

set_buffer_num(index, num)#

Pipeline Control#

setup()#

display_start()#

display_stop()#

dump_start(index)#

dump_stop(index)#

dump_frame(index, timeout_ms)#

dump_release(index)#

Data Access#

get_buffer_data(index)#

get_buffer_array(index)#

Property Query#

Display Frame Count Property#

display_frame_count (Property)#

OSD (On-Screen Display) Overlay#

set_osd_format(fourcc)#

osd_update(img)#

Rotation Enumeration (`Rotation`)#

Class: `V4l2Drm`#

`drm_init(drm_id=0)`#

`set_context(index, device, width, height, format, display)`#

`set_offset(index, x, y)`#

`set_crop(index, w, h, ox, oy)`#

`set_rotation(index, rotation)`#

`set_flip(index, hflip, vflip)`#

`set_buffer_num(index, num)`#

`setup()`#

`display_start()`#

`display_stop()`#

`dump_start(index)`#

`dump_stop(index)`#

`dump_frame(index, timeout_ms)`#

`dump_release(index)`#

`get_buffer_data(index)`#

`get_buffer_array(index)`#

`display_frame_count` (Property)#

`set_osd_format(fourcc)`#

`osd_update(img)`#