Note

This is the documentation for the latest development branch and may refer to features that are not available in released versions. If you are looking for the documentation for a specific release, use the drop-down menu on the left and select the desired version.

Pipeline Module API Manual

Overview

This manual is intended to guide developers in building a complete Media pipeline when developing AI Demos with MicroPython, enabling the functionality of acquiring images from the Camera and displaying AI inference results. This module encapsulates the default configuration for a single-camera dual-channel: one channel sends the Camera’s image directly to the Display module for display; the other channel uses the get_frame interface to acquire one frame of image for use by the AI program.

API Introduction

init

Description

PipeLine constructor. Initializes the resolution of the image acquired by the AI program and display-related parameters.

Syntax

from libs.PipeLine import PipeLine

pl=PipeLine(rgb888p_size=[1920,1080],display_mode='hdmi',display_size=None,osd_layer_num=1,debug_mode=0)

Parameters

Parameter Name	Description	Input / Output	Notes
rgb888p_size	Input image resolution for the AI program, list type, including width and height, e.g. [1920,1080]	Input	Default [224,224], determined by the AI program
display_mode	Display mode, supports hdmi and lcd, str type	Input	Default hdmi, determined by display configuration
display_size	Display resolution, list type, including width and height, e.g. [1920,1080]; if None, determined by the display screen; otherwise set according to input	Input	Default None, determined by the display screen
osd_layer_num	Number of OSD display layers, the number of layers the user program overlays on the original image	Input	Default 1
debug_mode	Debug timing mode, 0 for timing, 1 for no timing, int type	Input	Default 0

Return Value

Return Value	Description
PipeLine	PipeLine instance

create

Description

PipeLine initialization function. Initializes the Sensor/Display/OSD configuration in the Media pipeline.

Syntax

# Default configuration
pl.create()
# Users can also create their own instance and pass it in
from media.sensor import *
sensor=Sensor()
pl.create(sensor=sensor)

Parameters

Parameter Name	Description	Input / Output	Notes
sensor	Sensor instance, users can create a sensor instance externally and pass it in	Input	Optional, different development boards have default configurations
sensor_id	Sensor ID, users can configure this parameter to set the camera	Input	Optional, different development boards have default configurations
hmirror	Horizontal mirror parameter	Input	Optional, default is None, based on the default configuration of different development boards; when set, it is a bool type, set to True or False
vflip	Vertical flip parameter	Input	Optional, default is None, based on the default configuration of different development boards; when set, it is a bool type, set to True or False
fps	sensor frame rate parameter	Input	Optional, default 60, sets the frame rate of the Sensor
to_ide	Whether to transfer the screen display to the IDE for display	Input	Enabling this consumes more memory
crop_vertical	Whether to adapt to portrait screen display	Input	When the sensor output is a landscape image and the display is a portrait screen, cropping is used to achieve portrait display

Return Value

Return Value	Description
None

get_frame

Description

Acquire one frame of image for use by the AI program. The resolution of the acquired image is the rgb888p_size set by the PipeLine constructor, and the image format is Sensor.RGBP888. When returned, it is converted to ulab.numpy.ndarray format.

Syntax

img=pl.get_frame()

Return Value

Return Value	Description
img	Image data in ulab.numpy.ndarray format with a resolution of rgb888p_size

show_image

Description

Overlay and display the AI results drawn on pl.osd_img on the Display. pl.osd_img is a blank image with the format image.ARGB8888 initialized by the create interface, used for drawing AI results.

Syntax

pl.show_image()

Return Value

Return Value	Description
None

get_display_size

Description

Get the width and height of the current screen configuration.

Syntax

display_size=pl.get_display_size()

Return Value

Return Value	Description
list	Returns the display width and height of the screen configuration [display_width,display_height]

destroy

Description

Deinitialize the PipeLine instance.

Syntax

img=pl.destroy()

Return Value

Return Value	Description
None

Example Program

The following is an example program:

from libs.PipeLine import PipeLine
from libs.Utils import ScopedTiming
from media.media import *
import gc
import sys,os

if __name__ == "__main__":
    # Display mode, default is "hdmi", you can choose "hdmi" and "lcd"
    display_mode="hdmi"
    # Initialize PipeLine for image processing workflow
    pl = PipeLine(rgb888p_size=[1920,1080], display_mode=display_mode)
    pl.create()  # Create PipeLine instance
    display_size = pl.get_display_size()
    while True:
        with ScopedTiming("total",1):
            img = pl.get_frame()            # Get current frame data
            print(img.shape)
            gc.collect()                    # Garbage collection
    pl.destroy()                            # Destroy PipeLine instance

In the above code, an image with a resolution of rgb888p_size is obtained through the pl.get_frame() interface, with a type of ulab.numpy.ndarray and a CHW layout. Based on this code, you can focus on the operations of the AI inference part.