Note

This is the documentation for the latest development branch and may refer to features that are not available in released versions. If you are looking for the documentation for a specific release, use the drop-down menu on the left and select the desired version.

pipeline_module_api_manual

Overview

This manual is intended to guide developers in building a complete Media pipeline when developing AI Demos using MicroPython, achieving the functionality of acquiring images from the Camera and displaying AI inference results. This module encapsulates the default configuration for a single camera with dual channels: one channel sends the Camera image directly to the Display module for display; the other channel uses the get_frame interface to acquire one frame of image for use by the AI program.

API Introduction

init

Description

PipeLine constructor, initializes the resolution of images acquired by the AI program and display-related parameters.

Syntax

from libs.PipeLine import PipeLine

pl=PipeLine(rgb888p_size=[1920,1080],display_mode='hdmi',display_size=None,osd_layer_num=1,debug_mode=0)

Parameters

Parameter	Description	Input / Output	Notes
rgb888p_size	Input image resolution for the AI program, list type, including width and height, e.g., [1920,1080]	Input	Default [224,224], determined by the AI program
display_mode	Display mode, supports hdmi and lcd, str type	Input	Default hdmi, determined by display configuration
display_size	Display resolution, list type, including width and height, e.g., [1920,1080]; if None, determined by the display screen; otherwise set according to input	Input	Default None, determined by the display screen
osd_layer_num	Number of OSD display layers, the number of layers overlaid on the original image by the user program	Input	Default is 1
debug_mode	Debug timing mode, 0 for timing, 1 for no timing, int type	Input	Default is 0

Return Value

Return Value	Description
PipeLine	PipeLine instance

create

Description

PipeLine initialization function, initializes the Sensor/Display/OSD configuration in the Media pipeline.

Syntax

# Default configuration
pl.create()
# Users can also create their own instance and pass it in
from media.sensor import *
sensor=Sensor()
pl.create(sensor=sensor)

Parameters

Parameter	Description	Input / Output	Notes
sensor	Sensor instance, users can create a sensor instance externally and pass it in	Input	Optional, different development boards have default configurations
sensor_id	Sensor ID, users can configure this parameter to set the camera	Input	Optional, different development boards have default configurations
hmirror	Horizontal mirror parameter	Input	Optional, default is None, based on different development board default configuration; when set, it is a bool type, set to True or False
vflip	Vertical flip parameter	Input	Optional, default is None, based on different development board default configuration; when set, it is a bool type, set to True or False
fps	sensor frame rate parameter	Input	Optional, default 60, sets the frame rate of the Sensor
to_ide	Whether to transmit the screen display to the IDE for display	Input	Enabling it occupies more memory
crop_vertical	Whether to adapt to vertical screen display	Input	When the sensor output is a horizontal image and the display is a vertical screen, vertical screen display is achieved through cropping

Return Value

Return Value	Description
None

get_frame

Description

Acquire one frame of image for use by the AI program. The acquired image resolution is the rgb888p_size set by the PipeLine constructor, the image format is Sensor.RGBP888, and it is converted to ulab.numpy.ndarray format upon return.

Syntax

img=pl.get_frame()

Return Value

Return Value	Description
img	Image data in ulab.numpy.ndarray format with resolution of rgb888p_size

show_image

Description

Overlays and displays the AI results drawn on pl.osd_img on the Display. pl.osd_img is a blank image initialized by the create interface with the format image.ARGB8888, used for drawing AI results.

Syntax

pl.show_image()

Return Value

Return Value	Description
None

get_display_size

Description

Gets the width and height of the current screen configuration.

Syntax

display_size=pl.get_display_size()

Return Value

Return Value	Description
list	Returns the display width and height of the screen configuration [display_width,display_height]

destroy

Description

Deinitializes the PipeLine instance.

Syntax

img=pl.destroy()

Return Value

Return Value	Description
None

Example Program

The following is an example program:

from libs.PipeLine import PipeLine
from libs.Utils import ScopedTiming
from media.media import *
import gc
import sys,os

if __name__ == "__main__":
    # Display mode, default is "hdmi", you can choose "hdmi" and "lcd"
    display_mode="hdmi"
    # Initialize PipeLine for image processing workflow
    pl = PipeLine(rgb888p_size=[1920,1080], display_mode=display_mode)
    pl.create()  # Create PipeLine instance
    display_size = pl.get_display_size()
    while True:
        with ScopedTiming("total",1):
            img = pl.get_frame()            # Get current frame data
            print(img.shape)
            gc.collect()                    # Garbage collection
    pl.destroy()                            # Destroy PipeLine instance

In the above code, an image with a resolution of rgb888p_size is obtained through the pl.get_frame() interface, with a type of ulab.numpy.ndarray and a CHW layout. Based on this code, you can focus on the operations of the AI inference part.