Note

This is the documentation for the latest development branch and may refer to features that are not available in released versions. If you are looking for the documentation for a specific release, use the drop-down menu on the left and select the desired version.

nncase_runtime Module API Manual

Overview

This manual introduces the nncase_runtime module of the CanMV platform, guiding developers to invoke the KPU (Neural Network Processing Unit) and AI2D modules in the MicroPython environment to implement efficient deep learning inference and image processing functions.

Abbreviation	Description
nncase_runtime	k230 nncase runtime package, includes KPU module and AI2D module
nncase_runtime.kpu	kpu module
nncase_runtime.ai2d	ai2d module

API Introduction

from_numpy

Description

This function is used to create a runtime_tensor from a ulab.numpy object in MicroPython.

Syntax

runtime_tensor = nncase_runtime.from_numpy(ulab.numpy)

Parameters

Parameter Name	Description	Input/Output
ulab.numpy	numpy object	Input

Return Value

Return Value	Description
runtime_tensor	Returns the created runtime_tensor object.
Other	If it fails, a C++ exception will be thrown.

to_numpy

Description

Converts a runtime_tensor object to a ulab.numpy object.

Syntax

runtime_tensor = kpu.get_output_tensor(0)
result = runtime_tensor.to_numpy()

Parameters

None.

Return Value

Return Value	Description
ulab.numpy	Returns the ulab.numpy object converted from runtime_tensor.
Other	If it fails, a C++ exception will be thrown.

nncase_runtime.kpu

The kpu module provides basic functions for invoking KPU hardware to perform neural network model inference, including loading models, setting input data, executing inference, and obtaining output results.

load_kmodel

Description

Loads a neural network model in the compiled kmodel format.

Syntax

load_kmodel(read_bin)
load_kmodel(path)

Parameters

Parameter Name	Description	Input/Output
read_bin	Binary content of kmodel	Input
path	File path of kmodel	Input

Return Value

Return Value	Description
None	Loaded successfully.
Other	If it fails, a C++ exception will be thrown.

set_input_tensor

Description

Sets the input runtime_tensor for kmodel inference.

Syntax

set_input_tensor(index, runtime_tensor)

Parameters

Parameter Name	Description	Input/Output
index	Input index of kmodel	Input
runtime_tensor	Tensor containing input data information	Input

Return Value

Return Value	Description
None	Set successfully.
Other	If it fails, a C++ exception will be thrown.

get_input_tensor

Description

Gets the input runtime_tensor for kmodel inference.

Syntax

get_input_tensor(index)

Parameters

Parameter Name	Description	Input/Output
index	Input index of kmodel	Input

Return Value

Return Value	Description
runtime_tensor	Tensor containing input data information.
Other	If it fails, a C++ exception will be thrown.

set_output_tensor

Description

Sets the output result after kmodel inference.

Syntax

set_output_tensor(index, runtime_tensor)

Parameters

Parameter Name	Description	Input/Output
index	Output index of kmodel	Input
runtime_tensor	Tensor of the output result	Input

Return Value

Return Value	Description
None	Set successfully.
Other	If it fails, a C++ exception will be thrown.

get_output_tensor

Description

Gets the output result after kmodel inference.

Syntax

get_output_tensor(index)

Parameters

Parameter Name	Description	Input/Output
index	Output index of kmodel	Input

Return Value

Return Value	Description
runtime_tensor	Gets the runtime_tensor of the index-th output.
Other	If it fails, a C++ exception will be thrown.

run

Description

Starts the kmodel inference process.

Syntax

run()

Return Value

Return Value	Description
None	Inference successful
Other	If it fails, a C++ exception will be thrown.

inputs_size

Description

Gets the number of inputs of the kmodel.

Syntax

inputs_size()

Return Value

Return Value	Description
size_t	Number of inputs of kmodel.
Other	If it fails, a C++ exception will be thrown.

outputs_size

Description

Gets the number of outputs of the kmodel.

Syntax

outputs_size()

Return Value

Return Value	Description
size_t	Number of outputs of kmodel.
Other	If it fails, a C++ exception will be thrown.

get_input_desc

Description

Gets the input description information of the specified index.

Syntax

get_input_desc(index)

Parameters

Parameter Name	Description	Input/Output
index	Input index of kmodel	Input

Return Value

Return Value	Description
MemoryRange	Information of the index-th input, including dtype, start, size.

get_output_desc

Description

Gets the output description information of the specified index.

Syntax

get_output_desc(index)

Parameters

Parameter Name	Description	Input/Output
index	Output index of kmodel	Input

Return Value

Return Value	Description
MemoryRange	Information of the index-th output, including dtype, start, size.

nncase_runtime.ai2d

build

Description

Construct an ai2d_builder object.

Syntax

build(input_shape, output_shape)

Parameters

Name	Description
input_shape	Input shape
output_shape	Output shape

Return Value

Return Value	Description
ai2d_builder	Returns the ai2d_builder object for execution.
Other	If failed, a C++ exception will be thrown.

run

Description

Configure registers and start AI2D computation.

Syntax

ai2d_builder.run(input_tensor, output_tensor)

Parameters

Name	Description
input_tensor	Input tensor
output_tensor	Output tensor

Return Value

Return Value	Description
None	Success.
Other	If failed, a C++ exception will be thrown.

set_dtype

Description

Set the data type during AI2D computation.

Syntax

set_dtype(src_format, dst_format, src_type, dst_type)

Parameters

Name	Type	Description
src_format	ai2d_format	Input data format
dst_format	ai2d_format	Output data format
src_type	datatype_t	Input data type
dst_type	datatype_t	Output data type

set_crop_param

Description

Configure crop-related parameters.

Syntax

set_crop_param(crop_flag, start_x, start_y, width, height)

Parameters

Name	Type	Description
crop_flag	bool	Whether to enable the crop function
start_x	int	Starting pixel in the width direction
start_y	int	Starting pixel in the height direction
width	int	Width
height	int	Height

set_shift_param

【Description】

Used to configure shift-related parameters.

【Definition】

set_shift_param(shift_flag, shift_val)

【Parameters】

Name	Type	Description
shift_flag	bool	Whether to enable the shift function
shift_val	int	Number of bits for right shift

set_pad_param

【Description】

Used to configure pad-related parameters.

【Definition】

set_pad_param(pad_flag, paddings, pad_mode, pad_val)

【Parameters】

Name	Type	Description
pad_flag	bool	Whether to enable the pad function
paddings	list	Padding for each dimension, size=8, representing the number of front and back paddings for dim0 to dim4 respectively, where dim0/dim1 are fixed at {0, 0}
pad_mode	int	Only supports pad constant, configure 0
pad_val	list	Padding value for each channel

set_resize_param

【Description】

Used to configure resize-related parameters.

【Definition】

set_resize_param(resize_flag, interp_method, interp_mode)

【Parameters】

Name	Type	Description
resize_flag	bool	Whether to enable the resize function
interp_method	ai2d_interp_method	Resize interpolation method
interp_mode	ai2d_interp_mode	Resize mode

set_affine_param

【Description】

Used to configure affine-related parameters.

【Definition】

set_affine_param(affine_flag, interp_method, cord_round, bound_ind, bound_val, bound_smooth, M)

【Parameters】

Name	Type	Description
affine_flag	bool	Whether to enable the affine function
interp_method	ai2d_interp_method	Interpolation method used by Affine
cord_round	uint32_t	Integer boundary 0 or 1
bound_ind	uint32_t	Boundary pixel mode 0 or 1
bound_val	uint32_t	Boundary fill value
bound_smooth	uint32_t	Boundary smoothing 0 or 1
M	list	Vector corresponding to the affine transformation matrix. The affine transformation is \(Y=\[a_0, a_1; a_2, a_3\] \cdot X + \[b_0, b_1\] \), then M=[a_0,a_1,b_0,a_2,a_3,b_1 ]

shrink_memory_pool

【Description】

Cleans up the memory pool generated by nncase_runtime and releases memory.

【Syntax】

import gc
import nncase_runtime

gc.collect()
nncase_runtime.shrink_memory_pool()

Reference Example

import os
import ujson
import nncase_runtime as nn
import ulab.numpy as np
import image
import sys

# 加载kmodel
kmodel_path="/sdcard/examples/kmodel/face_detection_320.kmodel"
kpu=nn.kpu()
kpu.load_kmodel(kmodel_path)

# 读取一张图片，并将其transpose成chw数据
img_path="/sdcard/examples/utils/db_img/id_1.jpg"
img_data = image.Image(img_path).to_rgb888()
img_hwc=img_data.to_numpy_ref()
shape=img_hwc.shape
img_tmp = img_hwc.reshape((shape[0] * shape[1], shape[2]))
img_tmp_trans = img_tmp.transpose().copy()
img_chw=img_tmp_trans.reshape((shape[2],shape[0],shape[1]))

# 初始化ai2d预处理，并运行ai2d预处理，输出模型输入分辨率320*320
ai2d=nn.ai2d()
output_data = np.ones((1,3,320,320),dtype=np.uint8)
ai2d_output_tensor = nn.from_numpy(output_data)
ai2d.set_dtype(nn.ai2d_format.NCHW_FMT, nn.ai2d_format.NCHW_FMT, np.uint8, np.uint8)
ai2d.set_resize_param(True,nn.interp_method.tf_bilinear, nn.interp_mode.half_pixel)
ai2d_builder = ai2d.build([1,3,img_chw.shape[1],img_chw.shape[2]], [1,3,320,320])
ai2d_input_tensor = nn.from_numpy(img_chw)
ai2d_builder.run(ai2d_input_tensor, ai2d_output_tensor)

# kmodel运行，并获取输出，输出从tensor转成ulab.numpy.ndarray数据
kpu.set_input_tensor(0, ai2d_output_tensor)
kpu.run()
for i in range(kpu.outputs_size()):
    output_data = kpu.get_output_tensor(i)
    result = output_data.to_numpy()
    print(result.shape)

Conclusion

By using the nncase_runtime module, developers can implement efficient deep learning inference and image processing functions on the CanMV platform. This manual provides detailed descriptions of the API and usage examples, aiming to help developers get started quickly and efficiently develop related applications.