Note

This is the documentation for the latest development branch and may refer to features that are not available in released versions. If you are looking for the documentation for a specific release, use the drop-down menu on the left and select the desired version.

nncase_runtime Module Usage Instructions

Overview

This document introduces the CanMV nncase_runtime module, guiding developers on using MicroPython to call the KPU and AI2D modules.

Feature Introduction

Importing Libraries

Before using the nncase_runtime module, you need to import the relevant libraries:

import nncase_runtime as nn
import ulab.numpy as np

KPU Initialization

Initialize the model inference module:

kpu = nn.kpu()

AI2D Initialization

Initialize the image processing module:

ai2d = nn.ai2d()

Loading Model

There are two ways to load a model: through a file path or binary data.

# File path
model = nn.load_model('test.kmodel')

# Binary data
with open("test.kmodel", "rb") as f:
    data = f.read()
    kpu.load_kmodel(data)

Using KPU Alone for Inference

Setting Model Input

Before model inference, you need to set the corresponding model input data:

data = np.zeros((1, 3, 320, 320), dtype=np.uint8)
kpu_input = nn.from_numpy(data)
kpu.set_input_tensor(0, kpu_input)

# Model has multiple inputs
kpu.set_input_tensor(1, kpu_input_1)
kpu.set_input_tensor(2, kpu_input_2)

Running Inference and Getting Results

Run inference and get the results:

kpu.run()

result = kpu.get_output_tensor(i)  # Returns the i-th output tensor
data = result.to_numpy()  # Convert output tensor to numpy object

Using AI2D+KPU for Inference

Use AI2D to preprocess data collected by the camera, then use KPU for inference. For configuration of input devices such as the camera, please refer to the AI Demo User Manual.

Configuring AI2D Parameters

The AI2D features include crop, shift, pad, resize, and affine. Configure the corresponding parameters according to your requirements. Unused features can be ignored.

# Basic configuration: input and output layout, input and output dtype
ai2d.set_dtype(nncase_runtime.ai2d_format.NCHW_FMT,
               nncase_runtime.ai2d_format.NCHW_FMT,
               np.uint8, np.uint8)

# Feature configuration, taking pad and resize as examples
ai2d.set_pad_param(True, [0, 0, 0, 0, 1, 1, 2, 2], 0, [127, 127, 127])
ai2d.set_resize_param(True, nn.interp_method.tf_bilinear, nn.interp_mode.half_pixel)

# Execution configuration, needs to configure input and output shape
ai2d_builder = ai2d.build([1, 3, 224, 224], [1, 3, 256, 256])

AI2D+KPU Inference

Perform inference combining AI2D and KPU:

data = np.zeros((1, 3, 224, 224), dtype=np.uint8)
ai2d_input = nn.from_numpy(data)

# Get the input tensor of the KPU
kpu_input = kpu.get_input_tensor(0)

# Set the KPU's input tensor as the output of AI2D
ai2d_builder.run(ai2d_input, kpu_input)
kpu.run()

# Get the output tensor of the KPU
result = kpu.get_output_tensor(i)  # Returns the i-th output tensor
data = result.to_numpy()  # Convert the output tensor to a numpy object

Free Memory

Ensure that before the program ends, the reference count of all global variables is 0 to avoid memory leaks. You can also call gc.collect() at the beginning of the program to release memory that has not been freed.

import nncase_runtime as nn
import gc

del kpu
del ai2d
del ai2d_builder

# tensor = nn.from_numpy()
del tensor

# input_tensor = kpu.get_input_tensor(i)
del input_tensor

# output_tensor = kpu.get_output_tensor(i)
del output_tensor

gc.collect()
nn.shrink_memory_pool()

Summary

This module provides the basic framework for deep learning inference using KPU and AI2D. Developers can configure models and parameters according to specific needs, perform image processing and inference tasks, and pay attention to memory management to improve program stability and performance.