YOLO Application Guide#

Overview#

K230 has encapsulated YOLOv5, YOLOv8, YOLO11, and YOLO26 in the YOLO model, supporting five types of tasks: classification (classify), detection (detect), segmentation (segment), rotated object detection (obb), and keypoint detection (pose). Users can flexibly use parameters to invoke different models and modify the input mode (video/image) according to scenario requirements; they can also freely modify the image resolution captured by the camera to tune the YOLO deployment.

Model Conversion#

For the process of training models and converting kmodel, please refer to: YOLO Battle, the models converted according to the linked documentation are all available within this documentation.

YOLO Support#

The YOLO code is located in the k230_linux_sdk/buildroot-overlay/package/yolo directory. The code encapsulates the inference part of the YOLO model. Users only need to call the interface to obtain inference frames and feed them to the YOLO series models.

Code Structure#

The existing code structure is as follows:

|yolo
├── src
│    ├── ai_base.cc
│    ├── ai_base.h
│    ├── main.cc
│    ├── scoped_timing.h
│    ├── sensor_buf_manager.cc
│    ├── sensor_buf_manager.h
│    ├── sensor_set.h
│    ├── utils.h
│    ├── utils.cc
│    ├── utils.h
│    ├── yolo26.cc
│    ├── yolo26.h
│    ├── yolo11.cc
│    ├── yolo11.h
│    ├── yolov5.cc
│    ├── yolov5.h
│    ├── yolov8.cc
│    └── yolov8.h
├── utils
├── CMakeLists.txt
└── build_app.sh

Code Description#

The code files are described below:

File Name	Function
ai_base.h	Provides interfaces used during model inference
ai_bash.cc	Provides the interface implementation of the model inference methods defined in ai_bash.h
scoped_timing.h	Provides timing utilities to help with development and debugging
sensor_buf_manager.h	Interface for managing tensors in AI inference channels
sensor_buf_manager.cc	Implementation of the interfaces encapsulated in sensor_buf_manager.h
sensor_set.h	Parameter definitions for AI inference channels, setting the width and height of camera output images
utils.h	Provides common utility function interfaces such as binary data reading, image saving, and preprocessing configuration
utils.cc	Provides the implementation of utility functions defined in utils.h
yolo26.h	Provides interfaces for initialization, preprocessing, inference, postprocessing, and result drawing of the yolo26 model
yolo26.cc	Provides the interface implementation of the yolo26 model
yolo11.h	Provides interfaces for initialization, preprocessing, inference, postprocessing, and result drawing of the yolo11 model
yolo11.cc	Provides the interface implementation of the yolo11 model
yolov5.h	Provides interfaces for initialization, preprocessing, inference, postprocessing, and result drawing of the yolov5 model
yolov5.cc	Provides the interface implementation of the yolov5 model
yolov8.h	Provides interfaces for initialization, preprocessing, inference, postprocessing, and result drawing of the yolov8 model
yolov8.cc	Provides the interface implementation of the yolov8 model
main.cc	Main function implementation, which performs inference for different tasks (classify/detect/segment/obb/pose), different models (yolov5/yolov8/yolo11/yolo26), and different modes (image/video) of the YOLO series models

Among them, the utils directory contains sample models and images used for board deployment, and build_app.sh is the compilation script.

Application Steps#

Firmware Compilation#

Execute make menuconfig under k230_linux_sdk, select Target packages > canaan package > AI > yolo demo, choose Save->OK below, save and exit. This way, when compiling the firmware, the yolo example can be compiled into the firmware. After flashing the firmware, you can find the compiled yolo.elf executable file and test files under /app/yolo.

Code Compilation#

If you do not compile during firmware compilation, you can also choose to compile the yolo example separately. Enter the k230_linux_sdk/buildroot-overlay/package/yolo directory, execute the ./build_app.sh script, and the yolo.elf executable file generated by the compilation is in the k230_bin directory. You can copy the k230_bin directory to a development board that has already been flashed with the firmware.

Running Parameters#

The following describes the parameters when running on the board:

Parameter Name	Default Value	Description
`-model_type`	yolov8	Set the model type, the default value is yolov8, optional values: yolov5/yolov8/yolo11.
`-task_type`	detect	Set the task type, the default value is detect, optional values: classify/detect/segment/obb.
`-task_mode`	video	Set the task mode, the default value is video, optional values: image/video
`-image_path`	test.jpg	Set the image path, the default value is test.jpg.
`-kmodel_path`	yolov8n.kmodel	Set the kmodel path, the default value is yolov8n.kmodel.
`-labels_txt_filepath`	coco_labels.txt	Set the label text file path, the default value is coco_labels.txt, each label occupies a separate line.
`-conf_thres`	0.35	Set the confidence threshold, the default value is 0.35.
`-nms_thres`	0.65	Set the non-maximum suppression threshold, the default value is 0.65.
`-mask_thres`	0.5	Set the mask threshold, the default value is 0.5.
`-kp_num`	17	Set the number of keypoints, the default value is 17 (human skeleton keypoint scenario).
`-kp_dim`	3	Set the model keypoint dimension, only 2/3 are supported, the default value is 3 (human skeleton keypoint scenario).
`-debug_mode`	0	Set the debug mode, the default value is 0, optional values: 0/1, 0 means no debug, 1 means debug print.

Running Example#

Insert the flashed TF card into the K230 development board, power it on, connect the development board using a serial port, and run the video inference command and image inference command respectively to see the inference results. You can execute yolo.elf -help to view the parameter configuration.

Video Inference

#You can execute: ./video_run.sh
./yolo.elf -model_type yolov8 -task_type detect -task_mode video -kmodel_path yolov8n.kmodel -labels_txt_filepath coco_labels.txt -conf_thres 0.35 -nms_thres 0.65 -mask_thres 0.5 -debug_mode 0

Image Inference

#You can execute: ./image_run.sh
./yolo.elf -model_type yolov8 -task_type detect -task_mode image -image_path test.jpg -kmodel_path yolov8n.kmodel -labels_txt_filepath coco_labels.txt -conf_thres 0.35 -nms_thres 0.65 -mask_thres 0.5 -debug_mode 0

During the deployment process, you can replace the model, task type, task mode, threshold parameters, etc. as needed, where each label in the label text file occupies a separate line.

Notes#

Currently supported models are yolov5, yolov8, yolo11 and yolo26.
Currently supported task types: yolov5 supports three tasks: classify, detect, and segment; yolov8, yolo11, and yolo26 support five tasks: classify, detect, segment, obb, and pose.
Currently supported task modes are video and image.
During the tuning process, you can first adjust the threshold for tuning, and then modify the model quantization method and input resolution for tuning.
If the AI frame resolution is set to the same value as the model input resolution, more optimized inference speed can be achieved. The AI frame resolution is defined in sensor_set.h.