AI2D Application Guide#
Overview#
AI2D provides 5 hardware-accelerated preprocessing methods, including Crop/Shift/Resize/Pad/Affine. For example, Pad and Resize can be used to implement the letterbox operation commonly used in object detection tasks. AI2D is mainly used when writing deployment code for development boards, and provides a C++ version of the API. The sample source code given below is located in the k230_linux_sdk/buildroot-overlay/package/usage_ai2d directory. Execute build_app.sh to compile, and the build artifacts are in the k230_bin directory. Copy the artifacts to the development board to run them. Please pay attention to the usage notes for AI2D.
Note:
Affine and Resize functions are mutually exclusive and cannot be enabled simultaneously;
The input format of the Shift function can only be Raw16;
The Pad value is configured by channel, and the number of corresponding list elements must be equal to the number of channels;
In the current version, when only one AI2D function is needed, other parameters also need to be configured, just set the flag to false, and other fields do not need to be configured;
When multiple functions are configured, the execution order is Crop->Shift->Resize/Affine->Pad. Pay attention to matching when configuring parameters.
When using ai2d under linux, a kmodel needs to be loaded to complete part of the initialization operation.
Preprocessing Methods#
Resize Method#
The Resize method is a widely used operation in image preprocessing, mainly used to change the size of an image. Whether enlarging or shrinking an image, it can be achieved through this method. The source code is located in the k230_linux_sdk/buildroot-overlay/package/usage_ai2d/test_resize directory. The process of implementing Resize is as follows:
(from pictures/camera)") -->SetInput("Initialize AI2D input tensor")-->SetOutput("Initialize AI2D output tensor
based on the preprocessed shape")-->SetParam("Configure Resize parameters
ai2d_resize_param_t")-->InitBuilder("Construct ai2d_builder instance
Execute build_schedule")-->Run("Call invoke interface to run the configured preprocessing method")-->GetOutput("Get output data from the configured AI2D output tensor");
Here is an example code for using AI2D to implement the Resize process, which resizes the read-in image to a resolution of 640*320.
int main(int argc, char *argv[])
{
std::cout << "case " << argv[0] << " build " << __DATE__ << " " << __TIME__ << std::endl;
if (argc < 3)
{
std::cerr << "Usage: " << argv[0] << "<image> <debug_mode>" << std::endl;
return -1;
}
int debug_mode=atoi(argv[2]);
// 读入图片，并将数据处理成CHW和RGB格式
cv::Mat ori_img = cv::imread(argv[1]);
int ori_w = ori_img.cols;
int ori_h = ori_img.rows;
std::vector<uint8_t> chw_vec;
std::vector<cv::Mat> bgrChannels(3);
cv::split(ori_img, bgrChannels);
for (auto i = 2; i > -1; i--)
{
std::vector<uint8_t> data = std::vector<uint8_t>(bgrChannels[i].reshape(1, 1));
chw_vec.insert(chw_vec.end(), data.begin(), data.end());
}
// 创建AI2D输入tensor，并将CHW_RGB数据拷贝到tensor中，并回写到DDR
dims_t ai2d_in_shape{1, 3, ori_h, ori_w};
runtime_tensor ai2d_in_tensor = host_runtime_tensor::create(typecode_t::dt_uint8, ai2d_in_shape, hrt::pool_shared).expect("cannot create input tensor");
auto input_buf = ai2d_in_tensor.impl()->to_host().unwrap()->buffer().as_host().unwrap().map(map_access_::map_write).unwrap().buffer();
memcpy(reinterpret_cast<char *>(input_buf.data()), chw_vec.data(), chw_vec.size());
hrt::sync(ai2d_in_tensor, sync_op_t::sync_write_back, true).expect("write back input failed");
// resize参数
int out_w=640;
int out_h=320;
int size=out_w*out_h;
// 创建AI2D输出tensor
dims_t ai2d_out_shape{1, 3, out_h, out_w};
runtime_tensor ai2d_out_tensor = host_runtime_tensor::create(typecode_t::dt_uint8, ai2d_out_shape, hrt::pool_shared).expect("cannot create input tensor");
// 设置AI2D参数，AI2D支持5种预处理方法，crop/shift/pad/resize/affine。这里开启resize
ai2d_datatype_t ai2d_dtype{ai2d_format::NCHW_FMT, ai2d_format::NCHW_FMT, ai2d_in_tensor.datatype(), ai2d_out_tensor.datatype()};
ai2d_crop_param_t crop_param{false, 0, 0, 0, 0};
ai2d_shift_param_t shift_param{false, 0};
ai2d_pad_param_t pad_param{false, {{0, 0}, {0, 0}, {0, 0}, {0, 0}}, ai2d_pad_mode::constant, {0,0,0}};
ai2d_resize_param_t resize_param{true, ai2d_interp_method::tf_bilinear, ai2d_interp_mode::half_pixel};
ai2d_affine_param_t affine_param{false, ai2d_interp_method::cv2_bilinear, 0, 0, 127, 1, {0.5, 0.1, 0.0, 0.1, 0.5, 0.0}};
// 构造ai2d_builder
ai2d_builder builder(ai2d_in_shape, ai2d_out_shape, ai2d_dtype, crop_param, shift_param, pad_param, resize_param, affine_param);
builder.build_schedule();
// 执行ai2d，实现从ai2d_in_tensor->ai2d_out_tensor的预处理过程
builder.invoke(ai2d_in_tensor,ai2d_out_tensor).expect("error occurred in ai2d running");
//获取处理结果，并将结果存成图片
auto output_buf = ai2d_out_tensor.impl()->to_host().unwrap()->buffer().as_host().unwrap().map(map_access_::map_write).unwrap().buffer();
cv::Mat image_r = cv::Mat(out_h, out_w, CV_8UC1, output_buf.data());
cv::Mat image_g = cv::Mat(out_h, out_w, CV_8UC1, output_buf.data()+size);
cv::Mat image_b = cv::Mat(out_h, out_w, CV_8UC1, output_buf.data()+2*size);
std::vector<cv::Mat> color_vec(3);
color_vec.clear();
color_vec.push_back(image_b);
color_vec.push_back(image_g);
color_vec.push_back(image_r);
cv::Mat color_img;
cv::merge(color_vec, color_img);
cv::imwrite("test_resize.jpg", color_img);
return 0;
}
After compilation, run on the development board:
./test_resize.elf test.jpg 2
The comparison between the original image and the preprocessed image after execution is as follows:
Crop Method#
The Crop method is an operation used to extract (crop) a region of interest (ROI) from the original image. It can select a portion of the image as a new image based on specified coordinates and dimensions. The source code is located in k230_linux_sdk/buildroot-overlay/package/usage_ai2d/test_crop. The process of implementing Crop is as follows:
(from pictures/camera)") -->SetInput("Initialize AI2D input tensor")-->SetOutput("Initialize AI2D output tensor
based on the preprocessed shape")-->SetParam("Configure Crop parameters
ai2d_crop_param_t")-->InitBuilder("Construct ai2d_builder instance
Execute build_schedule")-->Run("Call invoke interface to run the configured preprocessing method")-->GetOutput("Get output data from the configured AI2D output tensor");
Here is an example code for using AI2D to implement the Crop process, which crops the read-in image at position [10,10] to a resolution of [400,400].
int main(int argc, char *argv[])
{
std::cout << "case " << argv[0] << " build " << __DATE__ << " " << __TIME__ << std::endl;
if (argc < 3)
{
std::cerr << "Usage: " << argv[0] << "<image> <debug_mode>" << std::endl;
return -1;
}
int debug_mode=atoi(argv[2]);
// 读入图片，并将数据处理成CHW和RGB格式
cv::Mat ori_img = cv::imread(argv[1]);
int ori_w = ori_img.cols;
int ori_h = ori_img.rows;
std::vector<uint8_t> chw_vec;
std::vector<cv::Mat> bgrChannels(3);
cv::split(ori_img, bgrChannels);
for (auto i = 2; i > -1; i--)
{
std::vector<uint8_t> data = std::vector<uint8_t>(bgrChannels[i].reshape(1, 1));
chw_vec.insert(chw_vec.end(), data.begin(), data.end());
}
// 创建AI2D输入tensor，并将CHW_RGB数据拷贝到tensor中，并回写到DDR
dims_t ai2d_in_shape{1, 3, ori_h, ori_w};
runtime_tensor ai2d_in_tensor = host_runtime_tensor::create(typecode_t::dt_uint8, ai2d_in_shape, hrt::pool_shared).expect("cannot create input tensor");
auto input_buf = ai2d_in_tensor.impl()->to_host().unwrap()->buffer().as_host().unwrap().map(map_access_::map_write).unwrap().buffer();
memcpy(reinterpret_cast<char *>(input_buf.data()), chw_vec.data(), chw_vec.size());
hrt::sync(ai2d_in_tensor, sync_op_t::sync_write_back, true).expect("write back input failed");
// crop参数
int crop_x=10;
int crop_y=10;
int crop_w=400;
int crop_h=400;
// 创建AI2D输出tensor
dims_t ai2d_out_shape{1, 3,crop_h, crop_w};
runtime_tensor ai2d_out_tensor = host_runtime_tensor::create(typecode_t::dt_uint8, ai2d_out_shape, hrt::pool_shared).expect("cannot create input tensor");
// 设置AI2D参数，AI2D支持5种预处理方法，crop/shift/pad/resize/affine。这里开启crop
ai2d_datatype_t ai2d_dtype{ai2d_format::NCHW_FMT, ai2d_format::NCHW_FMT, ai2d_in_tensor.datatype(), ai2d_out_tensor.datatype()};
ai2d_crop_param_t crop_param{true, crop_x, crop_y, crop_w, crop_h};
ai2d_shift_param_t shift_param{false, 0};
ai2d_pad_param_t pad_param{false, {{0, 0}, {0, 0}, {0, 0}, {0, 0}}, ai2d_pad_mode::constant, {114, 114, 114}};
ai2d_resize_param_t resize_param{false, ai2d_interp_method::tf_bilinear, ai2d_interp_mode::half_pixel};
ai2d_affine_param_t affine_param{false, ai2d_interp_method::cv2_bilinear, 0, 0, 127, 1, {0.5, 0.1, 0.0, 0.1, 0.5, 0.0}};
// 构造ai2d_builder
ai2d_builder builder(ai2d_in_shape, ai2d_out_shape, ai2d_dtype, crop_param, shift_param, pad_param, resize_param, affine_param);
builder.build_schedule();
// 执行ai2d，实现从ai2d_in_tensor->ai2d_out_tensor的预处理过程
builder.invoke(ai2d_in_tensor,ai2d_out_tensor).expect("error occurred in ai2d running");
//获取处理结果，并将结果存成图片
auto output_buf = ai2d_out_tensor.impl()->to_host().unwrap()->buffer().as_host().unwrap().map(map_access_::map_write).unwrap().buffer();
cv::Mat image_r = cv::Mat(crop_h, crop_w, CV_8UC1, output_buf.data());
cv::Mat image_g = cv::Mat(crop_h, crop_w, CV_8UC1, output_buf.data()+crop_h*crop_w);
cv::Mat image_b = cv::Mat(crop_h, crop_w, CV_8UC1, output_buf.data()+2*crop_h*crop_w);
std::vector<cv::Mat> color_vec(3);
color_vec.clear();
color_vec.push_back(image_b);
color_vec.push_back(image_g);
color_vec.push_back(image_r);
cv::Mat color_img;
cv::merge(color_vec, color_img);
cv::imwrite("test_crop.jpg", color_img);
return 0;
}
After compilation, run on the development board:
./test_crop.elf test.jpg 2
The comparison between the original image and the preprocessed image after execution is as follows:
Pad Method#
The Pad (filling) method is a technique used in the image preprocessing stage to fill the edges of an image. It changes the size of the image by adding pixel values around the image (top, bottom, left, right). These added pixel values can be customized. The source code is located in k230_linux_sdk/buildroot-overlay/package/usage_ai2d/test_pad. The process of implementing Pad is as follows:
(from pictures/camera)") -->SetInput("Initialize AI2D input tensor")-->SetOutput("Initialize AI2D output tensor
based on the preprocessed shape")-->SetParam("Configure Pad parameters
ai2d_pad_param_t")-->InitBuilder("Construct ai2d_builder instance
Execute build_schedule")-->Run("Call invoke interface to run the configured preprocessing method")-->GetOutput("Get output data from the configured AI2D output tensor");
Here is an example code for using AI2D to implement the Pad process, which pads the read-in image with 100, 100, 200, and 200 pixels on the top, bottom, left, and right respectively.
int main(int argc, char *argv[])
{
std::cout << "case " << argv[0] << " build " << __DATE__ << " " << __TIME__ << std::endl;
if (argc < 3)
{
std::cerr << "Usage: " << argv[0] << "<image> <debug_mode>" << std::endl;
return -1;
}
int debug_mode=atoi(argv[2]);
// 读入图片，并将数据处理成CHW和RGB格式
cv::Mat ori_img = cv::imread(argv[1]);
int ori_w = ori_img.cols;
int ori_h = ori_img.rows;
std::vector<uint8_t> chw_vec;
std::vector<cv::Mat> bgrChannels(3);
cv::split(ori_img, bgrChannels);
for (auto i = 2; i > -1; i--)
{
std::vector<uint8_t> data = std::vector<uint8_t>(bgrChannels[i].reshape(1, 1));
chw_vec.insert(chw_vec.end(), data.begin(), data.end());
}
// 创建AI2D输入tensor，并将CHW_RGB数据拷贝到tensor中，并回写到DDR
dims_t ai2d_in_shape{1, 3, ori_h, ori_w};
runtime_tensor ai2d_in_tensor = host_runtime_tensor::create(typecode_t::dt_uint8, ai2d_in_shape, hrt::pool_shared).expect("cannot create input tensor");
auto input_buf = ai2d_in_tensor.impl()->to_host().unwrap()->buffer().as_host().unwrap().map(map_access_::map_write).unwrap().buffer();
memcpy(reinterpret_cast<char *>(input_buf.data()), chw_vec.data(), chw_vec.size());
hrt::sync(ai2d_in_tensor, sync_op_t::sync_write_back, true).expect("write back input failed");
// padding参数
int pad_top=100;
int pad_bottom=100;
int pad_left=200;
int pad_right=200;
std::vector<int> pad_val={114,114,114};
int out_w=ori_w+pad_left+pad_right;
int out_h=ori_h+pad_top+pad_bottom;
int size=(ori_w+pad_left+pad_right)*(ori_h+pad_top+pad_bottom);
// 创建AI2D输出tensor
dims_t ai2d_out_shape{1, 3, out_h, out_w};
runtime_tensor ai2d_out_tensor = host_runtime_tensor::create(typecode_t::dt_uint8, ai2d_out_shape, hrt::pool_shared).expect("cannot create input tensor");
// 设置AI2D参数，AI2D支持5种预处理方法，crop/shift/pad/resize/affine。这里开启pad
ai2d_datatype_t ai2d_dtype{ai2d_format::NCHW_FMT, ai2d_format::NCHW_FMT, ai2d_in_tensor.datatype(), ai2d_out_tensor.datatype()};
ai2d_crop_param_t crop_param{false, 0, 0, 0, 0};
ai2d_shift_param_t shift_param{false, 0};
ai2d_pad_param_t pad_param{true, {{0, 0}, {0, 0}, {pad_top, pad_bottom}, {pad_left, pad_right}}, ai2d_pad_mode::constant, pad_val};
ai2d_resize_param_t resize_param{false, ai2d_interp_method::tf_bilinear, ai2d_interp_mode::half_pixel};
ai2d_affine_param_t affine_param{false, ai2d_interp_method::cv2_bilinear, 0, 0, 127, 1, {0.5, 0.1, 0.0, 0.1, 0.5, 0.0}};
// 构造ai2d_builder
ai2d_builder builder(ai2d_in_shape, ai2d_out_shape, ai2d_dtype, crop_param, shift_param, pad_param, resize_param, affine_param);
builder.build_schedule();
// 执行ai2d，实现从ai2d_in_tensor->ai2d_out_tensor的预处理过程
builder.invoke(ai2d_in_tensor,ai2d_out_tensor).expect("error occurred in ai2d running");
//获取处理结果，并将结果存成图片
auto output_buf = ai2d_out_tensor.impl()->to_host().unwrap()->buffer().as_host().unwrap().map(map_access_::map_write).unwrap().buffer();
cv::Mat image_r = cv::Mat(out_h, out_w, CV_8UC1, output_buf.data());
cv::Mat image_g = cv::Mat(out_h, out_w, CV_8UC1, output_buf.data()+size);
cv::Mat image_b = cv::Mat(out_h, out_w, CV_8UC1, output_buf.data()+2*size);
std::vector<cv::Mat> color_vec(3);
color_vec.clear();
color_vec.push_back(image_b);
color_vec.push_back(image_g);
color_vec.push_back(image_r);
cv::Mat color_img;
cv::merge(color_vec, color_img);
cv::imwrite("test_pad.jpg", color_img);
return 0;
}
After compilation, run on the development board:
./test_pad.elf test.jpg 2
The comparison between the original image and the preprocessed image after execution is as follows:
Affine Method#
The Affine (affine transformation) method is a technique used in image preprocessing to perform geometric transformations on an image. It can implement various geometric transformation operations such as rotation, translation, and scaling of an image, and can maintain the “straightness” of the image (i.e., straight lines remain straight after transformation) and “parallelism” (i.e., parallel lines remain parallel after transformation). The process of implementing Affine is as follows:
(from pictures/camera)") -->SetInput("Initialize AI2D input tensor")-->SetOutput("Initialize AI2D output tensor
based on the preprocessed shape")-->SetParam("Configure Affine parameters
ai2d_affine_param_t")-->InitBuilder("Construct ai2d_builder instance
Execute build_schedule")-->Run("Call invoke interface to run the configured preprocessing method")-->GetOutput("Get output data from the configured AI2D output tensor");
Here is an example code for using AI2D to implement the Affine process, which scales the read-in image by 0.5 times and translates it by 200 pixels in the x and y directions respectively.
int main(int argc, char *argv[])
{
std::cout << "case " << argv[0] << " build " << __DATE__ << " " << __TIME__ << std::endl;
if (argc < 3)
{
std::cerr << "Usage: " << argv[0] << "<image> <debug_mode>" << std::endl;
return -1;
}
int debug_mode=atoi(argv[2]);
// 读入图片，并将数据处理成CHW和RGB格式
cv::Mat ori_img = cv::imread(argv[1]);
int ori_w = ori_img.cols;
int ori_h = ori_img.rows;
std::vector<uint8_t> chw_vec;
std::vector<cv::Mat> bgrChannels(3);
cv::split(ori_img, bgrChannels);
for (auto i = 2; i > -1; i--)
{
std::vector<uint8_t> data = std::vector<uint8_t>(bgrChannels[i].reshape(1, 1));
chw_vec.insert(chw_vec.end(), data.begin(), data.end());
}
// 创建AI2D输入tensor，并将CHW_RGB数据拷贝到tensor中，并回写到DDR
dims_t ai2d_in_shape{1, 3, ori_h, ori_w};
runtime_tensor ai2d_in_tensor = host_runtime_tensor::create(typecode_t::dt_uint8, ai2d_in_shape, hrt::pool_shared).expect("cannot create input tensor");
auto input_buf = ai2d_in_tensor.impl()->to_host().unwrap()->buffer().as_host().unwrap().map(map_access_::map_write).unwrap().buffer();
memcpy(reinterpret_cast<char *>(input_buf.data()), chw_vec.data(), chw_vec.size());
hrt::sync(ai2d_in_tensor, sync_op_t::sync_write_back, true).expect("write back input failed");
// affine参数,创建仿射变换矩阵，缩放0.5倍，X/Y方向各平移200px
std::vector<float> affine_matrix = {0.5,0.0,200.0,
0.0,0.5,200.0};
int out_w=0.5*ori_w;
int out_h=0.5*ori_h;
int size=out_w*out_h;
// 创建AI2D输出tensor
dims_t ai2d_out_shape{1, 3, out_h, out_w};
runtime_tensor ai2d_out_tensor = host_runtime_tensor::create(typecode_t::dt_uint8, ai2d_out_shape, hrt::pool_shared).expect("cannot create input tensor");
// 设置AI2D参数，AI2D支持5种预处理方法，crop/shift/pad/resize/affine。这里开启affine
ai2d_datatype_t ai2d_dtype{ai2d_format::NCHW_FMT, ai2d_format::NCHW_FMT, ai2d_in_tensor.datatype(), ai2d_out_tensor.datatype()};
ai2d_crop_param_t crop_param{false, 0, 0, 0, 0};
ai2d_shift_param_t shift_param{false, 0};
ai2d_pad_param_t pad_param{false, {{0, 0}, {0, 0}, {0, 0}, {0, 0}}, ai2d_pad_mode::constant, {0,0,0}};
ai2d_resize_param_t resize_param{false, ai2d_interp_method::tf_bilinear, ai2d_interp_mode::half_pixel};
ai2d_affine_param_t affine_param{true, ai2d_interp_method::cv2_bilinear, 0, 0, 127, 1, affine_matrix};
// 构造ai2d_builder
ai2d_builder builder(ai2d_in_shape, ai2d_out_shape, ai2d_dtype, crop_param, shift_param, pad_param, resize_param, affine_param);
builder.build_schedule();
// 执行ai2d，实现从ai2d_in_tensor->ai2d_out_tensor的预处理过程
builder.invoke(ai2d_in_tensor,ai2d_out_tensor).expect("error occurred in ai2d running");
//获取处理结果，并将结果存成图片
auto output_buf = ai2d_out_tensor.impl()->to_host().unwrap()->buffer().as_host().unwrap().map(map_access_::map_write).unwrap().buffer();
cv::Mat image_r = cv::Mat(out_h, out_w, CV_8UC1, output_buf.data());
cv::Mat image_g = cv::Mat(out_h, out_w, CV_8UC1, output_buf.data()+size);
cv::Mat image_b = cv::Mat(out_h, out_w, CV_8UC1, output_buf.data()+2*size);
std::vector<cv::Mat> color_vec(3);
color_vec.clear();
color_vec.push_back(image_b);
color_vec.push_back(image_g);
color_vec.push_back(image_r);
cv::Mat color_img;
cv::merge(color_vec, color_img);
cv::imwrite("test_affine.jpg", color_img);
return 0;
}
After compilation, run on the development board:
./test_affine.elf test.jpg 2
The comparison between the original image and the preprocessed image after execution is as follows:
Shift Method#
The Shift method is a bitwise right-shift method in data preprocessing. Each right shift by one bit makes the original data become 1/2 of the original. It should be noted that the input data format of Shift must be RAW16. The process of implementing Affine is as follows:
(from pictures/camera)") -->SetInput("Initialize AI2D input tensor")-->SetOutput("Initialize AI2D output tensor
based on the preprocessed shape")-->SetParam("Configure Shift parameters
ai2d_shift_param_t")-->InitBuilder("Construct ai2d_builder instance
Execute build_schedule")-->Run("Call invoke interface to run the configured preprocessing method")-->GetOutput("Get output data from the configured AI2D output tensor");
Here is an example code for using AI2D to implement the Shift process. Here, a piece of data all set to 240 is created, and through Shift right-shifting by one bit, all the data becomes 120.
int main(int argc, char *argv[])
{
std::cout << "case " << argv[0] << " build " << __DATE__ << " " << __TIME__ << std::endl;
if (argc < 2)
{
std::cerr << "Usage: " << argv[0] << "<debug_mode:0,1,2>" << std::endl;
return -1;
}
int debug_mode=atoi(argv[1]);
// 创建一张位深为16的原始数据，初始化为240
cv::Mat ori_img(320, 320, CV_16UC3, cv::Scalar(240, 240, 240));
cv::imwrite("ori_img.jpg",ori_img);
//hwc,bgr
int ori_w = ori_img.cols;
int ori_h = ori_img.rows;
// 创建AI2D输入tensor
dims_t ai2d_in_shape{1,ori_h, ori_w,3};
runtime_tensor ai2d_in_tensor = host_runtime_tensor::create(typecode_t::dt_uint16, ai2d_in_shape, hrt::pool_shared).expect("cannot create input tensor");
auto input_buf = ai2d_in_tensor.impl()->to_host().unwrap()->buffer().as_host().unwrap().map(map_access_::map_write).unwrap().buffer();
memcpy(reinterpret_cast<uint16_t *>(input_buf.data()), ori_img.data , ori_h*ori_w*3*sizeof(uint16_t));
hrt::sync(ai2d_in_tensor, sync_op_t::sync_write_back, true).expect("write back input failed");
int out_w=ori_w;
int out_h=ori_h;
// 创建AI2D输出tensor
dims_t ai2d_out_shape{1,out_h, out_w,3};
runtime_tensor ai2d_out_tensor = host_runtime_tensor::create(typecode_t::dt_uint16, ai2d_out_shape, hrt::pool_shared).expect("cannot create input tensor");
// 设置AI2D参数，AI2D支持5种预处理方法，crop/shift/pad/resize/affine。这里开启shift,右移1位，数据变为原来的1/2
ai2d_datatype_t ai2d_dtype{ai2d_format::RAW16, ai2d_format::RAW16, ai2d_in_tensor.datatype(), ai2d_out_tensor.datatype()};
ai2d_crop_param_t crop_param{false, 0, 0, 0, 0};
ai2d_shift_param_t shift_param{true, 1};
ai2d_pad_param_t pad_param{false, {{0, 0}, {0, 0}, {0, 0}, {0, 0}}, ai2d_pad_mode::constant, {0,0,0}};
ai2d_resize_param_t resize_param{false, ai2d_interp_method::tf_bilinear, ai2d_interp_mode::half_pixel};
ai2d_affine_param_t affine_param{false, ai2d_interp_method::cv2_bilinear, 0, 0, 127, 1, {0.5, 0.1, 0.0, 0.1, 0.5, 0.0}};
// 构造ai2d_builder
ai2d_builder builder(ai2d_in_shape, ai2d_out_shape, ai2d_dtype, crop_param, shift_param, pad_param, resize_param, affine_param);
builder.build_schedule();
// 执行ai2d，实现从ai2d_in_tensor->ai2d_out_tensor的预处理过程
builder.invoke(ai2d_in_tensor,ai2d_out_tensor).expect("error occurred in ai2d running");
//获取处理结果，并将结果存成图片
auto output_buf = ai2d_out_tensor.impl()->to_host().unwrap()->buffer().as_host().unwrap().map(map_access_::map_write).unwrap().buffer();
cv::Mat image_r = cv::Mat(out_h, out_w, CV_16UC3, output_buf.data());
cv::imwrite("test_shift.jpg", image_r);
return 0;
}
After compilation, run on the development board:
./test_shift.elf 2
The comparison between the original image and the preprocessed image after execution is as follows: