Note

This is the documentation for the latest development branch and may refer to features that are not available in released versions. If you are looking for the documentation for a specific release, use the drop-down menu on the left and select the desired version.

K230 linux 2.5D GPU Usage Guide

Concept Introduction

Vector Graphics Basics

Vector graphics are images represented by geometric primitives such as points, lines, or polygons based on mathematical equations. Unlike bitmaps that represent images using pixels, vector graphics can be scaled infinitely without distortion. SVG is a typical vector graphics format, which is itself an XML text file describing the positions of various primitives. You can view the rendered effect by opening it in a browser. If you are completely unfamiliar with the VGLite API used by the K230 GPU, you can think of it as a simplified version of SVG.

K230 GPU supports multiple 2D primitives

• Lines

• Quadratic Bezier curves

• Cubic Bezier curves

• Circular curves (of course, cubic Bezier curves can also be used for approximation)

Note: These shapes are all lines; the GPU cannot directly draw lines, it can only draw closed shapes enclosed by these lines.

GPU Basics

On Linux, the GPU is interacted with primarily through the VGLite API. VGLite internally maintains a GPU command queue. When drawing needs to be completed, or when the queue is full, commands are submitted to the GPU hardware for rendering. The default length of the command queue is 65536, which can be modified by calling the vg_lite_set_command_buffer_size function.

Note: The VGLite API does not support use in multi-threaded contexts. If your application uses multiple threads, please ensure that only one thread uses the VGLite API.

The K230 GPU is a memory-to-memory device and does not have display output capability. If display is needed, it can be used in conjunction with DRM.

Using VGLite API

Development Environment Preparation

The VGLite API mainly includes the header file vg_lite.h and the library file libvg_lite.so. During compilation, use the linker parameter -lvg_lite. If you need to use other features such as saving images as png, you will also need vg_lite_util, and you can use the vg_lite_util.h header file and the libvg_lite_util.so library file.

Display

The K230 CanMV has an HDMI interface. On the little core Linux, DRM can be used for display. Letting the GPU driver load the DRM dumb buffer can reduce memory copies and achieve efficient rendering. For relevant code about GPU+DRM, you can refer to the vglite_drm demo.

It should be noted that DRM’s color format names are not completely consistent with vg_lite_buffer_format_t. For example, VGLITE_BGRA8888 represents a 32-bit color where red is in the lower 8 bits and alpha is in the upper 8 bits, corresponding to DRM_FORMAT_ARGB8888 in DRM.

As shown in the figure, the colors correctly displayed on the screen after vglite_drm runs: R(255)G(128)B(16)

Generally speaking, to achieve synchronous display, two buffers are required for ping-pong alternating display. However, to simplify the demonstration code, only one buffer is used here. Readers can refer to vglite_cube to implement a vertical-sync double buffer for continuous rendering on their own.

Drawing

Some Preparation

First, you need to initialize VGLite by calling vg_lite_init, which has two parameters tessellation_width and tessellation_height, used for the size of the rendering window. The larger the size, the higher the efficiency. If set to 0, the vector drawing function is not used and only BLIT is available. Typically, it is set to the size of the largest buffer.

Rendering requires a buffer, which can be imported from a DRM dumb, like this:

vg_lite_buffer_t buffer;
int buf_fd;
memset(&buffer, 0, sizeof(buffer));
buf_fd = drm_get_dmabuf_fd(0);
if (buf_fd < 0) {
    perror("get fd");
    return buf_fd;
}
memset(&buffer, 0, sizeof(buffer));
buffer.width = width;
buffer.height = height;
buffer.format = VG_LITE_ARGB8888;
buffer.stride = buffer.width * 4;
buffer.memory = buffer.memory = drm_get_map(0);
if (vg_lite_map(&buffer, VG_LITE_MAP_DMABUF, buf_fd)) {
    perror("import dma-buf");
    return -1;
}

You can also allocate an offscreen buffer from the GPU driver, like this:

vg_lite_buffer_t buffer;
memset(&buffer, 0, sizeof(buffer));
buffer.width = width;
buffer.height = height;
buffer.format = VG_LITE_ARGB8888;
if (vg_lite_allocate(&buffer)) {
    return -1;
}

Obviously, allocating an offscreen buffer is simpler; you only need to configure the resolution and pixel format. Importing from a DRM dumb requires you to calculate the stride (the number of bytes in one row of pixels) yourself. Of course, the advantage of importing from a DRM dumb is that it can be directly used for display.

Polygons

With a buffer, you can start drawing. A polygon consists of multiple straight lines. Taking a triangle as an example, you first need to determine the coordinates of the triangle’s three vertices, such as (0,0) (0,1) (1,0). The entire process is as follows:

1. Move the pen to (0,0)

2. Draw a line to (0,1)

3. Draw a line to (1,0)

4. Draw a line to (0,0)

5. Close the shape

Consulting the K230 GPU API reference, you can see that the opcode for move is 2, the opcode for line is 4, and the opcode for closing the path is 0. Using the first data format, you can construct the path_data array as follows:

uint8_t path_data[] = {
    2, 0, 0, // 移动到 (0,0)
    4, 0, 1, // 直线到 (0,1)
    4, 1, 0, // 直线到 (1,0)
    4, 0, 0, // 直线到 (0,0)
0};

path_data alone is not enough; the parameter required for rendering is path. In addition to path_data, path also contains information such as the data format. The data format can be specified as one of the following:

1. 8-bit signed integer

2. 16-bit signed integer

3. 32-bit signed integer

4. 32-bit floating point

Performance decreases from top to bottom. However, even with 8-bit signed integers, it does not mean the pixel range is limited to -128 to 127, because matrix transformations are needed to calculate the final coordinates.

Now construct the path and draw it to the buffer:

vg_lite_path_t path = {
    .bounding_box = {0., 1., 1., 0.}, // 图形的包围盒
    .quality = VG_LITE_HIGH, // 渲染质量
    .format = VG_LITE_S8, // 考虑到坐标很简单，所以 8bit 足够
    .uploaded = 0, // 路径没有被上传过 GPU，所以用0
    .path_length = sizeof(path_data), // 路径数据长度，以字节为单位
    .path = path_data, // 路径数据就放在这了
    .path_changed = 1, // 用来表示路径被更新过
    .pdata_internal = 0 // 表示路径数据不是由驱动分配的
};

With the variables above, you can perform the rendering. The steps are as follows:

1. Clear the buffer, that is, fill it with a single color. You can use vg_lite_clear to do this.

2. A transformation matrix. For more information about matrices, you can refer to content related to affine transformations. Here, a scaling matrix is used directly to enlarge the image by 100 times, so that a line with a final length of 1 will use 100 pixels in the image.

3. Call vg_lite_draw to “render” the path to the buffer.

4. Finally, use vg_lite_finish to submit the rendering.

For the convenience of error handling, the CHECK_ERROR macro is used to wrap functions that return vg_lite_error_t:

vg_lite_matrix_t matrix;
CHECK_ERROR(vg_lite_clear(&buffer, NULL, 0xffff0000)); // 使用蓝色填充整个 buffer
vg_lite_identity(&matrix); // 初始化为单位矩阵
vg_lite_translate(buffer.width / 2., buffer.height / 2., &matrix); // 移动到 buffer 中间位置
vg_lite_scale(100., 100., &matrix); // x y 方向都放d大100倍
CHECK_ERROR(vg_lite_draw(
    &buffer, &path,
    VG_LITE_FILL_NON_ZERO, //  填充规则，像素只要被覆盖就会被绘制
    &matrix,
    VG_LITE_BLEND_NONE, // 颜色混合规则，None 表示忽略透明度直接覆盖
    0xff0000ff // RGBA 颜色，这个值表示不透明的红色
));
CHECK_ERROR(vg_lite_finish()); // 提交到 GPU

The complete reference code can be found in vglite_drm.

You can easily notice that the coordinate system uses right as the positive x direction and down as the positive y direction. This is also the coordinate system used by SVG.

It should be noted that the “rendering” mentioned earlier for vg_lite_draw is in quotes because the actual rendering does not happen; only rendering instructions are written. The final rendering requires calling vg_lite_finish. This is beneficial for performance. In actual use, you can call vg_lite_draw multiple times, and execute vg_lite_finish just before the actual display, because vg_lite_finish involves a system call, which has a certain overhead, while vg_lite_draw does not and can be executed quickly.

When rendering is complete, the result can be displayed on the screen or saved as an image. Note that when saving an image, the data is read using the CPU, so you need to ensure that vg_lite_buffer_t::memory is readable. If the reader uses the DRM code above to create vg_lite_buffer_t, it cannot be read without mapping the DRM dumb.

Curves

K230 GPU supports three types of curves:

1. Quadratic Bezier curves

2. Cubic Bezier curves

3. Elliptical curves

Of course, elliptical curves can be approximated by cubic Bezier curves, so they can essentially be regarded as the same type of curve. As in the case of drawing polygons, you only need to modify the opcodes and data.

Below, we try to change the base of the triangle we just drew to a quadratic Bezier curve, with the midpoint at (1,1), to draw a pattern similar to a rounded corner. Change the path_data above to:

uint8_t path_data[] = {
    2, 0, 0,
    4, 0, 1,
    6, 1, 1, 1, 0, // 二次贝塞尔曲线，控制点(1,1)，画到(1,0)
    4, 0, 0,
0};

Of course, to better observe this curve, we can increase the scaling factor, for example to 500 times, and also reduce the translation so that the pattern is approximately in the center of the screen:

vg_lite_translate(buffer.width / 2., buffer.height / 2., &matrix);
vg_lite_scale(500., 500., &matrix);

The final drawn pattern looks like this:

Bitmap Fill

When you are not satisfied with a single-color fill, you can use a bitmap to fill. The bitmap file will be rendered to the target location. Of course, the bitmap also needs to be a vg_lite_buffer_t. If you need to load from local JPEG/PNG files, it is recommended to use an offscreen buffer to store the pixel content, and use vg_lite_blit or vg_lite_draw_pattern for rendering.

Gradients

For the VGLite implementation, a gradient is itself a special bitmap fill. The linear_grad related functions allocate a 1x256 buffer for BLIT. Of course, users can ignore the details above and just use it. Refer to the linearGrad demo. The specific process can be divided into the following calls:

1. vg_lite_init_grad initializes a gradient

2. vg_lite_set_grad sets colors and stops, supporting up to 16 stops

3. vg_lite_update_grad updates the gradient

4. vg_lite_get_grad_matrix obtains a pointer to the gradient’s transformation matrix

5. Adjust the transformation matrix, such as rotation and scaling. The default length is 256 pixels from left to right. If you need a gradient in other directions, you need to use this matrix to operate

6. vg_lite_draw_gradient draws the gradient