Note

This is the documentation for the latest development branch and may refer to features that are not available in released versions. If you are looking for a specific release, use the drop-down menu on the left.

How to Configure the Sensor Driver

Overview

In the K230 RTOS SDK, whether the Sensor driver is built into the firmware, which CSI it is connected to, whether CSI0 uses 2-lane or 4-lane, whether the on-chip MCLK is used, and how ISP parameters are packaged and loaded are jointly determined by MPP Kconfig and the VICAP API.

The configuration entry point is uniformly accessed from the SDK root directory:

make menuconfig
# Docker workflow: k230 make menuconfig

Related code and configuration locations:

Content	Path
Sensor / CSI / ISP installation switches	src/rtsmart/mpp/Kconfig → MPP Configuration → Sensor Configuration
Sensor drivers and mode tables	src/rtsmart/mpp/kernel/sensor/
On-board list_sensor command	src/rtsmart/mpp/kernel/sensor/src/sensor_dev.c
Sensor type enumeration	src/rtsmart/mpp/include/comm/k_sensor_comm.h
Application-layer Sensor information table / detection matching	src/rtsmart/mpp/userapps/src/sensor/mpi_sensor.c (kd_mpi_sensor_adapt_get_ex)
ISP calibration / tuning source files	src/rtsmart/mpp/userapps/src/sensor/config/
ISP installation rules	src/rtsmart/mpp/userapps/src/sensor/Makefile
ISP loading and parsing mode	src/rtsmart/mpp/userapps/src/vicap/src/mpi_vicap.c

For an overview of models and resolutions, see the Image Sensor Support List. To add a new model from scratch, see How to Add a Sensor.

Opening menuconfig

• First select the board-level defconfig (example):

  make k230_rtos_01studio_defconfig
  

• Enter the interactive configuration:

  make menuconfig
  

• Navigate to:

  MPP Configuration
    └── Sensor Configuration
  

• The settings take effect after rebuilding and flashing:

  make
  

Menu example (Sensor Configuration):

The image above is an example of the menuconfig interface. For the specific entries, refer to the current SDK’s mpp/Kconfig.

How to View the Currently Supported Sensors

Method 1: Check with menuconfig (before compilation)

Under Sensor Configuration, find Sensor Drivers Configuration. Each Enable <型号> entry corresponds to a driver already integrated into the SDK, for example:

• Enable GC2093

• Enable OV5647

• Checked: The driver is built into the kernel, its corresponding modes appear in the on-board list_sensor, and the matching ISP configurations are installed into the image according to the rules.

• Unchecked: This model will not appear in the current firmware.

The authoritative list is in src/rtsmart/mpp/Kconfig and increases with each version.

Method 2: Check the board-level defconfig

grep CONFIG_MPP_ENABLE_SENSOR_ configs/<your_board>_defconfig
grep CONFIG_MPP_CSI0_LINK_MODE_ configs/<your_board>_defconfig

Example:

CONFIG_MPP_CSI0_LINK_MODE_2LANE=y
CONFIG_MPP_ENABLE_SENSOR_IMX335=y
CONFIG_MPP_ENABLE_SENSOR_GC2093=y

Method 3: On-board list_sensor (confirm whether the driver is enabled)

list_sensor is an RT-Smart msh command exported by sensor_dev.c. It first prints the currently detected drivers, supported modes, and lane count for each CSI, then lists all configured Sensor types and marks them as SUPPORTED or NOT DETECTED.

After entering msh through the serial port, run:

list_sensor

Example output:

Detected Sensor Drivers:
CSI0: ov5647_csi0 (2 supported modes)
  Sensor Type List:
                  7 -> OV5647_MIPI_CSI0 (..., 2 lanes)
CSI1: no sensor driver detected
CSI2: no sensor driver detected

All Configured Sensor Types:
  [SUPPORTED]    CSI0                 7 -> OV5647_MIPI_CSI0 (..., 2 lanes)
  [NOT DETECTED]                     23 -> GC2093_MIPI_CSI0

Usage notes:

Symptom	Meaning
CSI<n>: <driver> (...)	A driver has been detected on this CSI. The number in parentheses indicates the number of driver modes.
CSI<n>: no sensor driver detected	No loaded Sensor driver has been detected on this CSI.
[SUPPORTED]	The configured type appears in the detected driver mode table.
[NOT DETECTED]	The type has been configured, but the corresponding driver / mode was not detected on the current board.
2 lanes / 4 lanes in an entry	The number of MIPI lanes used by the currently detected mode.
A type is present, but the preview is black	The driver has been loaded. The problem is more likely related to hardware wiring, the MIPI link, or ISP parameters.

When selecting a sensor at the application layer, use the name / type printed by list_sensor. Do not hard-code the volatile numeric sequence, as it changes with the Kconfig combination.

The types for CSI0 / CSI1 / CSI2 are isolated by CONFIG_MPP_ENABLE_CSI_DEV_n in the enumeration (k_sensor_comm.h), the list_sensor table, and the mpi_sensor.c adaptation table: when a CSI is not enabled, its corresponding types are not built into the firmware.

CSI0 LANE NUM（2LANE / 4LANE）

At the top of Sensor Hardware Configuration, there is an independent option CSI0 LANE NUM (default: 2LANE):

MPP Configuration
  └── Sensor Configuration
        ├── CSI0 LANE NUM (2LANE) --->
        │     ├── (X) 2LANE          # Default; the 4LANE option for each Sensor is hidden
        │     └── ( ) 4LANE          # CSI0 can use 2L+4L; each Sensor can enable 4LANE Configure
        ├── [*] Enable CSI0  --->    # GPIO / I2C / MCLK
        ├── [*] Enable CSI1          # Can be compiled to coexist; cannot be initialized simultaneously with CSI0 4LANE at runtime
        └── [*] Enable CSI2

CSI0 LANE NUM	Macro	Effect
2LANE (default)	CONFIG_MPP_CSI0_LINK_MODE_2LANE=y	Each *_ENABLE_4LANE_CONFIGURE / IMX415 is hidden and not compiled
4LANE	CONFIG_MPP_CSI0_LINK_MODE_4LANE=y	CSI0 can retain both 2-lane and 4-lane modes; Enable CSI1 remains selectable

Hardware note: By default, using CSI0 for 4-lane aggregation on the board consumes the PHY shared with CSI1. The software no longer hides CSI1 for this reason, so that customers can use CSI1 after fly-wire modification or board revision; with the original factory wiring on an unmodified board, do not enable CSI0 4LANE and CSI1 simultaneously.

How to switch to 4LANE mode

The models currently providing a per-sensor 4-lane switch include IMX335, OV13850, and IMX415 (the entire IMX415 sensor is visible only in 4LANE link mode).

1. make menuconfig → Sensor Configuration

2. CSI0 LANE NUM → select 4LANE

3. Confirm [*] Enable CSI0

4. Open the target Sensor and select Enable 4LANE Configure in its submenu (IMX335 / OV13850 are enabled by default under 4LANE; IMX415 is enabled by default)

5. (Optional) If CSI1 is still needed after modifying the board, you may continue to select Enable CSI1

CSI0 LANE NUM (4LANE) --->
[*] Enable CSI0
[ ] Enable CSI1          # May remain unselected by default; enable it after fly-wire modification or board revision
[*] Enable IMX335
  └── [*] IMX335 Enable 4LANE Configure

The same applies to OV13850 / IMX415.

Corresponding Kconfig macros:

Option	Macro
CSI0 2LANE	CONFIG_MPP_CSI0_LINK_MODE_2LANE
CSI0 4LANE	CONFIG_MPP_CSI0_LINK_MODE_4LANE
IMX335 4-lane mode	CONFIG_MPP_SENSOR_IMX335_ENABLE_4LANE_CONFIGURE
OV13850 4-lane mode	CONFIG_MPP_SENSOR_OV13850_ENABLE_4LANE_CONFIGURE
IMX415 4-lane mode	CONFIG_MPP_SENSOR_IMX415_ENABLE_4LANE_CONFIGURE

IMX335 submenu example:

How to confirm after enabling

• In .config / defconfig, it is CONFIG_MPP_CSI0_LINK_MODE_4LANE=y

• Under Detected Sensor Drivers in the on-device list_sensor, the corresponding *_CSI0_4LANE_* mode is present (along with the still-available CSI0 2LANE mode)

• When Enable CSI1 is not enabled, *_CSI1_* does not appear; after it is enabled, it appears

• On the application side, add -lane 4 when detecting the resolution, or specify the 4LANE type using -stype (see the next section)

• The module and ribbon cable must genuinely support 4-lane; if the software is enabled but the hardware supports only 2-lane, link errors or a black screen will occur

4K Sensors (such as OV13850 and IMX415) are also subject to the VICAP operating mode restrictions. See sensor_list.md.

How the Application Layer Matches Sensors by Resolution / FPS / Lane

Samples such as sample_vicap_sensor probe modes by parameters through kd_mpi_sensor_adapt_get_ex(config, info, lane_pref) (mpi_sensor.c); you can also use -stype to force a specified enumeration. The legacy interface kd_mpi_sensor_adapt_get() is retained for compatibility, but it always uses ANY and cannot specify a lane preference.

Command-Line Parameters (sample_vicap_sensor）

See VICAP Sensor Demo for the complete parameter table and examples. Common probe-related options include:

Parameter	Meaning	Default
-s <0|1|2>	CSI port	2
-width / -height / -fps	Expected resolution and frame rate	1920 / 1080 / 30
-lane <2|4>	Lane preference when resolutions are identical (lane_pref)	ANY (unrestricted; ties are ranked in favor of 2LANE)
-stype <id>	Force k_vicap_sensor_type	Not set
-ofmt <0|1|2|3>	CHN0: YUV / RGB888 / RGB888P / RAW	0
-dump <n>	Automatically dump n frames and then exit	Interactive
-tile <0|1>	Force SW_TILE (debugging)	0

Working mode summary: -ofmt 3 (RAW) defaults to ONLINE; YUV/RGB previews that exceed limits, such as 4K, use SW_TILE; -tile 1 can force SW_TILE.

Examples:

# IMX335 2592×1944@30，CSI0；默认 ANY，同分时选 2-lane
./sample_vicap_sensor.elf -c <connector> -s 0 -width 2592 -height 1944 -fps 30

# 同上分辨率，明确选 4-lane（需固件 CSI0 LANE NUM=4LANE 且已 Enable 4LANE Configure）
./sample_vicap_sensor.elf -c <connector> -s 0 -width 2592 -height 1944 -fps 30 -lane 4

# RAW Dump（ONLINE）并自动存 1 帧
./sample_vicap_sensor.elf -c <connector> -s 0 -ofmt 3 -dump 1

# 强制 type（以板端 list_sensor 打印为准）
./sample_vicap_sensor.elf -c <connector> -s 0 -stype <id>

kd_mpi_sensor_adapt_get_ex Matching Rules

The probing structure k_vicap_probe_config contains conditions such as csi_num, width, height, and fps, and no longer contains lane_pref. The caller should zero out the structure and pass the lane preference through the third parameter of kd_mpi_sensor_adapt_get_ex(); when using VICAP_MIPI_LANE_PREF_ANY=0, candidates are not filtered by lane, so 4LANE-only sensors such as IMX415 / OV13850 at 1080p can still be detected.

1. Candidate filtering

   • The CSI port must equal the requested csi_num

   • lane_pref=ANY: no filtering; =2LANE: retain non-4LANE (including 1LANE); =4LANE: retain only VICAP_MIPI_4LANE; illegal values are treated as ANY; 4LANE is supported only on CSI0

   • Only sensors that can be successfully opened become candidates (actually present on the board)

2. Ranking: higher fps → larger width → larger height → when resolutions are identical, 2LANE takes precedence over 4LANE

3. Matching priority

   1. When fps != 0, first find a match where width + height + fps are all equal

   2. Then find a match where width + height are equal (fps may differ)

   3. Then find a larger resolution not smaller than the requested width and height

   4. If none is found, fail

Therefore, for 2LANE / 4LANE at the same resolution: the default ANY ranking favors 2LANE; use -lane / lane_pref to force a specific option. On the compilation side, CSI0 LANE NUM determines whether the 4LANE type is compiled into the firmware; at application runtime, 4LANE cannot be requested on CSI1/CSI2, and CSI0 4LANE and CSI1 cannot be initialized simultaneously.

Common Options for CSI and Drivers

Default CSI

• Default SENSOR on CSI：Default CSI port (0 / 1 / 2).

Sensor Hardware Configuration

Option	Description
CSI0 LANE NUM	2LANE (default) / 4LANE; see the previous section
Enable CSI0	When enabled, configure PowerDown / Reset / I2C / MCLK
Enable CSI1	Can be compiled together with 4LANE; at runtime, CSI0 4LANE and CSI1 cannot be initialized simultaneously
Enable CSI2	Has no PHY conflict with 4LANE and can be enabled as needed

Common options for each CSI (after enabling):

Option	Description
PowerDown GPIO	Power pin; set to -1 if unavailable
Reset GPIO	Reset pin; set to -1 if unavailable
I2c Bus	Such as i2c0
MCLK num	On-chip MCLK1/2/3, or an external crystal oscillator

Sensor Sub-options

Option	Description
On CSI0/1/2 Use CHIP MCLK	Whether the Sensor uses the on-chip MCLK on the corresponding CSI
Enable 4LANE Configure	Additionally compile the CSI0 4-lane mode; appears only when CSI0 LANE NUM=4LANE

AutoFocus (optional)

Enable AutoFocus → A voice coil motor driver can then be selected (such as DW9714P); this is required only when the hardware supports AF.

ISP Configuration: Switching Between bin and XML/JSON

Source Files and Installation

ISP parameter source file directory:

src/rtsmart/mpp/userapps/src/sensor/config/

Naming convention (using IMX335 2592×1944 as an example):

File	Purpose
imx335-2592x1944_auto.json	Auto mode; one of the inputs for generating .bin
imx335-2592x1944_manual.json	Manual mode
imx335-2592x1944.xml	XML calibration / tuning parameters

The two installation switches at the top of Sensor Configuration:

Option	Macro	Function
Install binary ISP configurations	CONFIG_MPP_ISP_CONFIG_INSTALL_BIN	Generate and install *.bin into the image bin/ directory
Install JSON and XML ISP configurations	CONFIG_MPP_ISP_CONFIG_INSTALL_JSON_XML	Copy matching .json / .xml files into the image bin/ directory

Only Enabled Sensors have their prefix-matched ISP files installed (see userapps/src/sensor/Makefile).

Build artifacts are generally located in:

output/<defconfig>/images/bin/

The path on the device is usually /bin/, for example, /bin/imx335-2592x1944.bin and /bin/imx335-2592x1944.xml.

Why bin Mode Cannot “Swap ISP Files in Real Time”

Default logic (when no parsing mode is explicitly specified):

• kd_mpi_vicap_init() probes for /bin/<database_name>.bin.

• If the .bin file exists and loads successfully, bin mode is used (internally enumerated as VICAP_DATABASE_PARSE_HEADER).

• A bin file is the binary produced at build time by gen_isp_config_bin after baking in the xml/json; changing the .xml / .json files on the device at runtime will not automatically be reflected in the loaded bin.

• To incorporate new parameters into the bin, you must modify the source files → run make again to generate the bin → reflashing / replacing /bin/*.bin → and then restart the application.

Therefore: During the tuning phase, if ISP parameters need to be changed frequently, switch to XML/JSON parsing mode.

How to Switch to XML/JSON Mode (Code)

Call the following before kd_mpi_vicap_init():

#include "mpi_vicap_api.h"
#include "k_vicap_comm.h"

k_vicap_dev vicap_dev = VICAP_DEV_ID_0;

/* Force loading from /bin/<database>.xml and *_auto.json / *_manual.json */
ret = kd_mpi_vicap_set_database_parse_mode(vicap_dev, VICAP_DATABASE_PARSE_XML_JSON);
if (ret) {
    printf("set_database_parse_mode failed, ret=%d\n", ret);
    return ret;
}

ret = kd_mpi_vicap_init(vicap_dev);

Enum meanings (k_vicap_comm.h):

Enum	Value	Actual Behavior
VICAP_DATABASE_PARSE_XML_JSON	0	Use XML + JSON, without prioritizing bin
VICAP_DATABASE_PARSE_HEADER	1	Force bin; initialization fails if the bin does not exist

In the API comments, HEADER once meant “header file mode”; in the current implementation, it corresponds to binary bin configuration.

Many examples in the SDK (such as examples/ai/yolo/src/pipeline.cc) explicitly set XML/JSON before vicap_init:

ret = kd_mpi_vicap_set_database_parse_mode(vicap_dev, VICAP_DATABASE_PARSE_XML_JSON);
ret = kd_mpi_vicap_init(vicap_dev);

Meanwhile, when programs such as sample_vicap_sensor do not call this interface, they automatically use bin if a .bin file is present.

How to Change Parameters in XML/JSON Mode

• Keep Install JSON and XML ISP configurations set to y in menuconfig to ensure that xml/json files are included in the image.

• In the application, switch to VICAP_DATABASE_PARSE_XML_JSON as described in the previous section.

• Modify and replace the files on the device, for example:

  # Push from the host (adjust the path according to the actual file system)
  adb push imx335-2592x1944.xml /bin/
  adb push imx335-2592x1944_auto.json /bin/
  adb push imx335-2592x1944_manual.json /bin/
  

• Rerun the application (vicap_init is required again). The ISP is loaded during init, so the active bin memory image cannot be hot-swapped while the same stream remains active.

The default loading paths are constructed from database_name (such as imx335-2592x1944) and the ISP config path (default: /bin/):

/bin/imx335-2592x1944.xml
/bin/imx335-2592x1944_auto.json
/bin/imx335-2592x1944_manual.json

Mode Selection Recommendations

Scenario	Recommendation
Mass production / fast startup	bin (default probing is sufficient), with parameters baked in
ISP calibration / tuning / frequent parameter changes	Set the code to VICAP_DATABASE_PARSE_XML_JSON, and restart the application after changing xml/json
Explicit requirement to use bin	kd_mpi_vicap_set_database_parse_mode(..., VICAP_DATABASE_PARSE_HEADER)

For calibration and tuning details, see:

• How to Perform ISP Calibration

• How to Perform ISP Tuning

Configuration Checklist

• CSI0 LANE NUM: Defaults to 2LANE; change it to 4LANE when CSI0 4-lane is required (CSI1 can be compiled, but cannot be initialized simultaneously with CSI0 4LANE).

• Enable CSIx must match the board wiring (GPIO / I2C / MCLK).

• Check the target Enable <Sensor>; under 4LANE, also confirm Enable 4LANE Configure.

• For 2L/4L at the same resolution: the default is lane_pref=ANY (equal-score sorting favors 2LANE); use -lane 2|4 or -stype to force a preference.

• ISP: use bin for mass production; for tuning, switch to XML/JSON in code and ensure json/xml has been installed in /bin/.

• make → Flash → list_sensor to confirm the detected CSI / type / lane → Then run the VICAP / Sensor example.

Related Documentation

• Image Sensor Support List

• How to Add a Sensor

• How to Build the Firmware

• How to Run the Sample Programs