WebRTC Demo#

Introduction#

This example describes how to use the WebRTC protocol on the K230 board to stream real-time video to a browser. Through the following example, you will learn how to use the libpeer library API to set up a WebRTC service that streams the board’s camera feed, hardware-encoded as H.264, to a browser in real time, enabling low-latency LAN video transmission.

WebRTC LAN Camera: Describes how to stream the sensor image from the board to a browser via the WebRTC protocol, with simultaneous LCD/HDMI local preview. The board runs a built-in HTTP signaling server and Web UI page — simply open the URL in a browser to connect and view the live stream with one click.

Usage Instructions#

Code Location#

WebRTC Demo: /src/rtsmart/examples/mpp/sample_webrtc

Assuming the demo has been correctly compiled, after starting the development board, enter the /sdcard/app/examples/mpp directory, and you will see the following executable file:

sample_webrtc.elf: Corresponds to the WebRTC LAN Camera demo

Make sure this file exists and is correctly compiled, so that the example program can be run on the development board.

Build Configuration#

Use make menuconfig to enable libpeer and sample_webrtc:

RT-Smart UserSpace Libraries Configuration  →  Enable Build libpeer (WebRTC)  = Y
RT-Smart UserSpace Examples Configuration   →  Enable MPP examples           = Y
                                               →  Enable Build sample_webrtc   = Y

Example#

Example 1: WebRTC LAN Camera#

This example describes how to use the WebRTC protocol to stream the sensor image from the board to a browser in real time. The board starts an HTTP signaling server, and the browser accesses the built-in Web page to automatically complete SDP negotiation and establish a WebRTC connection for real-time video transmission.

Architecture#

VICAP (Camera)
  ├── CHN0 ──bind──> VO (LCD/HDMI preview)
  └── CHN1 ──bind──> VENC (H.264) ──> WebRTC ──> Browser
                                          ▲
                                   HTTP signaling (offer/answer)

Data flow:

VICAP captures the camera image. CHN0 is bound to VO for local preview, and CHN1 is bound to VENC for H.264 encoding.
H.264 encoded frames from VENC are sent to the WebRTC peer via peer_connection_send_video().
The HTTP signaling server handles the browser’s SDP offer/answer exchange to establish the WebRTC connection.

Parameter Description#

Parameter	Description	Parameter Range	Default Value
p	HTTP server port	-	8080
s	CSI device number	0-2	2
c	Display connector type	605274512=LCD, 757006876=HDMI	LCD (605274512)
W	Video encoding width	`[128, 1920]`	1280
H	Video encoding height	`[64, 1080]`	720
b	Video encoding bitrate (kbps)	-	2000

Running the Program#

After starting the WebRTC demo, the serial output will display a URL. Open this URL in a browser to view the board’s live camera feed.

./sample_webrtc.elf -p 8080 -s 2 -c 605274512 -W 1280 -H 720 -b 2000

Make sure the browser device and the board are on the same LAN. Press ctrl+c to exit the program.

msh /sdcard/app/examples/mpp>./sample_webrtc.elf -p 8080 -s 2 -c 605274512 -W 1280 -H 720 -b 2000
vb_set_config ok
debug: _ai_i2s_set_pitch_shift : semitones = 0
kd_mpi_venc_init start 0
venc[0] 1280*720 size:462848 cnt:30 srcfps:30 dstfps:30 rate:4000 rc_mode:1 type:96 profile:3
kd_mpi_venc_init end
set output err, set default format ISP_PIX_FMT_YUV420SP
[tuning] dev: 0
acq_win.width: 1920
acq_win.height: 1080
pipe_ctrl: 4294967289
sensor_fd: 12
sensor_type: 7
sensor_name: ov5647_csi0
database_name: ov5647-1920x1080
buffer_num: 0
buffer_size: 0
[tuning] chn: 0
out_win.width: 1280
out_win.height: 720
bit_width: 0
pix_format: 5
buffer_num: 6
buffer_size: 1382400
yraw_size: 0
uv_size: 0
v_size: 0
block_type: 1
wait_time: 500
chn_enable: 1
isp_3dnr_en is 0 g_isp_dev_ctx[dev_num].dev_attr.pipe_ctrl.bits.dnr3_enable is 1
VsiCamDeviceCreate hw:0-vt:0 created!
kd_mpi_isp_set_output_chn_format, width(1920), height(1080), pix_format(5)
[dw] init, version Feb 17 2025 00:06:56
[dw] open dewarp config file error
[dw] load config /bin/ov5647-1920x1080.bin failed, bypass
[dw] enter kd_mpi_sys_mmz_alloc((k_u64*)&settings->lut_phy_addr, &settings->lut_user_virt_addr, "dewarp/lut", "anonymous", settings->lut_size)
[dw] enter dev_fds[dev_num] < 0
[dw] enter ioctl(dev_fds[dev_num], K_DWVIOC_SETUP, settings)
[dw] split: 0 8191 8191 8191
[dw] LUT: 121x69=33396 phy(130ac000)
[dw] input:     1920x1080@YUV420SP align 0 bit 8
[dw] output[0]: 1280x720@YUV420SP align 12 bit 8 crop(0,0,0,0)

========================================
  libpeer LAN Camera Demo
========================================
  CSI: 2, Connector: 605274512
  Encode: 1280x720 @ 2000kbps H264
  Open in browser:
  http://10.100.228.46:8080
========================================

State: PEER_CONNECTION_NEW

Open http://10.100.228.46:8080 in a browser, and the page will automatically initiate a WebRTC connection request. After a successful connection, the serial output will show:

State: PEER_CONNECTION_CHECKING
State: PEER_CONNECTION_CONNECTED

At this point, the browser page will display the board’s live camera feed.

Browser Requirements#

Chrome/Edge enables mDNS privacy protection by default, which hides local IP addresses and causes WebRTC connections to fail. The web page will automatically detect this and show a warning.

Fix options (any one of the following):

Enter chrome://flags/#enable-webrtc-hide-local-ips-with-mdns in the address bar → Set to Disabled → Restart
Add --disable-features=WebRtcHideLocalIpsWithMdns as a browser launch argument
Use Firefox (mDNS privacy protection is not enabled by default)

Reconnection#

The browser page supports reconnection: when the WebRTC connection is dropped, click the reconnect button on the page to initiate a new SDP negotiation without refreshing the page or restarting the board-side program.

Tip

For the specific interfaces of the WebRTC module, please refer to the API Documentation