Note

This is the documentation for the latest development branch and may refer to features that are not available in released versions. If you are looking for the documentation for a specific release, use the drop-down menu on the left and select the desired version.

MJPEG Encoding and Web Video Streaming Routine Explanation

Overview

The MJPEG web video streaming routine obtains YUV420SP video frames from the Sensor, encodes them into JPEG via the MJPEGEncoder hardware, and continuously sends them using HTTP multipart/x-mixed-replace. After a phone or computer is connected to the same network as the development board, the live image can be viewed directly in a browser.

The complete routine is located in the CanMV firmware source code and in the IDE examples at:

02-Media/mjpeg_web_server.py

Main Configuration

Before running, the network parameters at the beginning of the routine need to be modified:

USE_WIFI = True
WIFI_SSID = "Test"
WIFI_PASSWORD = "12345678"

SERVER_PORT = 8080
FRAME_WIDTH = 1920
FRAME_HEIGHT = 1080
FRAME_ALIGNMENT = 12
JPEG_QUALITY = 50
STREAM_FPS = 30
USE_VIDEO_FRAME = True

Configuration	Description
USE_WIFI	True uses WLAN, False uses wired LAN
FRAME_WIDTH, FRAME_HEIGHT	Sensor output and JPEG encoding resolution
FRAME_ALIGNMENT	Sensor image plane alignment exponent; the VENC input should be set to 12, i.e., 4096 bytes
JPEG_QUALITY	JPEG quality, ranging from 1 to 99
STREAM_FPS	Upper limit of sending frame rate; the actual frame rate is also limited by encoding speed and network bandwidth
USE_VIDEO_FRAME	True directly encodes Sensor video frames; FHD continuous video streaming should remain True

When the script is re-run with a different SSID configured, it will first disconnect the old WLAN connection retained by the development board, and then connect to the new AP.

Encoding Flow

The Sensor output needs to use YUV420SP, and ensure that the physical address of each plane is aligned to 4096 bytes:

sensor = Sensor()
sensor.reset()
sensor.set_framesize(
    width=FRAME_WIDTH,
    height=FRAME_HEIGHT,
    alignment=FRAME_ALIGNMENT,
)
sensor.set_pixformat(Sensor.YUV420SP)
sensor.run()

encoder = MJPEGEncoder(quality=JPEG_QUALITY)

During continuous encoding, directly obtain py_video_frame_info:

def capture_jpeg(sensor, encoder):
    frame = sensor.snapshot(dump_frame=True)
    jpeg = encoder.encode(frame, timeout_ms=1000)
    del frame
    return jpeg

This approach submits the Sensor’s VB frame directly to VENC, avoiding copying a 1920×1080 image into an intermediate buffer. If you need to demonstrate image.Image input, you can change USE_VIDEO_FRAME to False at a lower resolution.

HTTP Interface

After the routine starts, the access address will be output via the serial port:

Open http://192.168.2.60:8080/ in a browser

Path	Content
/	Live video page
/stream	multipart/x-mixed-replace MJPEG data stream
/snapshot.jpg	Current single-frame JPEG image

The JavaScript in the web page will wait 1 second after the video stream connection fails and reconnect to /stream; no manual page refresh is required.

Large Frame Network Sending

The RT-Smart blocking socket send timeout is limited to 500 ms. A 1080P JPEG is relatively large; calling sendall() once directly may time out after backpressure occurs in the TCP send buffer, which manifests as the browser displaying only one frame before disconnecting.

The routine keeps the client socket in non-blocking mode, and sends the JPEG in 16 KiB chunks; on EAGAIN or temporary timeout, it continues to retry. As long as sending is still making progress, the connection will be maintained. The client connection is only closed if there is no sending progress for 5 consecutive seconds.

Running Steps

1. Modify WIFI_SSID and WIFI_PASSWORD, or set USE_WIFI to False to use a wired network.

2. Run 02-Media/mjpeg_web_server.py.

3. Wait for the serial port to output First JPEG size and the browser access address.

4. Open the address on a phone or computer within the same LAN.

Usage Limitations

• This routine adopts a synchronous single-client design. While one browser is accessing /stream, the server will not process another client’s request simultaneously.

• STREAM_FPS=30 is the target upper limit, and it does not guarantee that WLAN can continuously transmit 1080P 30 FPS. When there is significant latency, you should lower JPEG_QUALITY, STREAM_FPS, or the resolution.

• When the browser consumes data at a rate lower than the generation rate, TCP backpressure will naturally lower the actual frame rate, and will not infinitely buffer JPEG frames.

• When exiting the script, the socket, encoder, and Sensor will be closed in sequence to avoid leaving behind VENC channels or VB buffer pools.

For the complete interface description, please refer to the MJPEGEncoder API Manual.