Webcams - Revision history

Elias: Correct board to A7 and add build number; add section on one camera (shipped ustreamer) vs several (crowsnest)

2026-08-23T17:30:07Z

Correct board to A7 and add build number; add section on one camera (shipped ustreamer) vs several (crowsnest)

@@ Line 1: / Line 1: @@
 Recore can run one or more USB webcams for monitoring prints. This page covers how to choose a camera, how many will work at once, and how to diagnose the common failures.
-All figures here were measured on a Recore A5 running <code>rebuild-fluidd</code> (kernel 6.18.33) in August 2026, with three cameras attached.
+All figures on this page were measured on a '''Recore A7''' running '''<code>rebuild-fluidd-1adfb1e</code>''' (kernel 6.18.33) in August 2026, with three USB cameras attached.
 == The short version ==
@@ Line 8: / Line 8: @@
 * '''Roughly three cameras at 640x480, or two at 1280x720.''' The limit is USB bandwidth, not the processor.
 * If a camera will not start, check <code>dmesg</code> for <code>Failed to submit URB 0 (-28)</code> before suspecting anything else.
 == Why MJPEG matters ==
@@ Line 105: / Line 137: @@
 In dim light a camera on automatic exposure will lengthen its exposure, and exposure time puts a hard ceiling on frame rate: 140 ms per frame is at most 7 fps, no matter what else is configured.
-Measured on a Vimicro camera at 640x480:
+Measured on a Vimicro camera at 640x480 (Recore A7):
 {| class="wikitable"

Elias: Add H.264/HEVC section: measured efficiency and why camera-streamer cannot use them on A64

2026-08-23T17:23:39Z

Add H.264/HEVC section: measured efficiency and why camera-streamer cannot use them on A64

← Older revision		Revision as of 17:23, 23 August 2026
Line 276:		Line 276:

	So: '''a Recore with cameras and a USB WiFi dongle is a meaningfully heavier configuration than one using Ethernet.''' If cameras behave badly over WiFi but well over Ethernet, this is why.		So: '''a Recore with cameras and a USB WiFi dongle is a meaningfully heavier configuration than one using Ethernet.''' If cameras behave badly over WiFi but well over Ethernet, this is why.

			== H.264 and HEVC ==

			Some cameras offer H.264 or HEVC in addition to MJPEG. These are '''far''' more efficient than MJPEG, but they '''cannot currently be used on Recore'''. This section records why, so the question does not need re-investigating.

			=== The efficiency is dramatic ===

			Measured on a 4K microscope camera, same board, same bus, 45 frames per run:

			{\| class="wikitable"
			! Format !! Resolution !! Frame rate !! Per frame
			\|-
			\| MJPEG \|\| 1920x1080 \|\| 29.9 fps \|\| 395 kB
			\|-
			\| '''H.264''' \|\| 1920x1080 \|\| 29.8 fps \|\| '''18 kB''' (22x smaller)
			\|-
			\| H.264 \|\| 3840x2160 \|\| 26.3 fps \|\| 89 kB
			\|-
			\| '''HEVC''' \|\| 3840x2160 \|\| '''30.0 fps''' \|\| '''73 kB''' (25x smaller)
			\|-
			\| MJPEG \|\| 3840x2160 \|\| 12.7 fps \|\| 1871 kB
			\|}

			'''4K HEVC at a full 30 fps uses about five times less bandwidth than 1080p MJPEG.''' Note also that MJPEG at 4K cannot reach 30 fps at all - it manages 12.7 fps because it saturates the USB bus, while HEVC at the same resolution runs at the full rate.

			If these codecs worked, the bandwidth limits described above would largely disappear.

			=== Why it does not work ===

			The blocker is '''camera-streamer''', not crowsnest and not ustreamer.

			* '''ustreamer''' handles MJPEG only, by design. It is not the limitation, because crowsnest can launch camera-streamer instead.
			* '''camera-streamer''' does support H.264 and captures it correctly - it negotiates <code>1920x1080/H264</code> without trouble - but it then fails to start:

			<pre>
			device/buffer_list.c: CAMERA:capture: Using: 1920x1080/H264, buffers=3 <- capture works
			device/camera/camera_decoder.c: CAMERA: Cannot find 'H264' decoder
			device/camera/camera_output.c: CAMERA: Cannot find source for 'SNAPSHOT' for one of the formats 'JPEG, MJPG'.
			</pre>

			Two things combine to make this a hard stop:

			'''1. The snapshot output cannot be disabled.''' camera-streamer 0.4.2.4 provides <code>--camera-stream.disabled</code> and <code>--camera-video.disabled</code>, but there is no equivalent for snapshot. Snapshot must be JPEG or MJPG, so H.264 input has to be decoded to satisfy it. Disabling the other two outputs does not help - it still exits.

			'''2. The A64's video decoder is the wrong kind.''' The SoC does have a hardware decoder (<code>cedrus</code>), and it does handle H.264 and HEVC, but note the formats it accepts:

			<pre>
			$ v4l2-ctl -d /dev/video0 --list-formats-out
			[1]: 'S264' (H.264 Parsed Slice Data, compressed)
			[2]: 'S265' (HEVC Parsed Slice Data, compressed)
			</pre>

			"Parsed Slice Data" means this is a '''stateless''' decoder. Userspace has to parse the bitstream itself, extract the sequence and picture parameter sets, and drive the hardware slice by slice through the Media Request API. camera-streamer expects a '''stateful''' decoder - the kind found on the Raspberry Pi, where a stream goes in and frames come out. These are fundamentally different programming models, so camera-streamer cannot use cedrus even though the hardware is capable.

			This is not a missing driver, a missing package or a configuration mistake.

			=== What would be needed ===

			* '''Patch camera-streamer''' to allow the snapshot output to be disabled, then pass H.264 through to WebRTC without decoding at all. This is the smallest change and would deliver the bandwidth saving. The drawback is that WebRTC is the only viable output - Fluidd displays cameras with an ordinary image element, which cannot show H.264.
			* '''Use a decoder that speaks stateless V4L2''' - ffmpeg with the Request API, or libva - to convert H.264 to MJPEG on the board. This keeps compatibility with existing interfaces, but re-introduces a per-frame cost, which is much of what the codec was meant to avoid.

			Until one of those exists, '''use MJPEG''', and treat a camera's H.264 or HEVC support as a bonus that is not currently usable. A camera offering H.264 will almost always offer MJPEG too, so such cameras are still fine to buy.

	== See also ==		== See also ==

Elias: Best practice and measured results for USB webcams on Recore

2026-08-23T17:05:43Z

Best practice and measured results for USB webcams on Recore

New page

Recore can run one or more USB webcams for monitoring prints. This page covers how to choose a camera, how many will work at once, and how to diagnose the common failures.

All figures here were measured on a Recore A5 running <code>rebuild-fluidd</code> (kernel 6.18.33) in August 2026, with three cameras attached.

== The short version ==

* '''Buy a camera that supports MJPEG in hardware.''' This matters more than resolution, brand or price. A camera without it can cost 20x the USB bandwidth and forces the CPU to compress every frame.
* '''Roughly three cameras at 640x480, or two at 1280x720.''' The limit is USB bandwidth, not the processor.
* If a camera will not start, check <code>dmesg</code> for <code>Failed to submit URB 0 (-28)</code> before suspecting anything else.

== Why MJPEG matters ==

USB cameras deliver frames in one of two ways:

* '''MJPEG''' (<code>MJPG</code>) - the camera compresses each frame itself. The Recore copies compressed frames straight to the browser without touching them.
* '''Raw''' (<code>YUYV</code>) - the camera sends uncompressed pixels. Every frame must then be JPEG-compressed in software on the Recore's CPU.

A single 640x480 raw frame is 614 kB. The same frame as MJPEG is typically 20-40 kB. That difference applies to both the USB bus and the processor.

Measured on the same board, same bus:

{| class="wikitable"
! Camera !! MJPEG? !! Result
|-
| Logitech BRIO || yes || 1280x720 @ 54 fps, hardware passthrough
|-
| AUKEY PC-W1 || yes || 1280x720 @ 30 fps, hardware passthrough
|-
| Vimicro "Venus" || '''no''' || never obtained USB bandwidth alongside the others, even at 320x240
|}

The raw-only camera could not be made to work as a third camera at any resolution. It was not scaling its USB bandwidth request with frame size.

=== Checking a camera before buying ===

If you can plug it into any Linux machine:

<pre>
v4l2-ctl --list-formats-ext -d /dev/video0
</pre>

Look for this line:

<pre>
[1]: 'MJPG' (Motion-JPEG, compressed)
</pre>

If only <code>'YUYV'</code> appears, the camera has no hardware compression. It will work as a single camera at low resolution, but it is a poor choice for a Recore.

Some cameras also list <code>H264</code> or <code>HEVC</code>. These are not currently used - the streaming software (ustreamer) handles MJPEG only - but a camera offering H.264 will almost always offer MJPEG as well.

== How many cameras can run at once ==

The A64 has a '''single USB host controller''', and every USB device on a Recore sits behind one hub connected to it. USB 2.0 reserves bandwidth for cameras up front, when the stream starts, and the total is fixed.

Measured with three MJPEG cameras:

{| class="wikitable"
! Configuration !! Result
|-
| 3 cameras @ 640x480 || '''all three work''' - 27, 20 and 30 fps
|-
| 3 cameras @ 1280x720 || only the first starts; other two fail immediately
|-
| 2 cameras @ 1280x720 || both work - 53 and 30 fps
|-
| 2 cameras @ 1920x1080 || first works, second fails
|}

Bandwidth is claimed on a first-come basis. Whichever camera starts first gets what it asks for, and later cameras get whatever is left - which may be nothing.

=== The signature of running out of bandwidth ===

<pre>
$ dmesg | grep URB
uvcvideo 1-1.3:1.1: Failed to submit URB 0 (-28).
uvcvideo 1-1.1:1.1: Failed to submit URB 0 (-28).
</pre>

<code>-28</code> is <code>ENOSPC</code>. In this context it does not mean disk space - it means the USB controller could not reserve enough bandwidth for the stream.

This failure is '''immediate and unrelated to system load'''. If a camera fails this way it will fail on an idle machine just as reliably. The fix is to lower resolution or frame rate, or to use fewer cameras.

== CPU cost ==

With hardware MJPEG the CPU cost is close to nothing. Three cameras at 640x480, all three being watched in Fluidd:

<pre>
cpu0 9% cpu1 5% cpu2 8% cpu3 6% (softirq 0% on all cores)

ustreamer processes: ~3.6% each, about 11% of one core in total
klipper (python): 7.7% <- more than all three cameras combined
</pre>

Every frame captured reached the browser (<code>queued_fps</code> equalled <code>captured_fps</code> on all three streams), so nothing was being dropped.

A raw YUYV camera changes this picture completely, because ustreamer must then run a software JPEG encoder - it starts four worker threads for a single 640x480 stream.

'''Conclusion:''' on a Recore, multiple cameras are limited by USB bandwidth, not by the processor. Three cameras left the SoC around 90% idle.

== Low frame rate from a single camera ==

If one camera runs far below its rated frame rate, the cause is usually '''exposure time''', not bandwidth.

In dim light a camera on automatic exposure will lengthen its exposure, and exposure time puts a hard ceiling on frame rate: 140 ms per frame is at most 7 fps, no matter what else is configured.

Measured on a Vimicro camera at 640x480:

{| class="wikitable"
! Setting !! Frame rate
|-
| auto exposure (default) || 7.2 fps
|-
| <code>exposure_time_absolute=100</code> (~10 ms) || 19.9 fps
|-
| <code>exposure_time_absolute=30</code> (~3 ms) || 30.0 fps
|}

Note the trade-off: shorter exposure means a darker image. More light on the subject is the better fix where possible.

'''The diagnostic that distinguishes the two causes:''' lower the resolution and re-measure.
* Frame rate improves roughly in proportion - the limit was '''bandwidth'''.
* Frame rate does not change - the limit is '''exposure''' (or something else per-frame).

In the case above, 320x240 gave exactly the same 7 fps as 640x480, which ruled out bandwidth before anything was changed.

Setting exposure manually in crowsnest:

<pre>
[cam 1]
v4l2ctl: auto_exposure=1,exposure_time_absolute=100
</pre>

== Configuration ==

=== Use stable device paths ===

Do '''not''' use <code>/dev/video0</code>, <code>/dev/video2</code> and so on in configuration. These numbers change whenever a camera is added, removed or replugged - on the test board <code>/dev/webcam</code> moved from <code>video2</code> to <code>video5</code> to <code>video7</code> in a single afternoon as cameras were swapped, silently changing which camera was being streamed.

Use the by-id path instead:

<pre>
$ ls /dev/v4l/by-id/
usb-046d_Logitech_BRIO_E76A9283-video-index0
usb-Sonix_Technology_Co.__Ltd._AUKEY_PC-W1_SN0001-video-index0
</pre>

<code>index0</code> matters: a UVC camera exposes several device nodes and only the first is the capture device. The BRIO exposes four; the rest carry metadata.

=== Example crowsnest configuration ===

Three cameras, all MJPEG, at a resolution where all three fit:

<pre>
[crowsnest]
log_level: verbose
rollover_on_start: false
no_proxy: false

[cam 1]
mode: ustreamer
port: 8080
device: /dev/v4l/by-id/usb-046d_Logitech_BRIO_E76A9283-video-index0
resolution: 640x480
max_fps: 30
custom_flags: --format=MJPEG --drop-same-frames=30

[cam 2]
mode: ustreamer
port: 8081
device: /dev/v4l/by-id/usb-Sonix_Technology_Co.__Ltd._AUKEY_PC-W1_SN0001-video-index0
resolution: 640x480
max_fps: 30
custom_flags: --format=MJPEG --drop-same-frames=30

[cam 3]
mode: ustreamer
port: 8082
device: /dev/v4l/by-id/usb-Linux_Foundation_Webcam_gadget-video-index0
resolution: 640x480
max_fps: 30
custom_flags: --format=MJPEG --drop-same-frames=30
</pre>

Ports map to URLs: 8080 is <code>/webcam/</code>, 8081 is <code>/webcam2/</code>, 8082 is <code>/webcam3/</code>.

=== Adding cameras to Fluidd ===

Each camera has to be registered in Moonraker before it appears in the interface. In Fluidd: '''Settings -> Camera -> Add Camera''', then set the stream URL to <code>/webcam2/?action=stream</code> and the snapshot URL to <code>/webcam2/?action=snapshot</code>.

Choose service '''"MJPEG Stream"''' rather than '''"MJPEG Stream (Adaptive)"'''. In adaptive mode the browser fetches individual snapshots at its own pace instead of consuming the stream, and it throttles to <code>target_fps_idle</code> - 5 fps by default - whenever the printer is not printing. A camera capturing at 20 fps can appear to run at 2 fps for this reason alone, with nothing wrong on the Recore at all.

== Troubleshooting ==

{| class="wikitable"
! Symptom !! Likely cause !! Check
|-
| Second or third camera never starts || USB bandwidth || <code>dmesg | grep URB</code> for <code>-28</code>
|-
| Low fps, unchanged by lowering resolution || exposure time || <code>v4l2-ctl -d /dev/webcam --list-ctrls</code>
|-
| Low fps, improves when resolution is lowered || USB bandwidth || reduce resolution or camera count
|-
| Fluidd shows ~2-5 fps, board reports more || Fluidd adaptive mode || set service to plain "MJPEG Stream"
|-
| High CPU from ustreamer || camera has no MJPEG || <code>v4l2-ctl --list-formats-ext</code>
|-
| Camera works, then stops after replug || device path changed || use <code>/dev/v4l/by-id/</code>
|-
| All cameras stop after adding one || crowsnest crash - see below || <code>journalctl -u crowsnest</code>
|}

=== Useful commands ===

<pre>
# What formats and frame rates does this camera support?
v4l2-ctl --list-formats-ext -d /dev/video2

# What controls does it have (exposure, brightness, ...)?
v4l2-ctl --list-ctrls -d /dev/video2

# What is the stream actually doing right now?
curl -s http://localhost:8080/state

# Which USB devices are attached, and at what speed?
for d in /sys/bus/usb/devices/[0-9]*-[0-9]*; do
echo "$(basename $d) $(cat $d/speed)M $(cat $d/product 2>/dev/null)"
done
</pre>

The <code>/state</code> endpoint is the quickest way to see what is happening:

<pre>
"encoder": {"type": "HW"} <- hardware MJPEG, no CPU cost
"encoder": {"type": "CPU"} <- software encoding, camera has no MJPEG
"online": false <- camera not delivering frames at all
"captured_fps": 27 <- what the camera produces
"queued_fps": 27 <- what reaches viewers; lower means drops
</pre>

=== Known issue: one camera can prevent all others from starting ===

Crowsnest v5.0.11 crashes on startup if any attached camera exposes a V4L2 control without a <code>default</code> value, which is legal and occurs on some cameras:

<pre>
File "crowsnest/camera/types/uvc.py", line 72, in get_controls_string
line += f" default={data['default']}"
KeyError: 'default'
</pre>

The crash happens before '''any''' camera starts, so a single unusual camera takes down every stream on the machine. Changing the log level does not avoid it, because the controls are enumerated regardless.

Workaround, until fixed upstream - in <code>crowsnest/camera/types/uvc.py</code>:

<pre>
line += f" default={data.get('default', 'n/a')}"
</pre>

== Background: why the USB bus is the limit ==

The Allwinner A64 has one EHCI host controller. Every USB device on a Recore - cameras, WiFi dongles, USB storage, USB serial - shares it.

USB 2.0 gives cameras ''isochronous'' transfers: guaranteed, reserved bandwidth, claimed when the stream opens. The specification caps this at 80% of the bus, and the reservation is made up front based on the requested resolution and frame rate. This is why a camera either starts successfully or fails immediately, rather than degrading gradually.

It was previously suspected that USB interrupt handling on a single CPU core was the constraint. Measurement does not support this for cameras:

<pre>
$ grep ehci /proc/interrupts
57: 429 992573 0 0 GICv2 106 Level ehci_hcd:usb1
cpu0 cpu1
</pre>

The interrupt is handled on one core, but with three cameras streaming to three viewers that core was only 5% busy, and softirq time was zero across all cores.

'''However''', high single-core load ''has'' been measured on Recore when networking runs over a '''USB WiFi dongle''' - 73% of one core to receive 1.2 MB/s. In that case the USB completion handling and the entire network stack land on the same core. Cameras over Ethernet do not have this problem, because Ethernet on the A64 is a separate peripheral that does not use USB at all.

So: '''a Recore with cameras and a USB WiFi dongle is a meaningfully heavier configuration than one using Ethernet.''' If cameras behave badly over WiFi but well over Ethernet, this is why.

== See also ==

* [https://github.com/intelligent-agent/Rebuild/issues/76 Rebuild issue #76] - USB interrupt handling and multi-camera investigation, with measurements
* [https://docs.mainsail.xyz/crowsnest/ Crowsnest documentation]