Software ISP benchmarking

The Software ISP is particularly sensitive to performance regressions therefore it is a good idea to always benchmark the Software ISP before and after making changes to it and ensure that there are no performance regressions.

DebayerCpu class builtin benchmark

The DebayerCpu class has a builtin benchmark. This benchmark measures the time spent on processing (collecting statistics and debayering) only, it does not measure the time spent on capturing or outputting the frames.

The builtin benchmark always runs. So this can be used by simply running “cam” or “qcam” with a pipeline using the Software ISP.

When it runs it will skip measuring the first 30 frames to allow the caches and the CPU temperature (turbo-ing) to warm-up and then it measures 30 fps and shows the total and per frame processing time using an info level log message:

INFO Debayer debayer_cpu.cpp:907 Processed 30 frames in 244317us, 8143 us/frame

To get stable measurements it is advised to disable any other processes which may cause significant CPU usage (e.g. disable wifi, bluetooth and browsers). When possible it is also advisable to disable CPU turbo-ing and frequency-scaling.

For example when benchmarking on a Lenovo ThinkPad X1 Yoga Gen 8, with the charger plugged in, the CPU can be fixed to run at 2 GHz using:

sudo x86_energy_perf_policy --turbo-enable 0
sudo cpupower frequency-set -d 2GHz -u 2GHz

with these settings the builtin bench reports a processing time of ~7.8ms/frame on this laptop for FHD SGRBG10 (unpacked) bayer data.

Measuring power consumption

Since the Software ISP is often used on mobile devices it is also important to measure power consumption and ensure that that does not regress.

For example to measure power consumption on a Lenovo ThinkPad X1 Yoga Gen 8 it needs to be running on battery and it should be configured with its platform-profile (/sys/firmware/acpi/platform_profile) set to balanced and with its default turbo and frequency-scaling behavior to match real world usage.

Then start qcam to capture a FHD picture at 30 fps and position the qcam window so that it is fully visible. After this run the following command to monitor the power consumption:

watch -n 10 cat /sys/class/power_supply/BAT0/power_now /sys/class/hwmon/hwmon6/fan?_input

Note this not only measures the power consumption in µW it also monitors the speed of this laptop’s 2 fans. This is important because depending on the ambient temperature the 2 fans may spin up while testing and this will cause an additional power consumption of approx. 0.5 W messing up the measurement.

After starting qcam + the watch command let the laptop sit without using it for 2 minutes for the readings to stabilize. Then check that the fans have not turned on and manually take a couple of consecutive power readings and average these.

On the example Lenovo ThinkPad X1 Yoga Gen 8 laptop this results in a measured power consumption of approx. 13 W while running qcam versus approx. 4-5 W while setting idle with its OLED panel on.