The integration of pytest with real boards (test.py) was written by Stephen Warren of Nvidia, some 9 years ago. It has certainly stood the test of time. The original code has been tweaked for various purposes over the years, but considering the number of tests added in that time, the changes are very small. Here is a diffstat for the changes up until a recent rename:

 test/py/multiplexed_log.css           |  11 +-
 test/py/multiplexed_log.py            | 133 ++++++++++---
 test/py/test.py                       |  31 ++--
 test/py/u_boot_console_base.py        | 341 ++++++++++++++++++++++++++++------
 test/py/u_boot_console_exec_attach.py |  40 ++--
 test/py/u_boot_console_sandbox.py     |  54 ++++--
 test/py/u_boot_spawn.py               | 212 ++++++++++++++++++---
 test/py/u_boot_utils.py               | 197 ++++++++++++++++++--
 8 files changed, 848 insertions(+), 171 deletions(-)

When Stephen wrote the code, there was no Gitlab system in U-Boot (it used Travis). Tom Rini added Gitlab in 2019: test.py mostly just worked in that environment. One of the reasons the code has proven so stable is that it deals with boards at the console level, simply relying on shell-script hooks to talk start up and communicate with boards. These scripts can be made to do a lot of different things, such as powering boards on and off, sending U-Boot over USB, etc.

But perhaps it might be time to make a few changes. Let me give a bit of background first.

In 2020 I decided to try to get my collection of boards into some sort of lab. Picking out a board to manually test with it was quite annoying. I wrote Labman, a Python program which created various files based on a yaml description of the lab. Labman generates udev rules and an /etc/fstab file. It also creates small Python programs which know how to build U-Boot and write it to a board, including dealing with the reset/recovery sequences, SD-wire, etc. With all that in place, Tbot provides a way to get an interactive session on a board. It also provides a way to run U-Boot tests.

Early last year I decided to take another look at this. The best things about Labman were its unified lab description (including understanding how many ports each USB hub has and the address of each) and a ‘labman check’ option which quickly pointed to connection problems. The bad thing about Labman was…well, everything else. It was annoying to re-run the scripts and restart udev after each lab change. The Python code-generation was a strange way of dealing with the board-specific logic.

Tom Rini suggested looking at Labgrid. After a bit of investigation, it looked good to me. The specification of hubs is somewhat primitive and the split between the exporter and the environment is confusing. But the structure of it (coordinator, exporters and clients) is much better than Labman. The approach to connecting to boards (ssh) is better as well, since it starts ser2net automatically. Labman is a thin layer of code over some existing services. Labman is much better designed.

So overall I was pretty enthusiastic and set to work on creating an integration for U-Boot. So I can again build U-Boot, write it to a board and start it up with a simple command:

ellesmere:~/u$ ub-int rock5b
Building U-Boot in sourcedir for rock5b-rk3588
Bootstrapping U-Boot from dir /tmp/b/rock5b-rk3588
Writing U-Boot using method rockchip
DDR 9fa84341ce typ 24/09/06-09:51:11,fwver: v1.18

<...much unfortunate spam from secret binaries here...>

U-Boot Concept 2025.01-rc3-01976-g290829cc0d20 (Jul 20 2025 - 20:10:36 -0600)

Model: Radxa ROCK 5B
SoC:   RK3588
DRAM:  4 GiB
Core:  362 devices, 34 uclasses, devicetree: separate
MMC:   mmc@fe2c0000: 1, mmc@fe2d0000: 2, mmc@fe2e0000: 0
Loading Environment from nowhere... OK
In:    serial@feb50000
Out:   serial@feb50000
Err:   serial@feb50000
Model: Radxa ROCK 5B
SoC:   RK3588
Net:   No ethernet found.
Hit any key to stop autoboot:  0 
=> 

I’ve also used this integration to make my lab accessible to gitlab, so that any branch or pull-request can be tested on the lab, to make sure it has not broken U-Boot.

So, back to the topic. The Labgrid integration supports test.py and it works fine. A minor improvement is ‘lab mode’, where Labgrid handles getting U-Boot to a prompt, making it work with boards like the Beagleplay, which has a special autoboot message.

But the test.py interface is (at last) showing its age. It’s only real interface to Labgrid is via the u-boot-test-console script, which just runs the Labgrid client. Some tests restart the board, perhaps because they boot and OS or do something destructive to the running U-Boot. This results in U-Boot being built again, flashed to the board again and started again. When something breaks, it could be a lab failure or a test failure, but all we can do is show the output and let the user figure it out. The current lab works remarkably well given its fairly basic setup, but it is certainly not reliable. Sometimes a board will fail a test, but trying it again will pass, for example.

So I am thinking that it might make sense to integrate test.py and Labgrid a little more closely. Both are written in Python, so test.py could import some Labgrid modules, get the required target, start up the console and then let the tests run. If a test wants to restart, a function can do this in the most efficient and reliable way possible.

This might be more efficient and it might also provide better error messages. We would then not need the hook functions for the Labgrid case.

U-Boot has a new continuous integration (CI) lab page that provides a real-time look at the status of various development boards. The page, located at https://lab.u-boot.org/, offers a simple and clean interface that allows developers and curious people to quickly check on the health and activity of each board in the lab.

When you first visit the page, you’ll see a grid of all the available boards. Each board’s card displays its name and current status, making it easy to see which boards are online and which are not. A single click on any board will show a console view, taken from the last health check. This allows you see why boards are failing, for example.

This new lab page is a nice resource for the U-Boot community. It provides a transparent and accessible way to monitor this part of the CI system.

Check it out and get in touch if you have any suggestions or feedback! 🧪

In the world of software development, consistency is key. A recent update to U-Boot Concept takes a solid step in that direction by restructuring how it handles emulation targets. This change makes life easier for developers working across different processor architectures.

Previously there were inconsistencies in the configuration system (Kconfig). For example, enabling QEMU emulation for ARM systems used the ARCH_QEMU symbol, while x86 systems used VENDOR_EMULATION for a similar purpose. This could create confusion and added complexity when managing board configurations.

To resolve this, a new, architecture-neutral symbol, MACH_QEMU, has been introduced. This single, unified option replaces the separate symbols for both ARM and x86 emulation targets.

This small but important merge tidies up the codebase, creating a more consistent and intuitive developer experience. It also sets the stage for future work, with the potential to extend this unified approach to other architectures. It’s a great example of the continuous effort to keep U-Boot clean, efficient, and easy to maintain for everyone involved.

A few weeks ago I took a look at Qboot, a minimal x86 firmware for QEMU which can boot in milliseconds. Qboot was written by Paolo Bonzini and dates from 2015 and there is an LWN article with the original announcement.

I tried it on my machine and it booted in QEMU (with kvm) in about 20ms, from entering Qboot to entering Linux. Very impressive! I was intrigued as to what makes it so fast.

There is another repo, qemu-boot-time by Stefan Garzarella, which provides an easy means to benchmark the boot. It uses perf events in Linux to detect the start and end of Qboot.

Using x86 post codes, I added the same to U-Boot. Initially the boot time was 2.9 seconds! Terrible. Here is a script which works on my machine and measures the time taken for U-Boot boot, using the qemu-x86_64 target:

It turned out that almost two of the seconds were the U-Boot boot delay. Another 800ms was the PXE menu delay. With those removed the time dropped to 210ms, which is not bad.

Using CONFIG_NO_NET dropping CONFIG_VIDEO each shaved off about 50ms. I then tried passing the kernel and initrd through QEMU using the QFW interface. It only saved 15ms but it is something.

I figured that command-line processing would be quite slow. With CONFIG_CMDLINE disabled another 5ms was saved. A final 7ms came from disabling filesystems and EFI loader. Small gains.

In the end, my result is about 83ms (in bold below):

$ ./contrib/qemu-boot-timer.sh
starting perf
building U-Boot
running U-Boot in QEMU
waiting for a bit
qemu-system-x86_64: terminating on signal 15 from pid 2775874 (bash)
parsing perf results
1) pid 2779434
qemu_init_end: 51.924873
u_boot_start: 51.962744 (+0.037871)
u_boot_do_boot: 134.781048 (+82.818304)

One final note: the qemu-x86_64 target actually boots by starting SPL in 16-bit mode and then moving to 64-bit mode to start U-Boot proper. This was partly to avoid calling 16-bit video ROMs from 64-bit code. Now that bochs is used for the display, it should be easy enough to drop SPL for this target. I’m not sure how much time that would save.

Note: Some final hacks are tagged here.