Introduction

Pickman is a tool for cherry-picking patches from upstream U-Boot into a downstream branch. It walks the first-parent chain of the source branch, finds merge commits, and cherry-picks their contents one merge at a time using an AI agent.

This works well for normal merges containing a handful of commits. But upstream U-Boot occasionally has mega-merges — large merges whose second parent itself contains sub-merges. A single Merge branch ‘next’ can pull in dozens of sub-merges, each with their own set of commits, totalling hundreds of patches.

Feeding all of these to the AI agent at once overwhelms it. The context window fills up, conflicts become harder to diagnose, and a single failure can require restarting a very large batch. A better approach is to break the mega-merge into smaller pieces and process them one at a time.

The problem

Consider a typical mega-merge on the first-parent chain:

    A---B---M---C---D      (first-parent / mainline)
            |
            S1--S2--S3     (second parent, containing sub-merges)

When pickman encounters merge M, it collects all commits in prev..M and hands them to the agent. For a normal merge this is fine, a few commits at most. But when S1, S2 and S3 are themselves merges, each containing many commits, the batch can be very large.

The solution

Two new functions handle this:

• detect_sub_merges(merge_hash) – examines the second parent’s first-parent chain to find merge commits (sub-merges) within a larger merge. Returns a list of sub-merge hashes in chronological order, or an empty list if no sub-merges are present.
• decompose_mega_merge(dbs, prev_commit, merge_hash, sub_merges) – returns the next unprocessed batch from a mega-merge.

The second function handles three phases:

1. Mainline commits — commits between the previous position and the merge’s first parent (prev..M^1). These are on the mainline and are not part of any sub-merge.
2. Sub-merge batches — one sub-merge at a time, skipping any whose commits are already tracked in the database. Each call returns the commits for the next unprocessed sub-merge.
3. Remainder commits — any commits after the last sub-merge on the second parent’s chain.

The function tracks progress via the database: commits that have been applied (or are pending in an open MR) are skipped automatically. On the next invocation, pickman picks up where it left off.

The advance_to field in NextCommitsInfo tells the caller whether to update the source position. When processing sub-merge batches, it is None (stay put, more batches to come). When the mega-merge is fully processed, it contains the merge hash so the source advances past it.

The next-merges command also benefits: it now expands mega-merges in its output, showing each sub-merge with a numbered index:

    Next merges from us/next (15 from 3 first-parent):
      abc1234abcd Merge branch 'next' (12 sub-merges):
        1. def5678defg Merge branch 'dm-next'
        2. 1112222abcd Merge branch 'video-next'
        ...
      13. 333aaaa4444 Merge branch 'minor-fixes'
      14. 444bbbb5555 Merge branch 'doc-updates'

Rewind support

When something goes wrong partway through a mega-merge, it helps to be able to step backwards. The new rewind command walks back N merges on the first-parent chain from the current source position, deletes the corresponding commits from the database, and resets the source to the earlier position.

By default it performs a dry run, showing what would happen:

    $ pickman rewind us/next 2
    [dry run] Rewind 'us/next': abc123456789 -> def567890123
      Target: def5678901 Merge branch 'dm-next'
      Merges being rewound:
        abc1234abcd Merge branch 'video-next'
        bbb2345bcde Merge branch 'usb-next'
      Commits to delete from database: 47
    Use --force to execute this rewind

It also identifies any open MRs on GitLab whose cherry-pick branches correspond to commits in the rewound range, so that these can be cleaned up (normally deleted).

Summary

These changes allow pickman to handle the full range of upstream merge topologies with less manual intervention. Normal merges continue to work as before. Mega-merges are transparently decomposed into manageable batches, with progress tracked across runs and the ability to rewind when needed.

Introduction

U-Boot on x86_64 has traditionally relied on a Secondary Program Loader (SPL) to bootstrap into 64-bit mode. SPL starts in 16-bit real mode (as required by the x86 reset vector), transitions through 32-bit protected mode, sets up page tables, and finally jumps into the 64-bit U-Boot proper.

A recent series adds support for running U-Boot directly from ROM on x86_64 without SPL, using QEMU as the development platform. While this is a simpler configuration, getting it to work required solving several interesting architectural challenges at the intersection of x86 hardware, compiler conventions, and virtual-machine emulation.

Why Skip SPL?

The SPL adds complexity and boot time. On platforms like QEMU where the firmware image runs directly from a flat ROM, the extra SPL stage is unnecessary. A single-binary U-Boot that transitions from 16-bit to 64-bit mode internally is simpler to build, debug, and deploy.

This also serves as a foundation for future work on other x86_64 platforms that may not need SPL.

The Challenges

Read-only .data in ROM

When running from ROM (SPI flash), the .data section is in read-only flash memory. The existing x86_64 code stored the global data (gd) pointer in a global variable, which lives in .data. Writing to it before relocation to RAM would fault.

The solution was to use MSR_FS_BASE to hold the gd pointer address, mirroring how 32-bit x86 uses the FS segment descriptor base. This is a CPU register, so it works regardless of whether .data is writable:

    static inline void set_gd(volatile gd_t *gd_ptr)
    {
        gd_t *p = (gd_t *)gd_ptr;

        p->arch.gd_addr = p;
        asm volatile("wrmsr" : :
            "c" (MSR_FS_BASE),
            "a" ((unsigned int)(unsigned long)&p->arch.gd_addr),
            "d" ((unsigned int)((unsigned long)&p->arch.gd_addr >> 32))
            : "memory");
    }

This seemingly simple change had a knock-on effect: EFI runtime services that accesses gd->relocaddr crashes when the Linux kernel called SetVirtualAddressMap(), because the kernel repurposes the FS register for its own per-CPU data. The fix was to cache gd->relocaddr in an __efi_runtime_data variable during U-Boot initialisation, before the kernel takes over.

Mixing 32-bit and 64-bit Code

The x86 reset vector runs in 16-bit real mode. U-Boot’s existing 16-bit startup code (start16.S, resetvec.S) is designed for 32-bit builds. On x86_64, the main binary is compiled as position-independent 64-bit code (PIE), which is fundamentally incompatible with 16-bit/32-bit startup code.

The solution was to compile the 16-bit startup objects as 32-bit code and link them into a separate 32-bit ELF, then extract just the raw binary from it. The build system includes this binary in the final ROM image at the correct reset vector address. This required several build-system changes:

• Moving 16-bit binary rules from the top-level Makefile to arch/x86/Makefile
• Compiling startup objects with explicit -m32 flags on x86_64
• Linking them into a separate 32-bit ELF (u-boot-x86-start16.elf) distinct from the 64-bit main binary

32-bit to 64-bit Transition

A new assembly file (start_from_32.S) handles the transition from the 32-bit startup environment to 64-bit long mode:

1. Build identity-mapping page tables in RAM (1 GiB pages for simplicity)
2. Enable PAE (CR4.PAE) and load the page table base (CR3)
3. Set the long-mode-enable bit in MSR_EFER
4. Enable paging (CR0.PG), which activates long mode
5. Load a 64-bit GDT and perform a far jump to the 64-bit entry point

Why not write this in C? Well, the page tables must be created before enabling 64-bit long mode, but the C code is compiled as 64-bit and cannot execute until long mode is active. Since the setup is just a few store-loops filling PML4 and PDPT entries, assembly is simpler than compiling and linking a separate 32-bit C function just for the page tables.

One subtle requirement emerged during testing with KVM: the GDT used for the mode transition must be in RAM, not ROM. The CPU performs an implicit data read from the GDT during the far jump to load the 64-bit code-segment descriptor. While normal instruction fetches from ROM work fine, KVM cannot service this implicit GDT read from the ROM region (an EPT mapping limitation?). The symptom is a silent hang at the far-jump instruction with no exception or output. The fix is to copy the GDT from ROM to RAM with rep movsl before loading it with lgdt.

SSE: A Hidden Requirement

x86_64 GCC assumes SSE2 is always available (it is part of the x86_64 baseline) and freely generates SSE instructions such as movq %xmm0. If the SSE control bits are not set in the CPU control registers, these instructions cause an invalid-opcode exception (#UD), manifesting as a triple fault and boot loop after relocation.

The startup code must set CR4.OSFXSR and clear CR0.EM before any compiler-generated code runs.

Regparm and Calling Conventions

The x86 32-bit builds use -mregparm=3 to pass function arguments in registers rather than on the stack, improving performance and code size. However, this is a 32-bit-only GCC option and is incompatible with x86_64 (which already uses registers by default per the System V AMD64 ABI). A new Kconfig option (X86_NO_REGPARM) allows disabling this for 32-bit builds. The provides better interoperability with Rust, for example.

The Result

The new qemu-x86_64_nospl board is a single U-Boot binary that boots directly from the QEMU ROM, transitions to 64-bit mode, and can launch an operating system directly or via EFI. It is tested in CI alongside the existing SPL-based configurations. CI tests confirm that it can boot Linux correctly.

To try it with the build-qemu script:

./scripts/build-qemu -a x86 -rsX       # TCG (software emulation)
./scripts/build-qemu -a x86 -rsXk      # KVM (hardware virtualisation) 

Series Overview

The series consists of 12 patches:

1. Allow disabling regparm — adds X86_NO_REGPARM Kconfig option for x86_64 compatibility
2. MSR_FS_BASE for gd pointer — eliminates the writable .data dependency for the global-data pointer on x86_64
3. Cache gd->relocaddr for EFI — fixes the EFI runtime crash caused by the MSR_FS_BASE change
4. Build-system changes — restructure 16-bit startup code compilation to support mixed 32/64-bit linking
5. (continued)
6. (continued)
7. (continued)
8. 32-to-64-bit startup code — the assembly that transitions from 32-bit protected mode to 64-bit long mode
9. New defconfig — qemu-x86_64_nospl board configuration
10. MTRR setup — enable memory-type range register configuration for the no-SPL path
11. build-qemu integration — add –no-spl (-X) option to the QEMU helper script
12. CI coverage — add the new board to the continuous-integration test matrix

Debugging Tips

Debugging early x86 boot code in a virtual machine has its own set of tricks:

• QEMU exception logging: qemu-system-x86_64 -d int -D /tmp/log.log logs all CPU exceptions. Search for v=06 (#UD), v=0e (#PF), v=0d (#GP).
• Instruction tracing: add -d in_asm to trace executed instructions. Useful for finding where the CPU diverges from the expected path.
• KVM limitation: -d int does not work with KVM. Use the QEMU monitor (-monitor telnet:localhost:4444,server,nowait) or serial-port output (outb to 0x3f8) instead.
• Identifying SSE faults: look for v=06 (invalid opcode) in the exception log, then decode the instruction bytes (e.g. f3 0f 7e is the SSE movq instruction).

If you use U-Boot’s buildman tool frequently, you are likely familiar with the standard two-step dance. First, you run the build. Then, to really understand what happened—checking for code bloat, size changes, or new warnings—you run buildman -s to generate the summary.

While buildman effectively has two modes (building and summarising), treating them as mutually exclusive steps can be a hassle when you just want a quick health check on a branch.

A new patch aims to smooth out this workflow by adding a Build and Summary option.

The New -z Flag

The patch introduces a new command-line flag: -z (or --build-summary).

Previously, if you wanted to build a branch and immediately see the stats, you had to chain commands. Now, buildman can handle the compilation and immediately transition into generating the summary report within a single execution.

How it works

When you run buildman -z, the tool performs the build as usual. Once the build completes—regardless of whether it succeeded or failed—it automatically invokes the summary logic.

This is particularly useful for quickly iterating on a branch to reduce size growth.

The return code of the command still reflects the success or failure of the build.

Example

To build a branch for a specific board and see the size changes immediately:

buildman -b exph firefly-rk3399 -zSB

This simple addition removes friction from the development cycle, making it easier to keep an eye on binary sizes and warnings as you iterate.

Have you ever found yourself wishing for a bit more “modernity” while editing environment variables or command strings at the U-Boot prompt? Our latest patch series brings a suite of enhanced editing features to U-Boot, designed to make the command-line experience much more forgiving and efficient.

While these features improve the standard CLI, they were primarily designed to empower the expo framework, specifically for the new multi-line textedit widget.

---

Key Features at a Glance

This series introduces several “quality-of-life” improvements that seasoned terminal users will recognize:

• Multi-level Undo & Redo: Mistake? No problem. Use Ctrl+Z to undo and Ctrl+Shift+Z to redo.
• Emacs-style Yank/Paste: Text deleted with “kill” commands (like Ctrl+K) is saved to a yank buffer and can be pasted back with Ctrl+Y.
• Word Navigation: Move quickly through long strings using Ctrl+Left and Ctrl+Right arrows.
• Enhanced Multi-line Support: For expo-based editors, Home and End now intelligently navigate within the current line, and Ctrl+K kills text to the end of the line rather than the entire buffer.
• Sandbox SDL Support: We’ve added mapping for Home, End, and Ctrl+arrow keys in the sandbox console for easier testing and development.

---

Focus on Expo and Graphical Editing

The real star of this update is the integration with expo. We’ve added a new flag, SCENEOF_MULTILINE, which allows the textedit widget to handle multiple lines of text naturally—pressing Enter now inserts a newline rather than closing the widget.

To showcase this, we’ve updated the editenv command. You can now use the -e flag to trigger a graphical environment editor:

=> editenv -e my_long_variable

This opens a dedicated expo scene where you can use all the new undo/redo and navigation features in a clear, multi-line interface.

---

Configuration (Kconfig Options)

To enable these features in your build, you’ll need to look at these primary Kconfig options:

Option	Description
CONFIG_CMDLINE_EDITOR	The master switch for enhanced editing features. Enabled by default if EXPO is on.
CONFIG_CMDLINE_UNDO	Enables the undo/redo and yank/paste buffers.
CONFIG_CMDLINE_UNDO_COUNT	Configures the depth of the undo ring buffer (Default: 64).
CONFIG_EXPO_EDITENV	Enables the underlying graphical editor logic.
CONFIG_CMD_EDITENV_EXPO	Adds the -e flag to the editenv command to use the expo editor.

Note on Memory: Each undo level stores a full copy of the edit buffer. While 64 levels is the default, you can tune CONFIG_CMDLINE_UNDO_COUNT to fit the memory constraints of your specific board.

---

Technical Details

The implementation involves a new cli_editor_state structure that manages the redirection of character output and navigation callbacks. This allows the CLI logic to remain clean while supporting the specific needs of a graphical UI like expo. We’ve also addressed several memory leaks in expo object destruction and improved the video test infrastructure to help debug these graphical features more easily.

We’ve been working to improve the structure of the ext4l filesystem implementation in U-Boot, specifically targeting the large compatibility layer that allows us to reuse Linux kernel code.

We’ve just posted a new 33-patch series that reorganises the compatibility stubs, moving them out of the monolithic ext4_uboot.h and into their proper locations within include/linux/.

The Problem: A Monolithic Header

The ext4l filesystem uses a file named ext4_uboot.h to bridge the gap between U-Boot’s environment and the Linux kernel code it incorporates. To save time when porting ext4l, this header has become a dumping ground for every missing function, macro, and struct definition needed to make the code compile. It contains stubs for everything from inode operations to block device management, all mixed together.

This monolithic approach makes it difficult to see what was actually being stubbed and prevented other subsystems from using these compatibility definitions.

So why did we do it this way?

The header file has only been there a few months. Why didn’t we just put everything in the right place to begin with?

Well, it sounds easy, but actually it’s quite tricky. When porting ext4l there were thousands upon thousands of build errors and warnings. We could have brought in the Linux files one by one to try to get past the compilation step, i.e. to just be left with link errors. But the files continue a lot of stuff we don’t need in U-Boot, so that would have created lots of useless code to maintain.

Worse, most of that code would just add confusion, since there is no C implementation. Someone coming along later would see the prototype, call the function and wonder why it doesn’t work.

But the main issue was that the goal is to make the filesystem actually work. Until something is working, any stubbing or implementation work is speculative. The necessary pieces only fall into place when it is possible to mount a file system, list a directory, write to a file, etc.

So the approach chosen was to pull in, function by function, macro by macro, only what was needed to make the code build. This was still many weeks of work, not to mention around a week of time with Google Antimatter and Claude Code to build the initial prototype.

So ext4_uboot.h provided a feasible path to a functional implementation. The problem is, now we have an ext4_uboot.h hangover.

The Solution: Standard Linux Headers

This series takes a systematic approach to cleaning up this technical debt, building on half a dozen earlier series of a similar ilk. We move definitions from ext4_uboot.h to their standard Linux header locations in include/linux/

For example:

• File and inode operations (struct file_operations, struct inode_operations) are now in linux/fs.h
• Block device operations (blkdev_issue_discard, etc.) are now in linux/blkdev.h
• Wait queue primitives (DEFINE_WAIT) are in linux/wait.h
• DAX stubs are in linux/dax.h

We also created new headers where necessary, such as include/linux/fs/super_types.h for superblock operations, mirroring the kernel’s structure more closely.

Fixing list_sort

One notable fix in this series involves list_sort(). The function signature in U-Boot didn’t match the Linux kernel, lacking const qualifiers on the list heads in the comparison callback.

We updated list_sort() to match the kernel API:

typedef int (*list_cmp_func_t)(void *priv, const struct list_head *a,
                               const struct list_head *b);

This requires updating several callers across the codebase (UBIFS, EFI loader, STM32MP), but it ensures that we can use kernel code that relies on list_sort without modification.

Results

The result is a significant reduction in the size and complexity of ext4_uboot.h, shrinking it from over 550 lines to around 270 lines of essential, ext4-specific glue (at its peak the file held over 3000 lines!). The remaining code is much cleaner, with stubs residing where developers expect to find them.

This work completes the reorganisation of the ext4l compatibility layer, making future updates and maintenance significantly easier.

As the “Expo” menu system in U-Boot continues to mature, we are moving beyond simple menu selections and into more complex user interaction. One area needing significant attention is text input—specifically, how we handle multi-line text editing and the underlying video console cursor.

In a new 16-patch series, we overhaul the textedit object to support proper multiline editing and fundamentally change how we manage video console contexts.

The Cursor Conflict

Currently, the video console (vidconsole) operates with a single shared context for the cursor state. This works fine for a simple command line, but it becomes problematic when rendering complex UIs. If an Expo object needs to blink a cursor or accept input, it fights for control over the global cursor state. We typically have to rely on a clumsy save/restore mechanism (vidconsole_entry_save) to backup the console state before rendering a menu and restore it afterwards.

This series introduces independent vidconsole contexts.

Now, text-input objects (like textline and textedit) have their own dedicated vidconsole context. They maintain their own cursor position and state, completely independent of the main console or other objects. This allows us to delete the old entry_save and entry_restore mechanisms entirely, resulting in a nice cleanup of the video console drivers.

Multiline Navigation (Ctrl-P / Ctrl-N)

Editing text that spans multiple lines requires more than just moving forward and backward in a character buffer. Users expect to move visually up and down lines while maintaining their horizontal position—standard behaviour in any modern text editor.

We add support for visual line navigation using Ctrl-P (Previous line) and Ctrl-N (Next line).

Implementing this requires some interesting logic. Because the text wraps automatically based on the width of the container, “up” doesn’t mean “back 80 characters.” The implementation uses text measurement to calculate the pixel geometry of the wrapping. When a user moves up or down, we calculate the horizontal pixel offset of the current cursor, find the corresponding visual line, and place the cursor at the character index that matches that pixel alignment.

Key Changes Summary

• Independent Contexts: We add vidconsole_ctx to decouple cursor state for different objects.
• Cleanup: We remove vidconsole_entry_save/restore ops from console_normal and console_truetype.
• Navigation: We add logic to cread_line_process_ch and Expo to handle visual line jumping.
• Testing: We include tests in test/boot/expo.c covering keypress handling (Ctrl-B, Ctrl-F, Ctrl-W, etc.) and open/close cycles to ensure the display restores correctly.

This update paves the way for more sophisticated configuration screens within U-Boot, allowing users to edit longer configuration strings, scripts, or certificates directly from the bootloader UI.

U-Boot’s expo subsystem provides a way to create graphical menus and forms for user interaction. One key component is the textline object, which allows users to enter text, for example when typing a password or a filename.

This post describes recent work to support multiple vidconsole contexts, making it easier for expo to handle multiple independent text-input fields on the same display.

The Problem

The original textline implementation was designed around U-Boot’s command-line editor. When a textline is ‘opened’, the text is written out using vidconsole and the state is saved. When the user types a character, the state is restored, the character is written, and the state is saved again.

This approach has several limitations. Cursor movement is inefficient: when the user presses Ctrl-A to move to the start of the line, backspaces are written to reposition the cursor. Inserting a character then requires rewriting the entire string.

More fundamentally, all text operations share the same vidconsole context, making it impossible to have multiple, truly-independent text fields on the screen at once. Characters typed into a textline also appear on the text console, which is undesirable in a graphical interface.

Looking ahead, the approach cannot easily extend to a textedit widget that needs to handle multiple lines of text with up/down navigation. The tight coupling between expo and the CLI editor becomes a significant limitation.

The Solution

The solution involves threading a vidconsole context pointer through the entire video console stack. This enables each text-input object to maintain its own cursor position, font settings, CLI line state, and cursor appearance.

The changes span several layers. At the foundation, the vidconsole uclass now maintains a list of contexts and provides functions to allocate and free them. Each context contains cursor position, font metrics, and CLI state.

All vidconsole functions now accept an optional context parameter. When NULL is passed, the default context used by the command line is selected. This maintains backward compatibility while enabling the new functionality.

At the expo level, each textline object can now have its own vidconsole context, allocated when the object is created and freed when destroyed. The CLI line state, which tracks the text buffer and cursor position within the text, moves from the scene into each text-input object.

Implementation

The series begins by adding a context parameter to low-level functions like vidconsole_put_char() and vidconsole_set_cursor_pos(). This change propagates upward through vidconsole_put_string(), the escape sequence handler, and eventually to the expo rendering code.

With the plumbing in place, the CLI line state moves from being shared at the scene level to being owned by each text-input object. This means each textline tracks its own text buffer, cursor index, and editing state independently.

The vidconsole uclass gains an alist of contexts, managed through vidconsole_ctx_new() and vidconsole_ctx_dispose(). When expo creates a text-input object, it can request a dedicated context that will be used for all rendering operations on that field.

Finally, context allocation centralises in the uclass rather than being scattered across individual drivers. This reduces code-duplication and ensures consistent setup of cursor buffers, font metrics, and initial state.

Benefits

This refactoring enables forms with multiple independent text-input fields, each maintaining its own cursor and editing state. A login form could have separate username and password fields, each responding to input independently.

The clean separation between text-input objects and the global command-line state means that typing in an expo field no longer echoes characters to the text console. The user interface becomes self-contained.

Perhaps most importantly, this architecture provides the foundation for multi-line text editing widgets. A textedit control that supports scrolling through many lines of text, with up and down arrow navigation, becomes feasible once each widget owns its own context.

The size impact is modest: approximately 300 bytes on Thumb2 for boards that enable both command-line editing and expo. This is a reasonable cost for the added functionality and cleaner architecture.

Conclusion

Adding multiple vidconsole context support to U-Boot’s expo subsystem enables more sophisticated user interfaces with multiple independent text-input fields. While the implementation required threading a context pointer through many functions, the result is a cleaner architecture that supports future enhancements like multi-line editing and independent cursor blinking per field.

The changes maintain full backward compatibility. Existing code that passes NULL for the context continues to work exactly as before, using the default context shared with the command line.

U-Boot’s codman tool helps developers understand which source files and lines of code are actually compiled into a build. With the latest update, codman gains a powerful new capability: categorising source files by functional area and exporting analysis results to CSV for spreadsheet analysis.

The Challenge

U-Boot is a massive codebase with over 45,000 source files. When you build for a specific board, only a fraction of these files are compiled. But which fraction? And how does that code break down by function – how much is boot code, how much is driver code, how much is networking?

Previously, codman could tell you which files were used and show directory breakdowns, but there was no way to group files by their purpose. The new category system solves this.

Introducing Categories and Features

The category system uses a simple TOML configuration file (tools/codman/category.cfg) to define functional groupings:

[categories.load-boot]
description = "Loading & Boot"

[features.driver-model]
category = "drivers"
description = "Driver abstraction/model"
files = [
    "drivers/core/device-remove.c",
    "drivers/core/device.c",
    "drivers/core/dump.c",
   ...
]

[features.boot-linux-direct]
category = "load-boot"
description = "Boot Linux directly"
files = [
    "boot/bootm.c",
    "boot/bootm_final.c",
    "boot/bootm_os.c",
    "boot/bootmeth-uclass.c",
    "boot/bootmeth_cros.c",
    "boot/bootmeth_efi.c",
    "boot/bootmeth_efi_mgr.c",
    "boot/bootmeth_extlinux.c",
...
]

Files can be matched using:

• Exact paths: "boot/bootm.c"
• Glob patterns: "drivers/video/*.c"
• Directory prefixes: "lib/efi_loader/" (matches all files under the directory)

CSV Export for Spreadsheet Analysis

The new --csv option exports analysis results in a format ready for spreadsheet import:

codman -b qemu-x86 dirs -sf --csv report.csv

This produces output like:

Type,Path,Category,Feature,Files,Used,%Used,%Code,Lines,Used
dir,arch/x86/cpu,,,20,15,75,85,3816,3227
file,arch/x86/cpu/cpu.c,load-boot,boot-x86-bare,,,,88,399,353
file,arch/x86/cpu/irq.c,load-boot,boot-x86-bare,,,,100,366,366
...

For simpler output with just file data, use -F (files-only):

codman -b qemu-x86 dirs -sf --csv report.csv -F

Path,Category,Feature,%Code,Lines,Used
arch/x86/cpu/cpu.c,load-boot,boot-x86-bare,88,399,353
arch/x86/cpu/irq.c,load-boot,boot-x86-bare,100,366,366
...

Practical Applications

Understanding Build Composition

Import the CSV into a spreadsheet and create a pivot table by category to see how your build breaks down:

Category	Files	Lines	% of Total
load-boot	145	52K	21%
drivers	203	89K	36%
filesystem	67	31K	13%
…

Tracking Code Growth

Generate CSV reports over time to track how different functional areas grow or shrink as features are added or removed.

Identifying Uncategorised Files

Use -u to list files that don’t match any category:

codman -b qemu-x86 dirs -sf --csv report.csv -Fu

This helps identify gaps in your category configuration.

Creating graphs

Import the CSV into your favourite spreadsheet program and you can quickly create a useful graph:

Excluding External Code

External or vendored code can be excluded from reports using the [ignore] section:

[ignore]
files = [
    "lib/lwip/lwip/",    # External lwIP library
]

Ignored files are completely omitted from CSV output, keeping reports focused on code you maintain.

Getting Started

1. Build your board as usual: codman -b <board> dirs -sf --csv report.csv -F
2. Check for unmatched files: codman -b <board> dirs -sf -u
3. Edit tools/codman/category.cfg to add patterns for unmatched files
4. Regenerate the report and import into your favourite spreadsheet

What’s Next

The category system currently powers CSV output. Future work may extend categories to HTML reports and terminal output, enabling consistent functional-area analysis across all output formats.

We welcome contributions to expand category.cfg with better coverage of U-Boot’s functional areas. The goal is to have every source file mapped to a meaningful category, making it easy for anyone to understand what goes into a U-Boot build.

The PXE/extlinux boot mechanism is a cornerstone of U-Boot’s standard boot flow. It parses a configuration file (like extlinux.conf), presents a menu, and loads the kernel, initrd, and device tree for the chosen option. A typical configuration file looks like this:

menu title Boot Menu
timeout 50
default linux

label linux
    menu label Ubuntu Linux
    kernel /vmlinuz
    initrd /initrd.img
    fdt /dtb/board.dtb
    fdtoverlays /dtb/overlay1.dtbo /dtb/overlay2.dtbo
    append root=/dev/sda1 quiet

While functional, the traditional implementation relied heavily on callbacks: as the parser encountered a file path (like /vmlinuz above), it would immediately call a function to load it.

We have submitted a series of 26 patches that introduces a new, cleaner API for this process.

The Problem with Callbacks

The callback-based approach mixes parsing logic with file loading logic. This makes it difficult for a caller to know what files need to be loaded without actually loading them. It also complicates environments where file access requires special handling or where the caller wants precise control over memory layout.

Furthermore, analysing the existing parser revealed several memory leaks where token strings and menu structures were not being freed properly.

The New Three-Phase API

The new API separates the boot process into three distinct phases:

1. Parse (pxe_parse): The configuration file is parsed into a menu structure. Instead of loading files immediately, the parser populates a list of required files (kernel, initrd, FDT, overlays) for each label.
2. Load (pxe_load): The caller iterates over the file list and loads the files manually to their desired addresses. The pxe_load() helper is then called to record the size and location of each loaded file.
3. Boot (pxe_boot): Once everything is loaded, this function performs the final boot sequence using the pre-loaded file information.

This structure gives callers complete visibility and control. For example, a caller can now inspect the menu to see which files are required for a specific boot option before deciding where to load them.

Dealing with Includes

Configuration files often use the include directive to pull in other configs. In the new model, includes are not processed automatically. Instead, parsing populates an includes list. The caller is responsible for loading these included files and calling pxe_parse_include() to merge them into the menu. This explicit handling mirrors the control provided for boot files.

Improvements and Testing

Beyond the new API, this series includes several other improvements:

• Memory Leaks: Fixed leaks in token parsing, menu labels, and FDT overlay paths. These were trivial to find with the recent malloc additions.
• Explicit Sizing: The parser now accepts a file size parameter, removing the requirement for configuration files to be null-terminated strings.
• Documentation: Comprehensive documentation for the new API has been added to doc/develop/bootstd/pxe_api.rst.

Some recently added tests, such as pxe_test_sysboot_norun(), continue to verify the traditional end-to-end functionality via the sysboot command. We added a new test, pxe_test_files_api_norun(), to verify the new callback-free workflow. It parses a configuration file, manually loads the kernel, initrd, and device tree overlays, records their positions via pxe_load(), and then simulates a boot. This new test also explicitly checks for memory leaks, ensuring that pxe_cleanup() properly frees all allocated memory, including the new file lists.

While these changes increase code size slightly (around 300 bytes on Thumb2 mostly fixing memory leaks and avoid access outside the file bounds), the result is a more robust, flexible, and leak-free PXE subsystem.

Buildman is the Swiss Army knife of U-Boot development. It handles the heavy lifting of building hundreds of boards in parallel, fetching the correct toolchains, and—perhaps most importantly—analysing the impact of your patches across git history. Whether you are checking for code bloat or verifying that a refactor doesn’t break a board you don’t own, buildman is the tool that makes it possible.

However, if you have ever peered into the internals of buildman, you might have noticed that the Builder class had become… well, enormous. Over the years, it accumulated responsibilities ranging from managing threads and file I/O to formatting console output and tracking build statistics.

A recent series of 18 patches to address this technical debt. The goal: split the monolithic Builder class into smaller, more focused components.

The Problem: A Class That Did Too Much

The Builder class in tools/buildman/builder.py had grown to over 2200 lines of code. It was handling two distinct responsibilities:

1. Orchestrating the Build: Managing threads, checking out commits, running make, and saving artifacts.
2. Reporting Results: Processing build outcomes, calculating size deltas, tracking error lines, and formatting the summary for the user.

These two concerns were tightly coupled, making the code hard to read, test, and maintain.

The Solution: Introduce ResultHandler

This refactor extracts the result-processing logic (~900 lines) into a new class called ResultHandler. The separation is clean:

• Builder focuses on the act of building (I/O, git operations, process management).
• ResultHandler focuses on the presentation of the build (summaries, error reporting, bloat analysis).

How We Got There

Refactoring a core tool like buildman is risky—regressions here can break workflows for developers everywhere. We took a stepwise approach:

1. Extract Data Structures: We moved configuration handling (Config) and build outcomes (Outcome, BoardStatus, ErrLine) into their own modules (cfgutil.py, outcome.py). This broke the circular dependencies that often plague monolithic classes.
2. Formalise Options: Instead of passing a dozen boolean flags (like show_errors, show_bloat) around individually, we grouped them into a DisplayOptions named tuple.
3. Create the Handler: We introduced ResultHandler and progressively moved display methods into it—starting with simple size summaries and moving up to complex error delta calculations.
4. Clean Up: Finally, we enforced Python naming conventions by adding underscore prefixes to private members (self.col became self._col), clarifying the public API.

The Result

The Builder class is now down to ~1200 lines—a much more manageable size. The display logic lives in tools/buildman/resulthandler.py, where it can be evolved independently.

While this doesn’t change the user-facing behaviour of buildman today, it makes the codebase significantly healthier and ready for future improvements. Sometimes the most important feature is code you can actually understand!