If you have ever run make htmldocs in U-Boot, you are likely familiar with the “wall of text” it produces. Between the standard Sphinx output, sub-make messages, and custom progress indicators, the build process has traditionally been very noisy.

While verbose output can be useful for debugging the toolchain itself, it is a hindrance when you are just trying to write documentation. The sheer volume of text makes it difficult to spot legitimate warnings and errors, which often get buried in the scroll.

A recent 5-part series in Concept addresses this. The goal is simple: if the build prints something, it should be something that requires your attention.

What changed?

The series cleans up the output in three specific ways:

1. Enabling Quiet Mode: We now pass the -q flag to SPHINXOPTS and -s to the sub-make invocations. This suppresses the standard “reading sources” and “picking up dependencies” log lines.
2. Removing Informational Clutter: We dropped the explicit print statement regarding CJK (Chinese/Japanese/Korean) font support in conf.py. The functionality remains, but we don’t need to be told about it on every run.
3. Dropping Custom Progress Bars: The SPHINX and PARSE progress messages defined in the Makefiles have been removed.

The Result

The documentation build is now silent by default. This aligns the htmldocs target with the philosophy of the rest of the Kbuild system: silence is golden.

Now, when you run the docs build, you can be confident that any output appearing in your terminal is a warning or an error that needs to be fixed. This should make it significantly easier to maintain clean documentation and spot regressions in CI.

In the world of bootloaders, balance is everything. We need enough functionality to load an OS, but we must keep the footprint small and the code maintainable. For years, U-Boot has relied on a custom implementation of the EXT4 filesystem. However, as the EXT4 format evolves with features like metadata checksums and fast commits, keeping a custom implementation in sync with the Linux kernel becomes a monumental task.

Today, we are excited to share a major milestone in U-Boot’s filesystem support: the introduction of ext4l.

What is ext4l?

The ext4l driver (the ‘l’ stands for Linux) is a direct port of the Linux kernel’s EXT4 driver (from v6.18) into U-Boot. Instead of rewriting the logic, we have imported the actual kernel source files—including super.c, inode.c, mballoc.c, and the jbd2 journaling layer—and provided a thin compatibility shim to make them “feel at home” in U-Boot.

Why This Approach?

1. Feature Parity: By using the real kernel code, we gain immediate support for modern EXT4 features like extents, flex_bg, and CRC32C metadata verification.
2. Maintainability: When the Linux kernel community fixes a bug or adds an optimization, we can pull those changes into U-Boot with minimal friction.
3. Reliability: We are leveraging code that has been battle-tested on millions of production servers and devices.

The Engineering Journey: Stubs and Shims

Porting kernel code to a bootloader is no small feat. The Linux kernel assumes a rich environment of threads, complex memory management, and a full VFS (Virtual File System) layer. U-Boot, by comparison, is largely single-threaded and simplified.

To bridge this gap, we developed a sophisticated compatibility layer:

• ext4_uboot.h & stub.c: These files provide the “glue.” They map kernel structures like struct super_block and struct inode to U-Boot’s block I/O layer.
• Dummy Shrinkers & Mutexes: Since U-Boot doesn’t have the same memory pressure or concurrency requirements, we implemented “no-op” or dummy versions of kernel infrastructure like the memory shrinker and various locking primitives.
• Buffer Cache: We added a buffer_head I/O infrastructure to support caching, which significantly improves read performance during the boot process.

---

Key Milestones in the Patch Series

The journey to a successful mount was achieved through a systematic build-up of the filesystem stack:

Component	Description
The Foundation	Importing core files from Linux 6.18 (inode.c, extents.c, super.c).
JBD2 Support	Porting the Journaling Block Device layer to ensure compatibility with journaled filesystems.
The Glue	Implementing sb_getblk, sb_bread, and brelse to connect the kernel code to U-Boot’s disk drivers.
The Fixes	Resolving critical issues in rbtree implementations and implementing the Castagnoli CRC32C algorithm for checksums.
Success	The final commit ee3d2214 enables successful read-only mounting, allowing U-Boot to see the root dentry and traverse the filesystem.

Current Status

As of the latest commits, ext4l is capable of:

• Mounting EXT4 filesystems in read-only mode.
• Verifying metadata via CRC32C.
• Using the extent status cache for efficient file lookups.
• Toggling debug output via CONFIG_EXT4L_DEBUG.

While the long-term goal is to support write operations (the mballoc and transaction code is already building), we are prioritizing rock-solid read support for the current release.

How to Try It

You can enable the new driver in your board configuration by selecting:

CONFIG_FS_EXT4L=y

This will replace the legacy EXT4 implementation with the Linux-ported version. We encourage developers to test this on various hardware and provide feedback on the mailing list.

---

A special thanks to Claude Opus 4.5 for co-developing these patches and assisting with the extensive stubbing required to bring this to life.

Managing cherry-picks across multiple branches is one of the more tedious aspects of maintaining a large project like U-Boot. When you need to backport dozens of commits from an upstream branch while handling merge commits, resolving conflicts, and creating merge requests, the process can consume hours of developer time.

Today we’re introducing pickman, a new tool in the U-Boot Concept tree that automates cherry-pick workflows using AI assistance. Pickman combines database tracking, GitLab integration, and the Claude Agent SDK to transform what was once a manual, error-prone process into a streamlined, largely automated workflow.

The Problem

U-Boot maintainers regularly need to cherry-pick commits from upstream branches (like us/next) to integration branches. This involves:

• Identifying which commits haven’t been cherry-picked yet
• Handling merge commits that group related changes
• Resolving conflicts when commits don’t apply cleanly
• Creating merge requests for review
• Tracking which commits have been processed
• Responding to review comments on merge requests

With hundreds of commits to process, this becomes a significant time investment. Pickman aims to reduce this burden dramatically.

How Pickman Works

Database Tracking

Pickman maintains a local SQLite database (.pickman.db) that tracks:

• Source branches being monitored
• The last cherry-picked commit for each source
• Individual commit status (pending, applied, conflict, skipped)
• Merge requests created and their status
• Processed review comments

This persistent state allows pickman to resume work across sessions and avoid re-processing commits that have already been handled.

AI-Powered Cherry-Picking

The core innovation in pickman is its use of the Claude Agent SDK to handle the actual cherry-pick operations. When you run pickman apply, the tool:

1. Identifies the next set of commits to cherry-pick (typically grouped by merge commit)
2. Creates a new branch for the work
3. Invokes Claude to perform the cherry-picks
4. Claude handles any conflicts that arise, using its understanding of the codebase
5. Records the results in the database
6. Optionally pushes and creates a GitLab merge request

The AI agent can resolve many common conflicts automatically, understanding context like renamed files, moved code, and API changes. When it encounters something it can’t resolve, it reports the issue clearly.

GitLab Integration

Pickman integrates directly with GitLab to:

• Push branches and create merge requests automatically
• Monitor MR status (open, merged, closed)
• Fetch and process review comments
• Update the database when MRs are merged

This creates a closed loop where pickman can operate continuously, creating an MR, waiting for it to be reviewed and merged, then moving on to the next batch of commits.

Getting Started

Prerequisites

# Install the Claude Agent SDK
pip install claude-agent-sdk

# Install the GitLab API library
pip install python-gitlab

# Set up your GitLab token (or use ~/.config/pickman.conf)
export GITLAB_TOKEN="your-token-here"

Basic Usage

# Add a source branch to track
pickman add-source us/next abc123def

# See what commits are pending
pickman next-set us/next

# Apply the next batch of commits
pickman apply us/next --push

# Check comments on open merge requests, taking action as needed
pickman review --remote ci

# Run continuously until stopped
pickman poll us/next --interval 300

Key Commands

Command	Description
add-source	Register a new source branch to track
compare	Show differences between branches
next-set	Preview the next commits to be cherry-picked
apply	Cherry-pick commits using Claude
step	Process merged MRs and create new ones
poll	Run step continuously at an interval
review	Check MRs and handle review comments
count-merges	Show how many merges remain to process

The Workflow in Practice

A typical workflow using pickman looks like this:

# Initial setup - track the upstream branch starting from a known commit
$ pickman add-source us/next e7f94bcbcb0

# See how much work there is
$ pickman count-merges us/next
Found 47 merge commits to process

# Preview what's coming next
$ pickman next-set us/next
Next 3 commits to cherry-pick from us/next:
  - a1b2c3d: arm: Fix cache alignment issue
  - d4e5f6a: drivers: gpio: Add new driver
  - b7c8d9e: Merge "ARM improvements"

# Let pickman handle it
$ pickman apply us/next --push --remote ci --target master
Creating branch cherry-a1b2c3d...
Cherry-picking 3 commits...
Pushing to ci...
Creating merge request...
MR created: https://gitlab.com/project/-/merge_requests/123

# Later, process any review comments and continue
$ pickman poll us/next --remote ci --interval 300

The Human Review Loop

While pickman automates much of the cherry-pick process, human oversight remains central to the workflow. The tool is designed to work alongside maintainers, not replace them. Here’s how the review cycle works:

1. MR Creation

When pickman creates a merge request, it includes a detailed description with the source branch, list of commits, and a conversation log showing how Claude handled the cherry-picks. This gives reviewers full visibility into what happened.

2. Human Review

Maintainers review the MR just like any other contribution. They can:

• Examine the diff to verify the cherry-pick is correct
• Check that conflicts were resolved appropriately
• Request changes by leaving comments on specific lines
• Ask questions about why certain decisions were made

3. Automated Comment Handling

When reviewers leave comments, pickman’s review command detects them and invokes Claude to address the feedback:

$ pickman review --remote ci
Found 1 open pickman MR(s):
  !123: [pickman] arm: Fix cache alignment issue
Processing comments for MR !123...
  Comment from maintainer: "This variable name should match the upstream style"
  Addressing comment...
  Pushing updated branch...

Claude reads the comment, understands the requested change, modifies the code accordingly, and pushes an updated branch. The merge-request notes are updated with the new transcript. The database tracks which comments have been processed to avoid duplicate work.

4. Iteration

The review cycle can repeat multiple times. Each time a reviewer adds new comments, running pickman review or pickman poll will detect and address them. This continues until the reviewer is satisfied with the changes.

5. Approval and Merge

Once the maintainer is happy with the MR, he or she approves and merges it through GitLab’s normal interface. Pickman detects the merged status on its next step or poll cycle:

$ pickman step us/next --remote ci --target master
Checking for merged MRs...
  MR !123 has been merged - updating database
  Source us/next: abc123 -> def456
Creating next MR...
  Cherry-picking commits from def456...
  MR created: https://gitlab.com/project/-/merge_requests/124

The database is updated to record the new “last processed” commit, and pickman automatically moves on to the next batch of commits.

The Continuous Loop

With pickman poll, this entire cycle runs continuously:

$ pickman poll us/next --remote ci --target master --interval 300
Polling every 300 seconds (Ctrl+C to stop)...

[09:00] Checking for merged MRs... none found
[09:00] Checking for review comments... none found
[09:00] Open MR !123 pending review
[09:00] Sleeping 300 seconds...

[09:05] Checking for merged MRs... none found
[09:05] Checking for review comments... 2 new comments on !123
[09:05] Addressing comments...
[09:05] Pushed updates to MR !123
[09:05] Sleeping 300 seconds...

[09:10] Checking for merged MRs... !123 merged!
[09:10] Updated database: us/next now at def456
[09:10] Creating new MR for next commits...
[09:10] MR !124 created
[09:10] Sleeping 300 seconds...

The maintainer simply reviews each MR as it appears, adds comments when needed, and merges when satisfied. Pickman handles everything else automatically, creating a smooth continuous integration pipeline for cherry-picks. If manual intervention is needed, you can typically just make some edits and push an update to the branch.

Handling Merge Commits

One of pickman’s key features is intelligent handling of merge commits. Rather than cherry-picking merge commits directly (which often fails), pickman identifies the individual commits within a merge and processes them as a group. This ensures that related changes stay together in a single merge request.

The tool follows the first-parent chain to identify merge boundaries, which matches the typical workflow of merging topic branches into the main development branch.

History Tracking

Pickman maintains a .pickman-history file in the repo that records each cherry-pick operation, including:

• Date and source branch
• Branch name created
• List of commits processed
• The full conversation log with Claude

This provides an audit trail and helps with debugging when things don’t go as expected.

Future Directions

Pickman is an experimental tool. We will use the next few months to refine it and learn how best to envolve it.

Try It Out

Pickman is available in the U-Boot Concept tree under tools/pickman/. To run the tests:

$ ./tools/pickman/pickman test

---

We welcome feedback and contributions. If you maintain a branch that requires regular cherry-picks from upstream, give pickman a try and let us know how it works for your workflow.

Pickman was developed with significant assistance from Claude (Anthropic’s AI assistant) for both the implementation and the cherry-pick automation itself.

U-Boot has two testing worlds that rarely meet. Python tests are flexible and can set up complex scenarios – disk images, network configurations, boot environments. C tests are fast, debuggable, and run directly on hardware. What if we could combine them?

The Problem

Consider filesystem testing. You need to:

1. Create a disk image with specific files
2. Calculate MD5 checksums for verification
3. Mount it in U-Boot
4. Run read/write operations
5. Verify the results

The Python test framework handles steps 1-2 beautifully. But the actual test logic in Python looks like this:

output = ubman.run_command(f'{cmd}load host 0:0 {addr} /{filename}')
assert 'complete' in output
output = ubman.run_command(f'md5sum {addr} {hex(size)}')
assert expected_md5 in output

String parsing. Hoping the output format doesn’t change. No stepping through with a debugger when it fails (well actually it is possible, but it requires gdbserver). And try running this on real hardware without a console connection.

The Solution: Pass Arguments to C

What if Python could call a C test with parameters?

cmd = f'ut -f fs fs_test_load_norun fs_type={fs_type} fs_image={path} md5={expected}'
ubman.run_command(cmd)

And in C:

static int fs_test_load_norun(struct unit_test_state *uts)
{
    const char *fs_type = ut_str(0);
    const char *fs_image = ut_str(1);
    const char *expected_md5 = ut_str(2);

    ut_assertok(fs_set_blk_dev("host", "0", fs_type));
    ut_assertok(fs_read("/testfile", addr, 0, 0, &actread));
    ut_assertok(verify_md5(uts, expected_md5));

    return 0;
}

Real assertions. Real debugging. Real portability.

How It Works

1. Declare Arguments with Types

UNIT_TEST_ARGS(fs_test_load_norun, UTF_CONSOLE | UTF_MANUAL, fs,
               { "fs_type", UT_ARG_STR },
               { "fs_image", UT_ARG_STR },
               { "md5", UT_ARG_STR });

The UNIT_TEST_ARGS macro creates the test with argument definitions. Each argument has a name and type (UT_ARG_STR or UT_ARG_INT).

2. Parse on the Command Line

=> ut -f fs fs_test_load_norun fs_type=ext4 fs_image=/tmp/test.img md5=abc123

The ut command parses name=value pairs and populates uts->args[].

3. Access in C

const char *fs_type = uts->args[0].vstr;    // String access
int count = uts->args[1].vint;              // Integer access

Arguments are accessed by index in declaration order.

A Real Example: Filesystem Tests

Here’s the before and after for a filesystem size test.

Before (Pure Python):

def test_fs3(self, ubman, fs_obj_basic):
    fs_type, fs_img, _ = fs_obj_basic
    ubman.run_command(f'host bind 0 {fs_img}')
    output = ubman.run_command(f'{fs_type}size host 0:0 /{BIG_FILE}')
    ubman.run_command('printenv filesize')
    # Parse output, check values, hope nothing changed...

After (Hybrid):

def test_fs3(self, ubman, fs_obj_basic):
    fs_type, fs_img, _ = fs_obj_basic
    assert run_c_test(ubman, fs_type, fs_img, 'fs_test_size_big',
                      big=BIG_FILE)

static int fs_test_size_big_norun(struct unit_test_state *uts)
{
    const char *big = ut_str(2);
    loff_t size;

    ut_assertok(fs_size(big, &size));
    ut_asserteq_64((loff_t)SZ_1M * 2500, size);

    return 0;
}

The Python test is now 4 lines. The C test has real assertions and can easily be debugged.

The Private Buffer

Tests often need temporary storage – paths, formatted strings, intermediate results. Rather than allocating memory or using globals, each test gets a 256-byte private buffer:

static int my_test(struct unit_test_state *uts)
{
    // Build a path using the private buffer
    snprintf(uts->priv, sizeof(uts->priv), "/%s/%s", dir, filename);

    ut_assertok(fs_read(uts->priv, addr, 0, 0, &size));

    return 0;
}

No cleanup needed. The buffer is part of unit_test_state and exists for the life of each test.

Why Not Just Write Everything in C?

You could. But consider:

• Creating a 2.5GB sparse file with specific content: Python’s os and subprocess modules make this trivial
• Calculating MD5 checksums: One line in Python
• Setting up complex boot environments: Python’s pytest fixtures handle dependencies elegantly
• Parameterized tests: pytest’s @pytest.mark.parametrize runs the same test across ext4, FAT, exFAT automatically

The hybrid approach uses each language for what it does best.

Why Not Just Write Everything in Python?

• Debugging: GDB beats print statements
• Hardware testing: C tests run on real boards (and sandbox) without console parsing
• Speed: No string-parsing overhead; less back-and-forth across the Python->U-Boot console
• Assertions: ut_asserteq() gives precise failure locations
• Code coverage: C tests contribute to coverage metrics (once we get them!)

Getting Started

1. Declare your test with arguments:

static int my_test_norun(struct unit_test_state *uts)
{
    const char *input = ut_str(0);
    int expected = ut_int(1);

    // Your test logic here
    ut_asserteq(expected, some_function(input));

    return 0;
}
UNIT_TEST_ARGS(my_test_norun, UTF_CONSOLE | UTF_MANUAL, my_suite,
               { "input", UT_ARG_STR },
               { "expected", UT_ARG_INT });

2. Call from Python:

def test_something(self, ubman):
    ubman.run_command(f'ut -f my_suite my_test_norun input={value} expected={result}')

3. Check the result:

    output = ubman.run_command('echo $?')
    assert output.strip() == '0'

The Documentation

Full details are in the documentation. The filesystem tests in test/fs/fs_basic.c and test/py/tests/test_fs/test_basic.py serve as a complete working example.

This infrastructure was developed to convert U-Boot’s filesystem tests from pure Python to a hybrid model. The Python setup remains, but the test logic now lives in debuggable, portable C code.

Have you ever seen output like this from your test suite?

Running -858993444 bloblist tests

That’s not a buffer overflow or memory corruption. It’s a wierd interaction between linker alignment, compiler optimisations, and pointer arithmetic. Let me tell you how we tracked it down.

The Mystery

U-Boot uses ‘linker lists’ extensively – a pattern where the linker collects scattered data structures into contiguous arrays. Drivers, commands, and unit tests all use this mechanism. Each list has start (_1) and end (_3) markers, with entries (_2_*) in between.

To count entries, we use pointer subtraction:

#define ll_entry_count(_type, _list) \
    (ll_entry_end(_type, _list) - ll_entry_start(_type, _list))

Simple, right? The compiler divides the byte span by sizeof(struct) to get the count. Except sometimes it returns garbage.

The Clue: 0xCCCCCCCC

That -858993444 value caught my eye. In hex, it’s 0xCCCCCCCC – a suspiciously regular pattern. This isn’t random memory; it’s the result of a calculation.

GCC optimizes division by constants using multiplicative inverses. Instead of expensive division, it multiplies by a “magic number” and shifts. For dividing by 40 (a typical struct size), GCC generates something like:

movl    %eax, %edx
imulq   $-858993459, %rdx    ; magic number for /40
shrq    $34, %rdx

This optimization is mathematically correct – but only when the dividend is an exact multiple of the divisor. When it’s not, you get rubbish.

The Root Cause

U-Boot’s CONFIG_LINKER_LIST_ALIGN (32 bytes on sandbox) aligns each list’s start. But here’s the subtle bug: the end marker was also being aligned:

#define ll_entry_end(_type, _list)                    \
({                                                    \
    static char end[0] __aligned(32)                   \  // <-- Problem!
        __attribute__((used))                         \
        __section(".u_boot_list_"#_list"_3");         \
    (_type *)&end;                                    \
})

When the next list in memory has a higher alignment requirement, the linker inserts padding before our end marker. Our list might have 15 entries of 40 bytes each (600 bytes), but the span from start to end becomes 608 bytes due to padding.

608 / 40 = 15.2

Feed 608 into GCC’s magic-number division for 40, and out comes 0xCCCCCCCC.

The Fix

Change the end marker alignment to 1:

static char end[0] __aligned(1)  // No padding before end marker

Now the end marker sits immediately after the last entry, the span is an exact multiple, and pointer arithmetic works correctly.

Detection

We enhanced our check_linker_lists.py script to catch this:

1. Gap analysis: Compare gaps between consecutive symbols. Inconsistent gaps mean padding was inserted within the list.
2. Size comparison: Using nm -S to get symbol sizes, we check if gap > size. If so, padding exists.
3. Span verification: Check if (end - start) is a multiple of struct size. If not, pointer subtraction will fail.

./scripts/check_linker_lists.py u-boot -v

Lessons Learned

1. Magic number division is fragile: It only works for exact multiples. Any padding breaks it silently.
2. Zero-size markers inherit alignment from neighbours: The linker places them at aligned boundaries based on what follows.
3. Pointer arithmetic assumes contiguous arrays: This assumption is violated when padding sneaks in.
4. Garbage values often have patterns: 0xCCCCCCCC isn’t random – it’s a clue pointing to failed arithmetic.

The fix was one line. Finding it took considerably longer, but that’s firmware development for you.

This bug was found while working on U-Boot’s unit test infrastructure. The fix is in the patch “linker_lists: Fix end-marker alignment to prevent padding”.

The robust operation of any complex software system, especially one as foundational as U-Boot, hinges on the reliability of its core services. Among these, dynamic memory allocation via malloc() is paramount. While often taken for granted, failures in malloc() can be silent saboteurs, leading to unpredictable behaviour, security vulnerabilities, or outright system crashes. Here, we delve into the mechanisms for detecting and resolving these subtle yet critical issues, ensuring the integrity of U-Boot’s memory management.

The background for this topic is a recent series in Concept, which aims to improve tools and reliability in this area. It builds on the recent update to dlmalloc.

The Challenge of Dynamic Memory

In the constrained environment of an embedded bootloader, where resources are often tight and determinism is key, malloc() failures present unique challenges:

1. Subtlety: A failed malloc() call doesn’t always immediately manifest as a crash. It might return NULL, and if not meticulously checked, subsequent operations on this NULL pointer can lead to memory corruption, double-frees, or use-after-free vulnerabilities much later in execution.
2. Asynchronicity: With the use of the ‘cyclic’ feature, memory allocation can become a race condition, exacerbating the difficulty of reproducing and debugging issues.
3. Heap Fragmentation: Long-running systems or complex sequences of allocations and deallocations can lead to heap fragmentation, where sufficient total memory exists, but no contiguous block is large enough for a requested allocation. This is particularly insidious as it’s not a memory exhaustion issue per se, but an allocation-strategy issue.
4. Debugging Overhead: Traditional heap debugging tools can themselves consume significant memory and execution time, making them impractical for a bootloader.

---

Proactive Detection: The New Toolkit 🛠️

A new series in Concept introduces powerful new instrumentation and commands, moving U-Boot toward best-in-class memory debugging capabilities.

1. Real-Time Heap Statistics (malloc info)

The new malloc info command provides a clear, instantaneous snapshot of the heap’s health and usage patterns:

Statistic	Description
total bytes	Total size of the malloc() heap (set by CONFIG_SYS_MALLOC_LEN).
in use bytes	Current memory allocated and held by the application.
malloc count	Total number of calls to malloc().
free count	Total number of calls to free().
realloc count	Total number of calls to realloc().

This information is helpful for quickly identifying memory leaks (high malloc count with low/stagnant free count) or excessive memory churn (high total counts).

2. Caller Tracking and Heap Walk (malloc dump)

When enabled via CONFIG_MCHECK_HEAP_PROTECTION, the malloc dump command becomes the most potent debugging tool:

• Heap Walk: It systematically walks the entire heap, printing the address, size, and status (used or free) of every memory chunk.
• Allocation Traceability: For every allocated chunk, the header now stores a condensed backtrace string, showing the function and line number of the code that requested the memory:

  • 19a0e010   a0       log_init:453 <-board_init_r:774 <-sandbox_flow:

• Post-free() Analysis: This caller information is also preserved in the metadata of freed chunks. This is invaluable for detecting memory leaks, as you can see precisely which function allocated a chunk that is now free, or identifying potential double-free sources. Of course, free blocks can be reused, so this isn’t a panacea.

3. Heap Protection (mcheck)

The integration of the mcheck heap-protection feature embeds ‘canary’ data before and after each allocated chunk.

• Boundary Checking: These canaries are checked on free() and during heap walks. If a canary is corrupted, it instantly signals a buffer overflow or buffer underflow—a classic symptom of heap corruption.
• Detection: This shifts the memory integrity issue from a mysterious crash hours later to an immediate, localized fault, dramatically speeding up remediation.

---

How This Series Helps U-Boot Development

Note: Some of these features are currently largely available only on sandbox, U-Boot’s development and testing environment. In particular, there is currently no way to obtain line-number information at runtime on other architectures.

Overall, this series represents a qualitative shift in U-Boot’s memory diagnostics, providing a mechanism for detecting and finding the root cause of subtle memory bugs that were previously nearly impossible to find.

1. Pinpointing Leaks (Performance & Stability): Before this series, finding a memory leak was a slow process of elimination. Now, a simple malloc dump reveals which functions are responsible for the largest or most persistent allocated chunks, directly mapping resource usage to the source code (log_init:453 or membuf_new:420).
2. Tracking Heap Corruption (Reliability): Heap corruption is often caused by writing beyond the boundaries of an allocated buffer. With mcheck, this corruption is immediately detected. Furthermore, the malloc dump allows developers to see the call site of the corrupted chunk, leading you straight to the faulty allocation rather than searching half the code base.
3. Enabling Backtrace for Debugging: The series includes a refactor of the backtrace feature, ensuring that it no longer relies on malloc(). This guarantees that backtraces can be collected safely even when the allocator itself is in a compromised state (e.g., during an mcheck failure or stack smash), providing reliable context for crash reports.

Early results

This work has already yielded results. A huge memory leak involving the SCMI was discovered simply by looking at the malloc dump. A watchdog crash with ‘ut dm’ was pinpointed. Also it uncovered the very large number of allocations performed by the Truetype font engine, leading to a simple optimisation to reduce strain on the heap.

With this series U-Boot establishes the strong base for foundational diagnostics, transforming the challenge of memory debugging in a constrained environment into a manageable, data-driven process.

This final series in Concept closes out the complete implementation of TKey hardware-backed full disk encryption (FDE) in U-Boot.

The previous series established the core logic and UI flow. This final work wraps up the effort by providing end-to-end testing, a useful Python tool for key management on the host, along with documentation, making the feature ready for real-world secure-boot environments.

---

🛠️ The Final Pieces of the Puzzle

This series of 7 patches focuses on robust testing, external utility, and documentation.

1. Host-Side Key Management Tool

The core of this series is the new script, scripts/tkey_fde_key.py.

• Key Derivation: This script allows users on their host machine to use a physical TKey and a passphrase to derive the exact same disk encryption key that U-Boot derives at boot time.
• Disk Operations: It is an all-in-one utility that can then use the derived key to:

  • Encrypt a disk image (or partition) with LUKS.
  • Open/Unlock an already encrypted disk image for host access.
  • This capability is essential for creating encrypted root filesystems that U-Boot can later unlock using the TKey.

2. Comprehensive Testing & Infrastructure

To ensure this complex hardware-backed flow is reliable, the test infrastructure has been significantly upgraded:

• Full TKey Unlock Test: A major test, bootctl_logic_tkey, validates the entire TKey unlock state machine: passphrase entry, handling TKey removal and re-insertion prompts, app loading, key derivation, and final LUKS partition unlock.
• New Test Devices: The series adds two new disk images to the sandbox:

  • mmc13: A LUKS2-encrypted disk designed specifically for TKey-based unlock.
  • mmc14: A LUKS2-encrypted disk for testing the pre-derived master key path (via the -p flag in luks unlock), ensuring maximum flexibility.

• Keyfile Support: The Python filesystem helper is updated to support key files (instead of only passphrases) and the ability to specify a raw master key. This allows the automated tests to encrypt disks using the deterministic key generated by the TKey emulator.

3. Complete Documentation

The final piece is the doc/usage/tkey-fde.rst guide, which ties the entire workflow together.

• Workflow Explained: This new documentation covers the entire TKey FDE workflow, from how the key is derived from the User-Supplied Secret (USS) and the TKey’s Unique Device Identifier (UDI), to using the tkey_fde_key.py script and configuring U-Boot.
• Physical TKey Testing: It includes a guide on how to use a real TKey for testing by generating an override.bin key, allowing developers to switch seamlessly between the emulator and physical hardware.

---

What’s next?

This work has been an interesting expedition into LUKS and hardware-backed security. While it is complete from the U-Boot side, there are a few loose ends which could be looked at in future. For example, it would be useful to communicate the unlock key to Linux’s the ramdisk environment, so it doesn’t need to do another unlock (and prompt the user) on startup.

In any case, U-Boot now includes a complete, robust, and well-documented solution for hardware-backed full disk encryption, a significant step forward in the area of security.

A new series in Concept introduces the complete logic and UI enhancements required to use a TKey to unlock an encrypted disk (like a LUKS partition) before booting an operating system.

1. The TKey Unlock Flow

The TKey unlock process is complex because it involves iterative communication with the external hardware and handling scenarios like the TKey being in the wrong mode:

• State Machine: The series implements a simple state machine (using enum unlock_state) to manage the entire TKey process. This allows the UI to remain responsive while waiting for user input, TKey removal, TKey insertion, or the app loading process .
• App Loading: The user’s passphrase is used as the User-Supplied Secret (USS) to load a signing app onto the TKey. This is done incrementally (tkey_load_next) on each UI poll to prevent the UI from freezing.
• Key Derivation: Once the app is loaded, the TKey’s public key is used to derive the disk encryption key (via SHA256 hashing), which is then used to unlock the LUKS partition.

2. UI and UX Enhancements

The user interface has been significantly upgraded to support this secure flow:

• Visual Indicators: The series adds a lock image which appears next to any encrypted OS entry in the Bootctl UI, providing immediate visual feedback.
• Passphrase Prompt: A dedicated textline object is provided for the user to enter their passphrase/USS. The series ensures this text renders as asterisks (by setting the SCENEOF_PASSWORD flag) for security.
• User Messaging: The UI is enhanced to display dynamic messages to the user:

  • “Unlocking…” or “Preparing TKey… 50%” (progress updates during app load).
  • Error messages like “Incorrect passphrase.”
  • Hardware prompts like “Please remove TKey” or “Please insert TKey.”

• Autonomy: The series provides functions (bc_ui_show_pass, bc_ui_get_pass, etc.) to allow the core logic to control the visibility and content of the passphrase and message fields.

3. Stability and Testing

• LUKS2 Support: The logic is updated to ensure compatibility with LUKS2 partitions, which is crucial for modern Linux distributions.
• TKey Correction: A fix is included in the TKey driver logic to correctly handle the position and use of the USS hash during app loading.
• Test Environment: The series enables the Bootctl tests and includes configuration to use a TKey emulator within the Sandbox environment, ensuring the new TKey logic remains functional in the future.
• luks command enhancement: The luks unlock command gains a new -p flag, allowing a pre-derived master key to be passed, which is useful for testing or integration with external key management systems.

---

This series moves U-Boot’s Bootctl from simple boot selection toward a feature-rich, hardware-backed security manager, making it a viable candidate for handling encrypted system boots.

A new series just landed, focussed on making one part of U-Boot’s new graphical menus (Expo) considerably better with non-popup menus.

A long-standing and annoying limitation in expo is that couldn’t really type into text fields (like for passwords) if the menu wasn’t a pop-up (i.e. as used by the ‘cedit’ configuration editor). Now, with this work, you can actually use your keyboard!

---

The Highlights

1. The Series Enables Input! ⌨️

• When a text box is highlighted, key presses are correctly routed to the textline object. This is important for things like password or configuration fields embedded in a menu.
• No More Ghost Text: The series fixes a bug where the displayed text wouldn’t update visually because the text object was pointing at a static empty string instead of the textline’s live buffer. That’s squashed now—what gets typed is what shows up!

2. A Crazy Key Conflict is Resolved 🤯

• It turns out, pressing Up Arrow and hitting the Backspace/Delete Character shortcut (Ctrl-B) were sending the exact same signal to U-Boot. Messy!
• The Fix: This series moves the menu navigation keys to a different number range (0x100). Now, the system knows the difference between “move up” and “delete a character,” so everything works as it should!

3. Debugging is Easier Now (For the Devs) 🔍

• No More Silent-Mode Hassle: The series adds a new test flag (UTF_NO_SILENT) for tests that need the console to be active. This is vital for text-entry tests, as console rendering must be on for the textline state to update correctly.
• Better Names: The series makes the names of all the menu objects clearer (e.g., item0.label instead of a generic label). This hierarchical naming significantly simplifies tracing objects during debugging, particularly when dumping the expo.
• More Docs: New documentation covers how to write tests and debug the Expo system, including which command-line flags to use for watching the video frames in Sandbox.
• Hex Dumping: The series fixes a typo to enable the cedit dump command and switches its output to the standard U-Boot hexadecimal format.

---

Bottom line: The series makes Expo more flexible, ensures textlines work everywhere, and improves the code base for easier debugging and maintenance.

For over two decades—since 2002—U-Boot has relied on version 2.6.6 of Doug Lea’s malloc (dlmalloc, old docs) to handle dynamic memory allocation. While reliable, the codebase was showing its age.

In a massive 37-patch series, we have finally updated the core allocator to dlmalloc 2.8.6. This update brings modern memory efficiency algorithms, better security checks, and a cleaner codebase, all while fighting to keep the binary size footprint minimal for constrained devices.

Why the Update?

The move to version 2.8.6 isn’t just about bumping version numbers. The new implementation offers specific technical advantages:

• Improved Binning: The algorithm for sorting free chunks is more sophisticated, leading to better memory efficiency and less fragmentation.
• Overflow Protection: Robust checks via MAX_REQUEST prevent integer overflows during allocation requests, improving security.
• Reduced Data Usage: The old pre-initialised av_[] array (which lived in the data section) has been replaced with a _sm_ struct in the BSS (Block Started by Symbol) section. This change reduces usage in the data section by approximately 1.5KB, a significant win for boards with constraints on initialised data.

The Battle for Code Size

One of the biggest challenges in embedded development is code size. Out of the box, the newer dlmalloc was significantly larger than the old version—a non-starter for SPL (Secondary Program Loader) where every byte counts.

To combat this, the patch series a Kconfig option to strip down the allocator for space-constrained SPL builds (specifically via CONFIG_SYS_MALLOC_SMALL).

Key optimisations include:

1. NO_TREE_BINS: Disables complex binary-tree bins for large allocations (>256 bytes), falling back to a simple doubly-linked list. This trades O(log n) performance for O(n) but saves ~1.25KB of code.
2. SIMPLE_MEMALIGN: Simplifies the logic for aligned allocations, removing complex retry fallback mechanisms that are rarely needed in SPL. This saves ~100-150 bytes.
3. NO_REALLOC_IN_PLACE: Disables the logic that attempts to resize a memory block in place. Instead, it always allocates a new block and copies data. This saves ~500 bytes.

With these adjustments enabled, the new implementation is actually smaller than the old 2.6.6 version on architectures like Thumb2.

Keeping U-Boot Features Alive

This wasn’t a clean upstream import. Over the last 20 years, U-Boot accrued many custom modifications to dlmalloc. This series carefully ports these features to the new engine:

• Pre-relocation Malloc: Support for malloc_simple, allowing memory allocation before the main heap is initialised.
• Valgrind Support: Annotations that allow developers to run U-Boot sandbox under Valgrind to detect memory leaks and errors.
• Heap Protection: Integration with mcheck to detect buffer overruns and heap corruption.

New Documentation and Tests

To ensure stability, a new test suite (test/common/malloc.c) was added to verify edge cases, realloc behaviours, and large allocations. Additionally, comprehensive documentation has been added to doc/develop/malloc.rst, explaining the differences between pre- and post-relocation allocation and how to tune the new Kconfig options.

Next Steps

The legacy allocator is still available via CONFIG_SYS_MALLOC_LEGACY for compatibility testing, but new boards are encouraged to use the default 2.8.6 implementation.