The Best of Both Worlds: Hybrid Python/C Testing in U-Boot

U-Boot has two testing worlds that rarely meet. Python tests are flexible and can set up complex scenarios – disk images, network configurations, boot environments. C tests are fast, debuggable, and run directly on hardware. What if we could combine them?

The Problem

Consider filesystem testing. You need to:

1. Create a disk image with specific files
2. Calculate MD5 checksums for verification
3. Mount it in U-Boot
4. Run read/write operations
5. Verify the results

The Python test framework handles steps 1-2 beautifully. But the actual test logic in Python looks like this:

output = ubman.run_command(f'{cmd}load host 0:0 {addr} /{filename}')
assert 'complete' in output
output = ubman.run_command(f'md5sum {addr} {hex(size)}')
assert expected_md5 in output

String parsing. Hoping the output format doesn’t change. No stepping through with a debugger when it fails (well actually it is possible, but it requires gdbserver). And try running this on real hardware without a console connection.

The Solution: Pass Arguments to C

What if Python could call a C test with parameters?

cmd = f'ut -f fs fs_test_load_norun fs_type={fs_type} fs_image={path} md5={expected}'
ubman.run_command(cmd)

And in C:

static int fs_test_load_norun(struct unit_test_state *uts)
{
    const char *fs_type = ut_str(0);
    const char *fs_image = ut_str(1);
    const char *expected_md5 = ut_str(2);

    ut_assertok(fs_set_blk_dev("host", "0", fs_type));
    ut_assertok(fs_read("/testfile", addr, 0, 0, &actread));
    ut_assertok(verify_md5(uts, expected_md5));

    return 0;
}

Real assertions. Real debugging. Real portability.

How It Works

1. Declare Arguments with Types

UNIT_TEST_ARGS(fs_test_load_norun, UTF_CONSOLE | UTF_MANUAL, fs,
               { "fs_type", UT_ARG_STR },
               { "fs_image", UT_ARG_STR },
               { "md5", UT_ARG_STR });

The UNIT_TEST_ARGS macro creates the test with argument definitions. Each argument has a name and type (UT_ARG_STR or UT_ARG_INT).

2. Parse on the Command Line

=> ut -f fs fs_test_load_norun fs_type=ext4 fs_image=/tmp/test.img md5=abc123

The ut command parses name=value pairs and populates uts->args[].

3. Access in C

const char *fs_type = uts->args[0].vstr;    // String access
int count = uts->args[1].vint;              // Integer access

Arguments are accessed by index in declaration order.

A Real Example: Filesystem Tests

Here’s the before and after for a filesystem size test.

Before (Pure Python):

def test_fs3(self, ubman, fs_obj_basic):
    fs_type, fs_img, _ = fs_obj_basic
    ubman.run_command(f'host bind 0 {fs_img}')
    output = ubman.run_command(f'{fs_type}size host 0:0 /{BIG_FILE}')
    ubman.run_command('printenv filesize')
    # Parse output, check values, hope nothing changed...

After (Hybrid):

def test_fs3(self, ubman, fs_obj_basic):
    fs_type, fs_img, _ = fs_obj_basic
    assert run_c_test(ubman, fs_type, fs_img, 'fs_test_size_big',
                      big=BIG_FILE)

static int fs_test_size_big_norun(struct unit_test_state *uts)
{
    const char *big = ut_str(2);
    loff_t size;

    ut_assertok(fs_size(big, &size));
    ut_asserteq_64((loff_t)SZ_1M * 2500, size);

    return 0;
}

The Python test is now 4 lines. The C test has real assertions and can easily be debugged.

The Private Buffer

Tests often need temporary storage – paths, formatted strings, intermediate results. Rather than allocating memory or using globals, each test gets a 256-byte private buffer:

static int my_test(struct unit_test_state *uts)
{
    // Build a path using the private buffer
    snprintf(uts->priv, sizeof(uts->priv), "/%s/%s", dir, filename);

    ut_assertok(fs_read(uts->priv, addr, 0, 0, &size));

    return 0;
}

No cleanup needed. The buffer is part of unit_test_state and exists for the life of each test.

Why Not Just Write Everything in C?

You could. But consider:

• Creating a 2.5GB sparse file with specific content: Python’s os and subprocess modules make this trivial
• Calculating MD5 checksums: One line in Python
• Setting up complex boot environments: Python’s pytest fixtures handle dependencies elegantly
• Parameterized tests: pytest’s @pytest.mark.parametrize runs the same test across ext4, FAT, exFAT automatically

The hybrid approach uses each language for what it does best.

Why Not Just Write Everything in Python?

• Debugging: GDB beats print statements
• Hardware testing: C tests run on real boards (and sandbox) without console parsing
• Speed: No string-parsing overhead; less back-and-forth across the Python->U-Boot console
• Assertions: ut_asserteq() gives precise failure locations
• Code coverage: C tests contribute to coverage metrics (once we get them!)

Getting Started

1. Declare your test with arguments:

static int my_test_norun(struct unit_test_state *uts)
{
    const char *input = ut_str(0);
    int expected = ut_int(1);

    // Your test logic here
    ut_asserteq(expected, some_function(input));

    return 0;
}
UNIT_TEST_ARGS(my_test_norun, UTF_CONSOLE | UTF_MANUAL, my_suite,
               { "input", UT_ARG_STR },
               { "expected", UT_ARG_INT });

2. Call from Python:

def test_something(self, ubman):
    ubman.run_command(f'ut -f my_suite my_test_norun input={value} expected={result}')

3. Check the result:

    output = ubman.run_command('echo $?')
    assert output.strip() == '0'

The Documentation

Full details are in the documentation. The filesystem tests in test/fs/fs_basic.c and test/py/tests/test_fs/test_basic.py serve as a complete working example.

This infrastructure was developed to convert U-Boot’s filesystem tests from pure Python to a hybrid model. The Python setup remains, but the test logic now lives in debuggable, portable C code.