The Best of Both Worlds: Hybrid Python/C Testing in U-Boot
U-Boot has two testing worlds that rarely meet. Python tests are flexible and can set up complex scenarios – disk images, network configurations, boot environments. C tests are fast, debuggable, and run directly on hardware. What if we could combine them?
The Problem
Consider filesystem testing. You need to:
- Create a disk image with specific files
- Calculate MD5 checksums for verification
- Mount it in U-Boot
- Run read/write operations
- Verify the results
The Python test framework handles steps 1-2 beautifully. But the actual test logic in Python looks like this:
output = ubman.run_command(f'{cmd}load host 0:0 {addr} /{filename}')
assert 'complete' in output
output = ubman.run_command(f'md5sum {addr} {hex(size)}')
assert expected_md5 in output
String parsing. Hoping the output format doesn’t change. No stepping through with a debugger when it fails (well actually it is possible, but it requires gdbserver). And try running this on real hardware without a console connection.
The Solution: Pass Arguments to C
What if Python could call a C test with parameters?
cmd = f'ut -f fs fs_test_load_norun fs_type={fs_type} fs_image={path} md5={expected}'
ubman.run_command(cmd)
And in C:
static int fs_test_load_norun(struct unit_test_state *uts)
{
const char *fs_type = ut_str(0);
const char *fs_image = ut_str(1);
const char *expected_md5 = ut_str(2);
ut_assertok(fs_set_blk_dev("host", "0", fs_type));
ut_assertok(fs_read("/testfile", addr, 0, 0, &actread));
ut_assertok(verify_md5(uts, expected_md5));
return 0;
}
Real assertions. Real debugging. Real portability.
How It Works
1. Declare Arguments with Types
UNIT_TEST_ARGS(fs_test_load_norun, UTF_CONSOLE | UTF_MANUAL, fs,
{ "fs_type", UT_ARG_STR },
{ "fs_image", UT_ARG_STR },
{ "md5", UT_ARG_STR });
The UNIT_TEST_ARGS macro creates the test with argument definitions. Each argument has a name and type (UT_ARG_STR or UT_ARG_INT).
2. Parse on the Command Line
=> ut -f fs fs_test_load_norun fs_type=ext4 fs_image=/tmp/test.img md5=abc123
The ut command parses name=value pairs and populates uts->args[].
3. Access in C
const char *fs_type = uts->args[0].vstr; // String access int count = uts->args[1].vint; // Integer access
Arguments are accessed by index in declaration order.
A Real Example: Filesystem Tests
Here’s the before and after for a filesystem size test.
Before (Pure Python):
def test_fs3(self, ubman, fs_obj_basic):
fs_type, fs_img, _ = fs_obj_basic
ubman.run_command(f'host bind 0 {fs_img}')
output = ubman.run_command(f'{fs_type}size host 0:0 /{BIG_FILE}')
ubman.run_command('printenv filesize')
# Parse output, check values, hope nothing changed...
After (Hybrid):
def test_fs3(self, ubman, fs_obj_basic):
fs_type, fs_img, _ = fs_obj_basic
assert run_c_test(ubman, fs_type, fs_img, 'fs_test_size_big',
big=BIG_FILE)
static int fs_test_size_big_norun(struct unit_test_state *uts)
{
const char *big = ut_str(2);
loff_t size;
ut_assertok(fs_size(big, &size));
ut_asserteq_64((loff_t)SZ_1M * 2500, size);
return 0;
}
The Python test is now 4 lines. The C test has real assertions and can easily be debugged.
The Private Buffer
Tests often need temporary storage – paths, formatted strings, intermediate results. Rather than allocating memory or using globals, each test gets a 256-byte private buffer:
static int my_test(struct unit_test_state *uts)
{
// Build a path using the private buffer
snprintf(uts->priv, sizeof(uts->priv), "/%s/%s", dir, filename);
ut_assertok(fs_read(uts->priv, addr, 0, 0, &size));
return 0;
}
No cleanup needed. The buffer is part of unit_test_state and exists for the life of each test.
Why Not Just Write Everything in C?
You could. But consider:
- Creating a 2.5GB sparse file with specific content: Python’s
osandsubprocessmodules make this trivial - Calculating MD5 checksums: One line in Python
- Setting up complex boot environments: Python’s pytest fixtures handle dependencies elegantly
- Parameterized tests: pytest’s
@pytest.mark.parametrizeruns the same test across ext4, FAT, exFAT automatically
The hybrid approach uses each language for what it does best.
Why Not Just Write Everything in Python?
- Debugging: GDB beats print statements
- Hardware testing: C tests run on real boards (and sandbox) without console parsing
- Speed: No string-parsing overhead; less back-and-forth across the Python->U-Boot console
- Assertions:
ut_asserteq()gives precise failure locations - Code coverage: C tests contribute to coverage metrics (once we get them!)
Getting Started
1. Declare your test with arguments:
static int my_test_norun(struct unit_test_state *uts)
{
const char *input = ut_str(0);
int expected = ut_int(1);
// Your test logic here
ut_asserteq(expected, some_function(input));
return 0;
}
UNIT_TEST_ARGS(my_test_norun, UTF_CONSOLE | UTF_MANUAL, my_suite,
{ "input", UT_ARG_STR },
{ "expected", UT_ARG_INT });
2. Call from Python:
def test_something(self, ubman):
ubman.run_command(f'ut -f my_suite my_test_norun input={value} expected={result}')
3. Check the result:
output = ubman.run_command('echo $?')
assert output.strip() == '0'
The Documentation
Full details are in the documentation. The filesystem tests in test/fs/fs_basic.c and test/py/tests/test_fs/test_basic.py serve as a complete working example.
This infrastructure was developed to convert U-Boot’s filesystem tests from pure Python to a hybrid model. The Python setup remains, but the test logic now lives in debuggable, portable C code.
