Host-file Access with New virtio-fs

What is virtio-fs?

For those unfamiliar, virtio-fs is a modern shared filesystem designed specifically for virtualised environments. It allows a virtual machine (the “guest”) to access a directory on the host system, but it does so with a focus on performance and providing local filesystem semantics.

Unlike traditional methods like network filesystems (e.g., NFS, Samba) or even the older virtio-9p protocol, virtio-fs is engineered to take advantage of the fact that the guest and host are running on the same machine. By leveraging shared memory and a design based on FUSE (Filesystem in Userspace), it bypasses much of the communication overhead that can slow down other solutions. The result is a faster, more seamless file sharing experience that is ideal for development, testing, and booting from a root filesystem located on the host.

virtio-fs arrives in U-Boot Concept

The recent merge request in U-Boot Concept introduces a new virtio-fs driver within U-Boot. This initial implementation enables two key functions:

  • List directories on the host
  • Read files from the host

This is made possible by a new filesystem driver that integrates with U-Boot’s new FS, DIR, and FILE uclasses. A compatibility layer is included so that existing command-line functionalities continue to work as expected.

This new capability in U-Boot opens up more flexible and efficient workflows. For example, developers can now more easily load kernels, device tree blobs, or other artifacts directly from their development workstation into a QEMU guest running U-Boot, streamlining the entire test and debug cycle. For cloud use cases, reading configuration files from via virtio-fs is a common requirement.

Overall this lays a strong foundation for future enhancements to virtio-fs support within U-Boot, promising even tighter integration between guest environments and the host system.




Keeping Our Linker Lists in Line

U-Boot makes extensive use of linker-generated lists to discover everything from drivers to commands at runtime. This clever mechanism allows developers to add new features with a single macro, and the linker automatically assembles them into a contiguous array. The C code can then iterate through this array by finding its start and end markers, which are also provided by the linker.

For this to work, there’s a critical assumption: the array of structs is perfectly contiguous, with each element having the exact same size. But what happens when the linker, in its quest for optimisation, breaks this assumption?

A Little Wrinkle

We have known for a while about a subtle issue where the linker, in certain cases, would insert a few bytes of padding between elements in these lists. This is usually done to align the next element to a more efficient memory boundary (like 8 or 16 bytes).

While this is often harmless, it breaks U-Boot’s C code, which expects to find the next element by simply adding a fixed size to the address of the current one. This unexpected padding can lead to misaligned memory access, corrupted data, and hard-to-debug crashes.

Here is an example of what this looks like in the symbol table. Notice the gap between virtio_fs and virtio_fs_dir is 0x80 bytes, while the expected size is 0x78:

...
00000000011d0070 D _u_boot_list_2_driver_2_virtio_blk
00000000011d0160 D _u_boot_list_2_driver_2_virtio_fs
00000000011d01e0 D _u_boot_list_2_driver_2_virtio_fs_dir
...

This 8-byte padding (0x80 - 0x78) is the source of the problem.

A Script to the Rescue

To catch these alignment problems automatically, we’ve developed a new Python script, check_list_alignment.py, now in U-Boot Concept (merge).

The script works as follows:

  1. Runs nm -n on the final u-boot ELF file to get all symbols sorted by address.
  2. Automatically discovers all the different linker lists in use (e.g., driver, cmd, uclass_driver).
  3. For each list, calculates the gap between every consecutive element.
  4. Determines the most common gap size, assuming this is the correct sizeof(struct).
  5. Flags any gap that doesn’t match this common size.

Now, if the linker introduces any unexpected padding, the build will fail immediately with a clear error message:

$ ./scripts/check_list_alignment.py -v u-boot
List Name           # Symbols   Struct Size (hex)
-----------------   -----------   -----------------
...
driver                       65              0x78
  - Bad gap (0x80) before symbol: _u_boot_list_2_driver_2_virtio_fs_dir
...

FAILURE: Found 1 alignment problems

This simple check provides a powerful guarantee. It ensures the integrity of our linker lists, prevents a whole class of subtle bugs, and allows developers to continue using this powerful U-Boot feature with confidence.




Filesystems in U-Boot

U-Boot supports a fairly wide variety of filesystems, including ext4, ubifs, fat, exfat, zfs, btrfs. These are an important part of bootloader functionality, since reading files from bare partitions or disk offsets is neither scalable nor convenient.

The filesystem API is functional but could use an overhaul. The main interface is in fs/fs.c, which looks like this:

struct fstype_info {
	int fstype;
	char *name;
	/*
	 * Is it legal to pass NULL as .probe()'s  fs_dev_desc parameter? This
	 * should be false in most cases. For "virtual" filesystems which
	 * aren't based on a U-Boot block device (e.g. sandbox), this can be
	 * set to true. This should also be true for the dummy entry at the end
	 * of fstypes[], since that is essentially a "virtual" (non-existent)
	 * filesystem.
	 */
	bool null_dev_desc_ok;
	int (*probe)(struct blk_desc *fs_dev_desc,
		     struct disk_partition *fs_partition);
	int (*ls)(const char *dirname);
	int (*exists)(const char *filename);
	int (*size)(const char *filename, loff_t *size);
	int (*read)(const char *filename, void *buf, loff_t offset,
		    loff_t len, loff_t *actread);
	int (*write)(const char *filename, void *buf, loff_t offset,
		     loff_t len, loff_t *actwrite);
	void (*close)(void);
	int (*uuid)(char *uuid_str);
	/*
	 * Open a directory stream.  On success return 0 and directory
	 * stream pointer via 'dirsp'.  On error, return -errno.  See
	 * fs_opendir().
	 */
	int (*opendir)(const char *filename, struct fs_dir_stream **dirsp);
	/*
	 * Read next entry from directory stream.  On success return 0
	 * and directory entry pointer via 'dentp'.  On error return
	 * -errno.  See fs_readdir().
	 */
	int (*readdir)(struct fs_dir_stream *dirs, struct fs_dirent **dentp);
	/* see fs_closedir() */
	void (*closedir)(struct fs_dir_stream *dirs);
	int (*unlink)(const char *filename);
	int (*mkdir)(const char *dirname);
	int (*ln)(const char *filename, const char *target);
	int (*rename)(const char *old_path, const char *new_path);
};

At first glance this seems like a reasonable API. But where is the filesystem specified? The API seems to assume that this is already present somehow.

In fact there is a pair of separate functions responsible for selecting which filesystem the API acts on:

int fs_set_blk_dev(const char *ifname, const char *dev_part_str, int fstype)
int fs_set_blk_dev_with_part(struct blk_desc *desc, int part)

When you want to access a file, call either of these functions. It sets three ‘global’ variables, fs_dev_desc, fs_dev_part and fs_type . After each operation, a call to fs_close() resets things. This means you must select the block device before each operation. For example, see this code in bootmeth-uclass.c:

	if (IS_ENABLED(CONFIG_BOOTSTD_FULL) && bflow->fs_type)
		fs_set_type(bflow->fs_type);

	ret = fs_size(path, &size);
	log_debug("   %s - err=%d\n", path, ret);

	/* Sadly FS closes the file after fs_size() so we must redo this */
	ret2 = bootmeth_setup_fs(bflow, desc);
	if (ret2)
		return log_msg_ret("fs", ret2);

It is a bit clumsy. Obviously this interface is not set up to support caching. In fact the filesystem is mounted afresh each time it is accessed. In a bootloader this is normally not too much of a problem. Since the OS and associated files are normally packaged in a FIT, a single read is enough to obtain everything that is needed. But if multiple directories need to be searched to find that FIT, or if there are multiple files to read, the repeated mounting does slow things down.

If you have sharp eyes you might have seen another problem. The two functions above assume that they are dealing with a block device. In fact, struct blk_desc is the uclass-private data for a block device. What about when the filesystem is on the network? Also, with sandbox it is possible to access host files:

=> ls hostfs 0 /tmp/gimp
DIR    1044480 ..
DIR       4096 .
DIR       4096 2.10
=> 

Clearly, the files on the hostsystem are not accessed at the block level. How does that work?

The key to this is null_dev_desc_ok , which is true for the hostfs filesystem. There is a special case in the code to handle this.

int blk_get_device_part_str(const char *ifname, const char *dev_part_str,
			     struct blk_desc **desc,
			     struct disk_partition *info, int allow_whole_dev)
{
...
#if IS_ENABLED(CONFIG_SANDBOX) || IS_ENABLED(CONFIG_SEMIHOSTING)
	/*
	 * Special-case a pseudo block device "hostfs", to allow access to the
	 * host's own filesystem.
	 */
	if (!strcmp(ifname, "hostfs")) {
		strcpy((char *)info->type, BOOT_PART_TYPE);
		strcpy((char *)info->name, "Host filesystem");

		return 0;
	}
#endif

It isn’t great. I’ve been looking at virtio-fs lately, which also doesn’t use a block device.

There are other things that could be improved, too:

  • Filesystems must be specified explicitly by their device and partition number. It would be nice to have a unified ‘VFS’ like Linux (and Barebox) so filesystems could be mounted within a unified space.
  • Files cannot be accessed from a device, nor is there any way to maintain a reference to a file you are working with
  • Reading a file must done all at once, in most cases. It would be nice to have an interface to open, read and close the file.

Instead of adding yet more special cases, it may be time to overhaul the code a little.




Verified Boot for Embedded on RK3399

VBE has been a long-running project to create a smaller and faster alternative to EFI. It was originally introduced as a concept in 2022, along with a sandbox implementation and a simple firmware updater for fwupd.

In the intervening period an enormous about of effort has gone into getting this landed in U-Boot for a real board. This has resulted in 10 additional series, on top of the sandbox work:

  • A – Various MMC and SPL tweaks (14 patches, series)
  • B – binman: Enhance FIT support for firmware (20 patches, series)
  • C – binman: More patches to support VBE (15 patches, series)
  • D – A collection of minor tweaks in MMC and elsewhere (18 patches, series)
  • E – SPL improvements and other tweaks (19 patches, series)
  • F – VBE implementation itself, with SPL ‘relocating jump’ (22 patches, series)
  • G – VBE ‘ABrec’ implementation in TPL/SPL/VPL (19 patches, series)
  • H – xPL-stack cleanup (4 patches, series)
  • I – Convert rockchip to use Binman templates (7 patches, series), kindly taken over and landed by Jonas Karlman
  • J – Implementation for RK3399 (25 patches, series)

That’s a total of 163 patches!

The Firefly RK3399 board was chosen, since it has (just) enough SRAM and is fully open source.

The final series has not yet landed in the main tree and it is unclear whether it will. For now I have put it in the Concept tree. You can see a video of it booting below:

I have been thinking about why this took so long to (almost) land. Here is my list, roughly in order from most important to least:

  1. Each series had to land before the next could be sent, with it taking at least one release cycle (3 months) to land each one
  2. Some of the new features were difficult to implement, particularly the relocating SPL jump and the new Binman features
  3. Many of the patches seemed aimless or irrelevant when sent, since they had no useful purpose before VBE could fully land. This created resistance in review
  4. On the other hand, sending too many patches at once would cause people to ignore the series

Overall it was a very difficult process, even for someone who knows U-Boot well. It concerns me that it has become considerably harder to introduce major new things in U-Boot, compared to the days of sandbox or driver model. I don’t have much of a comparison with other firmware projects, but I’m interested in hearing other people’s point of view. Please add a comment if you have thoughts on this.

Anyway, I am pleased to be done with it. The only thing missing at present is ‘ABrec’ updates in fwupd. It should be fairly easy to do, but for the signature checking. Since fwupd has its own implementation of libfdt, that might be non-trivial.

More information on VBE: