robojerk/particle-os-tools
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
particle-os-tools/docs/apt-layer/composefs.md

10 KiB
Raw Permalink Blame History

ComposeFS Integration in apt-layer

TLDR - Quick Reference

Basic Commands

Create a ComposeFS image:

mkcomposefs <source-dir> <output.img> --digest-store=<object-store-dir>

Mount a ComposeFS image:

mount -t composefs -o basedir=<object-store-dir> <output.img> <mountpoint>
# or directly:
mount.composefs <output.img> <mountpoint> -o basedir=<object-store-dir>

Unmount:

umount <mountpoint>

Inspect an image:

composefs-info ls <image.composefs>                             # List files
composefs-info objects <image.composefs>                        # List backing files
composefs-info missing-objects <image.composefs> --basedir=<dir> # Check integrity
composefs-info dump <image.composefs>                           # Full metadata dump

Quick Example

# Create image with object store
mkcomposefs /path/to/rootfs myimage.composefs --digest-store=/path/to/objects

# Mount the image
mount -t composefs -o basedir=/path/to/objects myimage.composefs /mnt/composefs

# List contents
composefs-info ls myimage.composefs

# Unmount
umount /mnt/composefs

Note: In apt-layer, images are typically stored in /var/lib/apt-layer/images/ with object stores in the same directory. The above example uses generic paths for clarity.

Overview

apt-layer uses ComposeFS as its backend for atomic, deduplicated, and efficient filesystem layering—mirroring the approach used by rpm-ostree and Fedora Silverblue. ComposeFS is a Linux filesystem and image format designed for fast, space-efficient, and content-addressed deployment of system images.

Key Tools: The ComposeFS project provides a suite of tools including mkcomposefs for image creation, composefs-info for inspecting images, and mount.composefs for mounting. mount.composefs can be called directly or used by the standard mount -t composefs command.

Package Structure

Debian/Ubuntu Packages

ComposeFS is packaged in Debian/Ubuntu as three separate packages:

1. composefs (Main Tools Package)

Purpose: Userspace tools for ComposeFS operations
Contains:

  • /usr/bin/mkcomposefs - Create ComposeFS images
  • /usr/bin/composefs-info - Inspect and manage images
  • /usr/bin/mount.composefs - Mount helper for mount -t composefs
  • /usr/share/man/man1/ - Manual pages for all tools
  • /usr/share/doc/composefs/ - Documentation

2. libcomposefs1 (Runtime Library)

Purpose: Runtime shared library for ComposeFS
Contains:

  • /usr/lib/x86_64-linux-gnu/libcomposefs.so.1 - Runtime library
  • /usr/lib/x86_64-linux-gnu/libcomposefs.so.1.4.0 - Library version
  • Dependencies: glibc, libgcc, openssl-libs

3. libcomposefs-dev (Development Package)

Purpose: Development headers and pkg-config files
Contains:

  • /usr/include/libcomposefs/ - Header files
  • /usr/lib/x86_64-linux-gnu/libcomposefs.so - Development symlink
  • /usr/lib/x86_64-linux-gnu/pkgconfig/composefs.pc - pkg-config file
  • Dependencies: libcomposefs1 (= ${binary:Version})

Fedora/RHEL Packages

ComposeFS is packaged in Fedora/RHEL as three separate packages:

1. composefs (Main Tools Package)

Purpose: Userspace tools for ComposeFS operations
Contains: Same tools as Debian package

2. composefs-libs (Runtime Library)

Purpose: Runtime shared library for ComposeFS
Contains: Same library as Debian libcomposefs1

3. composefs-devel (Development Package)

Purpose: Development headers and pkg-config files
Contains: Same development files as Debian libcomposefs-dev

Installation Commands

Debian/Ubuntu:

sudo apt install -y composefs libcomposefs1

Fedora/RHEL:

sudo dnf install -y composefs composefs-libs

For development (optional):

# Debian/Ubuntu
sudo apt install -y libcomposefs-dev

# Fedora/RHEL  
sudo dnf install -y composefs-devel

Build Dependencies

The ComposeFS source package requires the following build dependencies:

Debian Build Dependencies:

  • debhelper-compat (= 13)
  • fsverity - File system verity support
  • fuse3 - FUSE filesystem support
  • go-md2man - Markdown to man page converter
  • libcap2-bin - Capability utilities
  • libfuse3-dev - FUSE development headers
  • libssl-dev - OpenSSL development headers
  • meson - Build system
  • pkgconf - Package configuration

Build System: ComposeFS uses the Meson build system for compilation and packaging.

Source Package Information

Commands

The composefs package provides the following tools:

Command Purpose Usage
mkcomposefs Create ComposeFS images mkcomposefs <source> <image> --digest-store=<dir>
composefs-info Inspect and manage images composefs-info [ls|objects|missing-objects|dump] <image>
mount.composefs Mount images (helper for mount -t composefs) mount.composefs <image> <mountpoint> -o basedir=<dir>

Important: There is no composefs executable. The package name is composefs, but the actual tools are mkcomposefs, composefs-info, and mount.composefs.

ComposeFS Workflow in apt-layer

1. Image Creation

To create a ComposeFS image from a directory tree:

mkcomposefs <rootfs-dir> <output.img> --digest-store=<object-store-dir>
  • <rootfs-dir>: Directory containing the root filesystem to layer
  • <output.img>: Output ComposeFS image (typically ends with .composefs). This file contains the image metadata (an EROFS image file)
  • --digest-store=<object-store-dir>: This option specifies a directory where mkcomposefs will copy (or reflink) regular files larger than 64 bytes from <rootfs-dir>. These files are content-addressed (named after their fsverity digest) and form the "backing store" for the ComposeFS image. This directory is then referenced as the basedir during mounting

2. Mounting a ComposeFS Image

To mount a ComposeFS image, apt-layer can either call mount.composefs directly or rely on the kernel's mount -t composefs interface, which will invoke mount.composefs as a helper.

Using mount.composefs directly:

mount.composefs <output.img> <mountpoint> -o basedir=<object-store-dir>[,basedir=<another-object-store-dir>...]

Using the standard mount command (which relies on mount.composefs as a helper):

mount -t composefs -o basedir=<object-store-dir>[,basedir=<another-object-store-dir>...] <output.img> <mountpoint>
  • <output.img>: Path to the ComposeFS image file (metadata)
  • <mountpoint>: Where to mount the filesystem
  • -o basedir=<object-store-dir>: This option is crucial. It points to the directory (or multiple colon-separated directories) containing the content-addressed backing files created with --digest-store during image creation. This provides the underlying content for the ComposeFS image

Optional mount.composefs options:

  • digest=DIGEST: Validates the image file against a specified fs-verity digest for integrity
  • verity: Ensures all files in the image and base directories have matching fs-verity digests. Requires kernel 6.6rc1+
  • idmap=PATH: Specifies a user namespace for ID mapping
  • upperdir/workdir: Allows for a writable overlay on top of the read-only ComposeFS image, similar to overlayfs

3. Unmounting

umount <mountpoint>

4. Listing and Removing Images

  • Listing: apt-layer lists ComposeFS images by scanning for .composefs files in its workspace. Additionally, apt-layer can use composefs-info ls <image.composefs> to inspect the contents of an image, or composefs-info missing-objects <image.composefs> --basedir=<object-store-dir> to verify the integrity of the object store. For advanced scenarios, composefs-info dump can provide a textual representation of the image's metadata (as defined by composefs-dump(5)), which can also be used as input for mkcomposefs --from-file
  • Removing: apt-layer removes images by deleting the corresponding .composefs file and cleaning up the associated content-addressed files in the --digest-store directory. This cleanup typically involves checking composefs-info objects to identify files that are no longer referenced by any active images before removal

Integration Notes

  • Package Structure: apt-layer supports the official ComposeFS packages from both Debian (composefs, libcomposefs1) and Fedora (composefs, composefs-libs) repositories
  • Specific Tools: While there isn't a single monolithic composefs CLI, specialized commands like composefs-info exist for introspection, and mount.composefs is the dedicated helper for mounting (callable directly or via mount -t composefs)
  • Dependencies: apt-layer requires mkcomposefs, composefs-info, mount.composefs (from the composefs package), mksquashfs, and unsquashfs for ComposeFS support
  • Distribution Detection: apt-layer automatically detects the distribution and provides appropriate installation commands for ComposeFS packages
  • Fallback: If mkcomposefs (and potentially mount.composefs) is not available, apt-layer can fall back to a shell script alternative (for development/testing only)
  • Compatibility: This approach matches rpm-ostree and Fedora Silverblue's use of ComposeFS for system layering

References