bootc-docs/composefs.md

bootc Composefs Integration - Technical Documentation

Overview

This document explains how bootc utilizes Composefs as an alternative storage backend to OSTree. The composefs-backend branch introduces direct container image deployment without OSTree conversion.

What is Composefs?

Composefs is a Linux kernel filesystem that mounts read-only EROFS images directly from userspace:

• Direct Container Support: Mount OCI images without conversion
• Kernel Performance: No userspace daemon required
• Built-in Security: fsverity integration for integrity verification
• Storage Efficiency: EROFS compression and deduplication

Kernel and Userspace Requirements

Minimum Kernel Requirements

• Basic EROFS support: Kernel 5.15+
• Composefs features: Kernel 6.5+ (recommended)
• Advanced overlay/verity: Kernel 6.6+ for certain integrity modes

Userspace Requirements

• composefs tools: Userspace composefs utilities
• EROFS support: Enhanced Read-Only File System support
• fsverity: File system verification support
• FUSE3: For fallback mounting (if kernel support unavailable)

Why Composefs is Needed

Main Branch (OSTree) Problems

1. Container Conversion: OCI images must be converted to OSTree format
2. Performance Overhead: Userspace operations for file access
3. Storage Inefficiency: OSTree object model creates overhead
4. Complex Deployment: Multi-step conversion process

Composefs-Backend Solutions

1. Direct Container Support: No conversion needed
2. Kernel Performance: Direct kernel mounting
3. Storage Efficiency: EROFS compression and layer deduplication
4. Simplified Architecture: Fewer moving parts

Architecture Comparison

Main Branch (OSTree Backend)

Container Image → OSTree Conversion → OSTree Repository → Deployment

Components:

• OSTree repository with converted images
• Deployment objects for state management
• Userspace file access through OSTree library
• OCI to OSTree conversion layer

Composefs-Backend Branch

Container Image → EROFS Image → Composefs Mount → Direct Access

Components:

• Composefs repository with EROFS images
• Direct kernel mounting via composefs
• Native container image support
• No conversion required

Technical Implementation

1. Repository Structure

Main Branch:

/sysroot/ostree/
├── repo/objects/          # OSTree objects
├── repo/refs/            # References
└── deploy/{id}/          # Deployment checkout
    ├── usr/              # System files
    ├── etc/              # Configuration
    └── var/              # Variable data

Composefs-Backend:

/sysroot/
├── composefs/            # EROFS images
├── state/deploy/{id}/    # Deployment states
│   ├── etc/             # Merged /etc
│   ├── var ->           # Symlink to shared /var
│   └── {id}.origin      # Metadata
└── boot/                # Bootloader config
    ├── loader/          # BLS entries
    └── grub2/           # GRUB config

2. Image Storage

Main Branch (OSTree):

// Must convert OCI to OSTree
let ostree_ref = ostree_container::OstreeImageReference {
    sigverify: SignatureSource::ContainerPolicy,
    imgref: ImageReference {
        transport: Transport::Registry,
        name: "quay.io/myorg/debian-bootc:v2.0".to_string(),
    },
};

let deployment = ostree_container::deploy::deploy(&sysroot, &ostree_ref, &deploy_opts)?;

Composefs-Backend:

// Direct OCI image pull
pub(crate) async fn pull_composefs_repo(transport: &String, image: &String) -> Result<(...)> {
    let repo = open_composefs_repo(&rootfs_dir)?;
    
    // Pull directly from registry
    let (id, verity) = composefs_oci_pull(
        &Arc::new(repo), 
        &format!("{transport}:{image}"), 
        None, 
        None
    ).await?;
    
    // Create EROFS filesystem
    let mut fs = create_composefs_filesystem(&repo, &hex::encode(id), None)?;
    let entries = fs.transform_for_boot(&repo)?;
    let id = fs.commit_image(&repo, None)?;
    
    Ok((repo, entries, id, fs))
}

3. Boot Configuration

Main Branch (BLS only):

let bls_config = BLSConfig {
    title: "bootc".to_string(),
    options: Some("root=ostree:ostree=1:deploy-id={id}".to_string()),
};

Composefs-Backend (BLS + UKI):

#[derive(ValueEnum, Debug, Copy, Clone)]
pub enum BootType {
    Bls,    // Boot Loader Specification
    Uki,    // Unified Kernel Image
}

let bls_config = BLSConfig {
    title: "bootc-composefs".to_string(),
    options: Some("composefs=sha256:{digest}".to_string()),
};

4. Kernel Integration

Main Branch:

// Uses root= parameter
let root_param = format!("root=ostree:ostree=1:deploy-id={deployment_id}");

Composefs-Backend:

// Uses composefs= parameter
pub const COMPOSEFS_CMDLINE: &str = "composefs";

pub(crate) struct ComposefsCmdline {
    pub insecure: bool,
    pub digest: Box<str>,
}

// Kernel parameter: composefs=sha256:abc123... or composefs=?sha256:abc123... (insecure)

State Management

Main Branch (OSTree)

// OSTree deployment objects
let deployments = sysroot.deployments();
let booted_deployment = sysroot.booted_deployment();

Composefs-Backend

// Custom state management
pub(crate) fn write_composefs_state(
    root_path: &Utf8PathBuf,
    deployment_id: Sha256HashValue,
    imgref: &ImageReference,
    staged: bool,
    boot_type: BootType,
    boot_digest: Option<String>,
) -> Result<()> {
    let state_path = root_path.join(format!("{STATE_DIR_RELATIVE}/{}", deployment_id.to_hex()));
    
    // Create deployment directory
    create_dir_all(state_path.join("etc"))?;
    
    // Copy pristine /etc from EROFS image
    copy_etc_to_state(&root_path, &deployment_id.to_hex(), &state_path)?;
    
    // Create symlink to shared /var
    let actual_var_path = root_path.join(SHARED_VAR_PATH);
    create_dir_all(&actual_var_path)?;
    
    symlink(
        path_relative_to(state_path.as_std_path(), actual_var_path.as_std_path())?,
        state_path.join("var"),
    )?;
    
    // Write deployment metadata
    let mut config = tini::Ini::new()
        .section("origin")
        .item(ORIGIN_CONTAINER, format!("ostree-unverified-image:{transport}{image_name}"));
    
    config = config
        .section(ORIGIN_KEY_BOOT)
        .item(ORIGIN_KEY_BOOT_TYPE, boot_type);
    
    if let Some(boot_digest) = boot_digest {
        config = config
            .section(ORIGIN_KEY_BOOT)
            .item(ORIGIN_KEY_BOOT_DIGEST, boot_digest);
    }
    
    // Write .origin file
    state_dir.atomic_write(
        format!("{}.origin", deployment_id.to_hex()),
        config.to_string().as_bytes(),
    )?;
    
    // Handle staged deployments
    if staged {
        std::fs::create_dir_all(COMPOSEFS_TRANSIENT_STATE_DIR)?;
        let staged_depl_dir = cap_std::fs::Dir::open_ambient_dir(
            COMPOSEFS_TRANSIENT_STATE_DIR,
            cap_std::ambient_authority(),
        )?;
        
        staged_depl_dir.atomic_write(
            COMPOSEFS_STAGED_DEPLOYMENT_FNAME,
            deployment_id.to_hex().as_bytes(),
        )?;
    }
    
    Ok(())
}

/etc Management

Main Branch (OSTree)

// OSTree 3-way merge
let pristine_etc = sysroot.deployment_get_origin_file(booted_deployment, "etc")?;
let current_etc = Dir::open_ambient_dir("/etc", ambient_authority())?;
let new_etc = sysroot.deployment_get_origin_file(staged_deployment, "etc")?;
let merged_etc = ostree_merge_etc(&pristine_etc, &current_etc, &new_etc)?;

Composefs-Backend

// Composefs 3-way merge with EROFS
pub(crate) async fn composefs_native_finalize() -> Result<()> {
    // Mount EROFS image for pristine /etc
    let sysroot = open_dir(CWD, "/sysroot")?;
    let composefs_fd = mount_composefs_image(&sysroot, &booted_composefs.verity, false)?;
    let erofs_tmp_mnt = TempMount::mount_fd(&composefs_fd)?;
    
    // Perform 3-way merge
    let pristine_etc = Dir::open_ambient_dir(
        erofs_tmp_mnt.dir.path().join("etc"), 
        ambient_authority()
    )?;
    let current_etc = Dir::open_ambient_dir("/etc", ambient_authority())?;
    
    let new_etc_path = Path::new(STATE_DIR_ABS)
        .join(&staged_composefs.verity)
        .join("etc");
    let new_etc = Dir::open_ambient_dir(new_etc_path, ambient_authority())?;
    
    let (pristine_files, current_files, new_files) =
        traverse_etc(&pristine_etc, &current_etc, &new_etc)?;
    
    let diff = compute_diff(&pristine_files, &current_files)?;
    merge(&current_etc, &current_files, &new_etc, &new_files, diff)?;
    
    Ok(())
}

Boot Process

Main Branch (OSTree)

1. Kernel Boot: root=ostree:ostree=1:deploy-id={id}
2. OSTree Mount: Mount OSTree repository
3. Deployment Checkout: Checkout deployment files
4. Userspace Access: Access files through OSTree library

Composefs-Backend

1. Kernel Boot: composefs=sha256:{digest}
2. EROFS Mount: Kernel directly mounts EROFS image
3. Direct Access: No userspace daemon required
4. Verity Verification: Built-in integrity checking

CLI Integration

Main Branch

// Direct OSTree operations
match opt {
    Opt::Upgrade(opts) => upgrade(opts).await,
    Opt::Switch(opts) => switch(opts).await,
    Opt::Rollback(opts) => rollback(opts).await,
}

Composefs-Backend

// Conditional backend selection
match opt {
    Opt::Upgrade(opts) => {
        #[cfg(feature = "composefs-backend")]
        if composefs_booted()?.is_some() {
            upgrade_composefs(opts).await
        } else {
            upgrade(opts).await
        }
    }
    Opt::Switch(opts) => {
        #[cfg(feature = "composefs-backend")]
        if composefs_booted()?.is_some() {
            switch_composefs(opts).await
        } else {
            switch(opts).await
        }
    }
}

Systemd Integration

Main Branch

Uses standard OSTree systemd services.

Composefs-Backend

New systemd service for composefs finalization:

[Unit]
Description=Composefs Finalize Staged Deployment
Documentation=man:bootc(1)
DefaultDependencies=no

RequiresMountsFor=/sysroot
After=local-fs.target
Before=basic.target final.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStop=/usr/bin/bootc composefs-finalize-staged
TimeoutStopSec=5m
ProtectHome=yes
ReadOnlyPaths=/etc

Key Differences

Aspect	Main Branch (OSTree)	Composefs-Backend
Storage	OSTree repository	EROFS images
Container Support	Conversion required	Direct OCI support
Boot Methods	BLS only	BLS + UKI
Kernel Integration	Limited	Native composefs
Performance	Userspace overhead	Kernel-native
Security	External fsverity	Built-in fsverity
Boot Process	root=ostree:...	composefs=sha256:...

External Commands

Composefs Operations

# Mount EROFS image
mount -t erofs /dev/loop0 /mnt

# Create EROFS image
mkfs.erofs -o image.erofs /source/directory

# Mount composefs
mount -t composefs /dev/loop0 /mnt

# Create composefs image
composefs-util create /source /destination

Container Registry

# Pull container image
podman pull quay.io/myorg/debian-bootc:v2.0

# Inspect image
podman inspect quay.io/myorg/debian-bootc:v2.0

Bootloader Configuration

# Update GRUB
grub2-mkconfig -o /boot/grub2/grub.cfg

# Update systemd-boot
bootctl update

Conclusion

The composefs-backend branch provides significant improvements over the main branch:

1. Better Performance: Kernel-native mounting eliminates userspace overhead
2. Direct Container Support: No conversion needed for OCI images
3. Enhanced Security: Built-in fsverity verification
4. Storage Efficiency: Better compression and deduplication
5. Simplified Architecture: Fewer moving parts, more reliable

The conditional compilation approach allows gradual adoption while maintaining backward compatibility with the OSTree backend.