bootc-docs/composefs.md

# bootc Composefs Integration - Technical Documentation

## Overview

This document explains how bootc utilizes Composefs as an alternative storage backend to OSTree. The composefs-backend branch introduces direct container image deployment without OSTree conversion.

## What is Composefs?

Composefs is a Linux kernel filesystem that mounts read-only EROFS images directly from userspace:

- **Direct Container Support**: Mount OCI images without conversion
- **Kernel Performance**: No userspace daemon required  
- **Built-in Security**: fsverity integration for integrity verification
- **Storage Efficiency**: EROFS compression and deduplication

## Kernel and Userspace Requirements

### Minimum Kernel Requirements
- **Basic EROFS support**: Kernel 5.15+
- **Composefs features**: Kernel 6.5+ (recommended)
- **Advanced overlay/verity**: Kernel 6.6+ for certain integrity modes

### Userspace Requirements
- **composefs tools**: Userspace composefs utilities
- **EROFS support**: Enhanced Read-Only File System support
- **fsverity**: File system verification support
- **FUSE3**: For fallback mounting (if kernel support unavailable)

## Why Composefs is Needed

### Main Branch (OSTree) Problems

1. **Container Conversion**: OCI images must be converted to OSTree format
2. **Performance Overhead**: Userspace operations for file access
3. **Storage Inefficiency**: OSTree object model creates overhead
4. **Complex Deployment**: Multi-step conversion process

### Composefs-Backend Solutions

1. **Direct Container Support**: No conversion needed
2. **Kernel Performance**: Direct kernel mounting
3. **Storage Efficiency**: EROFS compression and layer deduplication
4. **Simplified Architecture**: Fewer moving parts

## Architecture Comparison

### Main Branch (OSTree Backend)
```
Container Image → OSTree Conversion → OSTree Repository → Deployment
```

**Components:**
- OSTree repository with converted images
- Deployment objects for state management
- Userspace file access through OSTree library
- OCI to OSTree conversion layer

### Composefs-Backend Branch
```
Container Image → EROFS Image → Composefs Mount → Direct Access
```

**Components:**
- Composefs repository with EROFS images
- Direct kernel mounting via composefs
- Native container image support
- No conversion required

## Technical Implementation

### 1. Repository Structure

**Main Branch:**
```
/sysroot/ostree/
├── repo/objects/          # OSTree objects
├── repo/refs/            # References
└── deploy/{id}/          # Deployment checkout
    ├── usr/              # System files
    ├── etc/              # Configuration
    └── var/              # Variable data
```

**Composefs-Backend:**
```
/sysroot/
├── composefs/            # EROFS images
├── state/deploy/{id}/    # Deployment states
│   ├── etc/             # Merged /etc
│   ├── var ->           # Symlink to shared /var
│   └── {id}.origin      # Metadata
└── boot/                # Bootloader config
    ├── loader/          # BLS entries
    └── grub2/           # GRUB config
```

### 2. Image Storage

**Main Branch (OSTree):**
```rust
// Must convert OCI to OSTree
let ostree_ref = ostree_container::OstreeImageReference {
    sigverify: SignatureSource::ContainerPolicy,
    imgref: ImageReference {
        transport: Transport::Registry,
        name: "quay.io/myorg/debian-bootc:v2.0".to_string(),
    },
};

let deployment = ostree_container::deploy::deploy(&sysroot, &ostree_ref, &deploy_opts)?;
```

**Composefs-Backend:**
```rust
// Direct OCI image pull
pub(crate) async fn pull_composefs_repo(transport: &String, image: &String) -> Result<(...)> {
    let repo = open_composefs_repo(&rootfs_dir)?;
    
    // Pull directly from registry
    let (id, verity) = composefs_oci_pull(
        &Arc::new(repo), 
        &format!("{transport}:{image}"), 
        None, 
        None
    ).await?;
    
    // Create EROFS filesystem
    let mut fs = create_composefs_filesystem(&repo, &hex::encode(id), None)?;
    let entries = fs.transform_for_boot(&repo)?;
    let id = fs.commit_image(&repo, None)?;
    
    Ok((repo, entries, id, fs))
}
```

### 3. Boot Configuration

**Main Branch (BLS only):**
```rust
let bls_config = BLSConfig {
    title: "bootc".to_string(),
    options: Some("root=ostree:ostree=1:deploy-id={id}".to_string()),
};
```

**Composefs-Backend (BLS + UKI):**
```rust
#[derive(ValueEnum, Debug, Copy, Clone)]
pub enum BootType {
    Bls,    // Boot Loader Specification
    Uki,    // Unified Kernel Image
}

let bls_config = BLSConfig {
    title: "bootc-composefs".to_string(),
    options: Some("composefs=sha256:{digest}".to_string()),
};
```

### 4. Kernel Integration

**Main Branch:**
```rust
// Uses root= parameter
let root_param = format!("root=ostree:ostree=1:deploy-id={deployment_id}");
```

**Composefs-Backend:**
```rust
// Uses composefs= parameter
pub const COMPOSEFS_CMDLINE: &str = "composefs";

pub(crate) struct ComposefsCmdline {
    pub insecure: bool,
    pub digest: Box<str>,
}

// Kernel parameter: composefs=sha256:abc123... or composefs=?sha256:abc123... (insecure)
```

## State Management

### Main Branch (OSTree)
```rust
// OSTree deployment objects
let deployments = sysroot.deployments();
let booted_deployment = sysroot.booted_deployment();
```

### Composefs-Backend
```rust
// Custom state management
pub(crate) fn write_composefs_state(
    root_path: &Utf8PathBuf,
    deployment_id: Sha256HashValue,
    imgref: &ImageReference,
    staged: bool,
    boot_type: BootType,
    boot_digest: Option<String>,
) -> Result<()> {
    let state_path = root_path.join(format!("{STATE_DIR_RELATIVE}/{}", deployment_id.to_hex()));
    
    // Create deployment directory
    create_dir_all(state_path.join("etc"))?;
    
    // Copy pristine /etc from EROFS image
    copy_etc_to_state(&root_path, &deployment_id.to_hex(), &state_path)?;
    
    // Create symlink to shared /var
    let actual_var_path = root_path.join(SHARED_VAR_PATH);
    create_dir_all(&actual_var_path)?;
    
    symlink(
        path_relative_to(state_path.as_std_path(), actual_var_path.as_std_path())?,
        state_path.join("var"),
    )?;
    
    // Write deployment metadata
    let mut config = tini::Ini::new()
        .section("origin")
        .item(ORIGIN_CONTAINER, format!("ostree-unverified-image:{transport}{image_name}"));
    
    config = config
        .section(ORIGIN_KEY_BOOT)
        .item(ORIGIN_KEY_BOOT_TYPE, boot_type);
    
    if let Some(boot_digest) = boot_digest {
        config = config
            .section(ORIGIN_KEY_BOOT)
            .item(ORIGIN_KEY_BOOT_DIGEST, boot_digest);
    }
    
    // Write .origin file
    state_dir.atomic_write(
        format!("{}.origin", deployment_id.to_hex()),
        config.to_string().as_bytes(),
    )?;
    
    // Handle staged deployments
    if staged {
        std::fs::create_dir_all(COMPOSEFS_TRANSIENT_STATE_DIR)?;
        let staged_depl_dir = cap_std::fs::Dir::open_ambient_dir(
            COMPOSEFS_TRANSIENT_STATE_DIR,
            cap_std::ambient_authority(),
        )?;
        
        staged_depl_dir.atomic_write(
            COMPOSEFS_STAGED_DEPLOYMENT_FNAME,
            deployment_id.to_hex().as_bytes(),
        )?;
    }
    
    Ok(())
}
```

## /etc Management

### Main Branch (OSTree)
```rust
// OSTree 3-way merge
let pristine_etc = sysroot.deployment_get_origin_file(booted_deployment, "etc")?;
let current_etc = Dir::open_ambient_dir("/etc", ambient_authority())?;
let new_etc = sysroot.deployment_get_origin_file(staged_deployment, "etc")?;
let merged_etc = ostree_merge_etc(&pristine_etc, &current_etc, &new_etc)?;
```

### Composefs-Backend
```rust
// Composefs 3-way merge with EROFS
pub(crate) async fn composefs_native_finalize() -> Result<()> {
    // Mount EROFS image for pristine /etc
    let sysroot = open_dir(CWD, "/sysroot")?;
    let composefs_fd = mount_composefs_image(&sysroot, &booted_composefs.verity, false)?;
    let erofs_tmp_mnt = TempMount::mount_fd(&composefs_fd)?;
    
    // Perform 3-way merge
    let pristine_etc = Dir::open_ambient_dir(
        erofs_tmp_mnt.dir.path().join("etc"), 
        ambient_authority()
    )?;
    let current_etc = Dir::open_ambient_dir("/etc", ambient_authority())?;
    
    let new_etc_path = Path::new(STATE_DIR_ABS)
        .join(&staged_composefs.verity)
        .join("etc");
    let new_etc = Dir::open_ambient_dir(new_etc_path, ambient_authority())?;
    
    let (pristine_files, current_files, new_files) =
        traverse_etc(&pristine_etc, &current_etc, &new_etc)?;
    
    let diff = compute_diff(&pristine_files, &current_files)?;
    merge(&current_etc, &current_files, &new_etc, &new_files, diff)?;
    
    Ok(())
}
```

## Boot Process

### Main Branch (OSTree)
1. **Kernel Boot**: `root=ostree:ostree=1:deploy-id={id}`
2. **OSTree Mount**: Mount OSTree repository
3. **Deployment Checkout**: Checkout deployment files
4. **Userspace Access**: Access files through OSTree library

### Composefs-Backend
1. **Kernel Boot**: `composefs=sha256:{digest}`
2. **EROFS Mount**: Kernel directly mounts EROFS image
3. **Direct Access**: No userspace daemon required
4. **Verity Verification**: Built-in integrity checking

## CLI Integration

### Main Branch
```rust
// Direct OSTree operations
match opt {
    Opt::Upgrade(opts) => upgrade(opts).await,
    Opt::Switch(opts) => switch(opts).await,
    Opt::Rollback(opts) => rollback(opts).await,
}
```

### Composefs-Backend
```rust
// Conditional backend selection
match opt {
    Opt::Upgrade(opts) => {
        #[cfg(feature = "composefs-backend")]
        if composefs_booted()?.is_some() {
            upgrade_composefs(opts).await
        } else {
            upgrade(opts).await
        }
    }
    Opt::Switch(opts) => {
        #[cfg(feature = "composefs-backend")]
        if composefs_booted()?.is_some() {
            switch_composefs(opts).await
        } else {
            switch(opts).await
        }
    }
}
```

## Systemd Integration

### Main Branch
Uses standard OSTree systemd services.

### Composefs-Backend
New systemd service for composefs finalization:

```ini
[Unit]
Description=Composefs Finalize Staged Deployment
Documentation=man:bootc(1)
DefaultDependencies=no

RequiresMountsFor=/sysroot
After=local-fs.target
Before=basic.target final.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStop=/usr/bin/bootc composefs-finalize-staged
TimeoutStopSec=5m
ProtectHome=yes
ReadOnlyPaths=/etc
```

## Key Differences

| Aspect | Main Branch (OSTree) | Composefs-Backend |
|--------|---------------------|-------------------|
| **Storage** | OSTree repository | EROFS images |
| **Container Support** | Conversion required | Direct OCI support |
| **Boot Methods** | BLS only | BLS + UKI |
| **Kernel Integration** | Limited | Native composefs |
| **Performance** | Userspace overhead | Kernel-native |
| **Security** | External fsverity | Built-in fsverity |
| **Boot Process** | `root=ostree:...` | `composefs=sha256:...` |

## External Commands

### Composefs Operations
```bash
# Mount EROFS image
mount -t erofs /dev/loop0 /mnt

# Create EROFS image
mkfs.erofs -o image.erofs /source/directory

# Mount composefs
mount -t composefs /dev/loop0 /mnt

# Create composefs image
composefs-util create /source /destination
```

### Container Registry
```bash
# Pull container image
podman pull quay.io/myorg/debian-bootc:v2.0

# Inspect image
podman inspect quay.io/myorg/debian-bootc:v2.0
```

### Bootloader Configuration
```bash
# Update GRUB
grub2-mkconfig -o /boot/grub2/grub.cfg

# Update systemd-boot
bootctl update
```

## Conclusion

The composefs-backend branch provides significant improvements over the main branch:

1. **Better Performance**: Kernel-native mounting eliminates userspace overhead
2. **Direct Container Support**: No conversion needed for OCI images
3. **Enhanced Security**: Built-in fsverity verification
4. **Storage Efficiency**: Better compression and deduplication
5. **Simplified Architecture**: Fewer moving parts, more reliable

The conditional compilation approach allows gradual adoption while maintaining backward compatibility with the OSTree backend.