bootc-docs/COMPATIBILITY.md
robojerk adecc24db1
Some checks failed
Test bootc Documentation / test-base-image (push) Failing after 27s
Test bootc Documentation / test-documentation (push) Failing after 31s
Add critical grub2-mkconfig composefs compatibility fix
- Add comprehensive fix for grub2-mkconfig composefs incompatibility
- Document the critical issue from Red Hat Bugzilla #2308594
- Provide complete patch, automated script, and Containerfile integration
- Add troubleshooting documentation with verification steps
- Update COMPATIBILITY.md with critical issue warning
- Update README.md to highlight this critical fix

This fix is ESSENTIAL for all composefs-based bootc images to boot properly.
Without this fix, grub2-mkconfig will generate incorrect configurations
and cause boot failures.

Based on:
- Red Hat Bugzilla #2308594 (reported by Colin Walters)
- ostree issue #3198
- Affects all bootc/ostree systems using composefs
2025-09-15 15:38:16 -07:00

9.1 KiB

Compatibility Matrix

This document provides compatibility information for bootc on Debian systems.

Verified Compatibility Matrix

Component Minimum Version Tested Version Notes
bootc v1.1.4 v1.1.4 Main branch, composefs-backend experimental
Debian Base Sid (unstable) Sid Recommended for base image building
Debian Base 14 (Forky) 14 (Forky) Ideal (testing, latest OSTree)
Debian Base 13 (Trixie) 13 (Trixie) Stable, may need OSTree backports
Debian Base 12 (Bookworm) Not supported (OSTree too old)
Kernel 6.1 6.5+ Minimum 6.1, 6.5+ for composefs features
Podman 4.0 4.0+ Required for container operations
OSTree 2024.1 2024.3+ Required for deployments
composefs N/A Experimental Requires kernel 6.5+ for full features

Feature Compatibility by bootc Version

Feature bootc Version Kernel Requirement Notes
Basic install v1.1.4+ 6.1+ Manual installation recommended
Composefs backend v1.1.4+ 6.5+ Experimental, requires feature flag
--progress-fd v1.1.4+ Any Experimental flag
--no-signature-verification v1.1.4+ Any Bypasses security checks
--mutate-in-place v1.1.4+ Any In-place mutations
--json output v1.1.4+ Any JSON format output

Feature Compatibility

Core Features

  • bootc install - Manual installation recommended
  • bootc upgrade - Works with proper setup
  • bootc status - Full functionality
  • bootc switch - Image switching supported
  • bootc rollback - Rollback functionality
  • bootc usr-overlay - Temporary /usr modifications

Hidden/Internal Features

  • bootc image - Image management
  • bootc internals - Internal operations
  • bootc state - State management
  • ⚠️ bootc exec-in-host-mount-namespace - Requires privileged access
  • ⚠️ bootc composefs-finalize-staged - Composefs backend only

Experimental Features

  • ⚠️ --progress-fd - Available since bootc v1.1.4
  • ⚠️ --no-signature-verification - Bypasses security checks
  • ⚠️ --mutate-in-place - In-place mutations
  • ⚠️ --json - JSON output format
  • ⚠️ --target-no-signature-verification - Target signature bypass

Kernel Requirements Checklist

Basic bootc Requirements (Kernel 6.1+)

  • OSTree support: Basic OSTree operations
  • Container runtime: Podman/containerd support
  • systemd: Full systemd integration
  • AppArmor: Security labeling (Debian default)

Composefs Backend Requirements (Kernel 6.5+)

  • EROFS: Enhanced Read-Only File System support
  • composefs: Kernel composefs module
  • overlayfs: Overlay filesystem support
  • fsverity: File system verification
  • ⚠️ Advanced verity: Kernel 6.6+ for certain integrity modes

Userspace Requirements

  • composefs tools: Userspace composefs utilities
  • EROFS support: Enhanced Read-Only File System support
  • fsverity: File system verification support
  • FUSE3: For fallback mounting (if kernel support unavailable)

Kernel Feature Verification

# Check kernel version
uname -r

# Check for EROFS support
grep EROFS /proc/filesystems

# Check for composefs support
grep composefs /proc/filesystems

# Check for overlayfs support
grep overlay /proc/filesystems

# Check for fsverity support
grep fsverity /proc/filesystems

Feature Flags

  • composefs-backend - Enable composefs backend
  • install-to-disk - Direct disk installation
  • rhsm - Red Hat Subscription Manager (not applicable to Debian)

Debian-Specific Considerations

Package Availability

  • bootc: Not packaged in Debian main (compile from source)
  • ostree: Available in Debian repositories
  • podman: Available in Debian repositories
  • composefs: May require manual compilation

Build Dependencies

# Required for bootc compilation
sudo apt install -y build-essential git pkg-config libostree-dev libglib2.0-dev libgpgme-dev libseccomp-dev cargo rustc

# Required for composefs backend
sudo apt install -y libfuse3-dev libfuse3-3

Runtime Dependencies

# Required for bootc operation
sudo apt install -y ostree podman systemd

Known Issues

Critical: grub2-mkconfig Composefs Incompatibility

⚠️ CRITICAL ISSUE: grub2-mkconfig is incompatible with composefs and will cause boot failures.

Problem:

  • grub2-mkconfig doesn't know how to read /sysroot where the physical root filesystem lives
  • This affects all bootc/ostree systems using composefs
  • Reported in Red Hat Bugzilla #2308594

Solution: Apply this patch to /usr/sbin/grub2-mkconfig:

# Add this code after line 138 in grub2-mkconfig
# Path to the real rootfs; we may be in a container or using bootc/ostree
GRUB_PHYSICAL_FILESYSTEM=/
GRUB_FS="$(stat -f -c %T / || echo unknown)"
if test "x${GRUB_FS}" = "xoverlay" && test -d /sysroot; then
  GRUB_PHYSICAL_FILESYSTEM=/sysroot
fi

# Then change this line:
# GRUB_DEVICE="`${grub_probe} --target=device /`"
# To:
GRUB_DEVICE="`${grub_probe} --target=device ${GRUB_PHYSICAL_FILESYSTEM}`"

Workaround:

  • Apply the patch before running grub2-mkconfig
  • Or use alternative bootloader configuration methods
  • This issue affects all composefs-based bootc images

Debian-Specific Problems

  1. bootc install reliability: May fail on Debian due to Fedora-centric development
  2. Missing dependencies: Some bootc dependencies may not be available
  3. Compilation issues: Rust dependencies may not compile cleanly
  4. Runtime errors: Even if compiled, bootc may fail at runtime
  5. OSTree version issues: Debian 12 (Bookworm) has outdated OSTree packages

Workarounds

  1. Manual installation: Use provided scripts to bypass bootc install
  2. Hybrid approach: Manual install + bootc management
  3. Validation scripts: Manual validation of bootc-compatible images
  4. Containerfile fixes: Use symlinks instead of systemctl commands
  5. OSTree backporting: Backport newer OSTree libraries from Forky to Trixie

Testing Matrix

Experimental Configurations

  • ⚠️ Debian 13 (Trixie) + bootc v1.1.4 + Podman 4.0 (experimental, not fully tested), might need OSTree packages backported from Forky
  • ⚠️ Debian 14 (Forky) + bootc v1.1.4 + Podman 4.0 (experimental, not fully tested)
  • ⚠️ Debian 00 (sid)
  • ⚠️ Debian 12 (Bookworm) + manual installation + bootc management (old OSTree, experimental)
  • ⚠️ Debian 12 (Bookworm) + composefs backend (experimental, old OSTree)
  • Debian 11 (Bullseye) - Not tested, may have issues
  • No reproducible bootc images exist yet - All configurations are experimental

Untested Configurations

  • Other Debian derivatives (Ubuntu, etc.)

OSTree Backporting Guide

Backporting OSTree from Forky to Trixie

If you need newer OSTree libraries on Trixie, you can backport them from Forky:

# Add Forky sources temporarily
echo "deb http://deb.debian.org/debian testing main" | sudo tee /etc/apt/sources.list.d/forky.list

# Download newer OSTree packages
apt download ostree libostree-1-1 ostree-tests

# Install the newer packages
sudo dpkg -i ostree_*.deb libostree-1-1_*.deb

# Remove temporary sources
sudo rm /etc/apt/sources.list.d/forky.list
sudo apt update

Benefits of Newer OSTree

  • bootupd support: Required for bootloader management
  • Better bootc compatibility: Newer APIs and features
  • Performance improvements: Optimized deployment operations
  • Security updates: Latest security patches

Recommendations

For Production Use

⚠️ WARNING: bootc on Debian is EXPERIMENTAL and NOT production-ready

  1. DO NOT use in production - All configurations are experimental
  2. No reproducible bootc images exist - Everything is still experimental
  3. Use only for testing on virtual machines or test hardware
  4. Keep backups of critical data and configurations
  5. Expect failures - bootc is not stable on Debian yet
  6. Monitor bootc releases for Debian compatibility improvements
  7. Contribute fixes back to the bootc project

For Development

  1. Use the hybrid approach: Manual install + bootc management
  2. Validate images using provided validation scripts
  3. Test with different kernel versions for composefs features
  4. Contribute fixes back to the bootc project

Getting Help

Documentation

Community

Reporting Issues

When reporting issues, please include:

  • Debian version and kernel version
  • bootc version and build method
  • Complete error messages and logs
  • Steps to reproduce the issue