bootc-docs/internals/bootc-internals-technical-guide.md

bootc internals - Technical Guide

Overview

bootc internals is a hidden command that provides internal-only operations for system maintenance, debugging, and integration with other tools. These commands are not intended for regular users but are essential for system administrators, developers, and automated tooling.

Purpose

The internals commands serve several critical functions:

1. System Maintenance: Low-level system operations and cleanup
2. Integration: Proxy frontends for other tools and services
3. Debugging: Testing and diagnostic capabilities
4. Development: Internal tooling and schema generation
5. Systemd Integration: Generator and service management

Command Structure

pub(crate) enum InternalsOpts {
    SystemdGenerator { normal_dir, early_dir, late_dir },
    FixupEtcFstab,
    PrintJsonSchema { of: SchemaType },
    Fsverity(FsverityOpts),
    Fsck,
    Cleanup,
    Relabel { as_path, path },
    OstreeExt { args },
    Cfs { args },
    OstreeContainer { args },
    TestComposefs,
    LoopbackCleanupHelper { device },
    AllocateCleanupLoopback { file_path },
    BootcInstallCompletion { sysroot, stateroot },
    Reboot,
    PublishRhsmFacts,  // Feature-gated
    DumpCliJson,       // Feature-gated
    DirDiff { pristine_etc, current_etc, new_etc, perform_merge },
}

Core Commands

1. Systemd Generator

Purpose: Generate systemd units for bootc integration Usage: bootc internals systemd-generator <normal_dir> [early_dir] [late_dir]

InternalsOpts::SystemdGenerator {
    normal_dir: Utf8PathBuf,
    early_dir: Option<Utf8PathBuf>,
    late_dir: Option<Utf8PathBuf>,
} => {
    let unit_dir = &Dir::open_ambient_dir(normal_dir, cap_std::ambient_authority())?;
    crate::generator::generator(root, unit_dir)
}

Functionality:

• Generates systemd units for bootc services
• Handles fstab editing for composefs systems
• Creates bootc-fstab-edit.service when needed
• Integrates with systemd's generator system

Implementation:

pub(crate) fn fstab_generator_impl(root: &Dir, unit_dir: &Dir) -> Result<bool> {
    // Check if system is ostree-booted
    if !is_ostree_booted_in(root)? {
        return Ok(false);
    }

    // Check /etc/fstab for anaconda stamp
    if let Some(fd) = root.open_optional("etc/fstab")? {
        let mut from_anaconda = false;
        for line in fd.lines() {
            let line = line?;
            if line.contains(BOOTC_EDITED_STAMP) {
                return Ok(false); // Already processed
            }
            if line.contains(FSTAB_ANACONDA_STAMP) {
                from_anaconda = true;
            }
        }
        
        if from_anaconda {
            generate_fstab_editor(unit_dir)?;
            return Ok(true);
        }
    }
    Ok(false)
}

2. Filesystem Consistency Check (Fsck)

Purpose: Perform consistency checking on bootc systems Usage: bootc internals fsck

InternalsOpts::Fsck => {
    let sysroot = &get_storage().await?;
    crate::fsck::fsck(&sysroot, std::io::stdout().lock()).await?;
    Ok(())
}

Functionality:

• Validates OSTree repository integrity
• Checks composefs repository consistency
• Verifies deployment states
• Validates system configuration

Implementation:

pub async fn fsck(sysroot: &Storage, mut out: impl Write) -> Result<()> {
    let ostree = sysroot.get_ostree()?;
    let repo = ostree.repo();
    
    // Run all registered fsck checks
    for check in FSCK_CHECKS {
        match check(sysroot).await {
            Ok(Ok(())) => writeln!(out, "✓ {}", check.name())?,
            Ok(Err(e)) => writeln!(out, "✗ {}: {}", check.name(), e)?,
            Err(e) => writeln!(out, "✗ {}: {}", check.name(), e)?,
        }
    }
    Ok(())
}

3. Cleanup Operations

Purpose: Perform cleanup actions on bootc systems Usage: bootc internals cleanup

InternalsOpts::Cleanup => {
    let sysroot = get_storage().await?;
    crate::deploy::cleanup(&sysroot).await
}

Functionality:

• Remove old deployments
• Clean up unused container images
• Garbage collect OSTree repository
• Remove temporary files

4. Filesystem Verity (Fsverity)

Purpose: Manage fsverity integrity verification Usage: bootc internals fsverity <subcommand>

pub(crate) enum FsverityOpts {
    Measure { path: Utf8PathBuf },
    Enable { path: Utf8PathBuf },
}

// Implementation
InternalsOpts::Fsverity(args) => match args {
    FsverityOpts::Measure { path } => {
        let fd = std::fs::File::open(&path)?;
        let digest: fsverity::Sha256HashValue = fsverity::measure_verity(&fd)?;
        println!("{}", digest.to_hex());
        Ok(())
    }
    FsverityOpts::Enable { path } => {
        let fd = std::fs::File::open(&path)?;
        fsverity::enable_verity_raw::<fsverity::Sha256HashValue>(&fd)?;
        Ok(())
    }
}

Subcommands:

• measure: Calculate fsverity digest of a file
• enable: Enable fsverity verification on a file

5. Proxy Commands

OstreeExt Proxy

Purpose: Proxy frontend for ostree-ext CLI Usage: bootc internals ostree-ext [args...]

InternalsOpts::OstreeExt { args } => {
    ostree_ext::cli::run_from_iter(["ostree-ext".into()].into_iter().chain(args)).await
}

Cfs Proxy

Purpose: Proxy frontend for cfsctl CLI Usage: bootc internals cfs [args...]

InternalsOpts::Cfs { args } => {
    crate::cfsctl::run_from_iter(args.iter()).await
}

OstreeContainer Proxy

Purpose: Proxy frontend for legacy ostree container CLI Usage: bootc internals ostree-container [args...]

InternalsOpts::OstreeContainer { args } => {
    ostree_ext::cli::run_from_iter(
        ["ostree-ext".into(), "container".into()]
            .into_iter()
            .chain(args),
    ).await
}

6. System Maintenance

Fixup /etc/fstab

Purpose: Fix /etc/fstab for composefs systems Usage: bootc internals fixup-etc-fstab

InternalsOpts::FixupEtcFstab => {
    crate::deploy::fixup_etc_fstab(&root)
}

Relabel

Purpose: SELinux relabeling Usage: bootc internals relabel --as-path <path> <target_path>

InternalsOpts::Relabel { as_path, path } => {
    let root = &Dir::open_ambient_dir("/", cap_std::ambient_authority())?;
    let path = path.strip_prefix("/")?;
    let sepolicy = &ostree::SePolicy::new(&gio::File::for_path("/"), gio::Cancellable::NONE)?;
    crate::lsm::relabel_recurse(root, path, as_path.as_deref(), sepolicy)?;
    Ok(())
}

7. Testing and Development

Test Composefs

Purpose: Ensure composefs repository is initialized Usage: bootc internals test-composefs

InternalsOpts::TestComposefs => {
    let storage = get_storage().await?;
    let cfs = storage.get_ensure_composefs()?;
    let testdata = b"some test data";
    let testdata_digest = openssl::sha::sha256(testdata);
    let mut w = SplitStreamWriter::new(&cfs, None, Some(testdata_digest));
    w.write_inline(testdata);
    let object = cfs.write_stream(w, Some("testobject"))?.to_hex();
    // Verify expected digest
    assert_eq!(object, "expected_digest");
    Ok(())
}

Loopback Device Management

Purpose: Manage loopback devices for testing Usage:

• bootc internals loopback-cleanup-helper --device <device>
• bootc internals allocate-cleanup-loopback --file-path <path>

InternalsOpts::LoopbackCleanupHelper { device } => {
    crate::blockdev::run_loopback_cleanup_helper(&device).await
}

InternalsOpts::AllocateCleanupLoopback { file_path: _ } => {
    let temp_file = tempfile::NamedTempFile::new()?;
    let temp_path = temp_file.path();
    
    let loopback = crate::blockdev::LoopbackDevice::new(temp_path)?;
    println!("Created loopback device: {}", loopback.path());
    
    // Cleanup
    drop(loopback);
    println!("Successfully closed loopback device");
    Ok(())
}

8. Installation Completion

Purpose: Complete installation from ostree-ext Usage: bootc internals bootc-install-completion --sysroot <path> --stateroot <name>

InternalsOpts::BootcInstallCompletion { sysroot, stateroot } => {
    let rootfs = &Dir::open_ambient_dir("/", cap_std::ambient_authority())?;
    crate::install::completion::run_from_ostree(rootfs, &sysroot, &stateroot).await
}

9. System Operations

Reboot

Purpose: Initiate reboot (for testing) Usage: bootc internals reboot

InternalsOpts::Reboot => {
    crate::reboot::reboot()
}

Print JSON Schema

Purpose: Generate JSON schemas (for development) Usage: bootc internals print-json-schema --of <type>

InternalsOpts::PrintJsonSchema { of } => {
    let schema = match of {
        SchemaType::Host => schema_for!(crate::spec::Host),
        SchemaType::Progress => schema_for!(crate::progress_jsonl::Event),
    };
    let mut stdout = std::io::stdout().lock();
    serde_json::to_writer_pretty(&mut stdout, &schema)?;
    Ok(())
}

10. Feature-Gated Commands

RHSM Facts Publishing

Purpose: Publish subscription-manager facts Usage: bootc internals publish-rhsm-facts (requires rhsm feature)

#[cfg(feature = "rhsm")]
InternalsOpts::PublishRhsmFacts => {
    crate::rhsm::publish_facts(&root).await
}

CLI JSON Dump

Purpose: Dump CLI structure as JSON Usage: bootc internals dump-cli-json (requires docgen feature)

#[cfg(feature = "docgen")]
InternalsOpts::DumpCliJson => {
    use clap::CommandFactory;
    let cmd = Opt::command();
    let json = crate::cli_json::dump_cli_json(&cmd)?;
    println!("{}", json);
    Ok(())
}

11. Directory Diff Testing

Purpose: Test etc-diff/etc-merge functionality Usage: bootc internals dir-diff <pristine_etc> <current_etc> <new_etc> [--perform-merge]

InternalsOpts::DirDiff {
    pristine_etc,
    current_etc,
    new_etc,
    perform_merge,
} => {
    // Test 3-way merge functionality
    let pristine_dir = Dir::open_ambient_dir(&pristine_etc, cap_std::ambient_authority())?;
    let current_dir = Dir::open_ambient_dir(&current_etc, cap_std::ambient_authority())?;
    let new_dir = Dir::open_ambient_dir(&new_etc, cap_std::ambient_authority())?;
    
    let (pristine_files, current_files, new_files) =
        traverse_etc(&pristine_dir, &current_dir, &new_dir)?;
    
    if perform_merge {
        let diff = compute_diff(&pristine_files, &current_files)?;
        merge(&current_dir, &current_files, &new_dir, &new_files, diff)?;
    }
    Ok(())
}

Architecture

1. Command Routing

The internals commands are routed through the main CLI dispatcher:

match opt {
    Opt::Internals(opts) => match opts {
        InternalsOpts::SystemdGenerator { ... } => { ... }
        InternalsOpts::Fsck => { ... }
        InternalsOpts::Cleanup => { ... }
        // ... other commands
    }
}

2. Error Handling

All internals commands use consistent error handling:

#[context("Operation name")]
pub(crate) fn operation() -> Result<()> {
    // Implementation with proper error context
}

3. Integration Points

• Systemd: Generator integration for service management
• OSTree: Direct OSTree operations and repository management
• Composefs: Composefs repository operations
• SELinux: Security labeling and policy enforcement
• Filesystem: Low-level filesystem operations

Security Considerations

1. Privilege Requirements

Most internals commands require root privileges:

pub(crate) fn require_root(is_container: bool) -> Result<()> {
    ensure!(
        rustix::process::getuid().is_root(),
        if is_container {
            "The user inside the container from which you are running this command must be root"
        } else {
            "This command must be executed as the root user"
        }
    );
    Ok(())
}

2. Hidden Commands

All internals commands are hidden from help output:

#[clap(subcommand)]
#[clap(hide = true)]
Internals(InternalsOpts),

3. Internal Use Only

Commands are marked as internal-only and not intended for regular users.

Performance Characteristics

1. Systemd Generator

• Fast: Minimal overhead during boot
• Conditional: Only runs when needed
• Efficient: Direct filesystem operations

2. Fsck Operations

• Comprehensive: Checks all system components
• Parallel: Can run multiple checks concurrently
• Detailed: Provides specific error information

3. Cleanup Operations

• Safe: Only removes confirmed unused data
• Efficient: Batch operations where possible
• Atomic: Operations are atomic where possible

Troubleshooting

1. Common Issues

Generator Not Running

# Check systemd generator status
systemctl status bootc-systemd-generator

# Check generator logs
journalctl -u bootc-systemd-generator

Fsck Failures

# Run fsck manually
bootc internals fsck

# Check specific components
bootc internals fsck --component ostree

Cleanup Issues

# Run cleanup manually
bootc internals cleanup

# Check storage usage
df -h /sysroot

2. Debug Commands

# Enable debug logging
RUST_LOG=debug bootc internals <command>

# Check system state
bootc status

# Verify system integrity
bootc internals fsck

Best Practices

1. Usage Guidelines

• Internal Only: Don't use in production scripts
• Root Required: Always run as root
• Testing: Use in test environments first
• Documentation: Document any custom usage

2. Integration Patterns

• Systemd: Use generators for service integration
• Monitoring: Use fsck for health checks
• Maintenance: Use cleanup for storage management
• Development: Use test commands for validation

3. Error Handling

• Logging: Always check logs for errors
• Recovery: Have recovery procedures ready
• Testing: Test commands before production use
• Monitoring: Monitor system health after operations

This technical guide provides comprehensive understanding of the bootc internals system's architecture, implementation, and usage patterns.