deb-bootc-image-builder-new/docs/process.md

Fedora bootc-image-builder Complete Workflow Analysis

Overview

The Fedora bootc-image-builder is a sophisticated containerized tool that creates bootable disk images from bootc (bootable container) inputs. It's specifically designed for Fedora/CentOS/RHEL systems using DNF/RPM package management and supports various output formats including QCOW2, AMI, VMDK, VHD, GCE, and ISO images.

Complete Workflow Flowchart

flowchart TD
    A[User runs bootc-image-builder] --> B[Parse CLI Arguments]
    B --> C[Validate Container Storage Mount]
    C --> D[Load Configuration & Blueprint]
    D --> E[Validate Container Tags]
    E --> F[Get Container Size]
    F --> G[Create Podman Container Instance]
    G --> H[Extract OS Info from Container]
    H --> I[Initialize DNF in Container]
    I --> J[Create DNF Solver]
    J --> K[Generate OSBuild Manifest]
    
    K --> L[Load Distribution Definitions]
    L --> M[Create Package Set Chains]
    M --> N[DNF Dependency Resolution]
    N --> O[Resolve Container Specs]
    O --> P[Serialize Manifest]
    P --> Q[Execute OSBuild Pipeline]
    
    Q --> R{Image Type?}
    R -->|Disk Images| S[Create Partition Table]
    R -->|ISO Images| T[Create ISO Structure]
    
    S --> U[Generate Filesystem Layout]
    U --> V[Install GRUB2 Bootloader]
    V --> W[Copy Container Contents]
    W --> X[Apply Customizations]
    X --> Y[Generate Final Image]
    
    T --> Z[Create Installer Structure]
    Z --> AA[Package Installer Components]
    AA --> BB[Generate ISO Image]
    
    Y --> CC{Upload to Cloud?}
    BB --> CC
    CC -->|Yes| DD[AWS/GCP Upload]
    CC -->|No| EE[Save to Output Directory]
    DD --> EE
    EE --> FF[Complete]
    
    style A fill:#e1f5fe
    style FF fill:#c8e6c9
    style K fill:#fff3e0
    style Q fill:#f3e5f5
    style R fill:#fce4ec

Detailed Component Analysis

1. Container Orchestration Layer

• Podman Container: Manages the bootc container lifecycle
• Container Storage: Mounts /var/lib/containers/storage for image access
• Container Inspection: Extracts OS information, kernel details, and customization data
• Cross-Architecture Support: Uses qemu-user for cross-arch builds

2. Package Management System

• DNF Solver: Resolves package dependencies using dnfjson.Solver
• RPM Metadata Cache: Caches RPM metadata in /rpmmd volume
• Librepo Backend: Optional faster download backend
• Repository Configuration: Uses Fedora/CentOS/RHEL repositories

3. Distribution Definition System

• YAML Definitions: Loads package lists from data/defs/*.yaml files
• Version Matching: Supports exact and fuzzy version matching
• Image Type Specific: Different package sets for different image types
• Multi-Directory Support: Searches multiple definition directories

4. OSBuild Integration

• Manifest Generation: Creates comprehensive OSBuild manifests
• Pipeline Definition: Defines build, target, and export pipelines
• Stage Management: Orchestrates multiple build stages
• Serialization: Converts manifests to OSBuild-compatible format

5. Image Building Pipeline

Disk Image Generation:

1. Partition Table Creation: GPT/MBR partition layouts
2. Filesystem Setup: ext4/xfs/btrfs filesystem creation
3. Bootloader Installation: GRUB2 configuration and installation
4. Container Content Copy: Transfers bootc container contents
5. Customization Application: Applies user customizations
6. Final Assembly: Creates bootable disk image

ISO Image Generation:

1. Installer Structure: Creates Anaconda installer layout
2. Package Integration: Includes installer-specific packages
3. Boot Configuration: Sets up ISO boot parameters
4. Media Creation: Generates bootable ISO image

6. Cloud Integration

• AWS Upload: Direct AMI upload to AWS
• GCP Support: Google Cloud Platform integration
• Multi-Region: Support for multiple cloud regions
• Authentication: Handles cloud credentials and permissions

Key Tools and Dependencies

Core Tools:

• osbuild: Primary image building engine
• osbuild-ostree: OSTree integration for atomic updates
• osbuild-depsolve-dnf: DNF-based dependency resolution
• osbuild-lvm2: LVM2 support for advanced partitioning
• podman: Container runtime and management
• qemu-img: Image format conversion and manipulation

Package Management:

• dnf: Package manager for dependency resolution
• rpm: Package format handling
• librepo: Optional high-performance download backend
• subscription-manager: RHEL subscription handling

System Components:

• selinux-policy-targeted: SELinux policy enforcement
• distribution-gpg-keys: Package signature verification
• qemu-user: Cross-architecture emulation support

Configuration System

Blueprint Format:

• TOML/JSON Configuration: User customization files
• Container Metadata: Extracts configuration from container images
• Environment Variables: AWS credentials and other settings
• Command-line Flags: Extensive CLI options for fine-tuning

Hardcoded Defaults:

• Container Size Multiplier: containerSizeToDiskSizeMultiplier = 2
• Default Image Size: DEFAULT_SIZE = 10 * GibiByte
• Kernel Options: "rw", "console=tty0", "console=ttyS0"
• Distribution Paths: Multiple fallback paths for definitions

Build Process Stages

Stage 1: Initialization

1. Parse CLI arguments and validate inputs
2. Mount container storage and validate access
3. Load configuration and blueprint files
4. Validate container image tags and accessibility

Stage 2: Container Analysis

1. Get container size for disk sizing calculations
2. Create Podman container instance
3. Extract OS information (distro, version, kernel)
4. Initialize DNF package manager in container
5. Create DNF solver for dependency resolution

Stage 3: Manifest Generation

1. Load distribution-specific package definitions
2. Create package set chains for different image types
3. Resolve package dependencies using DNF solver
4. Resolve container specifications and architectures
5. Serialize manifest for OSBuild execution

Stage 4: Image Building

1. Execute OSBuild pipeline based on manifest
2. Create partition tables and filesystem layouts
3. Install bootloader and configure boot parameters
4. Copy container contents to target filesystem
5. Apply user customizations and configurations
6. Generate final bootable image

Stage 5: Output and Upload

1. Save images to output directory
2. Optionally upload to cloud providers (AWS/GCP)
3. Clean up temporary files and containers
4. Report build status and results

Supported Image Types

Disk Images:

• QCOW2: KVM/QEMU virtual machine images
• AMI: Amazon Machine Images for AWS
• VMDK: VMware virtual machine images
• VHD: Microsoft Hyper-V virtual machine images
• GCE: Google Compute Engine images
• Raw: Raw disk images for direct hardware deployment

ISO Images:

• Anaconda ISO: Fedora/CentOS/RHEL installer images
• Custom ISO: User-defined ISO configurations

Architecture Support

• x86_64: Primary architecture
• aarch64: ARM64 support
• ppc64le: PowerPC 64-bit little-endian
• s390x: IBM Z architecture
• riscv64: RISC-V 64-bit architecture

Testing Infrastructure

• Python-based Tests: Integration and functional testing
• Go Unit Tests: Component-level testing
• Container Testing: Tests run in isolated containers
• VM Testing: Can test resulting images in virtual machines
• Test Plans: FMF-based test organization and execution

This comprehensive workflow demonstrates the sophisticated orchestration of multiple tools and systems to create production-ready bootable images from container inputs, making it a powerful tool for modern container-native operating system deployment.