diff --git a/docs/process.md b/docs/process.md new file mode 100644 index 0000000..20f5298 --- /dev/null +++ b/docs/process.md @@ -0,0 +1,206 @@ +# Fedora bootc-image-builder Complete Workflow Analysis + +## Overview + +The Fedora bootc-image-builder is a sophisticated containerized tool that creates bootable disk images from bootc (bootable container) inputs. It's specifically designed for Fedora/CentOS/RHEL systems using DNF/RPM package management and supports various output formats including QCOW2, AMI, VMDK, VHD, GCE, and ISO images. + +## Complete Workflow Flowchart + +```mermaid +flowchart TD + A[User runs bootc-image-builder] --> B[Parse CLI Arguments] + B --> C[Validate Container Storage Mount] + C --> D[Load Configuration & Blueprint] + D --> E[Validate Container Tags] + E --> F[Get Container Size] + F --> G[Create Podman Container Instance] + G --> H[Extract OS Info from Container] + H --> I[Initialize DNF in Container] + I --> J[Create DNF Solver] + J --> K[Generate OSBuild Manifest] + + K --> L[Load Distribution Definitions] + L --> M[Create Package Set Chains] + M --> N[DNF Dependency Resolution] + N --> O[Resolve Container Specs] + O --> P[Serialize Manifest] + P --> Q[Execute OSBuild Pipeline] + + Q --> R{Image Type?} + R -->|Disk Images| S[Create Partition Table] + R -->|ISO Images| T[Create ISO Structure] + + S --> U[Generate Filesystem Layout] + U --> V[Install GRUB2 Bootloader] + V --> W[Copy Container Contents] + W --> X[Apply Customizations] + X --> Y[Generate Final Image] + + T --> Z[Create Installer Structure] + Z --> AA[Package Installer Components] + AA --> BB[Generate ISO Image] + + Y --> CC{Upload to Cloud?} + BB --> CC + CC -->|Yes| DD[AWS/GCP Upload] + CC -->|No| EE[Save to Output Directory] + DD --> EE + EE --> FF[Complete] + + style A fill:#e1f5fe + style FF fill:#c8e6c9 + style K fill:#fff3e0 + style Q fill:#f3e5f5 + style R fill:#fce4ec +``` + +## Detailed Component Analysis + +### 1. **Container Orchestration Layer** +- **Podman Container**: Manages the bootc container lifecycle +- **Container Storage**: Mounts `/var/lib/containers/storage` for image access +- **Container Inspection**: Extracts OS information, kernel details, and customization data +- **Cross-Architecture Support**: Uses qemu-user for cross-arch builds + +### 2. **Package Management System** +- **DNF Solver**: Resolves package dependencies using `dnfjson.Solver` +- **RPM Metadata Cache**: Caches RPM metadata in `/rpmmd` volume +- **Librepo Backend**: Optional faster download backend +- **Repository Configuration**: Uses Fedora/CentOS/RHEL repositories + +### 3. **Distribution Definition System** +- **YAML Definitions**: Loads package lists from `data/defs/*.yaml` files +- **Version Matching**: Supports exact and fuzzy version matching +- **Image Type Specific**: Different package sets for different image types +- **Multi-Directory Support**: Searches multiple definition directories + +### 4. **OSBuild Integration** +- **Manifest Generation**: Creates comprehensive OSBuild manifests +- **Pipeline Definition**: Defines build, target, and export pipelines +- **Stage Management**: Orchestrates multiple build stages +- **Serialization**: Converts manifests to OSBuild-compatible format + +### 5. **Image Building Pipeline** + +#### **Disk Image Generation**: +1. **Partition Table Creation**: GPT/MBR partition layouts +2. **Filesystem Setup**: ext4/xfs/btrfs filesystem creation +3. **Bootloader Installation**: GRUB2 configuration and installation +4. **Container Content Copy**: Transfers bootc container contents +5. **Customization Application**: Applies user customizations +6. **Final Assembly**: Creates bootable disk image + +#### **ISO Image Generation**: +1. **Installer Structure**: Creates Anaconda installer layout +2. **Package Integration**: Includes installer-specific packages +3. **Boot Configuration**: Sets up ISO boot parameters +4. **Media Creation**: Generates bootable ISO image + +### 6. **Cloud Integration** +- **AWS Upload**: Direct AMI upload to AWS +- **GCP Support**: Google Cloud Platform integration +- **Multi-Region**: Support for multiple cloud regions +- **Authentication**: Handles cloud credentials and permissions + +## Key Tools and Dependencies + +### **Core Tools**: +- **osbuild**: Primary image building engine +- **osbuild-ostree**: OSTree integration for atomic updates +- **osbuild-depsolve-dnf**: DNF-based dependency resolution +- **osbuild-lvm2**: LVM2 support for advanced partitioning +- **podman**: Container runtime and management +- **qemu-img**: Image format conversion and manipulation + +### **Package Management**: +- **dnf**: Package manager for dependency resolution +- **rpm**: Package format handling +- **librepo**: Optional high-performance download backend +- **subscription-manager**: RHEL subscription handling + +### **System Components**: +- **selinux-policy-targeted**: SELinux policy enforcement +- **distribution-gpg-keys**: Package signature verification +- **qemu-user**: Cross-architecture emulation support + +## Configuration System + +### **Blueprint Format**: +- **TOML/JSON Configuration**: User customization files +- **Container Metadata**: Extracts configuration from container images +- **Environment Variables**: AWS credentials and other settings +- **Command-line Flags**: Extensive CLI options for fine-tuning + +### **Hardcoded Defaults**: +- **Container Size Multiplier**: `containerSizeToDiskSizeMultiplier = 2` +- **Default Image Size**: `DEFAULT_SIZE = 10 * GibiByte` +- **Kernel Options**: `"rw"`, `"console=tty0"`, `"console=ttyS0"` +- **Distribution Paths**: Multiple fallback paths for definitions + +## Build Process Stages + +### **Stage 1: Initialization** +1. Parse CLI arguments and validate inputs +2. Mount container storage and validate access +3. Load configuration and blueprint files +4. Validate container image tags and accessibility + +### **Stage 2: Container Analysis** +1. Get container size for disk sizing calculations +2. Create Podman container instance +3. Extract OS information (distro, version, kernel) +4. Initialize DNF package manager in container +5. Create DNF solver for dependency resolution + +### **Stage 3: Manifest Generation** +1. Load distribution-specific package definitions +2. Create package set chains for different image types +3. Resolve package dependencies using DNF solver +4. Resolve container specifications and architectures +5. Serialize manifest for OSBuild execution + +### **Stage 4: Image Building** +1. Execute OSBuild pipeline based on manifest +2. Create partition tables and filesystem layouts +3. Install bootloader and configure boot parameters +4. Copy container contents to target filesystem +5. Apply user customizations and configurations +6. Generate final bootable image + +### **Stage 5: Output and Upload** +1. Save images to output directory +2. Optionally upload to cloud providers (AWS/GCP) +3. Clean up temporary files and containers +4. Report build status and results + +## Supported Image Types + +### **Disk Images**: +- **QCOW2**: KVM/QEMU virtual machine images +- **AMI**: Amazon Machine Images for AWS +- **VMDK**: VMware virtual machine images +- **VHD**: Microsoft Hyper-V virtual machine images +- **GCE**: Google Compute Engine images +- **Raw**: Raw disk images for direct hardware deployment + +### **ISO Images**: +- **Anaconda ISO**: Fedora/CentOS/RHEL installer images +- **Custom ISO**: User-defined ISO configurations + +## Architecture Support + +- **x86_64**: Primary architecture +- **aarch64**: ARM64 support +- **ppc64le**: PowerPC 64-bit little-endian +- **s390x**: IBM Z architecture +- **riscv64**: RISC-V 64-bit architecture + +## Testing Infrastructure + +- **Python-based Tests**: Integration and functional testing +- **Go Unit Tests**: Component-level testing +- **Container Testing**: Tests run in isolated containers +- **VM Testing**: Can test resulting images in virtual machines +- **Test Plans**: FMF-based test organization and execution + +This comprehensive workflow demonstrates the sophisticated orchestration of multiple tools and systems to create production-ready bootable images from container inputs, making it a powerful tool for modern container-native operating system deployment. \ No newline at end of file