Add docs/process.md
This commit is contained in:
parent
cd1c3b3bb0
commit
0c6fb782c3
1 changed files with 206 additions and 0 deletions
206
docs/process.md
Normal file
206
docs/process.md
Normal file
|
|
@ -0,0 +1,206 @@
|
||||||
|
# Fedora bootc-image-builder Complete Workflow Analysis
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The Fedora bootc-image-builder is a sophisticated containerized tool that creates bootable disk images from bootc (bootable container) inputs. It's specifically designed for Fedora/CentOS/RHEL systems using DNF/RPM package management and supports various output formats including QCOW2, AMI, VMDK, VHD, GCE, and ISO images.
|
||||||
|
|
||||||
|
## Complete Workflow Flowchart
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TD
|
||||||
|
A[User runs bootc-image-builder] --> B[Parse CLI Arguments]
|
||||||
|
B --> C[Validate Container Storage Mount]
|
||||||
|
C --> D[Load Configuration & Blueprint]
|
||||||
|
D --> E[Validate Container Tags]
|
||||||
|
E --> F[Get Container Size]
|
||||||
|
F --> G[Create Podman Container Instance]
|
||||||
|
G --> H[Extract OS Info from Container]
|
||||||
|
H --> I[Initialize DNF in Container]
|
||||||
|
I --> J[Create DNF Solver]
|
||||||
|
J --> K[Generate OSBuild Manifest]
|
||||||
|
|
||||||
|
K --> L[Load Distribution Definitions]
|
||||||
|
L --> M[Create Package Set Chains]
|
||||||
|
M --> N[DNF Dependency Resolution]
|
||||||
|
N --> O[Resolve Container Specs]
|
||||||
|
O --> P[Serialize Manifest]
|
||||||
|
P --> Q[Execute OSBuild Pipeline]
|
||||||
|
|
||||||
|
Q --> R{Image Type?}
|
||||||
|
R -->|Disk Images| S[Create Partition Table]
|
||||||
|
R -->|ISO Images| T[Create ISO Structure]
|
||||||
|
|
||||||
|
S --> U[Generate Filesystem Layout]
|
||||||
|
U --> V[Install GRUB2 Bootloader]
|
||||||
|
V --> W[Copy Container Contents]
|
||||||
|
W --> X[Apply Customizations]
|
||||||
|
X --> Y[Generate Final Image]
|
||||||
|
|
||||||
|
T --> Z[Create Installer Structure]
|
||||||
|
Z --> AA[Package Installer Components]
|
||||||
|
AA --> BB[Generate ISO Image]
|
||||||
|
|
||||||
|
Y --> CC{Upload to Cloud?}
|
||||||
|
BB --> CC
|
||||||
|
CC -->|Yes| DD[AWS/GCP Upload]
|
||||||
|
CC -->|No| EE[Save to Output Directory]
|
||||||
|
DD --> EE
|
||||||
|
EE --> FF[Complete]
|
||||||
|
|
||||||
|
style A fill:#e1f5fe
|
||||||
|
style FF fill:#c8e6c9
|
||||||
|
style K fill:#fff3e0
|
||||||
|
style Q fill:#f3e5f5
|
||||||
|
style R fill:#fce4ec
|
||||||
|
```
|
||||||
|
|
||||||
|
## Detailed Component Analysis
|
||||||
|
|
||||||
|
### 1. **Container Orchestration Layer**
|
||||||
|
- **Podman Container**: Manages the bootc container lifecycle
|
||||||
|
- **Container Storage**: Mounts `/var/lib/containers/storage` for image access
|
||||||
|
- **Container Inspection**: Extracts OS information, kernel details, and customization data
|
||||||
|
- **Cross-Architecture Support**: Uses qemu-user for cross-arch builds
|
||||||
|
|
||||||
|
### 2. **Package Management System**
|
||||||
|
- **DNF Solver**: Resolves package dependencies using `dnfjson.Solver`
|
||||||
|
- **RPM Metadata Cache**: Caches RPM metadata in `/rpmmd` volume
|
||||||
|
- **Librepo Backend**: Optional faster download backend
|
||||||
|
- **Repository Configuration**: Uses Fedora/CentOS/RHEL repositories
|
||||||
|
|
||||||
|
### 3. **Distribution Definition System**
|
||||||
|
- **YAML Definitions**: Loads package lists from `data/defs/*.yaml` files
|
||||||
|
- **Version Matching**: Supports exact and fuzzy version matching
|
||||||
|
- **Image Type Specific**: Different package sets for different image types
|
||||||
|
- **Multi-Directory Support**: Searches multiple definition directories
|
||||||
|
|
||||||
|
### 4. **OSBuild Integration**
|
||||||
|
- **Manifest Generation**: Creates comprehensive OSBuild manifests
|
||||||
|
- **Pipeline Definition**: Defines build, target, and export pipelines
|
||||||
|
- **Stage Management**: Orchestrates multiple build stages
|
||||||
|
- **Serialization**: Converts manifests to OSBuild-compatible format
|
||||||
|
|
||||||
|
### 5. **Image Building Pipeline**
|
||||||
|
|
||||||
|
#### **Disk Image Generation**:
|
||||||
|
1. **Partition Table Creation**: GPT/MBR partition layouts
|
||||||
|
2. **Filesystem Setup**: ext4/xfs/btrfs filesystem creation
|
||||||
|
3. **Bootloader Installation**: GRUB2 configuration and installation
|
||||||
|
4. **Container Content Copy**: Transfers bootc container contents
|
||||||
|
5. **Customization Application**: Applies user customizations
|
||||||
|
6. **Final Assembly**: Creates bootable disk image
|
||||||
|
|
||||||
|
#### **ISO Image Generation**:
|
||||||
|
1. **Installer Structure**: Creates Anaconda installer layout
|
||||||
|
2. **Package Integration**: Includes installer-specific packages
|
||||||
|
3. **Boot Configuration**: Sets up ISO boot parameters
|
||||||
|
4. **Media Creation**: Generates bootable ISO image
|
||||||
|
|
||||||
|
### 6. **Cloud Integration**
|
||||||
|
- **AWS Upload**: Direct AMI upload to AWS
|
||||||
|
- **GCP Support**: Google Cloud Platform integration
|
||||||
|
- **Multi-Region**: Support for multiple cloud regions
|
||||||
|
- **Authentication**: Handles cloud credentials and permissions
|
||||||
|
|
||||||
|
## Key Tools and Dependencies
|
||||||
|
|
||||||
|
### **Core Tools**:
|
||||||
|
- **osbuild**: Primary image building engine
|
||||||
|
- **osbuild-ostree**: OSTree integration for atomic updates
|
||||||
|
- **osbuild-depsolve-dnf**: DNF-based dependency resolution
|
||||||
|
- **osbuild-lvm2**: LVM2 support for advanced partitioning
|
||||||
|
- **podman**: Container runtime and management
|
||||||
|
- **qemu-img**: Image format conversion and manipulation
|
||||||
|
|
||||||
|
### **Package Management**:
|
||||||
|
- **dnf**: Package manager for dependency resolution
|
||||||
|
- **rpm**: Package format handling
|
||||||
|
- **librepo**: Optional high-performance download backend
|
||||||
|
- **subscription-manager**: RHEL subscription handling
|
||||||
|
|
||||||
|
### **System Components**:
|
||||||
|
- **selinux-policy-targeted**: SELinux policy enforcement
|
||||||
|
- **distribution-gpg-keys**: Package signature verification
|
||||||
|
- **qemu-user**: Cross-architecture emulation support
|
||||||
|
|
||||||
|
## Configuration System
|
||||||
|
|
||||||
|
### **Blueprint Format**:
|
||||||
|
- **TOML/JSON Configuration**: User customization files
|
||||||
|
- **Container Metadata**: Extracts configuration from container images
|
||||||
|
- **Environment Variables**: AWS credentials and other settings
|
||||||
|
- **Command-line Flags**: Extensive CLI options for fine-tuning
|
||||||
|
|
||||||
|
### **Hardcoded Defaults**:
|
||||||
|
- **Container Size Multiplier**: `containerSizeToDiskSizeMultiplier = 2`
|
||||||
|
- **Default Image Size**: `DEFAULT_SIZE = 10 * GibiByte`
|
||||||
|
- **Kernel Options**: `"rw"`, `"console=tty0"`, `"console=ttyS0"`
|
||||||
|
- **Distribution Paths**: Multiple fallback paths for definitions
|
||||||
|
|
||||||
|
## Build Process Stages
|
||||||
|
|
||||||
|
### **Stage 1: Initialization**
|
||||||
|
1. Parse CLI arguments and validate inputs
|
||||||
|
2. Mount container storage and validate access
|
||||||
|
3. Load configuration and blueprint files
|
||||||
|
4. Validate container image tags and accessibility
|
||||||
|
|
||||||
|
### **Stage 2: Container Analysis**
|
||||||
|
1. Get container size for disk sizing calculations
|
||||||
|
2. Create Podman container instance
|
||||||
|
3. Extract OS information (distro, version, kernel)
|
||||||
|
4. Initialize DNF package manager in container
|
||||||
|
5. Create DNF solver for dependency resolution
|
||||||
|
|
||||||
|
### **Stage 3: Manifest Generation**
|
||||||
|
1. Load distribution-specific package definitions
|
||||||
|
2. Create package set chains for different image types
|
||||||
|
3. Resolve package dependencies using DNF solver
|
||||||
|
4. Resolve container specifications and architectures
|
||||||
|
5. Serialize manifest for OSBuild execution
|
||||||
|
|
||||||
|
### **Stage 4: Image Building**
|
||||||
|
1. Execute OSBuild pipeline based on manifest
|
||||||
|
2. Create partition tables and filesystem layouts
|
||||||
|
3. Install bootloader and configure boot parameters
|
||||||
|
4. Copy container contents to target filesystem
|
||||||
|
5. Apply user customizations and configurations
|
||||||
|
6. Generate final bootable image
|
||||||
|
|
||||||
|
### **Stage 5: Output and Upload**
|
||||||
|
1. Save images to output directory
|
||||||
|
2. Optionally upload to cloud providers (AWS/GCP)
|
||||||
|
3. Clean up temporary files and containers
|
||||||
|
4. Report build status and results
|
||||||
|
|
||||||
|
## Supported Image Types
|
||||||
|
|
||||||
|
### **Disk Images**:
|
||||||
|
- **QCOW2**: KVM/QEMU virtual machine images
|
||||||
|
- **AMI**: Amazon Machine Images for AWS
|
||||||
|
- **VMDK**: VMware virtual machine images
|
||||||
|
- **VHD**: Microsoft Hyper-V virtual machine images
|
||||||
|
- **GCE**: Google Compute Engine images
|
||||||
|
- **Raw**: Raw disk images for direct hardware deployment
|
||||||
|
|
||||||
|
### **ISO Images**:
|
||||||
|
- **Anaconda ISO**: Fedora/CentOS/RHEL installer images
|
||||||
|
- **Custom ISO**: User-defined ISO configurations
|
||||||
|
|
||||||
|
## Architecture Support
|
||||||
|
|
||||||
|
- **x86_64**: Primary architecture
|
||||||
|
- **aarch64**: ARM64 support
|
||||||
|
- **ppc64le**: PowerPC 64-bit little-endian
|
||||||
|
- **s390x**: IBM Z architecture
|
||||||
|
- **riscv64**: RISC-V 64-bit architecture
|
||||||
|
|
||||||
|
## Testing Infrastructure
|
||||||
|
|
||||||
|
- **Python-based Tests**: Integration and functional testing
|
||||||
|
- **Go Unit Tests**: Component-level testing
|
||||||
|
- **Container Testing**: Tests run in isolated containers
|
||||||
|
- **VM Testing**: Can test resulting images in virtual machines
|
||||||
|
- **Test Plans**: FMF-based test organization and execution
|
||||||
|
|
||||||
|
This comprehensive workflow demonstrates the sophisticated orchestration of multiple tools and systems to create production-ready bootable images from container inputs, making it a powerful tool for modern container-native operating system deployment.
|
||||||
Loading…
Add table
Add a link
Reference in a new issue