diff --git a/osbuild-universe.md b/osbuild-universe.md new file mode 100644 index 0000000..29a6d4a --- /dev/null +++ b/osbuild-universe.md @@ -0,0 +1,468 @@ +# 🌌 OSBuild Universe: Complete Ecosystem Overview + +The OSBuild ecosystem is a comprehensive suite of tools and services for building, managing, and deploying operating system images. This document provides a complete overview of each component and how they work together to create a powerful image building platform. + +## 🏗️ **Core Architecture Overview** + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ OSBuild Ecosystem │ +├─────────────────────────────────────────────────────────────────┤ +│ Frontend (UI) │ CLI Tools │ API Services │ Build Engine │ +│ │ │ │ │ +│ image-builder- │ image- │ osbuild- │ osbuild │ +│ frontend │ builder-cli │ composer │ (core) │ +└─────────────────────────────────────────────────────────────────┘ + │ + ▼ + ┌─────────────────┐ + │ Image Types │ + │ │ + │ osbuild/images │ + └─────────────────┘ + │ + ▼ + ┌─────────────────┐ + │ CI/CD Actions │ + │ │ + │ bootc-image- │ + │ builder-action │ + └─────────────────┘ +``` + +## 🚀 **1. osbuild (Core Engine)** + +**Repository**: [osbuild/osbuild](https://github.com/osbuild/osbuild) +**Role**: Core pipeline execution engine +**Language**: Python +**Status**: Foundation component + +### **What It Is** +The core OSBuild engine that executes declarative pipelines to build operating system artifacts. It's the foundation upon which all other components are built. + +### **Key Features** +- **Pipeline-based architecture** - Declarative JSON manifests define build processes +- **Stage system** - Composable building blocks for OS construction +- **Assembler system** - Multiple output formats (QEMU, OSTree, tar, etc.) +- **Runner support** - Linux, QEMU, and other execution environments +- **Extensible** - Custom stages and assemblers can be added + +### **How It Works** +1. **Parse manifest** - JSON configuration defines the build pipeline +2. **Execute stages** - Each stage modifies the filesystem tree +3. **Run assembler** - Final stage creates the desired output format +4. **Output artifact** - QEMU image, OSTree commit, tar archive, etc. + +### **Example Usage** +```bash +# Build from manifest +osbuild --json manifest.json + +# List available stages +osbuild --list-stages + +# Validate manifest +osbuild --check manifest.json +``` + +--- + +## 🎼 **2. osbuild-composer** + +**Repository**: [osbuild/osbuild-composer](https://github.com/osbuild/osbuild-composer) +**Role**: HTTP service for image composition +**Language**: Go +**Status**: Production-ready service layer + +### **What It Is** +A high-level HTTP service that sits above the core osbuild engine, providing standardized image types and APIs for building operating system images. It's the "orchestrator" that translates user requests into osbuild pipelines. + +### **Key Features** +- **HTTP API service** - RESTful interface for image building +- **Predefined image types** - Standardized OS compositions +- **Multiple APIs** - lorax-composer compatibility, cloud APIs +- **Blueprint system** - Policy layer for user configuration +- **Distribution support** - RHEL, Fedora, CentOS, and others +- **Target uploads** - AWS, Azure, GCP, local storage + +### **Architecture** +``` +┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ +│ User Request │───▶│ Composer API │───▶│ osbuild Core │ +│ (Blueprint) │ │ │ │ (Pipeline) │ +└─────────────────┘ └─────────────────┘ └─────────────────┘ + │ + ▼ + ┌─────────────────┐ + │ Image Output │ + │ (QEMU, etc.) │ + └─────────────────┘ +``` + +### **Use Cases** +- **Enterprise deployments** - Standardized image building +- **Cloud infrastructure** - Automated image creation +- **CI/CD pipelines** - Programmatic image building +- **Multi-tenant services** - Shared image building infrastructure + +### **Example API Usage** +```bash +# Create a blueprint +composer-cli blueprints push my-blueprint.toml + +# Build an image +composer-cli compose start my-blueprint qcow2 + +# Check build status +composer-cli compose status + +# Download result +composer-cli compose image +``` + +--- + +## 🖼️ **3. osbuild/images** + +**Repository**: [osbuild/images](https://github.com/osbuild/images) +**Role**: Image type definitions and templates +**Language**: JSON, YAML +**Status**: Reference implementations + +### **What It Is** +A collection of predefined image types and templates that demonstrate how to build various operating system images using osbuild. These serve as both examples and production-ready image definitions. + +### **Key Features** +- **Standardized image types** - Consistent across distributions +- **Template system** - Reusable image configurations +- **Distribution support** - RHEL, Fedora, CentOS, Debian +- **Cloud-ready** - AWS, Azure, GCP, OpenStack formats +- **Container images** - OCI-compliant container builds + +### **Image Types Available** +- **QEMU images** - Virtual machine disk images +- **Cloud images** - AWS AMI, Azure VHD, GCP disk +- **Container images** - OCI containers for deployment +- **ISO images** - Bootable installation media +- **OSTree commits** - Atomic OS updates + +### **Example Image Definition** +```json +{ + "name": "debian-server", + "description": "Debian server image with minimal packages", + "distro": "debian-12", + "arch": "x86_64", + "image_type": "qcow2", + "repositories": [ + { + "name": "debian", + "baseurl": "https://deb.debian.org/debian", + "enabled": true + } + ], + "packages": [ + "openssh-server", + "systemd", + "curl" + ] +} +``` + +--- + +## 🖥️ **4. image-builder-cli** + +**Repository**: [osbuild/image-builder-cli](https://github.com/osbuild/image-builder-cli) +**Role**: Command-line interface for image building +**Language**: Go +**Status**: User-facing CLI tool + +### **What It Is** +A command-line interface that provides easy access to image building capabilities. It's designed to be user-friendly and can work with both local osbuild installations and remote composer services. + +### **Key Features** +- **Simple CLI interface** - Easy-to-use commands +- **Local and remote builds** - Works with local osbuild or composer services +- **Multiple output formats** - QEMU, cloud, container formats +- **Blueprint support** - Create and manage image blueprints +- **Build monitoring** - Track build progress and status + +### **Example Commands** +```bash +# Build a local image +image-builder build --blueprint my-blueprint.toml --output qcow2 + +# Connect to remote composer +image-builder --server https://composer.example.com build --blueprint server.toml + +# List available image types +image-builder list-types + +# Create a new blueprint +image-builder blueprint create my-server +``` + +### **Use Cases** +- **Developers** - Quick local image building +- **DevOps engineers** - Scripted image creation +- **System administrators** - On-demand image building +- **CI/CD pipelines** - Automated image creation + +--- + +## 🎨 **5. image-builder-frontend** + +**Repository**: [osbuild/image-builder-frontend](https://github.com/osbuild/image-builder-frontend) +**Role**: Web-based user interface +**Language**: React, TypeScript +**Status**: Modern web UI + +### **What It Is** +A modern web-based frontend that provides a graphical interface for building and managing operating system images. It's designed to be user-friendly and accessible to users who prefer GUI over command-line tools. + +### **Key Features** +- **Web-based interface** - Accessible from any browser +- **Visual blueprint editor** - Drag-and-drop interface for image configuration +- **Real-time build monitoring** - Watch builds progress in real-time +- **Image management** - Browse, download, and manage built images +- **User authentication** - Secure access control +- **Responsive design** - Works on desktop and mobile devices + +### **User Experience** +``` +┌─────────────────────────────────────────────────────────────┐ +│ Image Builder Dashboard │ +├─────────────────────────────────────────────────────────────┤ +│ [Create Blueprint] [Build Image] [Manage Images] │ +│ │ +│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────┐ │ +│ │ Blueprint │ │ Build Status │ │ Image │ │ +│ │ Editor │ │ Monitor │ │ Gallery │ │ +│ └─────────────────┘ └─────────────────┘ └─────────────┘ │ +└─────────────────────────────────────────────────────────────┘ +``` + +### **Target Users** +- **System administrators** - Visual image management +- **DevOps teams** - Collaborative image building +- **Non-technical users** - Simple image creation +- **Enterprise users** - Centralized image management + +--- + +## 🔄 **6. bootc-image-builder-action** + +**Repository**: [osbuild/bootc-image-builder-action](https://github.com/osbuild/bootc-image-builder-action) +**Role**: GitHub Actions integration +**Language**: YAML, Shell +**Status**: CI/CD automation + +### **What It Is** +A GitHub Actions workflow that automates the process of building bootc-compatible container images using osbuild. It's designed for continuous integration and deployment pipelines. + +### **Key Features** +- **GitHub Actions integration** - Automated image building +- **bootc compatibility** - Container-native booting support +- **Multi-architecture** - x86_64, ARM64 support +- **Automated testing** - Build validation and testing +- **Artifact publishing** - Automatic image distribution +- **Version tagging** - Semantic versioning support + +### **Workflow Example** +```yaml +name: Build bootc Image +on: + push: + branches: [main] + pull_request: + branches: [main] + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + + - name: Build bootc Image + uses: osbuild/bootc-image-builder-action@v1 + with: + blueprint: my-blueprint.toml + output-format: qcow2 + architecture: x86_64 + + - name: Upload Artifact + uses: actions/upload-artifact@v3 + with: + name: bootc-image + path: output/*.qcow2 +``` + +### **Use Cases** +- **Open source projects** - Automated image building +- **CI/CD pipelines** - Continuous image creation +- **Testing environments** - Automated testing with fresh images +- **Development workflows** - Rapid iteration and testing + +--- + +## 🔗 **How They Work Together** + +### **Complete Workflow Example** + +```mermaid +graph TD + A[User] --> B[image-builder-frontend] + A --> C[image-builder-cli] + B --> D[osbuild-composer] + C --> D + D --> E[osbuild core] + E --> F[osbuild/images] + F --> G[Output Image] + + H[CI/CD Pipeline] --> I[bootc-image-builder-action] + I --> D +``` + +### **1. User Interface Layer** +- **Frontend**: Web-based GUI for visual image building +- **CLI**: Command-line interface for automation and scripting + +### **2. Service Layer** +- **Composer**: HTTP API service that orchestrates builds +- **Image Types**: Predefined templates and configurations + +### **3. Execution Layer** +- **Core Engine**: Pipeline execution and artifact creation +- **Stages**: Modular building blocks for OS construction + +### **4. Automation Layer** +- **GitHub Actions**: CI/CD integration for automated builds +- **Workflows**: Predefined automation patterns + +--- + +## 🎯 **Integration with particle-os** + +### **Current Status** +particle-os has successfully implemented: +- ✅ **Core osbuild stages** - 10 Debian-specific stages +- ✅ **bootupd support** - Modern bootloader management +- ✅ **QEMU assembler** - Bootable disk image creation +- ✅ **OSTree integration** - Atomic OS updates +- ✅ **bootc support** - Container-native booting + +### **Future Integration Opportunities** +- 🔄 **osbuild-composer** - HTTP service for particle-os +- 🔄 **image-builder-cli** - CLI tools for particle-os +- 🔄 **image-builder-frontend** - Web UI for particle-os +- 🔄 **bootc-image-builder-action** - CI/CD automation + +### **Benefits of Integration** +1. **Standardized APIs** - Compatible with existing osbuild ecosystem +2. **Professional tooling** - Enterprise-grade management interfaces +3. **Automation support** - CI/CD integration for particle-os +4. **Community adoption** - Leverage existing osbuild user base +5. **Tool ecosystem** - Access to mature image building tools + +--- + +## 🚀 **Getting Started with the Ecosystem** + +### **For Developers** +1. **Start with osbuild core** - Understand pipeline execution +2. **Use image-builder-cli** - Command-line image building +3. **Explore osbuild/images** - Study existing image types +4. **Build custom stages** - Extend functionality + +### **For System Administrators** +1. **Deploy osbuild-composer** - Set up image building service +2. **Use image-builder-frontend** - Web-based management +3. **Create blueprints** - Define standard image configurations +4. **Automate with CI/CD** - Integrate with existing workflows + +### **For DevOps Teams** +1. **Integrate with CI/CD** - Use bootc-image-builder-action +2. **Automate image creation** - Script-based image building +3. **Standardize images** - Consistent deployment artifacts +4. **Monitor builds** - Track image creation and deployment + +--- + +## 🌟 **Key Advantages of the OSBuild Ecosystem** + +### **1. Unified Architecture** +- **Single core engine** - Consistent behavior across all tools +- **Standardized APIs** - Interoperable components +- **Modular design** - Mix and match components as needed + +### **2. Enterprise Ready** +- **Production proven** - Used in large-scale deployments +- **Security focused** - Secure image building practices +- **Compliance ready** - Meets enterprise requirements + +### **3. Developer Friendly** +- **Extensible** - Custom stages and assemblers +- **Well documented** - Comprehensive guides and examples +- **Active community** - Support and collaboration + +### **4. Cloud Native** +- **Container support** - OCI-compliant images +- **Cloud integration** - AWS, Azure, GCP support +- **Modern tooling** - GitOps and CI/CD ready + +--- + +## 🔮 **Future Directions** + +### **Short Term** +- **Enhanced cloud support** - More cloud provider integrations +- **Performance improvements** - Faster build times +- **Better monitoring** - Enhanced build visibility + +### **Medium Term** +- **Multi-architecture** - ARM64, RISC-V support +- **Advanced security** - Secure boot, TPM integration +- **Edge computing** - IoT and edge device support + +### **Long Term** +- **AI-powered optimization** - Intelligent build optimization +- **Distributed building** - Multi-node build clusters +- **Advanced automation** - Self-healing build systems + +--- + +## 📚 **Resources and References** + +### **Official Documentation** +- [OSBuild.org](https://www.osbuild.org) - Main project website +- [OSBuild Documentation](https://osbuild.org/docs) - Comprehensive guides +- [API Reference](https://osbuild.org/docs/api) - REST API documentation + +### **Community Resources** +- [GitHub Discussions](https://github.com/orgs/osbuild/discussions) - Community support +- [Matrix Chat](https://matrix.to/#/#image-builder:fedoraproject.org) - Real-time chat +- [Mailing Lists](https://lists.osbuild.org) - Email discussions + +### **Getting Help** +- **GitHub Issues** - Bug reports and feature requests +- **Community Forums** - User support and discussions +- **Documentation** - Self-service help and guides + +--- + +## 🎉 **Conclusion** + +The OSBuild ecosystem represents a **comprehensive, enterprise-ready solution** for building and managing operating system images. With its modular architecture, extensive tooling, and active community, it provides everything needed for modern image building workflows. + +For particle-os, this ecosystem offers: +- **Professional tooling** - Enterprise-grade management interfaces +- **Standardized workflows** - Compatible with existing osbuild deployments +- **Automation support** - CI/CD integration and automation +- **Community adoption** - Leverage mature, well-tested tools + +By integrating with the OSBuild ecosystem, particle-os can provide users with **professional-grade image building capabilities** while maintaining its unique Debian-focused approach and modern bootloader management features. + +--- + +*This document provides a comprehensive overview of the OSBuild ecosystem. For specific implementation details, refer to the individual project repositories and documentation.*