218 lines
No EOL
11 KiB
Markdown
218 lines
No EOL
11 KiB
Markdown
# Three-Tool Plan for Debian Build and Assembly System
|
|
|
|
## Executive Summary
|
|
|
|
This plan outlines the creation of a three-tool system that mirrors the functionality of Fedora's Pungi, Koji, and Mock, but is designed specifically for a Debian-based ecosystem. Each tool will be a stand-alone, purpose-built application that works with the others to provide a flexible, secure, and reproducible way to build, manage, and compose distributions like ParticleOS.
|
|
|
|
## The Three Tools
|
|
|
|
### 1. Mock Alternative: Deb-Mock
|
|
|
|
**Purpose**: A low-level utility to create clean, isolated build environments for single Debian packages. This tool is a direct functional replacement for Mock.
|
|
|
|
**Core Components**:
|
|
- **sbuild Integration**: A wrapper around the native Debian sbuild tool to standardize its command-line arguments and behavior.
|
|
- **Chroot Management**: Handles the creation, maintenance, and cleanup of the base chroot images used for building.
|
|
- **Build Metadata Capture**: Captures and standardizes all build output, including logs, .deb files, and .changes files, in a format that the Koji alternative can easily consume.
|
|
- **Reproducible Build Enforcement**: Ensures that all build dependencies are satisfied within the isolated environment and that no external packages can contaminate the build.
|
|
|
|
### 2. Koji Alternative: Deb-Orchestrator
|
|
|
|
**Purpose**: The central build hub that manages build requests, schedules tasks, and stores all build artifacts. This tool is a direct functional replacement for Koji.
|
|
|
|
**Core Components**:
|
|
- **Build Queue and Scheduler**: Manages incoming build requests from developers and automatically schedules them for building on available workers.
|
|
- **Worker Daemon**: A service that runs on each build host, polling the central queue and invoking Deb-Mock to execute a build task.
|
|
- **Artifact Manager**: A central repository that stores all artifacts produced by Deb-Mock, including the .deb files and build logs. It also provides a robust tagging system to organize different versions and releases.
|
|
- **Web Interface & API**: A user-friendly web application and a backend API to monitor build progress, review logs, and manage the system.
|
|
|
|
### 3. Pungi Alternative: Tumbi-Assembler
|
|
|
|
**Purpose**: The distribution composition tool that takes a set of built packages and assembles them into a complete, usable distribution. This tool is a direct functional replacement for Pungi.
|
|
|
|
**Core Components**:
|
|
- **Dependency Resolver**: A custom, high-level module that understands the complex dependencies of the final distribution and selects the correct versions of packages from the Deb-Orchestrator artifact store.
|
|
- **Distribution Blueprint**: Reads a configuration file (similar to a Pungi treefile) that defines the packages, groups, and configurations for the final OS image.
|
|
- **Composition Engine**: Orchestrates the process of using the gathered packages to build the final distribution artifacts. This involves:
|
|
- **APT Repository Creator**: Creates a temporary APT repository for the specific set of packages.
|
|
- **OSTree Generation**: Uses apt-ostree to create the atomic OSTree commit.
|
|
- **Live System Integration**: Uses live-build to create a bootable ISO.
|
|
- **Container Image Builder**: Uses bootc-deb to build container images from the OSTree commits.
|
|
|
|
## High-Level Workflow
|
|
|
|
The workflow will follow a logical progression, with each tool serving a specific function:
|
|
|
|
1. **A developer submits a new source package to Deb-Orchestrator.**
|
|
2. **Deb-Orchestrator schedules a worker to build the package using Deb-Mock.**
|
|
3. **Deb-Mock builds the package in a clean chroot and sends the resulting .deb file and logs back to Deb-Orchestrator's artifact manager.**
|
|
4. **Once all required packages for a release are available in the Deb-Orchestrator artifact store, Tumbi-Assembler is invoked.**
|
|
5. **Tumbi-Assembler reads its configuration, fetches the correct packages from Deb-Orchestrator, and then uses a series of integrated tools to compose the final distribution artifacts (OSTree, ISO, etc.).**
|
|
|
|
## Development Phases
|
|
|
|
### Phase 1: Deb-Mock Development (Weeks 1-6)
|
|
|
|
#### Objective
|
|
Create a robust, reproducible build environment tool that replaces Mock for Debian packages.
|
|
|
|
#### Tasks
|
|
- **sbuild Wrapper Development**: Create a standardized wrapper around sbuild
|
|
- **Chroot Management System**: Implement chroot creation, maintenance, and cleanup
|
|
- **Build Metadata Standardization**: Define and implement metadata capture format
|
|
- **Reproducible Build Testing**: Ensure builds are reproducible and isolated
|
|
|
|
#### Deliverables
|
|
- Functional Deb-Mock tool
|
|
- Standardized build environment management
|
|
- Build metadata capture system
|
|
- Reproducible build verification
|
|
|
|
### Phase 2: Deb-Orchestrator Development (Weeks 7-14)
|
|
|
|
#### Objective
|
|
Create the central build management system that replaces Koji for Debian packages.
|
|
|
|
#### Tasks
|
|
- **Build Queue System**: Implement build request management and scheduling
|
|
- **Worker Daemon**: Create worker service for build execution
|
|
- **Artifact Management**: Implement artifact storage and tagging system
|
|
- **Web Interface**: Develop user interface for build monitoring and management
|
|
- **API Development**: Create programmatic interface for system integration
|
|
|
|
#### Deliverables
|
|
- Functional Deb-Orchestrator system
|
|
- Build queue and scheduling system
|
|
- Worker daemon for build execution
|
|
- Web interface and API
|
|
- Artifact management and tagging
|
|
|
|
### Phase 3: Tumbi-Assembler Enhancement (Weeks 15-20)
|
|
|
|
#### Objective
|
|
Enhance Tumbi-Assembler to work with Deb-Orchestrator and create complete distribution artifacts.
|
|
|
|
#### Tasks
|
|
- **Deb-Orchestrator Integration**: Connect to Deb-Orchestrator for package retrieval
|
|
- **Dependency Resolution Enhancement**: Improve dependency resolution for distribution composition
|
|
- **Distribution Blueprint System**: Implement configuration-driven distribution definition
|
|
- **Composition Engine Enhancement**: Enhance composition engine for multiple output formats
|
|
- **Integration Testing**: Test complete workflow from package build to distribution
|
|
|
|
#### Deliverables
|
|
- Enhanced Tumbi-Assembler with Deb-Orchestrator integration
|
|
- Distribution blueprint system
|
|
- Complete composition engine
|
|
- Multiple output format support (OSTree, ISO, Container)
|
|
- End-to-end workflow testing
|
|
|
|
## Technology Mapping
|
|
|
|
### **Fedora Tools → Debian Alternatives**
|
|
|
|
| Fedora Tool | Purpose | Debian Alternative | Status |
|
|
|-------------|---------|-------------------|--------|
|
|
| **Mock** | Build environment | **Deb-Mock** (sbuild wrapper) | 🔄 **PLANNED** |
|
|
| **Koji** | Build management | **Deb-Orchestrator** | 🔄 **PLANNED** |
|
|
| **Pungi** | Distribution composition | **Tumbi-Assembler** | ✅ **IN PROGRESS** |
|
|
|
|
### **Core Technologies**
|
|
|
|
#### **Deb-Mock**
|
|
- **sbuild**: Native Debian package building
|
|
- **chroot**: Isolated build environments
|
|
- **debhelper**: Debian package building utilities
|
|
|
|
#### **Deb-Orchestrator**
|
|
- **Database**: Build queue and artifact storage
|
|
- **Web Framework**: User interface and API
|
|
- **Message Queue**: Build scheduling and coordination
|
|
|
|
#### **Tumbi-Assembler**
|
|
- **apt-ostree**: Atomic system composition
|
|
- **live-build**: Live system creation
|
|
- **bootc-deb**: Container image creation
|
|
- **Calamares**: Installer framework
|
|
|
|
## Architecture Overview
|
|
|
|
```
|
|
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
|
|
│ Deb-Mock │ │ Deb-Orchestrator │ │ Tumbi-Assembler │
|
|
│ │ │ │ │ │
|
|
│ • sbuild wrapper│◄──►│ • Build queue │◄──►│ • Distribution │
|
|
│ • Chroot mgmt │ │ • Worker daemon │ │ composition │
|
|
│ • Metadata │ │ • Artifact store │ │ • OSTree gen │
|
|
│ • Reproducible │ │ • Web interface │ │ • Live system │
|
|
│ builds │ │ • API │ │ • Container │
|
|
└─────────────────┘ └──────────────────┘ └─────────────────┘
|
|
```
|
|
|
|
## Success Criteria
|
|
|
|
### **Phase 1 Success Criteria**
|
|
- [ ] Deb-Mock successfully builds .deb packages in isolated environments
|
|
- [ ] Build environments are reproducible and clean
|
|
- [ ] Build metadata is captured and standardized
|
|
- [ ] Integration with sbuild is seamless
|
|
|
|
### **Phase 2 Success Criteria**
|
|
- [ ] Deb-Orchestrator manages build requests and scheduling
|
|
- [ ] Worker daemon executes builds using Deb-Mock
|
|
- [ ] Artifact management system stores and organizes build outputs
|
|
- [ ] Web interface provides build monitoring and management
|
|
- [ ] API allows programmatic system access
|
|
|
|
### **Phase 3 Success Criteria**
|
|
- [ ] Tumbi-Assembler integrates with Deb-Orchestrator
|
|
- [ ] Distribution composition creates complete system images
|
|
- [ ] Multiple output formats are supported (OSTree, ISO, Container)
|
|
- [ ] End-to-end workflow functions from package build to distribution
|
|
- [ ] Complete ParticleOS Atomic Desktop is created
|
|
|
|
## Risk Assessment
|
|
|
|
### **High Risk**
|
|
- **Deb-Mock Integration**: sbuild wrapper complexity and chroot management
|
|
- **Deb-Orchestrator Architecture**: Build queue and worker coordination
|
|
- **System Integration**: Three-tool coordination and data flow
|
|
|
|
### **Medium Risk**
|
|
- **Web Interface Development**: User interface complexity
|
|
- **Artifact Management**: Storage and retrieval system design
|
|
- **Dependency Resolution**: Complex Debian dependency handling
|
|
|
|
### **Low Risk**
|
|
- **Individual Tool Development**: Each tool can be developed independently
|
|
- **Technology Stack**: Well-established Debian tools and frameworks
|
|
- **Documentation**: Process and system documentation
|
|
|
|
## Timeline Summary
|
|
|
|
| Phase | Duration | Focus | Tools |
|
|
|-------|----------|-------|-------|
|
|
| Phase 1 | Weeks 1-6 | Deb-Mock Development | sbuild, chroot |
|
|
| Phase 2 | Weeks 7-14 | Deb-Orchestrator Development | Database, Web, API |
|
|
| Phase 3 | Weeks 15-20 | Tumbi-Assembler Enhancement | Integration, Composition |
|
|
|
|
**Total Duration**: 20 weeks (5 months)
|
|
|
|
## Deliverables
|
|
|
|
### **Final System**
|
|
- **Deb-Mock**: Reproducible Debian package building tool
|
|
- **Deb-Orchestrator**: Central build management system
|
|
- **Tumbi-Assembler**: Distribution composition tool
|
|
- **Complete Workflow**: End-to-end package build to distribution
|
|
|
|
### **Documentation**
|
|
- **Tool Documentation**: Individual tool usage and configuration
|
|
- **Integration Guide**: How the three tools work together
|
|
- **Workflow Guide**: Complete process from development to distribution
|
|
- **API Documentation**: Programmatic access to system components
|
|
|
|
---
|
|
|
|
**Status**: 🔄 **PLANNED**
|
|
|
|
This three-tool plan provides a direct replacement for Fedora's Pungi, Koji, and Mock ecosystem, adapted specifically for Debian-based distribution building and assembly. |