deb-mock/koji-and-mock-alternative.md

Three-Tool Plan for Debian Build and Assembly System

Executive Summary

This plan outlines the creation of a three-tool system that mirrors the functionality of Fedora's Pungi, Koji, and Mock, but is designed specifically for a Debian-based ecosystem. Each tool will be a stand-alone, purpose-built application that works with the others to provide a flexible, secure, and reproducible way to build, manage, and compose distributions like ParticleOS.

The Three Tools

1. Mock Alternative: Deb-Mock

Purpose: A low-level utility to create clean, isolated build environments for single Debian packages. This tool is a direct functional replacement for Mock.

Core Components:

• sbuild Integration: A wrapper around the native Debian sbuild tool to standardize its command-line arguments and behavior.
• Chroot Management: Handles the creation, maintenance, and cleanup of the base chroot images used for building.
• Build Metadata Capture: Captures and standardizes all build output, including logs, .deb files, and .changes files, in a format that the Koji alternative can easily consume.
• Reproducible Build Enforcement: Ensures that all build dependencies are satisfied within the isolated environment and that no external packages can contaminate the build.

2. Koji Alternative: Deb-Orchestrator

Purpose: The central build hub that manages build requests, schedules tasks, and stores all build artifacts. This tool is a direct functional replacement for Koji.

Core Components:

• Build Queue and Scheduler: Manages incoming build requests from developers and automatically schedules them for building on available workers.
• Worker Daemon: A service that runs on each build host, polling the central queue and invoking Deb-Mock to execute a build task.
• Artifact Manager: A central repository that stores all artifacts produced by Deb-Mock, including the .deb files and build logs. It also provides a robust tagging system to organize different versions and releases.
• Web Interface & API: A user-friendly web application and a backend API to monitor build progress, review logs, and manage the system.

3. Pungi Alternative: Tumbi-Assembler

Purpose: The distribution composition tool that takes a set of built packages and assembles them into a complete, usable distribution. This tool is a direct functional replacement for Pungi.

Core Components:

• Dependency Resolver: A custom, high-level module that understands the complex dependencies of the final distribution and selects the correct versions of packages from the Deb-Orchestrator artifact store.
• Distribution Blueprint: Reads a configuration file (similar to a Pungi treefile) that defines the packages, groups, and configurations for the final OS image.
• Composition Engine: Orchestrates the process of using the gathered packages to build the final distribution artifacts. This involves:
  • APT Repository Creator: Creates a temporary APT repository for the specific set of packages.
  • OSTree Generation: Uses apt-ostree to create the atomic OSTree commit.
  • Live System Integration: Uses live-build to create a bootable ISO.
  • Container Image Builder: Uses bootc-deb to build container images from the OSTree commits.

High-Level Workflow

The workflow will follow a logical progression, with each tool serving a specific function:

1. A developer submits a new source package to Deb-Orchestrator.
2. Deb-Orchestrator schedules a worker to build the package using Deb-Mock.
3. Deb-Mock builds the package in a clean chroot and sends the resulting .deb file and logs back to Deb-Orchestrator's artifact manager.
4. Once all required packages for a release are available in the Deb-Orchestrator artifact store, Tumbi-Assembler is invoked.
5. Tumbi-Assembler reads its configuration, fetches the correct packages from Deb-Orchestrator, and then uses a series of integrated tools to compose the final distribution artifacts (OSTree, ISO, etc.).

Development Phases

Phase 1: Deb-Mock Development (Weeks 1-6)

Objective

Create a robust, reproducible build environment tool that replaces Mock for Debian packages.

Tasks

• sbuild Wrapper Development: Create a standardized wrapper around sbuild
• Chroot Management System: Implement chroot creation, maintenance, and cleanup
• Build Metadata Standardization: Define and implement metadata capture format
• Reproducible Build Testing: Ensure builds are reproducible and isolated

Deliverables

• Functional Deb-Mock tool
• Standardized build environment management
• Build metadata capture system
• Reproducible build verification

Phase 2: Deb-Orchestrator Development (Weeks 7-14)

Objective

Create the central build management system that replaces Koji for Debian packages.

Tasks

• Build Queue System: Implement build request management and scheduling
• Worker Daemon: Create worker service for build execution
• Artifact Management: Implement artifact storage and tagging system
• Web Interface: Develop user interface for build monitoring and management
• API Development: Create programmatic interface for system integration

Deliverables

• Functional Deb-Orchestrator system
• Build queue and scheduling system
• Worker daemon for build execution
• Web interface and API
• Artifact management and tagging

Phase 3: Tumbi-Assembler Enhancement (Weeks 15-20)

Objective

Enhance Tumbi-Assembler to work with Deb-Orchestrator and create complete distribution artifacts.

Tasks

• Deb-Orchestrator Integration: Connect to Deb-Orchestrator for package retrieval
• Dependency Resolution Enhancement: Improve dependency resolution for distribution composition
• Distribution Blueprint System: Implement configuration-driven distribution definition
• Composition Engine Enhancement: Enhance composition engine for multiple output formats
• Integration Testing: Test complete workflow from package build to distribution

Deliverables

• Enhanced Tumbi-Assembler with Deb-Orchestrator integration
• Distribution blueprint system
• Complete composition engine
• Multiple output format support (OSTree, ISO, Container)
• End-to-end workflow testing

Technology Mapping

Fedora Tools → Debian Alternatives

Fedora Tool	Purpose	Debian Alternative	Status
Mock	Build environment	Deb-Mock (sbuild wrapper)	🔄 PLANNED
Koji	Build management	Deb-Orchestrator	🔄 PLANNED
Pungi	Distribution composition	Tumbi-Assembler	✅ IN PROGRESS

Core Technologies

Deb-Mock

• sbuild: Native Debian package building
• chroot: Isolated build environments
• debhelper: Debian package building utilities

Deb-Orchestrator

• Database: Build queue and artifact storage
• Web Framework: User interface and API
• Message Queue: Build scheduling and coordination

Tumbi-Assembler

• apt-ostree: Atomic system composition
• live-build: Live system creation
• bootc-deb: Container image creation
• Calamares: Installer framework

Architecture Overview

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   Deb-Mock      │    │ Deb-Orchestrator │    │ Tumbi-Assembler │
│                 │    │                  │    │                 │
│ • sbuild wrapper│◄──►│ • Build queue    │◄──►│ • Distribution  │
│ • Chroot mgmt   │    │ • Worker daemon  │    │   composition   │
│ • Metadata      │    │ • Artifact store │    │ • OSTree gen    │
│ • Reproducible  │    │ • Web interface  │    │ • Live system   │
│   builds        │    │ • API            │    │ • Container     │
└─────────────────┘    └──────────────────┘    └─────────────────┘

Success Criteria

Phase 1 Success Criteria

• Deb-Mock successfully builds .deb packages in isolated environments
• Build environments are reproducible and clean
• Build metadata is captured and standardized
• Integration with sbuild is seamless

Phase 2 Success Criteria

• Deb-Orchestrator manages build requests and scheduling
• Worker daemon executes builds using Deb-Mock
• Artifact management system stores and organizes build outputs
• Web interface provides build monitoring and management
• API allows programmatic system access

Phase 3 Success Criteria

• Tumbi-Assembler integrates with Deb-Orchestrator
• Distribution composition creates complete system images
• Multiple output formats are supported (OSTree, ISO, Container)
• End-to-end workflow functions from package build to distribution
• Complete ParticleOS Atomic Desktop is created

Risk Assessment

High Risk

• Deb-Mock Integration: sbuild wrapper complexity and chroot management
• Deb-Orchestrator Architecture: Build queue and worker coordination
• System Integration: Three-tool coordination and data flow

Medium Risk

• Web Interface Development: User interface complexity
• Artifact Management: Storage and retrieval system design
• Dependency Resolution: Complex Debian dependency handling

Low Risk

• Individual Tool Development: Each tool can be developed independently
• Technology Stack: Well-established Debian tools and frameworks
• Documentation: Process and system documentation

Timeline Summary

Phase	Duration	Focus	Tools
Phase 1	Weeks 1-6	Deb-Mock Development	sbuild, chroot
Phase 2	Weeks 7-14	Deb-Orchestrator Development	Database, Web, API
Phase 3	Weeks 15-20	Tumbi-Assembler Enhancement	Integration, Composition

Total Duration: 20 weeks (5 months)

Deliverables

Final System

• Deb-Mock: Reproducible Debian package building tool
• Deb-Orchestrator: Central build management system
• Tumbi-Assembler: Distribution composition tool
• Complete Workflow: End-to-end package build to distribution

Documentation

• Tool Documentation: Individual tool usage and configuration
• Integration Guide: How the three tools work together
• Workflow Guide: Complete process from development to distribution
• API Documentation: Programmatic access to system components

---

Status: 🔄 PLANNED

This three-tool plan provides a direct replacement for Fedora's Pungi, Koji, and Mock ecosystem, adapted specifically for Debian-based distribution building and assembly.