debian-forge/debian-forge-docs/osbuild-architecture.md

OSBuild Architecture for Debian Forge

Overview

This document details OSBuild's core architecture patterns that Debian Forge must maintain for 1:1 compatibility while extending the system for Debian atomic builds.

Core OSBuild Components

1. Pipeline System

• Stages: Modular build steps (e.g., org.osbuild.rpm, org.osbuild.ostree)
• Devices: Block device management (org.osbuild.loopback, org.osbuild.luks2)
• Mounts: Filesystem mounting (org.osbuild.bind, org.osbuild.ext4)
• Inputs: Source management (org.osbuild.containers, org.osbuild.files)
• Assemblers: Output format generation (org.osbuild.qemu, org.osbuild.tar)

2. Build Environment Management

• Buildroot Isolation: Uses bubblewrap for sandboxed build environments
• Object Store: FsCache for caching downloaded sources and intermediate artifacts
• API Communication: JSON-based inter-process communication via Unix sockets

3. OSBuild Composer Integration

• Web Service Layer: HTTP APIs for image composition and build orchestration
• Blueprint System: High-level image configuration definitions
• Build Orchestration: Translates user requests into OSBuild pipeline descriptions
• Multiple APIs: Supports lorax-composer API and custom endpoints

Debian Forge Extensions

1. Debian-Specific Stages

• org.osbuild.debootstrap: Base Debian filesystem creation
• org.osbuild.apt: Debian package management
• org.osbuild.apt.config: APT configuration and sources
• org.osbuild.ostree.commit: OSTree commit creation for Debian
• org.osbuild.ostree.deploy: OSTree deployment for Debian
• org.osbuild.sbuild: Debian package building in chroot
• org.osbuild.debian.source: Debian source package management

2. Build Orchestration System

• Resource Management: CPU, memory, and storage allocation
• Build Queue: Priority-based build scheduling
• State Machine: Build lifecycle management (PENDING → QUEUED → BUILDING → COMPLETED)
• Artifact Management: SQLite-based metadata and filesystem organization
• Environment Management: Isolated build environment creation and cleanup

3. Integration Points

• OSBuild Pipeline Execution: Direct integration with OSBuild core
• Composer API Compatibility: Extend composer for Debian atomic images
• Debos Logic Integration: Adapt debos actions to OSBuild stages

Compatibility Requirements

1:1 OSBuild Compatibility

• Stage Interface: Maintain identical stage input/output contracts
• Pipeline Format: Use standard OSBuild manifest format
• Build Environment: Preserve bubblewrap isolation and object store patterns
• API Patterns: Follow existing OSBuild API communication patterns

Debian-Specific Extensions

• Package Management: Replace RPM with APT/dpkg
• Build Tools: Replace Mock with sbuild/pbuilder
• Atomic System: Add OSTree composition and deployment
• Web Interface: Extend composer for Debian atomic workflows

Architecture Patterns

1. Stage Pattern

def run_stage(tree, options):
    # Stage implementation
    # Must follow OSBuild stage contract
    # Return metadata via api.metadata

2. Build Environment Pattern

# Use bubblewrap for isolation
# Mount essential filesystems (/dev, /sys, /proc)
# Follow OSBuild buildroot patterns

3. Object Store Pattern

# Use FsCache for source and artifact caching
# Maintain OSBuild cache structure
# Extend for Debian-specific artifacts

Integration Strategy

Phase 1: Core Compatibility

• Maintain OSBuild stage interface contracts
• Preserve build environment isolation patterns
• Keep object store and caching mechanisms

Phase 2: Debian Extensions

• Add Debian-specific stages following OSBuild patterns
• Extend build orchestration for Debian workflows
• Integrate debos logic into OSBuild architecture

Phase 3: Composer Integration

• Test osbuild-composer with Debian stages
• Extend blueprint system for Debian atomic images
• Maintain API compatibility with existing composer ecosystem

Testing Strategy

Compatibility Testing

• Validate all Debian stages follow OSBuild patterns
• Test build environment isolation and object store
• Verify pipeline execution compatibility

Integration Testing

• Test Debian stages in OSBuild pipelines
• Validate build orchestration with OSBuild core
• Test composer integration with Debian workflows

Performance Testing

• Benchmark Debian stages against OSBuild baselines
• Test resource allocation and build queue performance
• Validate stress testing under concurrent load

Future Considerations

OSBuild Updates

• Monitor OSBuild core changes for compatibility
• Plan for OSBuild version upgrades
• Maintain compatibility with new OSBuild features

Composer Evolution

• Track osbuild-composer development
• Plan integration with new composer features
• Maintain API compatibility across versions

Debian Ecosystem

• Monitor Debian package management evolution
• Plan for new Debian build tools and standards
• Maintain compatibility with Debian release cycles