🚀 particle-os

Complete Debian OSTree System Builder with osbuild Backend

particle-os is a production-ready Debian-based fork of ublue-os that provides comprehensive osbuild backend support for Debian ecosystems. This project adapts the Red Hat osbuild system to work seamlessly with Debian-based distributions, replacing RPM/DNF components with APT/DPKG equivalents.

🌟 What is osbuild?

osbuild is a pipeline-based build system for operating system artifacts that defines a universal pipeline description and execution engine. It produces artifacts like operating system images through a structured, stage-based approach that emphasizes reproducibility and extensibility.

🔧 How osbuild Works

osbuild follows a declarative pipeline architecture where:

1. Manifests define the complete build process as JSON
2. Stages are atomic, composable building blocks
3. Assemblers create final artifacts from stage outputs
4. Pipelines orchestrate stage execution and data flow

🏗️ Core Architecture

osbuild CLI → Manifest Parser → Pipeline Executor → Stage Runner → Assembler → Artifacts
     ↓              ↓                ↓               ↓            ↓          ↓
  Main Entry   JSON Schema    Pipeline Builder   Stage Exec   Output Gen   Final Files

🎯 Key Principles

• Stages are never broken, only deprecated - Same manifest always produces same output
• Explicit over implicit - No reliance on tree state
• Pipeline independence - Tree is empty at beginning of each pipeline
• Machine-generated manifests - No convenience functions for manual creation
• Confined build environment - Security against accidental misuse
• Distribution compatibility - Python 3.6+ support

🚀 What particle-os Provides

particle-os extends osbuild with 10 Debian-specific stages and Debian-specific assemblers to create a complete Debian OSTree system builder:

📋 Debian Stages

• org.osbuild.debian.debootstrap - Base system construction
• org.osbuild.debian.apt - Package management
• org.osbuild.debian.sources - APT sources configuration
• org.osbuild.debian.users - User account management
• org.osbuild.debian.locale - Locale configuration
• org.osbuild.debian.timezone - Timezone setup
• org.osbuild.debian.ostree - OSTree repository management
• org.osbuild.debian.bootc - Bootc integration
• org.osbuild.debian.systemd - OSTree-optimized systemd
• org.osbuild.debian.grub2 - GRUB2 bootloader configuration

🔧 Debian Assemblers

• org.osbuild.debian.qemu - Bootable disk image creation (raw, qcow2, vmdk, vdi)

🔗 osbuild and bootc-image-builder Integration

What is bootc-image-builder?

bootc-image-builder is a tool that uses osbuild to create bootable container images that can be deployed using bootc. It provides a higher-level interface for building OSTree-based systems with container-native booting capabilities.

How They Work Together

bootc-image-builder → osbuild → particle-os stages → Debian OSTree system
         ↓              ↓              ↓                    ↓
   Container config   Manifest    Debian stages      Bootable image

Integration Benefits

1. Container-native booting - Images can be deployed as containers
2. Atomic updates - OSTree provides rollback and update capabilities
3. Infrastructure as code - Declarative image definitions
4. CI/CD integration - Automated image building and testing
5. Multi-platform support - Build for different architectures and cloud providers

Example bootc-image-builder Usage

# Install bootc-image-builder
sudo apt install bootc-image-builder

# Build a Debian OSTree image using particle-os
bootc-image-builder build \
  --type qcow2 \
  --config config.toml \
  --output-dir ./outputs \
  debian-ostree.json

🚀 Quick Start

1. Installation

# Clone the repository
git clone https://git.raines.xyz/robojerk/deb-osbuild.git
cd deb-osbuild

# Set up development environment
./scripts/dev-setup.sh

# Install particle-os
make install

2. Basic Usage

Create a Simple Debian System

# Build a basic Debian system
osbuild examples/debian-basic.json

# This creates a compressed tar archive of a complete Debian system

Create a Bootable OSTree System

# Build a complete bootable Debian OSTree system
osbuild examples/debian-ostree-bootable.json

# This creates a bootable QCOW2 disk image with GRUB2 and bootc

3. Understanding Manifests

Basic Manifest Structure

{
  "version": "2",
  "pipelines": [
    {
      "name": "build",
      "runner": "org.osbuild.linux",
      "stages": [
        {
          "name": "org.osbuild.debian.debootstrap",
          "options": {
            "suite": "trixie",
            "mirror": "https://deb.debian.org/debian",
            "variant": "minbase"
          }
        }
      ]
    }
  ],
  "assembler": {
    "name": "org.osbuild.debian.qemu",
    "options": {
      "format": "qcow2",
      "filename": "debian-system.qcow2",
      "size": "15G"
    }
  }
}

Stage Execution Order

1. Sources → Configure APT repositories
2. Debootstrap → Create base Debian filesystem
3. APT → Install packages and dependencies
4. Users → Create user accounts and groups
5. Locale → Configure language and locale settings
6. Timezone → Set timezone configuration
7. Systemd → Configure systemd for OSTree
8. Bootc → Set up bootc for container-native booting
9. GRUB2 → Configure bootloader
10. OSTree → Create OSTree repository and commit

🔄 CI/CD Workflows

GitHub Actions

name: Build Debian OSTree Image

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build-image:
    runs-on: ubuntu-latest
    
    steps:
    - name: Checkout code
      uses: actions/checkout@v4
      
    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.11'
        
    - name: Install dependencies
      run: |
        sudo apt update
        sudo apt install -y python3-pip python3-venv
        python3 -m venv venv
        source venv/bin/activate
        pip install -r requirements.txt
        
    - name: Install particle-os
      run: |
        source venv/bin/activate
        make install
        
    - name: Run tests
      run: |
        source venv/bin/activate
        make test
        
    - name: Build Debian OSTree image
      run: |
        source venv/bin/activate
        osbuild examples/debian-ostree-bootable.json
        
    - name: Upload artifacts
      uses: actions/upload-artifact@v3
      with:
        name: debian-ostree-image
        path: debian-ostree-bootable.qcow2

GitLab CI

stages:
  - test
  - build
  - deploy

variables:
  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.pip-cache"

cache:
  paths:
    - .pip-cache/

test:
  stage: test
  image: python:3.11
  before_script:
    - pip install -r requirements.txt
    - make install
  script:
    - make test
  artifacts:
    reports:
      junit: test-results.xml

build:
  stage: build
  image: python:3.11
  before_script:
    - pip install -r requirements.txt
    - make install
  script:
    - osbuild examples/debian-ostree-bootable.json
  artifacts:
    paths:
      - "*.qcow2"
      - "*.raw"
    expire_in: 1 week
  only:
    - main

deploy:
  stage: deploy
  image: python:3.11
  script:
    - echo "Deploying image to production..."
    # Add your deployment logic here
  only:
    - main
  when: manual

Forgejo Actions

name: Build and Test Debian OSTree Image

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout
      uses: actions/checkout@v4
      
    - name: Setup Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.11'
        
    - name: Install system dependencies
      run: |
        sudo apt update
        sudo apt install -y python3-pip python3-venv build-essential
        
    - name: Install Python dependencies
      run: |
        python3 -m venv venv
        source venv/bin/activate
        pip install -r requirements.txt
        
    - name: Install particle-os
      run: |
        source venv/bin/activate
        make install
        
    - name: Run test suite
      run: |
        source venv/bin/activate
        make test
        
  build:
    needs: test
    runs-on: ubuntu-latest
    steps:
    - name: Checkout
      uses: actions/checkout@v4
      
    - name: Setup Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.11'
        
    - name: Install dependencies
      run: |
        sudo apt update
        sudo apt install -y python3-pip python3-venv
        python3 -m venv venv
        source venv/bin/activate
        pip install -r requirements.txt
        make install
        
    - name: Build Debian OSTree image
      run: |
        source venv/bin/activate
        osbuild examples/debian-ostree-bootable.json
        
    - name: Upload build artifacts
      uses: actions/upload-artifact@v3
      with:
        name: debian-ostree-images
        path: |
          *.qcow2
          *.raw
          *.vmdk

🔧 Advanced Usage

Custom Stage Configuration

Debian Sources Stage

{
  "name": "org.osbuild.debian.sources",
  "options": {
    "suite": "trixie",
    "mirror": "https://deb.debian.org/debian",
    "components": ["main", "contrib", "non-free"],
    "additional_sources": [
      "deb https://deb.debian.org/debian-security trixie-security main contrib non-free"
    ]
  }
}

User Management Stage

{
  "name": "org.osbuild.debian.users",
  "options": {
    "users": {
      "debian": {
        "password": "$6$rounds=656000$salt$hashedpassword",
        "shell": "/bin/bash",
        "groups": ["sudo", "users", "adm"],
        "uid": 1000,
        "gid": 1000,
        "home": "/home/debian",
        "comment": "Debian User"
      }
    },
    "default_shell": "/bin/bash",
    "default_home": "/home"
  }
}

GRUB2 Bootloader Stage

{
  "name": "org.osbuild.debian.grub2",
  "options": {
    "root_fs_uuid": "ROOT_UUID",
    "kernel_path": "/boot/vmlinuz",
    "initrd_path": "/boot/initrd.img",
    "bootloader_id": "debian",
    "timeout": 5,
    "default_entry": "0"
  }
}

QEMU Assembler Options

{
  "name": "org.osbuild.debian.qemu",
  "options": {
    "format": "qcow2",
    "filename": "debian-ostree.qcow2",
    "size": "20G",
    "ptuuid": "12345678-1234-1234-1234-123456789012"
  }
}

Supported Formats:

• raw - Raw disk image
• qcow2 - QEMU Copy On Write v2 (recommended)
• vmdk - VMware Virtual Machine Disk
• vdi - VirtualBox Virtual Disk Image

🧪 Testing and Development

Run Tests

# Run all tests
make test

# Run specific test files
pytest tests/test_ostree_stages.py
pytest tests/test_grub2_stage.py

# Run with coverage
pytest --cov=src tests/

Development Setup

# Set up development environment
make dev-setup

# Install in development mode
make install-dev

# Run linting
make lint

# Format code
make format

Demo Scripts

# Run complete bootable OSTree pipeline demonstration
./scripts/demo-bootable-ostree.py

# Test individual stages
./scripts/test-stages.py

📚 Examples

1. Basic Debian System (examples/debian-basic.json)

Creates a minimal Debian system with basic packages and user accounts.

2. OSTree System (examples/debian-ostree.json)

Builds a Debian system with OSTree repository management.

3. Complete System (examples/debian-complete.json)

Comprehensive Debian system with all basic stages.

4. Bootable OSTree System (examples/debian-ostree-bootable.json)

Complete bootable Debian OSTree system with GRUB2 and bootc.

🏗️ Architecture Deep Dive

Stage Implementation Pattern

Each stage follows a consistent pattern:

#!/usr/bin/python3
import os
import sys
import osbuild.api

def main(tree, options):
    """Stage-specific logic"""
    # Process options
    # Manipulate filesystem tree
    return 0

if __name__ == '__main__':
    args = osbuild.api.arguments()
    ret = main(args["tree"], args["options"])
    sys.exit(ret)

Build Process Flow

1. Manifest Loading → Parse and validate JSON manifest
2. Pipeline Construction → Build stage dependency graph
3. Source Resolution → Download and prepare input sources
4. Stage Execution → Run stages in dependency order
5. Assembly → Create final artifacts from stage outputs
6. Output → Export requested objects

Security and Isolation

• Process isolation using bubblewrap and systemd-nspawn
• Capability management with Linux capabilities
• Resource limits to prevent resource exhaustion
• Input validation for all external inputs
• Output sanitization for safe output generation

🔍 Troubleshooting

Common Issues

Stage Execution Failures

# Check stage logs
osbuild --monitor json examples/debian-basic.json

# Run with debug output
osbuild --debug examples/debian-basic.json

Permission Issues

# Ensure proper capabilities
sudo setcap cap_sys_admin,cap_sys_chroot,cap_dac_override+ep /usr/bin/osbuild

Package Installation Failures

# Check APT sources configuration
# Verify network connectivity
# Check package availability in repository

Debug Mode

# Enable debug output
export OSBUILD_DEBUG=1
osbuild examples/debian-basic.json

🌟 Key Features

• Declarative Manifests: JSON-based configuration with schema validation
• Stage-based Architecture: Atomic, composable building blocks
• OSTree Integration: Native OSTree support for atomic updates
• Bootc Support: Modern container-native bootloader interface
• GRUB2 Integration: Traditional bootloader with UEFI support
• Multi-format Output: Support for various image formats
• Security Focus: Process isolation and capability management
• Performance: Intelligent caching and parallel execution support

🎯 Use Cases

1. Distribution Building: Creating official Debian-based images
2. Custom Images: Building specialized Debian OSTree systems
3. CI/CD Pipelines: Automated image building and testing
4. Development: Testing and development environments
5. Production Deployment: Creating production-ready images
6. Education: Learning about OS image building and OSTree

🔮 Future Vision

particle-os aims to become the premier platform for building Debian-based OSTree systems, providing:

• Enterprise-grade reliability and performance
• Comprehensive tooling for all aspects of OS image building
• Active community of contributors and users
• Industry adoption in production environments
• Educational value for understanding modern OS architecture

🤝 Contributing

We welcome contributions! Please see our Development Guide for details on:

• Setting up the development environment
• Running tests
• Submitting pull requests
• Code style guidelines
• Architecture decisions

📄 License

This project is open source. See the LICENSE file for details.

🙏 Acknowledgments

• Red Hat for the original osbuild system
• Universal Blue for the inspiration and vision
• Debian Project for the excellent base system
• OSTree for the atomic update system
• bootc for the container-native bootloader

---

particle-os: Building the future of Debian, one image at a time. 🚀