deb-bootc-image-builder/docs/ci-cd-integration.md

# CI/CD Integration Guide for particle-os

particle-os is designed to be a first-class CI/CD tool for OS image building. This guide shows you how to integrate particle-os into any CI/CD system.

## 🚀 Why particle-os is CI/CD Friendly

### **Built-in CI/CD Features:**
- ✅ **Non-interactive mode** - No prompts, pure automation
- ✅ **JSON output** - Machine-readable build results  
- ✅ **Quiet mode** - Suppressed non-essential output
- ✅ **Proper exit codes** - Standard status codes for CI systems
- ✅ **Structured output** - Predictable, parseable results
- ✅ **Artifact management** - Organized output files

### **CI/CD Usage Pattern:**
```bash
# Basic CI/CD build
particle-os build --json --quiet recipes/debian-server.yml

# With custom output and cleanup
particle-os build --json --quiet --output /artifacts/server.img --clean recipes/debian-server.yml
```

## 🔧 GitHub Actions Integration

### **Basic OS Image Build Workflow:**

```yaml
# .github/workflows/build-os.yml
name: Build OS Images

on:
  push:
    branches: [main]
    paths: ['recipes/**', 'src/**']
  pull_request:
    branches: [main]

jobs:
  build-images:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        recipe: [debian-server, debian-desktop, debian-gaming]
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        
      - name: Setup Go
        uses: actions/setup-go@v4
        with:
          go-version: '1.21'
          
      - name: Build particle-os
        run: |
          cd bib
          go build -o particle-os cmd/particle_os/main.go
          
      - name: Build OS Image
        run: |
          sudo ./bib/particle-os build --json --quiet --clean recipes/${{ matrix.recipe }}.yml
          
      - name: Parse build results
        id: build-results
        run: |
          BUILD_OUTPUT=$(sudo ./bib/particle-os build --json --quiet --clean recipes/${{ matrix.recipe }}.yml)
          echo "build_output<<EOF" >> $GITHUB_OUTPUT
          echo "$BUILD_OUTPUT" >> $GITHUB_OUTPUT
          echo "EOF" >> $GITHUB_OUTPUT
          
          # Extract success status
          SUCCESS=$(echo "$BUILD_OUTPUT" | jq -r '.success')
          echo "success=$SUCCESS" >> $GITHUB_OUTPUT
          
          # Extract image path
          IMAGE_PATH=$(echo "$BUILD_OUTPUT" | jq -r '.image_path')
          echo "image_path=$IMAGE_PATH" >> $GITHUB_OUTPUT
          
      - name: Upload artifacts
        if: steps.build-results.outputs.success == 'true'
        uses: actions/upload-artifact@v3
        with:
          name: ${{ matrix.recipe }}-images
          path: |
            ${{ steps.build-results.outputs.image_path }}
            /tmp/particle-os-build/output/*.raw
            /tmp/particle-os-build/output/*.qcow2
            /tmp/particle-os-build/output/*.vmdk
          retention-days: 30
          
      - name: Build summary
        run: |
          echo "## Build Results for ${{ matrix.recipe }}" >> $GITHUB_STEP_SUMMARY
          echo "- **Status**: ${{ steps.build-results.outputs.success }}" >> $GITHUB_STEP_SUMMARY
          echo "- **Image**: ${{ steps.build-results.outputs.image_path }}" >> $GITHUB_STEP_SUMMARY
          echo "- **Recipe**: ${{ matrix.recipe }}" >> $GITHUB_STEP_SUMMARY
```

### **Advanced Multi-Stage Workflow:**

```yaml
# .github/workflows/advanced-os-build.yml
name: Advanced OS Image Pipeline

on:
  push:
    tags: ['v*']

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Validate recipes
        run: |
          cd bib && go build -o particle-os cmd/particle_os/main.go
          cd ..
          for recipe in recipes/*.yml; do
            echo "Validating $recipe..."
            ./bib/particle-os validate "$recipe"
          done
          
  build-base:
    needs: validate
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build base images
        run: |
          cd bib && go build -o particle-os cmd/particle_os/main.go
          cd ..
          sudo ./bib/particle-os build --json --quiet --clean recipes/debian-minimal.yml
          
  build-variants:
    needs: [validate, build-base]
    runs-on: ubuntu-latest
    strategy:
      matrix:
        variant: [server, desktop, gaming]
    steps:
      - uses: actions/checkout@v4
      - name: Build variant
        run: |
          cd bib && go build -o particle-os cmd/particle_os/main.go
          cd ..
          sudo ./bib/particle-os build --json --quiet --clean recipes/debian-${{ matrix.variant }}.yml
          
  test-images:
    needs: build-variants
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Download artifacts
        uses: actions/download-artifact@v3
        with:
          pattern: debian-*-images
          
      - name: Test image boot
        run: |
          # Basic image validation
          for img in *.qcow2; do
            echo "Testing $img..."
            qemu-img info "$img"
            # Add more validation as needed
          done
          
  release:
    needs: test-images
    runs-on: ubuntu-latest
    if: startsWith(github.ref, 'refs/tags/')
    steps:
      - uses: actions/checkout@v4
      - name: Download artifacts
        uses: actions/download-artifact@v3
        with:
          pattern: debian-*-images
          
      - name: Create release
        uses: softprops/action-gh-release@v1
        with:
          files: |
            debian-*-images/*.qcow2
            debian-*-images/*.raw
            debian-*-images/*.vmdk
          body: |
            ## OS Images Built with particle-os
            
            This release contains Debian-based OS images built using particle-os.
            
            ### Available Images:
            - **debian-server**: Server-optimized Debian
            - **debian-desktop**: Desktop Debian with GNOME
            - **debian-gaming**: Gaming-optimized Debian
            
            ### Build Details:
            - Built with particle-os v${{ github.ref_name }}
            - Base: Debian Trixie
            - Formats: raw, qcow2, vmdk
```

## 🔧 GitLab CI Integration

### **Basic GitLab CI Pipeline:**

```yaml
# .gitlab-ci.yml
stages:
  - validate
  - build
  - test
  - deploy

variables:
  WORK_DIR: "/tmp/particle-os-build"

validate-recipes:
  stage: validate
  script:
    - cd bib && go build -o particle-os cmd/particle_os/main.go
    - cd ..
    - for recipe in recipes/*.yml; do
        echo "Validating $recipe..."
        ./bib/particle-os validate "$recipe"
      done
  artifacts:
    paths:
      - bib/particle-os
    expire_in: 1 hour

build-server:
  stage: build
  script:
    - sudo ./bib/particle-os build --json --quiet --clean recipes/debian-server.yml
  artifacts:
    paths:
      - /tmp/particle-os-build/output/
    expire_in: 1 week
  dependencies:
    - validate-recipes

build-desktop:
  stage: build
  script:
    - sudo ./bib/particle-os build --json --quiet --clean recipes/debian-desktop.yml
  artifacts:
    paths:
      - /tmp/particle-os-build/output/
    expire_in: 1 week
  dependencies:
    - validate-recipes

test-images:
  stage: test
  script:
    - echo "Testing built images..."
    - for img in /tmp/particle-os-build/output/*.qcow2; do
        echo "Validating $img..."
        qemu-img info "$img"
      done
  dependencies:
    - build-server
    - build-desktop

deploy-images:
  stage: deploy
  script:
    - echo "Deploying images to artifact storage..."
    - # Add your deployment logic here
  dependencies:
    - test-images
  only:
    - main
```

## 🔧 Jenkins Integration

### **Jenkins Pipeline Script:**

```groovy
// Jenkinsfile
pipeline {
    agent any
    
    environment {
        WORKSPACE_DIR = '/tmp/particle-os-build'
        RECIPES_DIR = 'recipes'
    }
    
    stages {
        stage('Setup') {
            steps {
                sh 'cd bib && go build -o particle-os cmd/particle_os/main.go'
                sh 'cd ..'
            }
        }
        
        stage('Validate Recipes') {
            steps {
                script {
                    def recipes = sh(
                        script: "find ${RECIPES_DIR} -name '*.yml' -type f",
                        returnStdout: true
                    ).trim().split('\n')
                    
                    for (recipe in recipes) {
                        echo "Validating ${recipe}..."
                        sh "./bib/particle-os validate ${recipe}"
                    }
                }
            }
        }
        
        stage('Build Images') {
            parallel {
                stage('Build Server') {
                    steps {
                        sh '''
                            sudo ./bib/particle-os build --json --quiet --clean recipes/debian-server.yml
                            # Parse JSON output for Jenkins
                            BUILD_OUTPUT=$(sudo ./bib/particle-os build --json --quiet --clean recipes/debian-server.yml)
                            echo "BUILD_OUTPUT=${BUILD_OUTPUT}" > build.env
                        '''
                        archiveArtifacts artifacts: 'build.env'
                    }
                }
                
                stage('Build Desktop') {
                    steps {
                        sh '''
                            sudo ./bib/particle-os build --json --quiet --clean recipes/debian-desktop.yml
                            BUILD_OUTPUT=$(sudo ./bib/particle-os build --json --quiet --clean recipes/debian-desktop.yml)
                            echo "BUILD_OUTPUT=${BUILD_OUTPUT}" > build.env
                        '''
                        archiveArtifacts artifacts: 'build.env'
                    }
                }
            }
        }
        
        stage('Test Images') {
            steps {
                sh '''
                    echo "Testing built images..."
                    for img in /tmp/particle-os-build/output/*.qcow2; do
                        echo "Validating $img..."
                        qemu-img info "$img"
                    done
                '''
            }
        }
        
        stage('Archive Artifacts') {
            steps {
                archiveArtifacts artifacts: '/tmp/particle-os-build/output/**/*'
            }
        }
    }
    
    post {
        always {
            cleanWs()
        }
        success {
            echo "All OS images built successfully!"
        }
        failure {
            echo "OS image build failed!"
        }
    }
}
```

## 🔧 Generic CI/CD Integration

### **Shell Script for Any CI/CD System:**

```bash
#!/bin/bash
# build-os-images.sh - Generic CI/CD script for particle-os

set -euo pipefail

# Configuration
RECIPE_DIR="recipes"
OUTPUT_DIR="/artifacts"
BUILD_LOG="/tmp/build.log"

# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color

log() {
    echo -e "${GREEN}[$(date +'%Y-%m-%d %H:%M:%S')] $1${NC}"
}

warn() {
    echo -e "${YELLOW}[$(date +'%Y-%m %H:%M:%S')] WARNING: $1${NC}"
}

error() {
    echo -e "${RED}[$(date +'%Y-%m-%d %H:%M:%S')] ERROR: $1${NC}"
    exit 1
}

# Build particle-os binary
build_particle_os() {
    log "Building particle-os binary..."
    cd bib
    if ! go build -o particle-os cmd/particle_os/main.go; then
        error "Failed to build particle-os binary"
    fi
    cd ..
    log "particle-os binary built successfully"
}

# Validate all recipes
validate_recipes() {
    log "Validating all recipes..."
    local failed_recipes=()
    
    for recipe in "$RECIPE_DIR"/*.yml; do
        if [[ -f "$recipe" ]]; then
            log "Validating $recipe..."
            if ! ./bib/particle-os validate "$recipe"; then
                failed_recipes+=("$recipe")
            fi
        fi
    done
    
    if [[ ${#failed_recipes[@]} -gt 0 ]]; then
        error "Recipe validation failed for: ${failed_recipes[*]}"
    fi
    
    log "All recipes validated successfully"
}

# Build OS image from recipe
build_image() {
    local recipe="$1"
    local recipe_name=$(basename "$recipe" .yml)
    
    log "Building image from recipe: $recipe_name"
    
    # Build with CI/CD flags
    local build_output
    if ! build_output=$(sudo ./bib/particle-os build --json --quiet --clean "$recipe" 2>&1); then
        error "Build failed for $recipe_name"
    fi
    
    # Parse JSON output
    local success
    local image_path
    local exit_code
    
    success=$(echo "$build_output" | jq -r '.success // false')
    image_path=$(echo "$build_output" | jq -r '.image_path // ""')
    exit_code=$(echo "$build_output" | jq -r '.exit_code // 1')
    
    if [[ "$success" != "true" ]] || [[ "$exit_code" != "0" ]]; then
        error "Build failed for $recipe_name (exit code: $exit_code)"
    fi
    
    if [[ -z "$image_path" ]]; then
        error "No image path returned for $recipe_name"
    fi
    
    log "Build successful for $recipe_name: $image_path"
    
    # Copy to artifacts directory
    if [[ -n "$OUTPUT_DIR" ]]; then
        mkdir -p "$OUTPUT_DIR/$recipe_name"
        cp "$image_path" "$OUTPUT_DIR/$recipe_name/"
        
        # Copy all output formats
        local work_dir
        work_dir=$(echo "$build_output" | jq -r '.work_directory // ""')
        if [[ -n "$work_dir" ]] && [[ -d "$work_dir/output" ]]; then
            cp "$work_dir/output"/* "$OUTPUT_DIR/$recipe_name/" 2>/dev/null || true
        fi
        
        log "Artifacts copied to $OUTPUT_DIR/$recipe_name"
    fi
    
    # Output build info for CI/CD system
    echo "BUILD_SUCCESS_$recipe_name=true" >> "$GITHUB_ENV" 2>/dev/null || true
    echo "IMAGE_PATH_$recipe_name=$image_path" >> "$GITHUB_ENV" 2>/dev/null || true
}

# Main build process
main() {
    log "Starting OS image build process..."
    
    # Check prerequisites
    if ! command -v go &> /dev/null; then
        error "Go is not installed"
    fi
    
    if ! command -v jq &> /dev/null; then
        error "jq is not installed"
    fi
    
    # Build particle-os
    build_particle_os
    
    # Validate recipes
    validate_recipes
    
    # Build all images
    local build_count=0
    local success_count=0
    
    for recipe in "$RECIPE_DIR"/*.yml; do
        if [[ -f "$recipe" ]]; then
            build_count=$((build_count + 1))
            if build_image "$recipe"; then
                success_count=$((success_count + 1))
            fi
        fi
    done
    
    # Summary
    log "Build process completed: $success_count/$build_count images built successfully"
    
    if [[ $success_count -eq $build_count ]]; then
        log "All images built successfully!"
        exit 0
    else
        error "Some images failed to build"
    fi
}

# Run main function
main "$@"
```

## 🔧 Docker Integration

### **Dockerfile for CI/CD:**

```dockerfile
# Dockerfile.ci
FROM debian:bookworm-slim

# Install dependencies
RUN apt-get update && apt-get install -y \
    golang-go \
    jq \
    qemu-utils \
    podman \
    && rm -rf /var/lib/apt/lists/*

# Set working directory
WORKDIR /app

# Copy source code
COPY . .

# Build particle-os
RUN cd bib && go build -o particle-os cmd/particle_os/main.go

# Create recipes directory
RUN mkdir -p /recipes

# Set entrypoint
ENTRYPOINT ["./bib/particle-os"]

# Default command
CMD ["--help"]
```

### **Docker Compose for CI/CD:**

```yaml
# docker-compose.ci.yml
version: '3.8'

services:
  particle-os-builder:
    build:
      context: .
      dockerfile: Dockerfile.ci
    volumes:
      - ./recipes:/recipes
      - ./artifacts:/artifacts
      - /var/lib/containers:/var/lib/containers
    environment:
      - RECIPES_DIR=/recipes
      - OUTPUT_DIR=/artifacts
    command: ["build", "--json", "--quiet", "--clean", "/recipes/debian-server.yml"]
    privileged: true  # Required for container operations
```

## 🔧 Best Practices for CI/CD

### **1. Always Use CI/CD Flags:**
```bash
# ✅ Good - CI/CD friendly
particle-os build --json --quiet --clean recipes/debian-server.yml

# ❌ Bad - Human-friendly only
particle-os build recipes/debian-server.yml
```

### **2. Parse JSON Output:**
```bash
# Extract build information
BUILD_OUTPUT=$(particle-os build --json --quiet recipes/debian-server.yml)
SUCCESS=$(echo "$BUILD_OUTPUT" | jq -r '.success')
IMAGE_PATH=$(echo "$BUILD_OUTPUT" | jq -r '.image_path')
EXIT_CODE=$(echo "$BUILD_OUTPUT" | jq -r '.exit_code')

if [[ "$SUCCESS" == "true" ]] && [[ "$EXIT_CODE" == "0" ]]; then
    echo "Build successful: $IMAGE_PATH"
else
    echo "Build failed"
    exit 1
fi
```

### **3. Handle Artifacts Properly:**
```bash
# Create organized artifact structure
mkdir -p artifacts/$RECIPE_NAME
cp "$IMAGE_PATH" artifacts/$RECIPE_NAME/
cp /tmp/particle-os-build/output/* artifacts/$RECIPE_NAME/
```

### **4. Clean Up After Builds:**
```bash
# Always use --clean flag in CI/CD
particle-os build --json --quiet --clean recipes/debian-server.yml

# Or manually clean up
sudo rm -rf /tmp/particle-os-build
```

### **5. Set Proper Exit Codes:**
```bash
# Check exit code from particle-os
if particle-os build --json --quiet recipes/debian-server.yml; then
    echo "Build successful"
    exit 0
else
    echo "Build failed"
    exit 1
fi
```

## 🚀 Next Steps

Now that you have CI/CD integration examples, you can:

1. **Choose your CI/CD system** and implement the appropriate example
2. **Customize the workflows** for your specific needs
3. **Add more validation** and testing steps
4. **Implement artifact management** for your deployment pipeline
5. **Add monitoring** and alerting for build failures

particle-os is designed to work seamlessly in any CI/CD environment, providing reliable, reproducible OS image builds with machine-readable output perfect for automation.