deb-bootc-image-builder/docs/validation-guide.md

Validation Guide

Overview

This document consolidates all validation information for the Debian bootc-image-builder project. It covers testing strategies, validation procedures, quality assurance, and best practices for ensuring the reliability and correctness of the system.

Testing Strategy

Testing Pyramid

1. Unit Tests (Foundation)

   • Individual component testing
   • Function and method validation
   • Mock and stub usage
   • Fast execution (< 1 second per test)

2. Integration Tests (Middle)

   • Component interaction testing
   • API endpoint validation
   • Database and external service integration
   • Moderate execution time (1-10 seconds per test)

3. End-to-End Tests (Top)

   • Complete workflow validation
   • Real environment testing
   • Image building and deployment
   • Longer execution time (10+ seconds per test)

Test Coverage Goals

• Unit Tests: 90%+ code coverage
• Integration Tests: 80%+ API coverage
• End-to-End Tests: Critical path coverage
• Overall Coverage: 85%+ combined coverage

Unit Testing

Go Testing Framework

Test Structure

package debos

import (
    "testing"
    "github.com/stretchr/testify/assert"
)

func TestFunctionName(t *testing.T) {
    // Arrange
    input := "test input"
    expected := "expected output"
    
    // Act
    result := FunctionName(input)
    
    // Assert
    assert.Equal(t, expected, result)
}

Test Naming Convention

• TestFunctionName: Test for specific function
• TestFunctionName_Scenario: Test for specific scenario
• TestFunctionName_Error: Test for error conditions

Mock and Stub Usage

// Mock interface
type MockDebosRunner struct {
    ExecuteFunc func(template *DebosTemplate, outputDir string) (*DebosResult, error)
}

func (m *MockDebosRunner) Execute(template *DebosTemplate, outputDir string) (*DebosResult, error) {
    if m.ExecuteFunc != nil {
        return m.ExecuteFunc(template, outputDir)
    }
    return nil, nil
}

// Test with mock
func TestBuilderWithMock(t *testing.T) {
    mockRunner := &MockDebosRunner{
        ExecuteFunc: func(template *DebosTemplate, outputDir string) (*DebosResult, error) {
            return &DebosResult{Success: true}, nil
        },
    }
    
    // Test implementation
}

Test Categories

1. Functionality Tests

• Input Validation: Test with valid and invalid inputs
• Edge Cases: Test boundary conditions
• Error Handling: Test error scenarios and recovery
• Success Paths: Test normal operation flows

2. Performance Tests

• Memory Usage: Test memory allocation and cleanup
• Execution Time: Test function performance
• Resource Usage: Test CPU and I/O usage
• Scalability: Test with different input sizes

3. Security Tests

• Input Sanitization: Test for injection attacks
• Access Control: Test permission validation
• Data Validation: Test for malicious input
• Secure Defaults: Test default security settings

Integration Testing

API Testing

HTTP Endpoint Testing

func TestAPIIntegration(t *testing.T) {
    // Setup test server
    server := httptest.NewServer(http.HandlerFunc(handleRequest))
    defer server.Close()
    
    // Test API calls
    resp, err := http.Get(server.URL + "/api/test")
    assert.NoError(t, err)
    assert.Equal(t, http.StatusOK, resp.StatusCode)
}

Database Integration Testing

func TestDatabaseIntegration(t *testing.T) {
    // Setup test database
    db := setupTestDB(t)
    defer cleanupTestDB(t, db)
    
    // Test database operations
    result, err := db.Query("SELECT * FROM test_table")
    assert.NoError(t, err)
    // Validate results
}

External Service Testing

Container Runtime Testing

func TestContainerIntegration(t *testing.T) {
    // Test with real container runtime
    if testing.Short() {
        t.Skip("Skipping container integration test in short mode")
    }
    
    // Test container operations
    container, err := createTestContainer()
    assert.NoError(t, err)
    defer cleanupContainer(container)
    
    // Validate container functionality
}

Package Repository Testing

func TestPackageRepository(t *testing.T) {
    // Test with real Debian repositories
    if testing.Short() {
        t.Skip("Skipping repository test in short mode")
    }
    
    // Test package resolution
    packages, err := resolvePackages([]string{"systemd", "bash"})
    assert.NoError(t, err)
    assert.NotEmpty(t, packages)
}

End-to-End Testing

Image Building Validation

Complete Build Pipeline

#!/bin/bash
# End-to-end build test script

set -e

echo "Starting end-to-end build test..."

# 1. Setup test environment
echo "Setting up test environment..."
mkdir -p test-output
cd test-output

# 2. Generate manifest
echo "Generating manifest..."
bootc-image-builder manifest --config /dev/null debian:trixie > manifest.json

# 3. Build image
echo "Building image..."
bootc-image-builder build --type qcow2 --output . debian:trixie

# 4. Validate output
echo "Validating output..."
if [ -f "*.qcow2" ]; then
    echo "✅ Image file created successfully"
    
    # Check file size
    file_size=$(stat -c%s *.qcow2)
    if [ $file_size -gt 1000000 ]; then
        echo "✅ Image file size is reasonable: $file_size bytes"
    else
        echo "❌ Image file size is too small: $file_size bytes"
        exit 1
    fi
else
    echo "❌ Image file not found"
    exit 1
fi

echo "🎉 End-to-end test completed successfully!"

Image Validation

#!/bin/bash
# Image validation script

set -e

IMAGE_FILE="$1"
if [ -z "$IMAGE_FILE" ]; then
    echo "Usage: $0 <image-file>"
    exit 1
fi

echo "Validating image: $IMAGE_FILE"

# Check file exists and is readable
if [ ! -r "$IMAGE_FILE" ]; then
    echo "❌ Image file not readable: $IMAGE_FILE"
    exit 1
fi

# Check file size
file_size=$(stat -c%s "$IMAGE_FILE")
echo "📏 Image file size: $file_size bytes"

# Validate image format
if command -v qemu-img >/dev/null 2>&1; then
    echo "🔍 Validating image format with qemu-img..."
    qemu-img info "$IMAGE_FILE"
    
    # Check if image is valid
    if qemu-img check "$IMAGE_FILE" 2>/dev/null; then
        echo "✅ Image format validation passed"
    else
        echo "❌ Image format validation failed"
        exit 1
    fi
else
    echo "⚠️  qemu-img not available, skipping format validation"
fi

# Test image mounting (if possible)
echo "🔧 Testing image mounting..."
if command -v guestmount >/dev/null 2>&1; then
    mount_point=$(mktemp -d)
    if guestmount -a "$IMAGE_FILE" -i "$mount_point" 2>/dev/null; then
        echo "✅ Image mounting successful"
        
        # Check basic filesystem structure
        if [ -d "$mount_point/etc" ] && [ -d "$mount_point/boot" ]; then
            echo "✅ Basic filesystem structure validated"
        else
            echo "❌ Basic filesystem structure missing"
            exit 1
        fi
        
        guestunmount "$mount_point"
        rmdir "$mount_point"
    else
        echo "❌ Image mounting failed"
        exit 1
    fi
else
    echo "⚠️  guestmount not available, skipping mount test"
fi

echo "🎉 Image validation completed successfully!"

OSTree Validation

Repository Validation

#!/bin/bash
# OSTree repository validation

set -e

REPO_PATH="$1"
if [ -z "$REPO_PATH" ]; then
    echo "Usage: $0 <ostree-repo-path>"
    exit 1
fi

echo "Validating OSTree repository: $REPO_PATH"

# Check if repository exists
if [ ! -d "$REPO_PATH" ]; then
    echo "❌ Repository directory not found: $REPO_PATH"
    exit 1
fi

# Check repository structure
if [ -d "$REPO_PATH/refs" ] && [ -d "$REPO_PATH/objects" ]; then
    echo "✅ Repository structure validated"
else
    echo "❌ Invalid repository structure"
    exit 1
fi

# List available branches
echo "📋 Available branches:"
ostree refs --repo="$REPO_PATH" || {
    echo "❌ Failed to list repository references"
    exit 1
}

# Check repository integrity
echo "🔍 Checking repository integrity..."
ostree fsck --repo="$REPO_PATH" || {
    echo "❌ Repository integrity check failed"
    exit 1
}

echo "🎉 OSTree repository validation completed successfully!"

Quality Assurance

Code Quality

Linting and Formatting

# Go formatting
go fmt ./...

# Go linting
golangci-lint run

# Code complexity analysis
gocyclo -over 15 .

# Security scanning
gosec ./...

Static Analysis

# Go vet
go vet ./...

# Staticcheck
staticcheck ./...

# Ineffassign
ineffassign ./...

Performance Testing

Benchmark Tests

func BenchmarkFunctionName(b *testing.B) {
    for i := 0; i < b.N; i++ {
        FunctionName("test input")
    }
}

Memory Profiling

# Run with memory profiling
go test -memprofile=mem.prof ./...

# Analyze memory profile
go tool pprof mem.prof

CPU Profiling

# Run with CPU profiling
go test -cpuprofile=cpu.prof ./...

# Analyze CPU profile
go tool pprof cpu.prof

Continuous Integration

CI/CD Pipeline

GitHub Actions Example

name: Test and Validate

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v3
    
    - name: Set up Go
      uses: actions/setup-go@v3
      with:
        go-version: '1.21'
    
    - name: Install dependencies
      run: go mod download
    
    - name: Run unit tests
      run: go test -v -race -coverprofile=coverage.txt ./...
    
    - name: Run integration tests
      run: go test -v -tags=integration ./...
    
    - name: Upload coverage
      uses: codecov/codecov-action@v3
      with:
        file: ./coverage.txt

Pre-commit Hooks

#!/bin/bash
# Pre-commit validation script

set -e

echo "Running pre-commit validation..."

# Format code
go fmt ./...

# Run tests
go test -v ./...

# Run linting
golangci-lint run

# Check for security issues
gosec ./...

echo "✅ Pre-commit validation passed"

Validation Checklist

Before Release

• All unit tests pass
• All integration tests pass
• End-to-end tests pass
• Code coverage meets targets
• Security scan passes
• Performance benchmarks meet targets
• Documentation is complete and accurate
• Changelog is updated
• Version numbers are correct

Before Deployment

• Staging environment tests pass
• Load testing completed
• Security testing completed
• Backup and recovery tested
• Monitoring and alerting configured
• Rollback plan tested

Troubleshooting

Common Test Issues

Test Environment Problems

# Check Go environment
go env

# Verify dependencies
go mod verify

# Clean test cache
go clean -testcache

# Check for race conditions
go test -race ./...

Integration Test Failures

# Check external services
systemctl status docker
systemctl status containerd

# Verify network connectivity
ping -c 1 deb.debian.org

# Check disk space
df -h

Performance Test Issues

# Check system resources
htop
iostat
iotop

# Monitor system during tests
watch -n 1 'ps aux | grep test'

Resources

Testing Tools

• Go Testing: Built-in testing framework
• Testify: Assertion and mocking library
• GolangCI-Lint: Comprehensive linting
• Gosec: Security scanning
• pprof: Performance profiling

Documentation

• Go Testing Documentation
• Testify Documentation
• GolangCI-Lint Documentation

---

Status: Active Development
Last Updated: August 2025
Maintainer: Debian Bootc Image Builder Team