deb-bootc-image-builder/docs/TESTING_STRATEGY.md

Testing Strategy - deb-bootc-image-builder

Date: August 17, 2025
Status: 🚧 Ready for Testing - Binary Needs Recompilation

---

🎯 Testing Overview

This document outlines the comprehensive testing strategy for deb-bootc-image-builder once the binary is recompiled with the sudo fixes. The goal is to validate that all critical functionality works correctly and identify any remaining issues.

---

🧪 Testing Phases

Phase 1: Basic Functionality Testing 🔥 HIGHEST PRIORITY

Goal: Verify that the core infrastructure works correctly

1.1 Container Operations

# Test container listing
./bib/particle-os container list

# Test container inspection
./bib/particle-os container inspect debian:trixie-slim

# Test container extraction (without full build)
./bib/particle-os build --work-dir /tmp/test-container recipes/minimal-debug.yml --verbose

Expected Results:

• ✅ Container listing works
• ✅ Container inspection provides correct metadata
• ✅ Container extraction completes successfully
• ✅ Rootfs is created with correct ownership

1.2 Recipe System

# Test recipe listing
./bib/particle-os list

# Test recipe validation
./bib/particle-os validate recipes/minimal-debug.yml
./bib/particle-os validate recipes/simple-cli-bootable.yml

# Test recipe parsing
./bib/particle-os build --work-dir /tmp/test-recipe recipes/minimal-debug.yml --verbose

Expected Results:

• ✅ Recipe listing shows all available recipes
• ✅ Recipe validation passes for valid recipes
• ✅ Recipe parsing works correctly
• ✅ Build starts successfully

Phase 2: Stage Execution Testing 🔥 HIGHEST PRIORITY

Goal: Verify that all stages execute correctly with the sudo fixes

2.1 Package Management Stage

# Test apt stage with minimal recipe
./bib/particle-os build --work-dir /tmp/test-apt recipes/minimal-debug.yml --verbose

Expected Results:

• ✅ Container extraction completes
• ✅ Package installation works (locales, tzdata, bash, coreutils)
• ✅ Package cache cleanup works
• ✅ Stage completes successfully

2.2 Locale Stage

# Test locale stage with minimal-debug-locale.yml
./bib/particle-os build --work-dir /tmp/test-locale recipes/minimal-debug-locale.yml --verbose

Expected Results:

• ✅ Locale stage executes without permission errors
• ✅ /etc/locale.gen is written correctly
• ✅ locale-gen command runs successfully
• ✅ Default locale is set correctly
• ✅ Stage completes successfully

2.3 Timezone Stage

# Test timezone stage with a recipe that includes it
./bib/particle-os build --work-dir /tmp/test-timezone recipes/simple-cli-bootable.yml --verbose

Expected Results:

• ✅ Timezone stage executes without permission errors
• ✅ /etc/timezone is written correctly
• ✅ /etc/localtime symlink is created correctly
• ✅ Stage completes successfully

2.4 Users Stage

# Test users stage with a recipe that includes it
./bib/particle-os build --work-dir /tmp/test-users recipes/simple-cli-bootable.yml --verbose

Expected Results:

• ✅ Users stage executes without permission errors
• ✅ User accounts are created correctly
• ✅ Passwords are set correctly
• ✅ Groups are assigned correctly
• ✅ Stage completes successfully

Phase 3: End-to-End Workflow Testing 🔥 HIGH PRIORITY

Goal: Verify that complete recipes execute successfully

3.1 Minimal Debug Recipe

# Test complete minimal recipe
./bib/particle-os build --work-dir /tmp/test-minimal recipes/minimal-debug.yml --verbose

Expected Results:

• ✅ All stages execute successfully
• ✅ Final image creation is reached
• ✅ Bootable image is created
• ✅ Build completes with success

3.2 Simple CLI Bootable Recipe

# Test complete CLI recipe
./bib/particle-os build --work-dir /tmp/test-cli recipes/simple-cli-bootable.yml --verbose

Expected Results:

• ✅ All stages execute successfully
• ✅ Package installation completes
• ✅ System configuration applies correctly
• ✅ Final image creation is reached
• ✅ Bootable image is created
• ✅ Build completes with success

Phase 4: Image Creation Testing 📋 MEDIUM PRIORITY

Goal: Verify that all image formats and creation methods work

4.1 QEMU Stage Testing

# Test QEMU stage with different formats
./bib/particle-os build --work-dir /tmp/test-qemu recipes/qemu-test.yml --verbose

Expected Results:

• ✅ Raw image creation works
• ✅ QCOW2 image creation works (if qemu-img available)
• ✅ VMDK image creation works (if qemu-img available)
• ✅ VDI image creation works (if qemu-img available)
• ✅ Size parsing works correctly
• ✅ Filename generation works correctly

4.2 Bootable Image Creation

# Test bootable image creation
./bib/particle-os build --work-dir /tmp/test-bootable recipes/simple-cli-bootable.yml --verbose

Expected Results:

• ✅ GPT partition table is created
• ✅ ext4 filesystem is formatted
• ✅ Rootfs content is copied correctly
• ✅ Bootloader (extlinux) is installed
• ✅ Boot configuration files are created
• ✅ Image is recognized as bootable by QEMU

Phase 5: Error Handling Testing 📋 MEDIUM PRIORITY

Goal: Verify that error handling works correctly

5.1 Invalid Recipe Testing

# Test with invalid recipe
./bib/particle-os build --work-dir /tmp/test-error invalid-recipe.yml --verbose

Expected Results:

• ✅ Clear error message is displayed
• ✅ Helpful debugging information is provided
• ✅ Work directory cleanup occurs
• ✅ Proper exit code is returned

5.2 Resource Limitation Testing

# Test with insufficient disk space
./bib/particle-os build --work-dir /tmp/test-space recipes/minimal-debug.yml --verbose

Expected Results:

• ✅ Disk space check prevents build
• ✅ Clear error message about space requirements
• ✅ Helpful suggestions for resolution
• ✅ Build fails gracefully

---

🔧 Test Environment Setup

Prerequisites

# Install required tools
sudo apt update
sudo apt install -y parted e2fsprogs syslinux-common extlinux qemu-utils

# Verify tools are available
which parted mkfs.ext4 extlinux qemu-img

# Check sudo access
sudo -n true || echo "Sudo access required"

Disk Space Requirements

# Check available space (need minimum 5GB)
df -h /tmp
df -h /home

# Use custom work directory if needed
mkdir -p /home/joe/particle-os-test

Test Recipes

• recipes/minimal-debug.yml - Basic apt stage testing
• recipes/minimal-debug-locale.yml - Locale stage testing
• recipes/simple-cli-bootable.yml - Complete workflow testing
• recipes/qemu-test.yml - QEMU stage testing (create this)

---

📊 Test Results Tracking

Test Matrix

Test Category	Test Case	Expected Result	Actual Result	Status	Notes
Container	List containers	Shows available containers
Container	Inspect container	Provides metadata
Container	Extract container	Creates rootfs
Recipe	List recipes	Shows all recipes
Recipe	Validate recipe	Passes validation
Stage	apt stage	Installs packages
Stage	locale stage	Configures locale
Stage	timezone stage	Configures timezone
Stage	users stage	Creates users
QEMU	raw format	Creates raw image
QEMU	qcow2 format	Creates qcow2 image
Image	Bootable creation	Creates bootable image
Error	Invalid recipe	Handles gracefully
Error	Insufficient space	Prevents build

Success Criteria

• Phase 1: All basic functionality tests pass
• Phase 2: All stage execution tests pass
• Phase 3: All end-to-end workflow tests pass
• Phase 4: All image creation tests pass
• Phase 5: All error handling tests pass

---

🚨 Known Issues to Monitor

Permission Issues (Should Be Fixed)

• ❌ Locale stage permission denied errors
• ❌ Timezone stage permission denied errors
• ❌ Users stage permission denied errors

Disk Space Issues (Should Be Handled)

• ❌ "No space left on device" errors
• ❌ Build failures due to insufficient space

Stage Execution Issues (Should Be Fixed)

• ❌ Stage failures stopping entire build
• ❌ Poor error messages and debugging info

---

🎯 Testing Priorities

Immediate (After Binary Recompilation)

1. Test stage execution - Verify sudo fixes work
2. Test end-to-end workflow - Validate complete builds
3. Test error handling - Verify graceful failure handling

Short Term (1-2 weeks)

1. Test all image formats - Validate QEMU stage
2. Test bootable image creation - Verify bootloader installation
3. Test edge cases - Invalid recipes, resource limitations

Medium Term (2-4 weeks)

1. Performance testing - Build times and resource usage
2. Integration testing - CI/CD workflows
3. User acceptance testing - Real-world usage scenarios

---

📝 Test Execution Commands

Quick Test Suite

#!/bin/bash
# Quick test suite for basic functionality

echo "🧪 Running quick test suite..."

# Test 1: Basic functionality
echo "📋 Test 1: Basic functionality"
./bib/particle-os --version
./bib/particle-os --help
./bib/particle-os list

# Test 2: Container operations
echo "📋 Test 2: Container operations"
./bib/particle-os container list
./bib/particle-os container inspect debian:trixie-slim

# Test 3: Recipe validation
echo "📋 Test 3: Recipe validation"
./bib/particle-os validate recipes/minimal-debug.yml

echo "✅ Quick test suite completed"

Full Test Suite

#!/bin/bash
# Full test suite for comprehensive validation

echo "🧪 Running full test suite..."

# Create test work directory
TEST_DIR="/tmp/particle-os-test-$(date +%s)"
mkdir -p "$TEST_DIR"

# Test all stages individually
echo "📋 Testing individual stages..."
./bib/particle-os build --work-dir "$TEST_DIR/apt" recipes/minimal-debug.yml --verbose
./bib/particle-os build --work-dir "$TEST_DIR/locale" recipes/minimal-debug-locale.yml --verbose
./bib/particle-os build --work-dir "$TEST_DIR/cli" recipes/simple-cli-bootable.yml --verbose

# Cleanup
rm -rf "$TEST_DIR"

echo "✅ Full test suite completed"

---

🔍 Debugging and Troubleshooting

Common Issues and Solutions

Stage Execution Failures

# Enable verbose logging
./bib/particle-os build --verbose recipes/minimal-debug.yml

# Check work directory
ls -la /tmp/particle-os-build/

# Check stage logs
find /tmp/particle-os-build/stages/ -name "*.log" -exec cat {} \;

Permission Issues

# Check sudo access
sudo -l

# Check file ownership in rootfs
ls -la /tmp/particle-os-build/rootfs/etc/

# Check chroot permissions
sudo chroot /tmp/particle-os-build/rootfs ls -la /etc/

Disk Space Issues

# Check available space
df -h /tmp
df -h /home

# Use disk space management script
./scripts/manage-disk-space.sh status
./scripts/manage-disk-space.sh cleanup

---

📚 Additional Resources

Documentation

• HOW-TO-USE.md: Command-line usage guide
• HOW-TO-USE-AS-CICD.md: CI/CD integration guide
• todo: Detailed project status and roadmap

Test Scripts

• test-improvements.sh: Basic functionality testing
• test-sudo-fix.sh: Sudo approach validation
• scripts/manage-disk-space.sh: Disk space management

---

Last Updated: August 17, 2025
Status: 🚧 Ready for Testing - Binary Needs Recompilation
Next Milestone: Execute Testing Strategy After Binary Recompilation
Testing Readiness: 100% (strategy complete, waiting for binary)