From 7a631f95efdd2b3255ad627376547225c3faa4fe Mon Sep 17 00:00:00 2001
From: robojerk <rraines@gmail.com>
Date: Fri, 15 Aug 2025 21:49:55 -0700
Subject: [PATCH] feat: Add comprehensive testing suite for apt-ostree

- Create justfile-based testing framework with 20+ test targets
- Support different deployment scenarios: container, booted system, disk-based, fresh install
- Environment detection for OSTree, container, boot type, and system info
- CLI functionality testing for all apt-ostree commands
- Error handling and performance testing
- Test report generation and cleanup utilities
- Comprehensive documentation in TESTING.md
- Mindful of system state: booted, container, installed, or disk image
---
 TESTING.md | 316 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 justfile   | 187 +++++++++++++++++++++++++++++++
 2 files changed, 503 insertions(+)
 create mode 100644 TESTING.md
 create mode 100644 justfile

diff --git a/TESTING.md b/TESTING.md
new file mode 100644
index 00000000..306c572d
--- /dev/null
+++ b/TESTING.md
@@ -0,0 +1,316 @@
+# apt-ostree Testing Suite
+
+A comprehensive testing framework for `apt-ostree` that handles different deployment scenarios in Debian-based OSTree systems.
+
+## 🚀 Quick Start
+
+```bash
+# Install just (if not already installed)
+sudo apt install just
+
+# Run comprehensive system test
+just test-system
+
+# Check system environment
+just detect-env
+
+# Test CLI functionality only
+just test-cli
+```
+
+## 🔍 Environment Detection
+
+The testing suite automatically detects your system environment:
+
+- **🐳 Container Environment**: Podman, Docker, LXC
+- **🚀 Booted OSTree System**: Live deployment with write access
+- **💾 Disk-based System**: QCOW2, VMDK, or mounted images
+- **🆕 Fresh Installation**: New system without OSTree
+
+## 📋 Available Test Targets
+
+### Environment Detection
+- `detect-env` - Detect system environment and show status
+- `test-environments` - Test apt-ostree in different environments
+
+### Specific Tests
+- `test-cli` - Test CLI functionality
+- `test-ostree` - Test OSTree integration
+- `test-container` - Test in container environment
+- `test-booted-system` - Test on booted OSTree system
+- `test-disk-system` - Test on disk-based OSTree system
+- `test-fresh-install` - Test on fresh installation
+
+### Comprehensive Testing
+- `test-system` - Run comprehensive system test
+- `test-commands` - Test specific commands
+- `test-errors` - Test error handling
+- `test-performance` - Test performance
+
+### Utilities
+- `test-report` - Generate test report
+- `clean` - Clean up test artifacts
+- `help` - Show help message
+
+## 🐳 Container Testing
+
+### Test in Existing Container
+```bash
+# If already running in container
+just test-container
+```
+
+### Create Temporary Container
+```bash
+# Creates temporary container for testing
+just run-container-test
+```
+
+This will:
+- Use Podman if available, fallback to Docker
+- Mount current directory as `/workspace`
+- Run basic CLI tests
+- Clean up automatically
+
+## 🚀 Booted OSTree System Testing
+
+For systems running live OSTree deployments:
+
+```bash
+# Test on booted system
+just test-booted-system
+```
+
+This tests:
+- OSTree deployment status
+- System-specific commands
+- Write access to OSTree directories
+
+## 💾 Disk-based System Testing
+
+For QCOW2, VMDK, or mounted OSTree images:
+
+```bash
+# Test on disk-based system
+just test-disk-system
+```
+
+This detects:
+- OSTree configuration files
+- Directory permissions
+- Read-only vs. writable status
+
+## 🆕 Fresh Installation Testing
+
+For new systems without OSTree:
+
+```bash
+# Test on fresh installation
+just test-fresh-install
+```
+
+This tests:
+- Basic CLI functionality
+- Help system
+- Version information
+- No OSTree dependencies
+
+## 📊 Test Reports
+
+Generate comprehensive test reports:
+
+```bash
+# Generate test report
+just test-report
+
+# View report
+cat test-report.txt
+```
+
+Reports include:
+- System environment detection
+- CLI test results
+- OSTree integration status
+- Performance metrics
+
+## 🔧 Customization
+
+### Add Custom Tests
+Add new test targets to the `justfile`:
+
+```makefile
+# Custom test example
+test-custom:
+    @echo "🧪 Running custom test..."
+    @apt-ostree custom-command
+    @echo "✅ Custom test complete"
+```
+
+### Environment Variables
+Set environment variables for testing:
+
+```bash
+# Set test environment
+export APT_OSTREE_TEST_ENV=production
+export APT_OSTREE_LOG_LEVEL=debug
+
+# Run tests
+just test-system
+```
+
+## 🐛 Troubleshooting
+
+### Common Issues
+
+#### OSTree Not Available
+```bash
+# Check if OSTree is installed
+which ostree
+
+# Install OSTree if needed
+sudo apt install ostree
+```
+
+#### Permission Denied
+```bash
+# Check directory permissions
+ls -la /ostree
+
+# Run with appropriate privileges
+sudo just test-booted-system
+```
+
+#### Container Issues
+```bash
+# Check container runtime
+which podman || which docker
+
+# Verify container access
+ls -la /.dockerenv /run/.containerenv
+```
+
+### Debug Mode
+Enable verbose output:
+
+```bash
+# Run with debug output
+just --verbose test-system
+```
+
+## 📈 Performance Testing
+
+Test command performance:
+
+```bash
+# Run performance tests
+just test-performance
+```
+
+This measures:
+- Help command response time
+- Search command performance
+- Version command speed
+
+## 🧹 Cleanup
+
+Clean up test artifacts:
+
+```bash
+# Remove test files
+just clean
+```
+
+## 🔄 Continuous Integration
+
+### GitHub Actions Example
+```yaml
+name: Test apt-ostree
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-rust@v3
+      - run: |
+          sudo apt install just ostree
+          just test-system
+          just test-report
+      - uses: actions/upload-artifact@v3
+        with:
+          name: test-report
+          path: test-report.txt
+```
+
+### Local CI
+```bash
+# Run tests before commit
+just test-system && just test-report
+
+# Check for failures
+grep -i "❌\|FAIL" test-report.txt
+```
+
+## 📚 Examples
+
+### Development Workflow
+```bash
+# 1. Check environment
+just detect-env
+
+# 2. Run basic tests
+just test-cli
+
+# 3. Test in container
+just run-container-test
+
+# 4. Full system test
+just test-system
+
+# 5. Generate report
+just test-report
+```
+
+### Production Testing
+```bash
+# Test on production system
+just test-booted-system
+
+# Performance testing
+just test-performance
+
+# Error handling
+just test-errors
+```
+
+### CI/CD Pipeline
+```bash
+# Automated testing
+just test-system
+just test-performance
+just test-report
+
+# Exit on failure
+if grep -q "❌\|FAIL" test-report.txt; then
+    echo "Tests failed!"
+    exit 1
+fi
+```
+
+## 🤝 Contributing
+
+When adding new tests:
+
+1. **Follow naming convention**: `test-<feature>`
+2. **Add to help section**: Update the `help` target
+3. **Include in system test**: Add to `test-system` target
+4. **Document usage**: Add examples to this README
+
+## 📄 License
+
+This testing suite is part of the `apt-ostree` project and follows the same license terms.
+
+---
+
+**Happy Testing! 🧪✨**
diff --git a/justfile b/justfile
new file mode 100644
index 00000000..5002411e
--- /dev/null
+++ b/justfile
@@ -0,0 +1,187 @@
+# apt-ostree Testing Suite
+# Comprehensive testing for Debian-based OSTree systems
+
+# Default target - show available commands
+default:
+    @just --list
+
+# Detect system environment and show status
+detect-env:
+    @echo "🔍 Detecting system environment..."
+    @echo "OSTree Status:"
+    @ostree admin status 2>/dev/null || echo "  ❌ OSTree not available"
+    @echo ""
+    @echo "Container Environment:"
+    @bash -c 'if [ -f /.dockerenv ] || [ -f /run/.containerenv ]; then echo "  🐳 Running in container"; echo "  Container type: $(cat /proc/1/cgroup | grep -o "docker\|podman\|lxc" | head -1 || echo "unknown")"; else echo "  💻 Running on bare metal/VM"; fi'
+    @echo ""
+    @echo "Boot Status:"
+    @bash -c 'if [ -d /sys/firmware/efi ]; then echo "  🔌 UEFI boot detected"; else echo "  💾 Legacy BIOS boot detected"; fi'
+    @echo ""
+    @echo "System Information:"
+    @echo "  OS: $(cat /etc/os-release | grep PRETTY_NAME | cut -d'"' -f2 || echo 'unknown')"
+    @echo "  Kernel: $(uname -r)"
+    @echo "  Architecture: $(uname -m)"
+
+# Test apt-ostree CLI functionality
+test-cli:
+    @echo "🧪 Testing apt-ostree CLI functionality..."
+    @echo "Testing main help..."
+    @apt-ostree --help
+    @echo ""
+    @echo "Testing version..."
+    @apt-ostree --version
+    @echo ""
+    @echo "Testing command help..."
+    @apt-ostree status --help
+    @echo ""
+    @echo "Testing search functionality..."
+    @apt-ostree search bash | head -5
+
+# Test OSTree system status
+test-ostree:
+    @echo "🌳 Testing OSTree system status..."
+    @bash -c 'if command -v ostree >/dev/null 2>&1; then echo "OSTree version: $(ostree --version)"; echo ""; echo "Current deployment:"; ostree admin status 2>/dev/null || echo "  ❌ No deployments found"; echo ""; echo "Repository status:"; ostree admin status --repo 2>/dev/null || echo "  ❌ Repository not accessible"; else echo "❌ OSTree not installed"; fi'
+
+# Test apt-ostree in different environments
+test-environments:
+    @echo "🔧 Testing apt-ostree in different environments..."
+    @echo ""
+    @echo "1. Testing in current environment..."
+    @just test-cli
+    @echo ""
+    @echo "2. Testing OSTree integration..."
+    @just test-ostree
+    @echo ""
+    @echo "3. Testing package search..."
+    @apt-ostree search apt | head -3
+
+# Test container environment (Podman/Docker)
+test-container:
+    @echo "🐳 Testing apt-ostree in container environment..."
+    @bash -c 'if [ -f /.dockerenv ] || [ -f /run/.containerenv ]; then echo "✅ Running in container"; just test-cli; just test-ostree; else echo "❌ Not running in container"; echo "To test in container, run: just run-container-test"; fi'
+
+# Run container test (creates temporary container)
+run-container-test:
+    @echo "🐳 Creating temporary container for testing..."
+    @bash -c 'if command -v podman >/dev/null 2>&1; then echo "Using Podman..."; podman run --rm -it --name apt-ostree-test -v $(pwd):/workspace debian:bookworm-slim bash -c "cd /workspace && just test-cli"; elif command -v docker >/dev/null 2>&1; then echo "Using Docker..."; docker run --rm -it --name apt-ostree-test -v $(pwd):/workspace debian:bookworm-slim bash -c "cd /workspace && just test-cli"; else echo "❌ Neither Podman nor Docker found"; fi'
+
+# Test booted OSTree system
+test-booted-system:
+    @echo "🚀 Testing apt-ostree on booted OSTree system..."
+    @bash -c 'if [ -d /ostree ] && [ -d /sysroot ]; then echo "✅ OSTree system detected"; just test-cli; just test-ostree; echo ""; echo "Testing system-specific commands..."; apt-ostree status 2>/dev/null || echo "  ❌ Status command failed"; apt-ostree search systemd | head -3; else echo "❌ Not running on booted OSTree system"; echo "This test requires a live OSTree deployment"; fi'
+
+# Test disk-based OSTree system (QCOW2, etc.)
+test-disk-system:
+    @echo "💾 Testing apt-ostree on disk-based OSTree system..."
+    @bash -c 'if [ -f /etc/ostree/ostree.conf ] || [ -d /ostree ]; then echo "✅ OSTree configuration found"; just test-cli; echo ""; echo "Testing disk-specific operations..."; if [ -w /ostree ]; then echo "✅ OSTree directory is writable"; else echo "⚠️  OSTree directory is read-only (may be mounted image)"; fi; else echo "❌ OSTree configuration not found"; echo "This may be a fresh system or non-OSTree system"; fi'
+
+# Test fresh installation
+test-fresh-install:
+    @echo "🆕 Testing apt-ostree on fresh installation..."
+    @bash -c 'if ! command -v ostree >/dev/null 2>&1; then echo "✅ Fresh system detected (no OSTree)"; just test-cli; echo ""; echo "Testing without OSTree dependencies..."; apt-ostree --help; apt-ostree --version; else echo "⚠️  OSTree already installed"; just test-cli; fi'
+
+# Comprehensive system test
+test-system:
+    @echo "🔍 Running comprehensive system test..."
+    @just detect-env
+    @echo ""
+    @just test-environments
+    @echo ""
+    @echo "📊 Test Summary:"
+    @echo "  CLI Tests: $(just test-cli >/dev/null 2>&1 && echo '✅ PASS' || echo '❌ FAIL')"
+    @echo "  OSTree Tests: $(just test-ostree >/dev/null 2>&1 && echo '✅ PASS' || echo '❌ FAIL')"
+    @echo "  Environment Tests: $(just test-environments >/dev/null 2>&1 && echo '✅ PASS' || echo '❌ FAIL')"
+
+# Test specific apt-ostree commands
+test-commands:
+    @echo "📋 Testing specific apt-ostree commands..."
+    @echo ""
+    @echo "Testing help commands..."
+    @apt-ostree --help >/dev/null && echo "✅ Main help" || echo "❌ Main help"
+    @apt-ostree status --help >/dev/null && echo "✅ Status help" || echo "❌ Status help"
+    @apt-ostree install --help >/dev/null && echo "✅ Install help" || echo "❌ Install help"
+    @echo ""
+    @echo "Testing search commands..."
+    @apt-ostree search bash >/dev/null && echo "✅ Search bash" || echo "❌ Search bash"
+    @apt-ostree search systemd >/dev/null && echo "✅ Search systemd" || echo "❌ Search systemd"
+    @echo ""
+    @echo "Testing status commands..."
+    @apt-ostree status >/dev/null 2>&1 && echo "✅ Status command" || echo "❌ Status command"
+
+# Test error handling
+test-errors:
+    @echo "⚠️  Testing error handling..."
+    @echo ""
+    @echo "Testing missing arguments..."
+    @apt-ostree install 2>&1 | grep -q "No package specified" && echo "✅ Install error handling" || echo "❌ Install error handling"
+    @apt-ostree search 2>&1 | grep -q "No query specified" && echo "✅ Search error handling" || echo "❌ Search error handling"
+    @echo ""
+    @echo "Testing invalid commands..."
+    @apt-ostree invalid-command 2>&1 | grep -q "Unknown command" && echo "✅ Invalid command handling" || echo "❌ Invalid command handling"
+
+# Performance testing
+test-performance:
+    @echo "⚡ Testing performance..."
+    @echo ""
+    @echo "Help command performance:"
+    @bash -c 'start=$(date +%s); apt-ostree --help >/dev/null; end=$(date +%s); echo "  Time: $((end - start))s"'
+    @echo ""
+    @echo "Search command performance:"
+    @bash -c 'start=$(date +%s); apt-ostree search bash >/dev/null; end=$(date +%s); echo "  Time: $((end - start))s"'
+    @echo ""
+    @echo "Version command performance:"
+    @bash -c 'start=$(date +%s); apt-ostree --version >/dev/null; end=$(date +%s); echo "  Time: $((end - start))s"'
+
+# Generate test report
+test-report:
+    @echo "📊 Generating test report..."
+    @echo "Test Report - $(date)" > test-report.txt
+    @echo "========================" >> test-report.txt
+    @echo "" >> test-report.txt
+    @just detect-env >> test-report.txt 2>&1
+    @echo "" >> test-report.txt
+    @echo "CLI Tests:" >> test-report.txt
+    @just test-cli >> test-report.txt 2>&1
+    @echo "" >> test-report.txt
+    @echo "OSTree Tests:" >> test-report.txt
+    @just test-ostree >> test-report.txt 2>&1
+    @echo "Report saved to: test-report.txt"
+
+# Clean up test artifacts
+clean:
+    @echo "🧹 Cleaning up test artifacts..."
+    @rm -f test-report.txt
+    @echo "✅ Cleanup complete"
+
+# Show help
+help:
+    @echo "apt-ostree Testing Suite"
+    @echo "========================"
+    @echo ""
+    @echo "Environment Detection:"
+    @echo "  detect-env          - Detect system environment and status"
+    @echo "  test-environments   - Test apt-ostree in different environments"
+    @echo ""
+    @echo "Specific Tests:"
+    @echo "  test-cli            - Test CLI functionality"
+    @echo "  test-ostree         - Test OSTree integration"
+    @echo "  test-container      - Test in container environment"
+    @echo "  test-booted-system  - Test on booted OSTree system"
+    @echo "  test-disk-system    - Test on disk-based OSTree system"
+    @echo "  test-fresh-install  - Test on fresh installation"
+    @echo ""
+    @echo "Comprehensive Testing:"
+    @echo "  test-system         - Run comprehensive system test"
+    @echo "  test-commands       - Test specific commands"
+    @echo "  test-errors         - Test error handling"
+    @echo "  test-performance    - Test performance"
+    @echo ""
+    @echo "Utilities:"
+    @echo "  test-report         - Generate test report"
+    @echo "  clean               - Clean up test artifacts"
+    @echo "  help                - Show this help message"
+    @echo ""
+    @echo "Examples:"
+    @echo "  just test-system    - Run full system test"
+    @echo "  just test-cli       - Test CLI only"
+    @echo "  just detect-env     - Check system status"