particle-os-tools/src/apt-layer

Particle-OS apt-layer Tool - Modular Structure

This directory contains the modular source code for the Particle-OS apt-layer Tool, organized into logical scriptlets that are compiled into a single unified script.

Particle-OS is aiming to be a near 1:1 implementation of the Fedora ublue-os, but using Ubuntu/Debian as a base. ublue-os uses rpm-ostree which obviously won't work on Ubuntu/Debian systems. apt-layer.sh is a proof of concept to make a 1:1 feature complete imitation of rpm-ostree (knowing we're using apt and not dnf, keeping those limitations in mind).

📁 Directory Structure

src/apt-layer/
├── compile.sh              # Compilation script (merges all scriptlets)
├── config/                 # Configuration files (JSON)
│   ├── apt-layer-settings.json     # Main configuration
│   └── package-validation.json     # Package validation rules
├── scriptlets/             # Individual scriptlet files
│   ├── 00-header.sh        # Shared utility functions, global cleanup, system detection
│   ├── 01-dependencies.sh  # Dependency checking and validation
│   ├── 02-transactions.sh  # Transaction management and rollback
│   ├── 03-traditional.sh   # Traditional chroot-based layer creation
│   ├── 04-container.sh     # Container-based layer creation (Apx-style)
│   ├── 05-live-overlay.sh  # Live system layering (rpm-ostree style)
│   ├── 06-oci-integration.sh # OCI export/import functionality
│   ├── 07-bootloader.sh    # Bootloader integration
│   ├── 09-atomic-deployment.sh # Atomic deployment system
│   ├── 10-rpm-ostree-compat.sh # rpm-ostree compatibility layer
│   ├── 15-ostree-atomic.sh # OSTree atomic package management
│   ├── 24-dpkg-direct-install.sh # Direct dpkg Installation (Performance Optimization)
│   └── 99-main.sh          # Main dispatch and help
├── README.md               # This file
└── CHANGELOG.md            # Version history and changes

🚀 Usage

Compiling the Unified Script

# Navigate to the apt-layer directory
cd src/apt-layer

# Run the compilation script
bash compile.sh

This will generate apt-layer.sh in the project root directory.

Development Workflow

1. Edit Individual Scriptlets: Modify the specific scriptlet files in scriptlets/
2. Test Changes: Make your changes and test individual components
3. Compile: Run bash compile.sh to merge all scriptlets
4. Deploy: The unified apt-layer.sh is ready for distribution

📋 Scriptlet Descriptions

Core Scriptlets (All Implemented)

• 00-header.sh: Shared utility functions, global cleanup, and system detection helpers
• 01-dependencies.sh: Package dependency validation and kernel module checking
• 02-transactions.sh: Transaction management with automatic rollback
• 03-traditional.sh: Traditional chroot-based layer creation
• 04-container.sh: Container-based layer creation (Apx-style) ✅ IMPLEMENTED
• 05-live-overlay.sh: Live system layering (rpm-ostree style) ✅ IMPLEMENTED
• 06-oci-integration.sh: OCI export/import functionality ✅ IMPLEMENTED
• 07-bootloader.sh: Bootloader integration (UEFI/GRUB/systemd-boot) ✅ IMPLEMENTED
• 09-atomic-deployment.sh: Atomic deployment system ✅ IMPLEMENTED
• 10-rpm-ostree-compat.sh: rpm-ostree compatibility layer ✅ IMPLEMENTED
• 15-ostree-atomic.sh: OSTree atomic package management ✅ IMPLEMENTED
• 24-dpkg-direct-install.sh: Direct dpkg Installation (Performance Optimization) ✅ IMPLEMENTED
• 99-main.sh: Main command dispatch and help system

🔧 Benefits of This Structure

✅ Modular Development

• Each component can be developed and tested independently
• Easy to locate and modify specific functionality
• Clear separation of concerns

✅ Unified Deployment

• Single apt-layer.sh file for end users
• No complex dependency management
• Professional distribution format

✅ Maintainable Code

• Logical organization by functionality
• Easy to add new features
• Clear documentation per component

✅ Version Control Friendly

• Small, focused files are easier to review
• Clear commit history per feature
• Reduced merge conflicts

🏗️ Architecture Overview

Core Components

1. Transaction Management: Atomic operations with automatic rollback
2. ComposeFS Integration: Uses the modular ComposeFS backend
3. Dependency Validation: Comprehensive dependency checking
4. Error Handling: Robust error handling and recovery

Layer Creation Methods

1. Traditional: Chroot-based package installation ✅ IMPLEMENTED

   • Uses chroot for isolation
   • Direct package installation in isolated environment
   • Suitable for most use cases

2. Container (Apx-style): Isolated container-based installation ✅ IMPLEMENTED

   • Uses container technology (Podman/Docker/systemd-nspawn)
   • Complete isolation from host system
   • Reproducible and secure package installation
   • Named after the Apx package manager's isolation approach

3. Live Overlay: Runtime package installation ✅ IMPLEMENTED

   • Installs packages on running system using overlayfs
   • Provides immediate package availability
   • Supports commit/rollback operations

Integration Points

• ComposeFS Backend: Uses the modular composefs-alternative.sh
• Particle-OS Config: Integrates with unified configuration system
• Bootloader: Automatic boot entry management ✅ IMPLEMENTED
• OCI: Container image export/import ✅ IMPLEMENTED

🚀 Quick Start

Basic Layer Creation

# Create a gaming layer
sudo ./apt-layer.sh particle-os/base/24.04 particle-os/gaming/24.04 steam wine

# List available images
sudo ./apt-layer.sh --list

# Show image information
sudo ./apt-layer.sh --info particle-os/gaming/24.04

Container-based Layer Creation (Apx-style Isolation)

Apx-style isolation follows the same architectural pattern as the Apx package manager, which uses OSTree + Container Isolation. This approach provides a secure, isolated environment for package installation while maintaining the immutable, layered system architecture.

Architecture Overview:

• Base System: Immutable base layer (OSTree in Apx, ComposeFS in apt-layer)
• Isolation Layer: Container-based package installation
• Result: New immutable layer with container-derived changes

Key Benefits:

• Isolation: Package installations run in isolated containers, preventing conflicts with the host system
• Reproducibility: Same packages installed in the same base image always produce identical results
• Security: Container isolation prevents malicious packages from affecting the host
• Clean Environment: Each installation starts with a fresh, clean base image
• Multi-runtime Support: Works with Podman, Docker, or systemd-nspawn
• Immutable Layers: Results in immutable, atomic layers just like OSTree

How it Works (Same Pattern as Apx):

1. Starts with immutable base layer (ComposeFS instead of OSTree)
2. Creates a temporary container from the base image
3. Installs packages inside the isolated container environment
4. Extracts the changes and creates a new immutable layer
5. Cleans up the temporary container
6. Results in an immutable layer that can be deployed atomically

Comparison with rpm-ostree and Apx:

Aspect	rpm-ostree	Apx	apt-layer (Apx-style)
Base System	OSTree commits	OSTree commits	ComposeFS layers
Isolation Method	Direct RPM installation	Container isolation	Container isolation
Package Format	RPM packages	RPM packages	DEB packages
Installation Process	Direct in OSTree tree	Container → OSTree commit	Container → ComposeFS layer
Dependency Resolution	DNF/RPM	DNF/RPM	APT
Transaction Safety	OSTree atomic	OSTree atomic	ComposeFS atomic
Reproducibility	OSTree commit hashes	OSTree commit hashes	ComposeFS layer hashes
Cross-Platform	Red Hat/Fedora	Red Hat/Fedora	Ubuntu/Debian

Key Architectural Similarities:

• Apx: OSTree base + Container isolation + OSTree commits = Immutable layers
• apt-layer: ComposeFS base + Container isolation + ComposeFS layers = Immutable layers
• rpm-ostree: OSTree base + Direct installation + OSTree commits = Immutable layers

The Pattern is the Same: All three approaches create immutable, layered systems. The difference is in the underlying technologies:

• rpm-ostree: Uses OSTree + RPM directly
• Apx: Uses OSTree + Container isolation + RPM
• apt-layer: Uses ComposeFS + Container isolation + APT

What is ComposeFS? ComposeFS is a modern filesystem technology that provides immutable, layered filesystem capabilities similar to OSTree. It creates read-only, deduplicated layers that can be composed together to form a complete filesystem. Each layer contains the changes from the previous layer, creating an immutable, versioned filesystem structure. ComposeFS layers are content-addressable (identified by cryptographic hashes) and can be efficiently shared and distributed, making it ideal for container images and immutable system deployments.

How ComposeFS Works:

1. Base Layer: Starts with an immutable base filesystem layer
2. Layer Creation: New layers contain only the changes (deltas) from the previous layer
3. Composition: Multiple layers are composed together to create the final filesystem
4. Immutability: Each layer is read-only and cannot be modified once created
5. Efficiency: Deduplication and compression reduce storage requirements
6. Distribution: Layers can be independently distributed and cached

We're not departing from how rpm-ostree works - we're following the same immutable layering pattern but with Ubuntu/Debian technologies instead of Red Hat technologies.

# Container-based layer creation (Apx-style)
sudo ./apt-layer.sh --container particle-os/base/24.04 particle-os/gaming/24.04 steam wine

# Container-based layer with multiple packages
sudo ./apt-layer.sh --container particle-os/base/24.04 particle-os/dev/24.04 vscode git nodejs npm

# Container status and information
sudo ./apt-layer.sh --container-status

Live System Installation

# Live system installation (rpm-ostree style)
sudo ./apt-layer.sh --live-install firefox

# Live overlay management
sudo ./apt-layer.sh --live-overlay start
sudo ./apt-layer.sh --live-overlay status
sudo ./apt-layer.sh --live-overlay commit
sudo ./apt-layer.sh --live-overlay rollback

Direct dpkg Installation (Performance Optimization)

# Direct dpkg installation (faster, more controlled)
sudo ./apt-layer.sh --dpkg-install curl wget

# Container-based dpkg installation
sudo ./apt-layer.sh --container-dpkg particle-os/base/24.04 particle-os/dev/24.04 vscode git

# Live system dpkg installation
sudo ./apt-layer.sh --live-dpkg firefox

# Download-only mode (for offline installation)
DPKG_DOWNLOAD_ONLY=true sudo ./apt-layer.sh --dpkg-install curl

# Force installation with dependency issues
DPKG_FORCE_DEPENDS=true sudo ./apt-layer.sh --dpkg-install package-name

# Use specific chroot directory
DPKG_CHROOT_DIR=/path/to/chroot sudo ./apt-layer.sh --dpkg-install package-name

OCI Integration

# OCI export
sudo ./apt-layer.sh --oci-export ubuntu-ublue/gaming/24.04 my-registry/gaming:latest

# OCI import
sudo ./apt-layer.sh --oci-import my-registry/gaming:latest ubuntu-ublue/gaming/24.04

# OCI status
sudo ./apt-layer.sh --oci-status

Bootloader Management

# Bootloader management
sudo ./apt-layer.sh bootloader status
sudo ./apt-layer.sh bootloader list-entries
sudo ./apt-layer.sh bootloader set-default particle-os/gaming/24.04

Kernel arguments (rpm-ostree compatibility)

sudo ./apt-layer.sh kargs add rd.break=pre-mount
sudo ./apt-layer.sh kargs list
sudo ./apt-layer.sh kargs remove rd.break=pre-mount

OSTree Atomic Package Management

# Atomic OSTree package management
sudo ./apt-layer.sh ostree compose install firefox vlc
sudo ./apt-layer.sh ostree compose remove package-name
sudo ./apt-layer.sh ostree compose update

# View atomic history
sudo ./apt-layer.sh ostree log
sudo ./apt-layer.sh ostree diff commit1 commit2
sudo ./apt-layer.sh ostree status
sudo ./apt-layer.sh ostree rollback commit-id
sudo ./apt-layer.sh ostree cleanup

rpm-ostree Compatibility

# Full rpm-ostree command compatibility
sudo ./apt-layer.sh install firefox
sudo ./apt-layer.sh upgrade
sudo ./apt-layer.sh rebase particle-os/gaming/24.04
sudo ./apt-layer.sh rollback
sudo ./apt-layer.sh status
sudo ./apt-layer.sh diff
sudo ./apt-layer.sh db list
sudo ./apt-layer.sh cleanup

🔧 Configuration

The apt-layer tool integrates with the Particle-OS configuration system and includes a comprehensive JSON-based configuration system:

Particle-OS Integration

# Configuration is automatically loaded from:
# /usr/local/etc/particle-config.sh

# Key configuration variables:
WORKSPACE="/var/lib/particle-os"
COMPOSEFS_SCRIPT="/usr/local/bin/composefs-alternative.sh"
CONTAINER_RUNTIME="podman"

JSON Configuration System

The tool includes embedded JSON configuration files for enterprise-grade configurability:

• apt-layer-settings.json: Global settings, feature toggles, and defaults
• security-policy.json: Security policies, signature requirements, and blocked packages
• users.json: RBAC user definitions and access control
• audit-settings.json: Audit logging policies and compliance frameworks
• backup-policy.json: Backup frequency, retention, and encryption settings
• signing-policy.json: Layer signing methods and trusted keys
• oci-settings.json: OCI registry configuration and authentication
• package-management.json: Repository policies and dependency resolution
• maintenance.json: Automated maintenance and cleanup policies

All configuration files are automatically embedded in the compiled script and can be overridden via command-line arguments for enterprise deployment flexibility.

🛠️ Development Guidelines

Adding New Scriptlets

1. Create the scriptlet file in scriptlets/ with appropriate naming
2. Add to compile.sh in the correct order
3. Update this README with the new scriptlet description
4. Test thoroughly before committing

Scriptlet Naming Convention

• 00-header.sh: Shared utility functions, global cleanup, and system detection
• 01-XX.sh: Dependencies and validation
• 02-XX.sh: Core functionality
• 03-XX.sh: Layer creation methods
• 04-XX.sh: Advanced features
• 05-XX.sh: Live system features
• 06-XX.sh: OCI integration
• 07-XX.sh: Bootloader integration
• 09-XX.sh: Atomic deployment
• 10-XX.sh: Compatibility layers
• 15-XX.sh: OSTree atomic features
• 24-XX.sh: Performance optimizations
• 99-main.sh: Main dispatch (always last)

Error Handling

All scriptlets should:

• Use the unified logging system (log_info, log_error, etc.)
• Include proper error handling and cleanup
• Integrate with transaction management when appropriate

📚 Related Documentation

• ComposeFS Modular System: Backend filesystem layer
• BootC Modular System: Container-native boot system
• Particle-OS Configuration: Unified configuration system

🎯 Development Phases

✅ Phase 1: Core Stability (COMPLETED)

• Modular architecture implementation
• Transaction management system
• Traditional layer creation
• ComposeFS backend integration

✅ Phase 2: Enhanced Features (COMPLETED)

• Container-based layer creation
• OCI integration
• Live system layering

✅ Phase 3: Bootloader Integration (COMPLETED)

• Multi-bootloader support (UEFI/GRUB/systemd-boot)
• Kernel arguments management
• Boot entry management
• Atomic deployment integration

✅ Phase 4: OSTree Atomic Package Management (COMPLETED)

• OSTree atomic commits for package operations
• Atomic deployment with rollback capabilities
• Versioned package history
• Direct dpkg installation optimization
• Live overlay system with DNS fixes

✅ Phase 5: rpm-ostree Compatibility (COMPLETED)

• Full rpm-ostree command compatibility
• Atomic deployment system
• Live overlay system
• Bootloader integration
• OCI integration

🎯 Current Status

✅ COMPLETED MAJOR MILESTONES:

• OSTree/Atomic Workflow Implemented:
  • All apt-layer ostree compose commands (install, remove, update) create atomic, versioned commits
  • apt-layer ostree log, diff, status, rollback, cleanup fully implemented and tested
  • Overlay and dpkg install workflow robust, with DNS fixes for WSL and offline .deb install support
  • Log function bug fixed (commit history now displays correctly)
• Testing & Validation:
  • All atomic/OSTree commands tested and confirmed functional
  • Overlay and atomic install workflows validated, including rollback readiness

🔄 NEXT PRIORITIES:

• Further test rollback and deployment activation
• Document overlay/atomic best practices and known caveats
• Continue integration and optimization of atomic/OSTree workflow
• Add more automated tests for edge cases (optional)

🛠️ COMPILATION SYSTEM ENHANCEMENTS:

• Add source file dependency validation - Validate that all required functions exist in source
  • Add validation that all required functions exist in source scriptlets
  • Add dependency validation during compilation
  • Add error checking for missing source files
  • Add function dependency graph validation
  • Add cross-scriptlet function reference checking

🎯 Scope Reduction Summary

As of July 2025, Particle-OS apt-layer has been successfully reduced to core rpm-ostree-like features only. All advanced, enterprise, cloud, multi-tenant, admin, compliance, and security features have been archived to archive/apt-layer/scriptlets/.

Current Focus:

• Atomic deployment, rollback, status, diff, cleanup - Core rpm-ostree functionality
• Live overlay and container-based layering - Immutable system management
• Bootloader and kargs management - System boot configuration
• OCI/ComposeFS integration - Container and filesystem integration
• Direct dpkg install - Performance optimization for apt/deb systems
• OSTree atomic package management - True atomic package operations