robojerk d295f9bb4d Major milestone: Complete apt-ostree bootc compatibility and OCI integration

- ✅ Real package installation (replaced mock installation)
- ✅ Real OSTree commit creation from installed packages
- ✅ OCI image creation from both commits and rootfs
- ✅ Full bootc compatibility with proper labels
- ✅ Comprehensive test suite (test-bootc-apt-ostree.sh)
- ✅ Container tool validation (skopeo, podman)
- ✅ Updated compatibility reports for Ubuntu Questing
- ✅ Fixed OCI schema version and field naming issues
- ✅ Temporary directory lifecycle fixes
- ✅ Serde rename attributes for OCI JSON compliance

Ready for Aurora-style workflow deployment!

2025-07-20 21:06:44 +00:00

23 KiB

Raw Blame History

apt-ostree Architecture

Executive Summary

apt-ostree is a Debian/Ubuntu equivalent of rpm-ostree, providing a hybrid image/package system that combines the strengths of APT package management with OSTree's atomic, immutable deployment model. The architecture mirrors rpm-ostree's design while adapting it to the Debian/Ubuntu ecosystem.

Core Architectural Principles

1. "From Scratch" Philosophy

Principle: Every change regenerates the target filesystem completely from scratch.

Benefits:

Avoids hysteresis (state-dependent behavior)
Ensures reproducible results
Maintains system consistency
Simplifies debugging and testing

Implementation:

OSTree commit-based filesystem management
Atomic transaction processing
Complete state regeneration for each operation
Rollback capability through multiple deployments

2. Hybrid System Design

Principle: Combine the best of both package management and image-based deployments.

Components:

APT Package Management: Traditional package installation via libapt-pkg
OSTree Image Management: Atomic, immutable filesystem deployments
Layered Architecture: Base image + user packages
Atomic Operations: All changes are transactional

3. Daemon-Client Architecture

Principle: Centralized daemon with D-Bus communication for privileged operations.

Benefits:

Privilege separation and security
Concurrent operation support
State persistence across operations
Resource management and optimization

4. 100% CLI Compatibility

Principle: Identical user experience to rpm-ostree.

Benefits:

Familiar interface for users
Easy migration from rpm-ostree
Consistent behavior across systems
Reduced learning curve

System Architecture

High-Level Architecture

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   CLI Client    │    │   GUI Client    │    │   API Client    │
│  (apt-ostree)   │    │  (GNOME/KDE)    │    │  (Python/Go)   │
└─────────┬───────┘    └─────────┬───────┘    └─────────┬───────┘
          │                      │                      │
          └──────────────────────┼──────────────────────┘
                                 │
                    ┌─────────────▼─────────────┐
                    │      D-Bus Interface      │
                    │   (org.aptostree.dev)     │
                    └─────────────┬─────────────┘
                                  │
                    ┌─────────────▼─────────────┐
                    │   apt-ostreed Daemon      │
                    │   (Privileged Service)    │
                    └─────────────┬─────────────┘
                                  │
          ┌───────────────────────┼───────────────────────┐
          │                       │                       │
    ┌─────▼─────┐         ┌───────▼──────┐         ┌─────▼─────┐
    │ libapt-pkg│         │   libostree  │         │  System   │
    │ (APT/DEB) │         │ (Filesystem) │         │ Services  │
    └───────────┘         └──────────────┘         └───────────┘

Component Architecture

1. Client Layer

Purpose: User interface and command processing

Components:

CLI Client (apt-ostree): Command-line interface
GUI Clients: GNOME Software, KDE Discover integration (planned)
API Clients: Python, Go, and other language bindings (planned)

Responsibilities:

Command-line argument parsing
D-Bus communication with daemon
Progress reporting and user feedback
Error handling and user guidance
Fallback to direct system calls if daemon fails

2. D-Bus Interface Layer

Purpose: Inter-process communication and service discovery

Interface: org.aptostree.dev

Key Objects:

/org/aptostree/dev/Sysroot: System root management
/org/aptostree/dev/OS: Operating system operations

Key Methods:

install_packages: Install packages with atomic commits
remove_packages: Remove packages with rollback support
upgrade_system: Upgrade system with automatic policies
rollback: Rollback to previous deployment
show_status: Show system status and deployment information
list_packages: List installed packages
search_packages: Search for packages
show_package_info: Show package information

3. Daemon Layer

Purpose: Centralized system service for privileged operations

Components:

Main Daemon (apt-ostreed): Core daemon process
Transaction Manager: Atomic operation management
State Manager: System state tracking
Resource Manager: Resource allocation and cleanup

Responsibilities:

Privileged system operations
Transaction management and atomicity
State persistence and recovery
Concurrent operation handling
Resource management and optimization

4. Integration Layer

Purpose: Coordination between APT and OSTree systems

Components:

APT Integration: Package management via libapt-pkg
OSTree Integration: Filesystem management via libostree
System Integration: Bootloader and system service integration

Responsibilities:

Package installation and removal
Filesystem commit creation and management
Deployment switching and rollback
Boot configuration management
System state synchronization

Detailed Component Analysis

1. APT Manager (`src/apt.rs`)

Purpose: APT package management using libapt-pkg

Key Features:

Real DEB package download and extraction
APT database management in OSTree context
Package metadata extraction from control files
Dependency resolution using APT's native resolver
Package script execution with Bubblewrap sandboxing

Implementation:

pub struct AptManager {
    // APT integration
    cache: Option<*mut apt_pkg_Cache>,
    depcache: Option<*mut apt_pkg_DepCache>,
    
    // OSTree integration
    ostree_repo: Option<*mut ostree_Repo>,
    sysroot: Option<*mut ostree_Sysroot>,
    
    // Package operations
    pub fn install_packages(&mut self, packages: &[String]) -> Result<(), Error>;
    pub fn remove_packages(&mut self, packages: &[String]) -> Result<(), Error>;
    pub fn upgrade_system(&mut self) -> Result<(), Error>;
    
    // Package information
    pub fn list_packages(&self) -> Result<Vec<PackageInfo>, Error>;
    pub fn search_packages(&self, query: &str) -> Result<Vec<PackageInfo>, Error>;
    pub fn show_package_info(&self, package: &str) -> Result<PackageInfo, Error>;
}

Key Methods:

install_packages: Download and install packages with atomic commits
remove_packages: Remove packages with rollback support
upgrade_system: Upgrade all packages with automatic policies
list_packages: List all installed packages
search_packages: Search for packages by name or description
show_package_info: Show detailed package information

2. OSTree Manager (`src/ostree.rs`)

Purpose: OSTree deployment management and filesystem operations

Key Features:

OSTree commit creation and management
Deployment switching and rollback
Filesystem assembly from commits and layers
Boot configuration management
State tracking and synchronization

Implementation:

pub struct OstreeManager {
    // OSTree integration
    repo: Option<*mut ostree_Repo>,
    sysroot: Option<*mut ostree_Sysroot>,
    
    // Deployment operations
    pub fn create_commit(&mut self, ref_name: &str) -> Result<(), Error>;
    pub fn deploy_commit(&mut self, ref_name: &str) -> Result<(), Error>;
    pub fn rollback_deployment(&mut self) -> Result<(), Error>;
    
    // Filesystem operations
    pub fn assemble_filesystem(&mut self) -> Result<(), Error>;
    pub fn mount_deployment(&mut self, deployment: &str) -> Result<(), Error>;
    
    // Boot configuration
    pub fn update_boot_config(&mut self) -> Result<(), Error>;
    pub fn set_kernel_args(&mut self, args: &[String]) -> Result<(), Error>;
}

Key Methods:

create_commit: Create new OSTree commit with package changes
deploy_commit: Deploy specific commit to bootable state
rollback_deployment: Rollback to previous deployment
assemble_filesystem: Assemble filesystem from commits and layers
update_boot_config: Update bootloader configuration

3. System Integration (`src/system.rs`)

Purpose: Coordination between APT and OSTree systems

Key Features:

APT-OSTree state synchronization
Transaction management
Error handling and recovery
System state monitoring

Implementation:

pub struct SystemIntegration {
    // Component managers
    apt_manager: AptManager,
    ostree_manager: OstreeManager,
    
    // State management
    state: SystemState,
    
    // Integration operations
    pub fn install_packages(&mut self, packages: &[String]) -> Result<(), Error>;
    pub fn remove_packages(&mut self, packages: &[String]) -> Result<(), Error>;
    pub fn upgrade_system(&mut self) -> Result<(), Error>;
    pub fn rollback_system(&mut self) -> Result<(), Error>;
    
    // State synchronization
    pub fn sync_state(&mut self) -> Result<(), Error>;
    pub fn save_state(&mut self) -> Result<(), Error>;
    pub fn load_state(&mut self) -> Result<(), Error>;
}

Key Methods:

install_packages: Coordinated package installation with atomic commits
remove_packages: Coordinated package removal with rollback
upgrade_system: Coordinated system upgrade
rollback_system: Coordinated system rollback
sync_state: Synchronize APT and OSTree state

4. Package Manager (`src/package_manager.rs`)

Purpose: High-level package operations

Key Features:

Package installation with atomic commits
Package removal with rollback support
System upgrades with automatic policies
Package search and information display

Implementation:

pub struct PackageManager {
    // System integration
    system: SystemIntegration,
    
    // High-level operations
    pub fn install_packages(&mut self, packages: &[String]) -> Result<(), Error>;
    pub fn remove_packages(&mut self, packages: &[String]) -> Result<(), Error>;
    pub fn upgrade_system(&mut self) -> Result<(), Error>;
    pub fn search_packages(&self, query: &str) -> Result<Vec<PackageInfo>, Error>;
    pub fn show_package_info(&self, package: &str) -> Result<PackageInfo, Error>;
    
    // Transaction management
    pub fn start_transaction(&mut self, operation: &str) -> Result<String, Error>;
    pub fn commit_transaction(&mut self, transaction_id: &str) -> Result<(), Error>;
    pub fn rollback_transaction(&mut self, transaction_id: &str) -> Result<(), Error>;
}

Key Methods:

install_packages: High-level package installation
remove_packages: High-level package removal
upgrade_system: High-level system upgrade
search_packages: Package search functionality
show_package_info: Package information display

5. OSTree Detection (`src/ostree_detection.rs`)

Purpose: Environment detection and validation

Key Features:

Comprehensive OSTree environment detection
Multiple detection methods for reliability
Clear error messages for non-OSTree systems
Environment validation

Implementation:

pub struct OstreeDetection {
    // Detection methods
    pub fn detect_filesystem(&self) -> bool;
    pub fn detect_boot(&self) -> bool;
    pub fn detect_kernel_params(&self) -> bool;
    pub fn detect_library(&self) -> bool;
    pub fn detect_service(&self) -> bool;
    
    // Comprehensive detection
    pub fn detect_ostree_environment(&self) -> Result<bool, Error>;
    
    // Error reporting
    pub fn get_detection_error(&self) -> String;
}

Detection Methods:

Filesystem Detection: Check for /ostree directory
Boot Detection: Check for /run/ostree-booted file
Kernel Parameter Detection: Check for ostree in /proc/cmdline
Library Detection: Try to load OSTree sysroot
Service Detection: Check for daemon availability

6. Permissions (`src/permissions.rs`)

Purpose: Root privilege checks and error handling

Key Features:

Robust root privilege validation
User-friendly error messages
Security model enforcement
Privilege separation

Implementation:

pub struct Permissions {
    // Privilege checks
    pub fn check_root_privileges(&self) -> Result<(), Error>;
    pub fn check_daemon_privileges(&self) -> Result<(), Error>;
    
    // Error handling
    pub fn get_privilege_error(&self) -> String;
    pub fn get_user_guidance(&self) -> String;
}

Key Methods:

check_root_privileges: Validate root privileges for operations
check_daemon_privileges: Validate daemon privileges
get_privilege_error: Get user-friendly privilege error message
get_user_guidance: Get guidance for privilege issues

Daemon-Client Architecture

Daemon (`src/bin/apt-ostreed.rs`)

Purpose: Centralized system service for privileged operations

Key Features:

D-Bus service exposing system management interface
Privileged package and deployment operations
Transaction management with atomicity guarantees
Progress reporting and cancellation support

Implementation:

pub struct AptOstreeDaemon {
    // D-Bus interface
    connection: Option<Connection>,
    object_manager: Option<ObjectManager>,
    
    // System integration
    package_manager: PackageManager,
    
    // Transaction management
    transactions: HashMap<String, Transaction>,
    
    // D-Bus methods
    pub fn handle_install_packages(&mut self, invocation: MethodInvocation) -> Result<(), Error>;
    pub fn handle_remove_packages(&mut self, invocation: MethodInvocation) -> Result<(), Error>;
    pub fn handle_upgrade_system(&mut self, invocation: MethodInvocation) -> Result<(), Error>;
    pub fn handle_rollback(&mut self, invocation: MethodInvocation) -> Result<(), Error>;
    pub fn handle_show_status(&mut self, invocation: MethodInvocation) -> Result<(), Error>;
}

Key Methods:

handle_install_packages: Handle package installation requests
handle_remove_packages: Handle package removal requests
handle_upgrade_system: Handle system upgrade requests
handle_rollback: Handle rollback requests
handle_show_status: Handle status requests

Client (`src/main.rs`)

Purpose: Command-line interface for user interaction

Key Features:

100% rpm-ostree CLI compatibility
D-Bus client communication with daemon
Fallback to direct system calls if daemon fails
User-friendly error messages and help

Implementation:

pub struct AptOstreeClient {
    // D-Bus client
    connection: Option<Connection>,
    
    // Command handling
    pub fn handle_command(&mut self, args: &[String]) -> Result<(), Error>;
    
    // D-Bus communication
    pub fn connect_to_daemon(&mut self) -> Result<(), Error>;
    pub fn call_daemon_method(&mut self, method: &str, params: &[&str]) -> Result<(), Error>;
    
    // Fallback handling
    pub fn fallback_to_direct_calls(&mut self, args: &[String]) -> Result<(), Error>;
}

Key Methods:

handle_command: Handle CLI command execution
connect_to_daemon: Connect to D-Bus daemon
call_daemon_method: Call daemon method via D-Bus
fallback_to_direct_calls: Fallback to direct system calls

CLI Commands (100% Complete)

Core Commands (21/21 - 100% Complete)

install: Package installation with atomic commits
deploy: Deployment management and switching
apply-live: Live application of changes
cancel: Transaction cancellation
cleanup: Old deployment cleanup
compose: Tree composition
status: System status with rich formatting
upgrade: System upgrades with automatic policies
rollback: Deployment rollback
db: Package database queries (diff, list, version)
search: Enhanced package search
override: Package overrides (replace, remove, reset, list)
refresh-md: Repository metadata refresh
reload: Configuration reload
reset: State reset
rebase: Tree switching
initramfs-etc: Initramfs file management
usroverlay: Transient overlayfs to /usr
kargs: Kernel argument management
uninstall: Package removal (alias for remove)
initramfs: Initramfs management

Command Architecture

All commands follow the same architecture:

CLI Parsing: Parse command-line arguments
Daemon Communication: Request operation via D-Bus
Fallback Handling: Use direct system calls if daemon fails
Progress Reporting: Show operation progress
Result Display: Display operation results

Transaction System

Transaction Lifecycle

Initiation: Client requests operation via D-Bus
Validation: Daemon validates request and creates transaction
Execution: Transaction executes with progress reporting
Completion: Transaction completes with success/failure status
Cleanup: Resources are cleaned up and state is updated

Transaction Types

InstallTransaction

Purpose: Install packages with atomic commits
Operations: Package download, installation, dependency resolution, OSTree commit creation
Rollback: Remove installed packages and restore previous state

RemoveTransaction

Purpose: Remove packages with rollback support
Operations: Package removal, dependency cleanup, OSTree commit creation
Rollback: Restore removed packages

UpgradeTransaction

Purpose: Upgrade system to latest version
Operations: Check for updates, download packages, install updates, OSTree commit creation
Rollback: Revert to previous system version

RollbackTransaction

Purpose: Rollback to previous deployment
Operations: Switch to previous deployment, update boot configuration
Rollback: Switch back to current deployment

Atomic Operations

Principle: All operations are atomic - they either complete entirely or not at all.

Implementation:

Transaction-based execution
Rollback points at each stage
State preservation during operations
Cleanup on failure

Example Flow:

// Start transaction
let transaction_id = start_transaction("install")?;

// Create rollback points
create_rollback_point("pre-package-download")?;
create_rollback_point("pre-package-install")?;
create_rollback_point("pre-ostree-commit")?;

// Execute operations
if !download_packages(packages).await? {
    rollback_to("pre-package-download")?;
    return Err(Error::PackageDownloadFailed);
}

if !install_packages(packages).await? {
    rollback_to("pre-package-install")?;
    return Err(Error::PackageInstallFailed);
}

if !create_ostree_commit().await? {
    rollback_to("pre-ostree-commit")?;
    return Err(Error::OstreeCommitFailed);
}

// Commit transaction
commit_transaction(transaction_id)?;

Security Model

Privilege Separation

Principle: Separate privileged operations from unprivileged user operations.

Implementation:

Daemon runs with elevated privileges
Client operations are unprivileged
D-Bus communication for privileged operations
PolicyKit integration for authentication

Bubblewrap Sandboxing

Principle: Execute package scripts in controlled, isolated environments.

Implementation:

Package script execution in sandboxed environment
Namespace isolation for security
Controlled filesystem access
Privilege restrictions

Sandboxing Features:

Namespace Isolation: Process, mount, network namespaces
Bind Mounts: Controlled filesystem access
Security Controls: Privilege restrictions
Atomic Context: Script execution in atomic context

OSTree Environment Detection

Principle: Ensure operations only run in valid OSTree environments.

Implementation:

Multiple detection methods for reliability
Clear error messages for non-OSTree systems
Environment validation before operations
User guidance for environment setup

Performance Characteristics

Optimization Strategies

OSTree Deduplication: Leverage OSTree's content-addressable storage
Incremental Updates: Only download and apply changes
Parallel Processing: Concurrent package operations
Caching: Cache frequently accessed data

Resource Usage

Memory: Scales with package count and complexity
Disk: Optimized through OSTree deduplication
Network: Minimized through delta updates
CPU: Optimized through parallel processing

Performance Metrics

Package Installation: ~100-500ms per package
System Upgrade: ~30-60 seconds for typical updates
Deployment Switch: ~5-15 seconds
Rollback: ~5-15 seconds

Deployment Model

OSTree Integration

Principle: Use OSTree for atomic, immutable filesystem management.

Features:

Atomic commit-based deployments
Rollback capability through multiple deployments
Bootloader integration for deployment switching
State tracking and management

Package Layering

Principle: Layer user packages on top of immutable base image.

Features:

Base image remains immutable
User packages layered on top
Clear separation of base and user content
Atomic layer application and removal

Boot Configuration

Principle: Manage boot configuration for deployment switching.

Features:

GRUB/systemd-boot integration
Kernel argument management
Initramfs management
Boot-time deployment selection

Systemd Services

Core Services

apt-ostreed.service: Main daemon service with OSTree detection
apt-ostree-bootstatus.service: Boot-time status logging
apt-ostreed-automatic.service: Automatic system updates (planned)
apt-ostree-countme.service: Usage reporting (planned)

Service Configuration

D-Bus service activation
OSTree environment detection
Automatic update policies
Boot-time status reporting

Future Architecture

OCI Integration

Planned Features:

Direct container image support
OCI image integration
Container runtime integration
Hybrid container/host management

Cloud Integration

Planned Features:

Cloud deployment support
Multi-cloud compatibility
Cloud-native workflows
Infrastructure as code integration

Enterprise Features

Planned Features:

Role-based access control (RBAC)
Audit logging and compliance
Enterprise policy management
Advanced security features

Multi-Architecture Support

Planned Features:

ARM64 support
Multi-arch package handling
Architecture-specific optimizations
Cross-architecture compatibility

23 KiB Raw Blame History

apt-ostree Architecture

Executive Summary

Core Architectural Principles

1. "From Scratch" Philosophy

2. Hybrid System Design

3. Daemon-Client Architecture

4. 100% CLI Compatibility

System Architecture

High-Level Architecture

Component Architecture

1. Client Layer

2. D-Bus Interface Layer

3. Daemon Layer

4. Integration Layer

Detailed Component Analysis

1. APT Manager (src/apt.rs)

2. OSTree Manager (src/ostree.rs)

3. System Integration (src/system.rs)

4. Package Manager (src/package_manager.rs)

5. OSTree Detection (src/ostree_detection.rs)

6. Permissions (src/permissions.rs)

Daemon-Client Architecture

Daemon (src/bin/apt-ostreed.rs)

Client (src/main.rs)

CLI Commands (100% Complete)

Core Commands (21/21 - 100% Complete)

Command Architecture

Transaction System

Transaction Lifecycle

Transaction Types

InstallTransaction

RemoveTransaction

UpgradeTransaction

RollbackTransaction

Atomic Operations

Security Model

Privilege Separation

Bubblewrap Sandboxing

OSTree Environment Detection

Performance Characteristics

Optimization Strategies

Resource Usage

Performance Metrics

Deployment Model

OSTree Integration

Package Layering

Boot Configuration

Systemd Services

Core Services

Service Configuration

Future Architecture

OCI Integration

Cloud Integration

Enterprise Features

Multi-Architecture Support

23 KiB

Raw Blame History

1. APT Manager (`src/apt.rs`)

2. OSTree Manager (`src/ostree.rs`)

3. System Integration (`src/system.rs`)

4. Package Manager (`src/package_manager.rs`)

5. OSTree Detection (`src/ostree_detection.rs`)

6. Permissions (`src/permissions.rs`)

Daemon (`src/bin/apt-ostreed.rs`)

Client (`src/main.rs`)