apt-ostree/.notes/architecture/apt-ostree.md

# apt-ostree Architecture

## Executive Summary

apt-ostree is a Debian/Ubuntu equivalent of rpm-ostree, providing a hybrid image/package system that combines the strengths of APT package management with OSTree's atomic, immutable deployment model. The architecture mirrors rpm-ostree's design while adapting it to the Debian/Ubuntu ecosystem.

## Core Architectural Principles

### 1. "From Scratch" Philosophy

**Principle**: Every change regenerates the target filesystem completely from scratch.

**Benefits**:
- Avoids hysteresis (state-dependent behavior)
- Ensures reproducible results
- Maintains system consistency
- Simplifies debugging and testing

**Implementation**:
- OSTree commit-based filesystem management
- Atomic transaction processing
- Complete state regeneration for each operation
- Rollback capability through multiple deployments

### 2. Hybrid System Design

**Principle**: Combine the best of both package management and image-based deployments.

**Components**:
- **APT Package Management**: Traditional package installation via libapt-pkg
- **OSTree Image Management**: Atomic, immutable filesystem deployments
- **Layered Architecture**: Base image + user packages
- **Atomic Operations**: All changes are transactional

### 3. Daemon-Client Architecture

**Principle**: Centralized daemon with D-Bus communication for privileged operations.

**Benefits**:
- Privilege separation and security
- Concurrent operation support
- State persistence across operations
- Resource management and optimization

### 4. 100% CLI Compatibility

**Principle**: Identical user experience to rpm-ostree.

**Benefits**:
- Familiar interface for users
- Easy migration from rpm-ostree
- Consistent behavior across systems
- Reduced learning curve

## System Architecture

### High-Level Architecture

```
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   CLI Client    │    │   GUI Client    │    │   API Client    │
│  (apt-ostree)   │    │  (GNOME/KDE)    │    │  (Python/Go)   │
└─────────┬───────┘    └─────────┬───────┘    └─────────┬───────┘
          │                      │                      │
          └──────────────────────┼──────────────────────┘
                                 │
                    ┌─────────────▼─────────────┐
                    │      D-Bus Interface      │
                    │   (org.aptostree.dev)     │
                    └─────────────┬─────────────┘
                                  │
                    ┌─────────────▼─────────────┐
                    │   apt-ostreed Daemon      │
                    │   (Privileged Service)    │
                    └─────────────┬─────────────┘
                                  │
          ┌───────────────────────┼───────────────────────┐
          │                       │                       │
    ┌─────▼─────┐         ┌───────▼──────┐         ┌─────▼─────┐
    │ libapt-pkg│         │   libostree  │         │  System   │
    │ (APT/DEB) │         │ (Filesystem) │         │ Services  │
    └───────────┘         └──────────────┘         └───────────┘
```

### Component Architecture

#### 1. Client Layer

**Purpose**: User interface and command processing

**Components**:
- **CLI Client** (`apt-ostree`): Command-line interface
- **GUI Clients**: GNOME Software, KDE Discover integration (planned)
- **API Clients**: Python, Go, and other language bindings (planned)

**Responsibilities**:
- Command-line argument parsing
- D-Bus communication with daemon
- Progress reporting and user feedback
- Error handling and user guidance
- Fallback to direct system calls if daemon fails

#### 2. D-Bus Interface Layer

**Purpose**: Inter-process communication and service discovery

**Interface**: `org.aptostree.dev`

**Key Objects**:
- `/org/aptostree/dev/Sysroot`: System root management
- `/org/aptostree/dev/OS`: Operating system operations

**Key Methods**:
- `install_packages`: Install packages with atomic commits
- `remove_packages`: Remove packages with rollback support
- `upgrade_system`: Upgrade system with automatic policies
- `rollback`: Rollback to previous deployment
- `show_status`: Show system status and deployment information
- `list_packages`: List installed packages
- `search_packages`: Search for packages
- `show_package_info`: Show package information

#### 3. Daemon Layer

**Purpose**: Centralized system service for privileged operations

**Components**:
- **Main Daemon** (`apt-ostreed`): Core daemon process
- **Transaction Manager**: Atomic operation management
- **State Manager**: System state tracking
- **Resource Manager**: Resource allocation and cleanup

**Responsibilities**:
- Privileged system operations
- Transaction management and atomicity
- State persistence and recovery
- Concurrent operation handling
- Resource management and optimization

#### 4. Integration Layer

**Purpose**: Coordination between APT and OSTree systems

**Components**:
- **APT Integration**: Package management via libapt-pkg
- **OSTree Integration**: Filesystem management via libostree
- **System Integration**: Bootloader and system service integration

**Responsibilities**:
- Package installation and removal
- Filesystem commit creation and management
- Deployment switching and rollback
- Boot configuration management
- System state synchronization

## Detailed Component Analysis

### 1. APT Manager (`src/apt.rs`)

**Purpose**: APT package management using libapt-pkg

**Key Features**:
- Real DEB package download and extraction
- APT database management in OSTree context
- Package metadata extraction from control files
- Dependency resolution using APT's native resolver
- Package script execution with Bubblewrap sandboxing

**Implementation**:
```rust
pub struct AptManager {
    // APT integration
    cache: Option<*mut apt_pkg_Cache>,
    depcache: Option<*mut apt_pkg_DepCache>,
    
    // OSTree integration
    ostree_repo: Option<*mut ostree_Repo>,
    sysroot: Option<*mut ostree_Sysroot>,
    
    // Package operations
    pub fn install_packages(&mut self, packages: &[String]) -> Result<(), Error>;
    pub fn remove_packages(&mut self, packages: &[String]) -> Result<(), Error>;
    pub fn upgrade_system(&mut self) -> Result<(), Error>;
    
    // Package information
    pub fn list_packages(&self) -> Result<Vec<PackageInfo>, Error>;
    pub fn search_packages(&self, query: &str) -> Result<Vec<PackageInfo>, Error>;
    pub fn show_package_info(&self, package: &str) -> Result<PackageInfo, Error>;
}
```

**Key Methods**:
- `install_packages`: Download and install packages with atomic commits
- `remove_packages`: Remove packages with rollback support
- `upgrade_system`: Upgrade all packages with automatic policies
- `list_packages`: List all installed packages
- `search_packages`: Search for packages by name or description
- `show_package_info`: Show detailed package information

### 2. OSTree Manager (`src/ostree.rs`)

**Purpose**: OSTree deployment management and filesystem operations

**Key Features**:
- OSTree commit creation and management
- Deployment switching and rollback
- Filesystem assembly from commits and layers
- Boot configuration management
- State tracking and synchronization

**Implementation**:
```rust
pub struct OstreeManager {
    // OSTree integration
    repo: Option<*mut ostree_Repo>,
    sysroot: Option<*mut ostree_Sysroot>,
    
    // Deployment operations
    pub fn create_commit(&mut self, ref_name: &str) -> Result<(), Error>;
    pub fn deploy_commit(&mut self, ref_name: &str) -> Result<(), Error>;
    pub fn rollback_deployment(&mut self) -> Result<(), Error>;
    
    // Filesystem operations
    pub fn assemble_filesystem(&mut self) -> Result<(), Error>;
    pub fn mount_deployment(&mut self, deployment: &str) -> Result<(), Error>;
    
    // Boot configuration
    pub fn update_boot_config(&mut self) -> Result<(), Error>;
    pub fn set_kernel_args(&mut self, args: &[String]) -> Result<(), Error>;
}
```

**Key Methods**:
- `create_commit`: Create new OSTree commit with package changes
- `deploy_commit`: Deploy specific commit to bootable state
- `rollback_deployment`: Rollback to previous deployment
- `assemble_filesystem`: Assemble filesystem from commits and layers
- `update_boot_config`: Update bootloader configuration

### 3. System Integration (`src/system.rs`)

**Purpose**: Coordination between APT and OSTree systems

**Key Features**:
- APT-OSTree state synchronization
- Transaction management
- Error handling and recovery
- System state monitoring

**Implementation**:
```rust
pub struct SystemIntegration {
    // Component managers
    apt_manager: AptManager,
    ostree_manager: OstreeManager,
    
    // State management
    state: SystemState,
    
    // Integration operations
    pub fn install_packages(&mut self, packages: &[String]) -> Result<(), Error>;
    pub fn remove_packages(&mut self, packages: &[String]) -> Result<(), Error>;
    pub fn upgrade_system(&mut self) -> Result<(), Error>;
    pub fn rollback_system(&mut self) -> Result<(), Error>;
    
    // State synchronization
    pub fn sync_state(&mut self) -> Result<(), Error>;
    pub fn save_state(&mut self) -> Result<(), Error>;
    pub fn load_state(&mut self) -> Result<(), Error>;
}
```

**Key Methods**:
- `install_packages`: Coordinated package installation with atomic commits
- `remove_packages`: Coordinated package removal with rollback
- `upgrade_system`: Coordinated system upgrade
- `rollback_system`: Coordinated system rollback
- `sync_state`: Synchronize APT and OSTree state

### 4. Package Manager (`src/package_manager.rs`)

**Purpose**: High-level package operations

**Key Features**:
- Package installation with atomic commits
- Package removal with rollback support
- System upgrades with automatic policies
- Package search and information display

**Implementation**:
```rust
pub struct PackageManager {
    // System integration
    system: SystemIntegration,
    
    // High-level operations
    pub fn install_packages(&mut self, packages: &[String]) -> Result<(), Error>;
    pub fn remove_packages(&mut self, packages: &[String]) -> Result<(), Error>;
    pub fn upgrade_system(&mut self) -> Result<(), Error>;
    pub fn search_packages(&self, query: &str) -> Result<Vec<PackageInfo>, Error>;
    pub fn show_package_info(&self, package: &str) -> Result<PackageInfo, Error>;
    
    // Transaction management
    pub fn start_transaction(&mut self, operation: &str) -> Result<String, Error>;
    pub fn commit_transaction(&mut self, transaction_id: &str) -> Result<(), Error>;
    pub fn rollback_transaction(&mut self, transaction_id: &str) -> Result<(), Error>;
}
```

**Key Methods**:
- `install_packages`: High-level package installation
- `remove_packages`: High-level package removal
- `upgrade_system`: High-level system upgrade
- `search_packages`: Package search functionality
- `show_package_info`: Package information display

### 5. OSTree Detection (`src/ostree_detection.rs`)

**Purpose**: Environment detection and validation

**Key Features**:
- Comprehensive OSTree environment detection
- Multiple detection methods for reliability
- Clear error messages for non-OSTree systems
- Environment validation

**Implementation**:
```rust
pub struct OstreeDetection {
    // Detection methods
    pub fn detect_filesystem(&self) -> bool;
    pub fn detect_boot(&self) -> bool;
    pub fn detect_kernel_params(&self) -> bool;
    pub fn detect_library(&self) -> bool;
    pub fn detect_service(&self) -> bool;
    
    // Comprehensive detection
    pub fn detect_ostree_environment(&self) -> Result<bool, Error>;
    
    // Error reporting
    pub fn get_detection_error(&self) -> String;
}
```

**Detection Methods**:
1. **Filesystem Detection**: Check for `/ostree` directory
2. **Boot Detection**: Check for `/run/ostree-booted` file
3. **Kernel Parameter Detection**: Check for `ostree` in `/proc/cmdline`
4. **Library Detection**: Try to load OSTree sysroot
5. **Service Detection**: Check for daemon availability

### 6. Permissions (`src/permissions.rs`)

**Purpose**: Root privilege checks and error handling

**Key Features**:
- Robust root privilege validation
- User-friendly error messages
- Security model enforcement
- Privilege separation

**Implementation**:
```rust
pub struct Permissions {
    // Privilege checks
    pub fn check_root_privileges(&self) -> Result<(), Error>;
    pub fn check_daemon_privileges(&self) -> Result<(), Error>;
    
    // Error handling
    pub fn get_privilege_error(&self) -> String;
    pub fn get_user_guidance(&self) -> String;
}
```

**Key Methods**:
- `check_root_privileges`: Validate root privileges for operations
- `check_daemon_privileges`: Validate daemon privileges
- `get_privilege_error`: Get user-friendly privilege error message
- `get_user_guidance`: Get guidance for privilege issues

## Daemon-Client Architecture

### Daemon (`src/bin/apt-ostreed.rs`)

**Purpose**: Centralized system service for privileged operations

**Key Features**:
- D-Bus service exposing system management interface
- Privileged package and deployment operations
- Transaction management with atomicity guarantees
- Progress reporting and cancellation support

**Implementation**:
```rust
pub struct AptOstreeDaemon {
    // D-Bus interface
    connection: Option<Connection>,
    object_manager: Option<ObjectManager>,
    
    // System integration
    package_manager: PackageManager,
    
    // Transaction management
    transactions: HashMap<String, Transaction>,
    
    // D-Bus methods
    pub fn handle_install_packages(&mut self, invocation: MethodInvocation) -> Result<(), Error>;
    pub fn handle_remove_packages(&mut self, invocation: MethodInvocation) -> Result<(), Error>;
    pub fn handle_upgrade_system(&mut self, invocation: MethodInvocation) -> Result<(), Error>;
    pub fn handle_rollback(&mut self, invocation: MethodInvocation) -> Result<(), Error>;
    pub fn handle_show_status(&mut self, invocation: MethodInvocation) -> Result<(), Error>;
}
```

**Key Methods**:
- `handle_install_packages`: Handle package installation requests
- `handle_remove_packages`: Handle package removal requests
- `handle_upgrade_system`: Handle system upgrade requests
- `handle_rollback`: Handle rollback requests
- `handle_show_status`: Handle status requests

### Client (`src/main.rs`)

**Purpose**: Command-line interface for user interaction

**Key Features**:
- 100% rpm-ostree CLI compatibility
- D-Bus client communication with daemon
- Fallback to direct system calls if daemon fails
- User-friendly error messages and help

**Implementation**:
```rust
pub struct AptOstreeClient {
    // D-Bus client
    connection: Option<Connection>,
    
    // Command handling
    pub fn handle_command(&mut self, args: &[String]) -> Result<(), Error>;
    
    // D-Bus communication
    pub fn connect_to_daemon(&mut self) -> Result<(), Error>;
    pub fn call_daemon_method(&mut self, method: &str, params: &[&str]) -> Result<(), Error>;
    
    // Fallback handling
    pub fn fallback_to_direct_calls(&mut self, args: &[String]) -> Result<(), Error>;
}
```

**Key Methods**:
- `handle_command`: Handle CLI command execution
- `connect_to_daemon`: Connect to D-Bus daemon
- `call_daemon_method`: Call daemon method via D-Bus
- `fallback_to_direct_calls`: Fallback to direct system calls

## CLI Commands (100% Complete)

### Core Commands (21/21 - 100% Complete)

- **install**: Package installation with atomic commits
- **deploy**: Deployment management and switching
- **apply-live**: Live application of changes
- **cancel**: Transaction cancellation
- **cleanup**: Old deployment cleanup
- **compose**: Tree composition
- **status**: System status with rich formatting
- **upgrade**: System upgrades with automatic policies
- **rollback**: Deployment rollback
- **db**: Package database queries (diff, list, version)
- **search**: Enhanced package search
- **override**: Package overrides (replace, remove, reset, list)
- **refresh-md**: Repository metadata refresh
- **reload**: Configuration reload
- **reset**: State reset
- **rebase**: Tree switching
- **initramfs-etc**: Initramfs file management
- **usroverlay**: Transient overlayfs to /usr
- **kargs**: Kernel argument management
- **uninstall**: Package removal (alias for remove)
- **initramfs**: Initramfs management

### Command Architecture

All commands follow the same architecture:
1. **CLI Parsing**: Parse command-line arguments
2. **Daemon Communication**: Request operation via D-Bus
3. **Fallback Handling**: Use direct system calls if daemon fails
4. **Progress Reporting**: Show operation progress
5. **Result Display**: Display operation results

## Transaction System

### Transaction Lifecycle

1. **Initiation**: Client requests operation via D-Bus
2. **Validation**: Daemon validates request and creates transaction
3. **Execution**: Transaction executes with progress reporting
4. **Completion**: Transaction completes with success/failure status
5. **Cleanup**: Resources are cleaned up and state is updated

### Transaction Types

#### InstallTransaction
- **Purpose**: Install packages with atomic commits
- **Operations**: Package download, installation, dependency resolution, OSTree commit creation
- **Rollback**: Remove installed packages and restore previous state

#### RemoveTransaction
- **Purpose**: Remove packages with rollback support
- **Operations**: Package removal, dependency cleanup, OSTree commit creation
- **Rollback**: Restore removed packages

#### UpgradeTransaction
- **Purpose**: Upgrade system to latest version
- **Operations**: Check for updates, download packages, install updates, OSTree commit creation
- **Rollback**: Revert to previous system version

#### RollbackTransaction
- **Purpose**: Rollback to previous deployment
- **Operations**: Switch to previous deployment, update boot configuration
- **Rollback**: Switch back to current deployment

### Atomic Operations

**Principle**: All operations are atomic - they either complete entirely or not at all.

**Implementation**:
- Transaction-based execution
- Rollback points at each stage
- State preservation during operations
- Cleanup on failure

**Example Flow**:
```rust
// Start transaction
let transaction_id = start_transaction("install")?;

// Create rollback points
create_rollback_point("pre-package-download")?;
create_rollback_point("pre-package-install")?;
create_rollback_point("pre-ostree-commit")?;

// Execute operations
if !download_packages(packages).await? {
    rollback_to("pre-package-download")?;
    return Err(Error::PackageDownloadFailed);
}

if !install_packages(packages).await? {
    rollback_to("pre-package-install")?;
    return Err(Error::PackageInstallFailed);
}

if !create_ostree_commit().await? {
    rollback_to("pre-ostree-commit")?;
    return Err(Error::OstreeCommitFailed);
}

// Commit transaction
commit_transaction(transaction_id)?;
```

## Security Model

### Privilege Separation

**Principle**: Separate privileged operations from unprivileged user operations.

**Implementation**:
- Daemon runs with elevated privileges
- Client operations are unprivileged
- D-Bus communication for privileged operations
- PolicyKit integration for authentication

### Bubblewrap Sandboxing

**Principle**: Execute package scripts in controlled, isolated environments.

**Implementation**:
- Package script execution in sandboxed environment
- Namespace isolation for security
- Controlled filesystem access
- Privilege restrictions

**Sandboxing Features**:
- **Namespace Isolation**: Process, mount, network namespaces
- **Bind Mounts**: Controlled filesystem access
- **Security Controls**: Privilege restrictions
- **Atomic Context**: Script execution in atomic context

### OSTree Environment Detection

**Principle**: Ensure operations only run in valid OSTree environments.

**Implementation**:
- Multiple detection methods for reliability
- Clear error messages for non-OSTree systems
- Environment validation before operations
- User guidance for environment setup

## Performance Characteristics

### Optimization Strategies

1. **OSTree Deduplication**: Leverage OSTree's content-addressable storage
2. **Incremental Updates**: Only download and apply changes
3. **Parallel Processing**: Concurrent package operations
4. **Caching**: Cache frequently accessed data

### Resource Usage

- **Memory**: Scales with package count and complexity
- **Disk**: Optimized through OSTree deduplication
- **Network**: Minimized through delta updates
- **CPU**: Optimized through parallel processing

### Performance Metrics

- **Package Installation**: ~100-500ms per package
- **System Upgrade**: ~30-60 seconds for typical updates
- **Deployment Switch**: ~5-15 seconds
- **Rollback**: ~5-15 seconds

## Deployment Model

### OSTree Integration

**Principle**: Use OSTree for atomic, immutable filesystem management.

**Features**:
- Atomic commit-based deployments
- Rollback capability through multiple deployments
- Bootloader integration for deployment switching
- State tracking and management

### Package Layering

**Principle**: Layer user packages on top of immutable base image.

**Features**:
- Base image remains immutable
- User packages layered on top
- Clear separation of base and user content
- Atomic layer application and removal

### Boot Configuration

**Principle**: Manage boot configuration for deployment switching.

**Features**:
- GRUB/systemd-boot integration
- Kernel argument management
- Initramfs management
- Boot-time deployment selection

## Systemd Services

### Core Services

- **apt-ostreed.service**: Main daemon service with OSTree detection
- **apt-ostree-bootstatus.service**: Boot-time status logging
- **apt-ostreed-automatic.service**: Automatic system updates (planned)
- **apt-ostree-countme.service**: Usage reporting (planned)

### Service Configuration

- D-Bus service activation
- OSTree environment detection
- Automatic update policies
- Boot-time status reporting



## Future Architecture

### OCI Integration

**Planned Features**:
- Direct container image support
- OCI image integration
- Container runtime integration
- Hybrid container/host management

### Cloud Integration

**Planned Features**:
- Cloud deployment support
- Multi-cloud compatibility
- Cloud-native workflows
- Infrastructure as code integration

### Enterprise Features

**Planned Features**:
- Role-based access control (RBAC)
- Audit logging and compliance
- Enterprise policy management
- Advanced security features

### Multi-Architecture Support

**Planned Features**:
- ARM64 support
- Multi-arch package handling
- Architecture-specific optimizations
- Cross-architecture compatibility