apt-ostree/docs/apt-ostree-daemon-plan/implementation/deployment-guide.md

🚀 apt-ostree Deployment Guide

🎯 Overview

This document provides comprehensive guidance for deploying apt-ostree in production environments, including system requirements, installation, configuration, monitoring, and maintenance procedures. The deployment approach ensures reliability, security, and optimal performance.

📋 System Requirements

Hardware Requirements

Minimum Requirements

• CPU: 2 cores (x86_64/amd64)
• RAM: 4GB
• Storage: 20GB available space
• Network: Internet connectivity for package updates

Recommended Requirements

• CPU: 4+ cores (x86_64/amd64)
• RAM: 8GB+
• Storage: 50GB+ available space
• Network: High-speed internet connection
• SSD: For optimal performance

Software Requirements

Operating System

• Debian: 13 (Trixie) or later
• Ubuntu: 25.04 (Plucky Puffin) or later
• Kernel: Linux 6.12+ with OSTree support

System Dependencies

# Essential system packages
sudo apt update
sudo apt install -y \
    systemd \
    dbus \
    polkit \
    ostree \
    apt \
    dpkg \
    systemd-boot \
    dracut

# Development tools (for building from source)
sudo apt install -y \
    build-essential \
    pkg-config \
    cmake \
    git \
    curl

🏗️ Installation Methods

Method 1: Package Installation (Recommended)

Add Repository

# Add apt-ostree repository
echo "deb [signed-by=/usr/share/keyrings/apt-ostree-archive-keyring.gpg] \
    https://apt.ostree.dev/debian trixie main" | \
    sudo tee /etc/apt/sources.list.d/apt-ostree.list

# Add repository key
curl -fsSL https://apt.ostree.dev/debian/apt-ostree-archive-keyring.gpg | \
    sudo gpg --dearmor -o /usr/share/keyrings/apt-ostree-archive-keyring.gpg

# Update package lists
sudo apt update

Install apt-ostree

# Install apt-ostree
sudo apt install -y apt-ostree

# Verify installation
apt-ostree --version

# Check system status
apt-ostree status

Method 2: Build from Source

Clone Repository

# Clone the repository
git clone https://github.com/your-org/apt-ostree.git
cd apt-ostree

# Checkout stable release
git checkout v1.0.0

Build and Install

# Install Rust toolchain
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.cargo/env

# Build the project
cargo build --release

# Install system-wide
sudo cp target/release/apt-ostree /usr/local/bin/
sudo cp target/release/aptostreed /usr/local/bin/

# Install systemd services
sudo cp systemd/aptostreed.service /etc/systemd/system/
sudo cp systemd/aptostreed.socket /etc/systemd/system/

# Reload systemd
sudo systemctl daemon-reload

⚙️ Configuration

Main Configuration File

Configuration Location

# Main configuration file
sudo mkdir -p /etc/apt-ostree
sudo nano /etc/apt-ostree/config.toml

Configuration Example

# /etc/apt-ostree/config.toml
[system]
# System-wide settings
ostree_path = "/ostree"
sysroot_path = "/"
default_os = "debian"

[daemon]
# Daemon configuration
host = "127.0.0.1"
port = 8080
max_connections = 100
idle_timeout = 300
log_level = "info"

[security]
# Security settings
polkit_enabled = true
require_authorization = true
allowed_users = ["root", "admin"]
allowed_groups = ["sudo", "wheel"]

[packages]
# Package management settings
auto_update = false
update_interval = 86400
cache_dir = "/var/cache/apt-ostree"
max_cache_size = "10GB"

[ostree]
# OSTree settings
repo_mode = "bare-user"
compression = "gzip"
commit_timeout = 300
max_commits = 100

[logging]
# Logging configuration
log_file = "/var/log/apt-ostree/aptostreed.log"
max_log_size = "100MB"
max_log_files = 5
log_format = "json"

[monitoring]
# Monitoring settings
metrics_enabled = true
metrics_port = 9090
health_check_interval = 60

Environment-Specific Configuration

Development Environment

# /etc/apt-ostree/config.dev.toml
[system]
ostree_path = "/tmp/ostree-dev"
sysroot_path = "/tmp/sysroot-dev"

[daemon]
log_level = "debug"
host = "0.0.0.0"

[security]
polkit_enabled = false
require_authorization = false

[packages]
auto_update = true
update_interval = 3600

Production Environment

# /etc/apt-ostree/config.prod.toml
[system]
ostree_path = "/ostree"
sysroot_path = "/"

[daemon]
log_level = "warn"
host = "127.0.0.1"

[security]
polkit_enabled = true
require_authorization = true

[packages]
auto_update = false
update_interval = 86400

[monitoring]
metrics_enabled = true
health_check_interval = 30

Polkit Configuration

Policy Files

<!-- /usr/share/polkit-1/actions/org.projectatomic.aptostree.policy -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE policyconfig PUBLIC
 "-//freedesktop//DTD PolicyKit Policy Configuration 1.0//EN"
 "http://www.freedesktop.org/standards/PolicyKit/1/policyconfig.dtd">
<policyconfig>
  <vendor>Project Atomic</vendor>
  <vendor_url>https://github.com/projectatomic/apt-ostree</vendor_url>
  
  <action id="org.projectatomic.aptostree.status">
    <description>Get system status</description>
    <message>Authentication is required to get system status</message>
    <defaults>
      <allow_any>yes</allow_any>
      <allow_inactive>yes</allow_inactive>
      <allow_active>yes</allow_active>
    </defaults>
  </action>
  
  <action id="org.projectatomic.aptostree.upgrade">
    <description>Upgrade system</description>
    <message>Authentication is required to upgrade the system</message>
    <defaults>
      <allow_any>auth_admin</allow_any>
      <allow_inactive>auth_admin</allow_inactive>
      <allow_active>auth_admin</allow_active>
    </defaults>
  </action>
  
  <action id="org.projectatomic.aptostree.install">
    <description>Install packages</description>
    <message>Authentication is required to install packages</message>
    <defaults>
      <allow_any>auth_admin</allow_any>
      <allow_inactive>auth_admin</allow_inactive>
      <allow_active>auth_admin</allow_active>
    </defaults>
  </action>
</policyconfig>

Rules Configuration

// /etc/polkit-1/rules.d/50-apt-ostree.rules
polkit.addRule(function(action, subject) {
    if (action.id == "org.projectatomic.aptostree.status") {
        return polkit.Result.YES;
    }
    
    if (action.id == "org.projectatomic.aptostree.upgrade" ||
        action.id == "org.projectatomic.aptostree.install") {
        if (subject.isInGroup("sudo") || subject.isInGroup("wheel")) {
            return polkit.Result.YES;
        }
    }
    
    return polkit.Result.NO;
});

🚀 Service Management

Systemd Service Configuration

Daemon Service

# /etc/systemd/system/aptostreed.service
[Unit]
Description=apt-ostree Daemon
Documentation=man:aptostreed(8)
After=network.target ostree.service
Requires=ostree.service
Wants=polkit.service

[Service]
Type=notify
ExecStart=/usr/bin/aptostreed --config /etc/apt-ostree/config.toml
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=5
User=root
Group=root
RuntimeDirectory=apt-ostree
RuntimeDirectoryMode=0755
StateDirectory=apt-ostree
StateDirectoryMode=0755
LogsDirectory=apt-ostree
LogsDirectoryMode=0755

# Security settings
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/ostree /var/lib/apt-ostree /var/cache/apt-ostree
CapabilityBoundingSet=CAP_SYS_ADMIN CAP_DAC_OVERRIDE

[Install]
WantedBy=multi-user.target

Socket Configuration

# /etc/systemd/system/aptostreed.socket
[Unit]
Description=apt-ostree Daemon Socket
Documentation=man:aptostreed(8)
Before=aptostreed.service

[Socket]
ListenStream=127.0.0.1:8080
SocketUser=root
SocketGroup=root
SocketMode=0660

[Install]
WantedBy=sockets.target

Service Management Commands

Start and Enable Services

# Enable and start services
sudo systemctl enable aptostreed.socket
sudo systemctl enable aptostreed.service
sudo systemctl start aptostreed.socket
sudo systemctl start aptostreed.service

# Check service status
sudo systemctl status aptostreed.service
sudo systemctl status aptostreed.socket

# View service logs
sudo journalctl -u aptostreed.service -f

Service Control

# Restart services
sudo systemctl restart aptostreed.service

# Reload configuration
sudo systemctl reload aptostreed.service

# Stop services
sudo systemctl stop aptostreed.service
sudo systemctl stop aptostreed.socket

# Disable services
sudo systemctl disable aptostreed.service
sudo systemctl disable aptostreed.socket

🔒 Security Configuration

User and Group Management

Create Dedicated User

# Create apt-ostree user
sudo useradd -r -s /bin/false -d /var/lib/apt-ostree aptostree

# Create necessary directories
sudo mkdir -p /var/lib/apt-ostree
sudo mkdir -p /var/cache/apt-ostree
sudo mkdir -p /var/log/apt-ostree

# Set ownership
sudo chown -R aptostree:aptostree /var/lib/apt-ostree
sudo chown -R aptostree:aptostree /var/cache/apt-ostree
sudo chown -R aptostree:aptostree /var/log/apt-ostree

# Set permissions
sudo chmod 750 /var/lib/apt-ostree
sudo chmod 750 /var/cache/apt-ostree
sudo chmod 750 /var/log/apt-ostree

Configure sudo Access

# Add user to sudo group
sudo usermod -a -G sudo aptostree

# Configure sudoers
echo "aptostree ALL=(ALL) NOPASSWD: /usr/bin/apt-ostree" | \
    sudo tee /etc/sudoers.d/apt-ostree

# Verify configuration
sudo visudo -c

Network Security

Firewall Configuration

# Configure UFW firewall
sudo ufw allow from 127.0.0.1 to any port 8080
sudo ufw allow from 192.168.1.0/24 to any port 8080

# Enable firewall
sudo ufw enable

# Check firewall status
sudo ufw status

Network Isolation

# Create network namespace
sudo ip netns add aptostree

# Move interface to namespace
sudo ip link set eth0 netns aptostree

# Configure namespace networking
sudo ip netns exec aptostree ip addr add 192.168.100.1/24 dev eth0
sudo ip netns exec aptostree ip link set eth0 up

📊 Monitoring and Logging

Logging Configuration

Log Rotation

# /etc/logrotate.d/apt-ostree
/var/log/apt-ostree/*.log {
    daily
    missingok
    rotate 7
    compress
    delaycompress
    notifempty
    create 644 aptostree aptostree
    postrotate
        systemctl reload aptostreed.service
    endscript
}

Structured Logging

// Example log entry
{
    "timestamp": "2024-01-15T10:30:00Z",
    "level": "info",
    "component": "daemon",
    "operation": "package_install",
    "user_id": 1000,
    "package": "vim",
    "version": "2:9.0.1378-1",
    "duration_ms": 1500,
    "success": true,
    "message": "Package vim installed successfully"
}

Metrics Collection

Prometheus Metrics

// Metrics endpoint configuration
use prometheus::{Registry, Counter, Histogram, Gauge};

pub struct Metrics {
    pub operations_total: Counter,
    pub operation_duration: Histogram,
    pub active_connections: Gauge,
    pub package_installations: Counter,
}

impl Metrics {
    pub fn new(registry: &Registry) -> Self {
        let operations_total = Counter::new(
            "apt_ostree_operations_total",
            "Total number of operations"
        ).unwrap();
        
        let operation_duration = Histogram::new(
            "apt_ostree_operation_duration_seconds",
            "Operation duration in seconds"
        ).unwrap();
        
        let active_connections = Gauge::new(
            "apt_ostree_active_connections",
            "Number of active connections"
        ).unwrap();
        
        let package_installations = Counter::new(
            "apt_ostree_package_installations_total",
            "Total number of package installations"
        ).unwrap();
        
        registry.register(Box::new(operations_total.clone())).unwrap();
        registry.register(Box::new(operation_duration.clone())).unwrap();
        registry.register(Box::new(active_connections.clone())).unwrap();
        registry.register(Box::new(package_installations.clone())).unwrap();
        
        Self {
            operations_total,
            operation_duration,
            active_connections,
            package_installations,
        }
    }
}

Health Check Endpoint

// Health check implementation
use axum::{routing::get, Router, Json};
use serde_json::json;

pub async fn health_check() -> Json<serde_json::Value> {
    let health_status = check_system_health().await;
    
    Json(json!({
        "status": if health_status.is_healthy { "healthy" } else { "unhealthy" },
        "timestamp": chrono::Utc::now().to_rfc3339(),
        "version": env!("CARGO_PKG_VERSION"),
        "uptime": get_system_uptime(),
        "components": {
            "ostree": health_status.ostree_healthy,
            "apt": health_status.apt_healthy,
            "database": health_status.database_healthy,
            "filesystem": health_status.filesystem_healthy
        },
        "details": health_status.details
    }))
}

pub fn create_health_router() -> Router {
    Router::new()
        .route("/health", get(health_check))
        .route("/ready", get(health_check))
}

🔧 Maintenance Procedures

Regular Maintenance Tasks

Daily Tasks

# Check service status
sudo systemctl status aptostreed.service

# Check log files
sudo tail -f /var/log/apt-ostree/aptostreed.log

# Check disk space
df -h /ostree /var/cache/apt-ostree

# Check system resources
htop

Weekly Tasks

# Clean old packages
sudo apt-ostree cleanup

# Update package lists
sudo apt-ostree refresh-md

# Check for updates
sudo apt-ostree upgrade --preview

# Rotate log files
sudo logrotate -f /etc/logrotate.d/apt-ostree

Monthly Tasks

# Full system health check
sudo apt-ostree status --verbose

# Check OSTree repository health
sudo ostree fsck

# Review and clean old deployments
sudo apt-ostree cleanup --old-deployments

# Update system packages
sudo apt-ostree upgrade

Backup and Recovery

Backup Procedures

# Backup configuration
sudo tar -czf /backup/apt-ostree-config-$(date +%Y%m%d).tar.gz \
    /etc/apt-ostree /etc/systemd/system/aptostreed*

# Backup OSTree repository
sudo ostree admin backup --repo=/ostree/repo /backup/ostree-backup-$(date +%Y%m%d)

# Backup package cache
sudo tar -czf /backup/apt-cache-$(date +%Y%m%d).tar.gz /var/cache/apt-ostree

# Backup logs
sudo tar -czf /backup/apt-ostree-logs-$(date +%Y%m%d).tar.gz /var/log/apt-ostree

Recovery Procedures

# Restore configuration
sudo tar -xzf /backup/apt-ostree-config-$(date +%Y%m%d).tar.gz -C /

# Restore OSTree repository
sudo ostree admin restore --repo=/ostree/repo /backup/ostree-backup-$(date +%Y%m%d)

# Restart services
sudo systemctl restart aptostreed.service

# Verify recovery
sudo apt-ostree status

🚨 Troubleshooting

Common Issues

Service Won't Start

# Check service status
sudo systemctl status aptostreed.service

# Check logs
sudo journalctl -u aptostreed.service -n 50

# Check configuration
sudo aptostreed --config /etc/apt-ostree/config.toml --validate

# Check dependencies
sudo systemctl status ostree.service
sudo systemctl status dbus.service

Permission Issues

# Check file permissions
ls -la /ostree /var/lib/apt-ostree /var/cache/apt-ostree

# Check user/group ownership
id aptostree
groups aptostree

# Fix permissions
sudo chown -R aptostree:aptostree /ostree
sudo chown -R aptostree:aptostree /var/lib/apt-ostree
sudo chown -R aptostree:aptostree /var/cache/apt-ostree

Network Issues

# Check network connectivity
ping -c 3 8.8.8.8

# Check service binding
sudo netstat -tlnp | grep 8080

# Check firewall
sudo ufw status

# Test local connectivity
curl -v http://127.0.0.1:8080/health

Debug Mode

Enable Debug Logging

# Edit configuration
sudo nano /etc/apt-ostree/config.toml

# Set log level to debug
log_level = "debug"

# Restart service
sudo systemctl restart aptostreed.service

# Monitor logs
sudo journalctl -u aptostreed.service -f

Command Line Debugging

# Run daemon in foreground with debug
sudo aptostreed --config /etc/apt-ostree/config.toml --debug --foreground

# Test CLI commands with verbose output
apt-ostree --debug status --verbose

# Check system information
apt-ostree status --json

📈 Performance Optimization

System Tuning

Kernel Parameters

# /etc/sysctl.conf
# Increase file descriptor limits
fs.file-max = 1000000

# Optimize memory management
vm.swappiness = 10
vm.dirty_ratio = 15
vm.dirty_background_ratio = 5

# Network tuning
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 5000

# Apply changes
sudo sysctl -p

Resource Limits

# /etc/security/limits.conf
aptostree soft nofile 65536
aptostree hard nofile 65536
aptostree soft nproc 32768
aptostree hard nproc 32768

Application Optimization

Connection Pooling

// Connection pool configuration
use deadpool::managed::{Manager, Pool, PoolError};

pub struct ConnectionPool {
    pool: Pool<ConnectionManager>,
}

impl ConnectionPool {
    pub async fn new(max_connections: usize) -> Result<Self, PoolError> {
        let manager = ConnectionManager::new();
        let pool = Pool::builder(manager)
            .max_size(max_connections)
            .build()?;
        
        Ok(Self { pool })
    }
    
    pub async fn get_connection(&self) -> Result<PooledConnection, PoolError> {
        self.pool.get().await
    }
}

Caching Strategy

// Cache configuration
use moka::future::Cache;

pub struct CacheManager {
    package_cache: Cache<String, PackageInfo>,
    deployment_cache: Cache<String, DeploymentInfo>,
}

impl CacheManager {
    pub fn new() -> Self {
        let package_cache = Cache::builder()
            .max_capacity(10000)
            .time_to_live(Duration::from_secs(3600))
            .build();
            
        let deployment_cache = Cache::builder()
            .max_capacity(1000)
            .time_to_live(Duration::from_secs(1800))
            .build();
        
        Self {
            package_cache,
            deployment_cache,
        }
    }
}

🎯 Next Steps

Immediate Actions

1. Review system requirements and ensure compatibility
2. Choose installation method (package vs source)
3. Configure basic settings and security policies
4. Test basic functionality and verify installation

Short-term Goals

1. Set up monitoring and alerting systems
2. Configure backup and recovery procedures
3. Implement performance monitoring and optimization
4. Establish maintenance schedules and procedures

Long-term Vision

1. Automated deployment and configuration management
2. Multi-site deployment and load balancing
3. Advanced monitoring and predictive maintenance
4. Integration with existing infrastructure management tools

---

This deployment guide provides comprehensive instructions for deploying apt-ostree in production environments. For detailed architecture information, refer to the architecture documents in the docs/apt-ostree-daemon-plan/architecture/ directory.