apt-ostree/.notes/oci/apt-ostree.md

apt-ostree OCI Container Integration

Overview

apt-ostree integrates with OCI (Open Container Initiative) containers to provide container-native deployment capabilities for Debian/Ubuntu systems. This document explains how apt-ostree implements OCI container integration.

Core OCI Integration

Container Image Generation

apt-ostree can generate OCI container images from OSTree commits:

// OCI integration in src/container.rs
use std::fs;
use std::path::Path;
use serde_json::{json, Value};

pub struct OciManager;

impl OciManager {
    // Generate OCI container image from OSTree commit
    pub fn generate_oci_image(
        commit_checksum: &str,
        image_name: &str,
        image_tag: &str,
    ) -> Result<String, Box<dyn std::error::Error>> {
        // 1. Extract OSTree commit to filesystem
        let ostree_manager = OstreeManager::new()?;
        let commit_tree = ostree_manager.read_commit(commit_checksum)?;
        
        // 2. Create OCI image layers
        let layers = Self::create_oci_layers(&commit_tree)?;
        
        // 3. Generate OCI manifest
        let manifest = Self::generate_oci_manifest(&layers, image_name, image_tag)?;
        
        // 4. Create OCI image archive
        Self::create_oci_archive(&manifest, &layers, image_name, image_tag)
    }
    
    // Create OCI layers from filesystem
    fn create_oci_layers(
        filesystem_tree: &OstreeRepoFile,
    ) -> Result<Vec<Value>, Box<dyn std::error::Error>> {
        let mut layers = Vec::new();
        
        // Create layer from filesystem
        let layer_path = Self::create_filesystem_layer(filesystem_tree)?;
        
        // Calculate layer digest
        let layer_digest = Self::calculate_layer_digest(&layer_path)?;
        
        // Create layer descriptor
        let layer_desc = json!({
            "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
            "digest": layer_digest,
            "size": fs::metadata(&layer_path)?.len()
        });
        
        layers.push(layer_desc);
        Ok(layers)
    }
    
    // Generate OCI manifest
    fn generate_oci_manifest(
        layers: &[Value],
        image_name: &str,
        image_tag: &str,
    ) -> Result<Value, Box<dyn std::error::Error>> {
        let manifest = json!({
            "schemaVersion": 2,
            "config": {
                "mediaType": "application/vnd.oci.image.config.v1+json",
                "digest": "sha256:config",
                "size": 0
            },
            "layers": layers
        });
        
        Ok(manifest)
    }
    
    // Create filesystem layer
    fn create_filesystem_layer(
        filesystem_tree: &OstreeRepoFile,
    ) -> Result<std::path::PathBuf, Box<dyn std::error::Error>> {
        // Create temporary directory for layer
        let layer_path = tempfile::tempdir()?.path().to_path_buf();
        
        // Extract filesystem tree to layer
        let ostree_manager = OstreeManager::new()?;
        ostree_manager.extract_tree_to_path(filesystem_tree, &layer_path)?;
        
        // Create tar.gz archive
        let archive_path = layer_path.with_extension("tar.gz");
        Self::create_tar_gz(&layer_path, &archive_path)?;
        
        Ok(archive_path)
    }
    
    // Calculate layer digest
    fn calculate_layer_digest(layer_path: &Path) -> Result<String, Box<dyn std::error::Error>> {
        use std::io::Read;
        use sha2::{Sha256, Digest};
        
        let mut file = fs::File::open(layer_path)?;
        let mut buffer = Vec::new();
        file.read_to_end(&mut buffer)?;
        
        let mut hasher = Sha256::new();
        hasher.update(&buffer);
        let result = hasher.finalize();
        
        Ok(format!("sha256:{:x}", result))
    }
    
    // Create tar.gz archive
    fn create_tar_gz(
        source_path: &Path,
        archive_path: &Path,
    ) -> Result<(), Box<dyn std::error::Error>> {
        use std::process::Command;
        
        let output = Command::new("tar")
            .args(&["-czf", archive_path.to_str().unwrap(), "-C", source_path.to_str().unwrap(), "."])
            .output()?;
        
        if !output.status.success() {
            return Err(format!(
                "Failed to create tar.gz archive: {}",
                String::from_utf8_lossy(&output.stderr)
            ).into());
        }
        
        Ok(())
    }
}

Container Registry Integration

apt-ostree integrates with OCI container registries:

// Registry integration in src/container.rs
use std::process::Command;

impl OciManager {
    // Push OCI image to registry
    pub fn push_oci_image(
        image_path: &str,
        registry_url: &str,
        image_name: &str,
        image_tag: &str,
        username: Option<&str>,
        password: Option<&str>,
    ) -> Result<(), Box<dyn std::error::Error>> {
        let mut cmd = Command::new("skopeo");
        cmd.arg("copy");
        
        // Add credentials if provided
        if let (Some(user), Some(pass)) = (username, password) {
            cmd.args(&["--dest-creds", &format!("{}:{}", user, pass)]);
        }
        
        cmd.args(&[
            &format!("oci:{}", image_path),
            &format!("docker://{}/{}/{}:{}", registry_url, image_name, image_tag)
        ]);
        
        let output = cmd.output()?;
        
        if !output.status.success() {
            return Err(format!(
                "Failed to push OCI image: {}",
                String::from_utf8_lossy(&output.stderr)
            ).into());
        }
        
        Ok(())
    }
    
    // Pull OCI image from registry
    pub fn pull_oci_image(
        registry_url: &str,
        image_name: &str,
        image_tag: &str,
        local_path: &str,
        username: Option<&str>,
        password: Option<&str>,
    ) -> Result<(), Box<dyn std::error::Error>> {
        let mut cmd = Command::new("skopeo");
        cmd.arg("copy");
        
        // Add credentials if provided
        if let (Some(user), Some(pass)) = (username, password) {
            cmd.args(&["--src-creds", &format!("{}:{}", user, pass)]);
        }
        
        cmd.args(&[
            &format!("docker://{}/{}/{}:{}", registry_url, image_name, image_tag),
            &format!("oci:{}", local_path)
        ]);
        
        let output = cmd.output()?;
        
        if !output.status.success() {
            return Err(format!(
                "Failed to pull OCI image: {}",
                String::from_utf8_lossy(&output.stderr)
            ).into());
        }
        
        Ok(())
    }
    
    // List available images in registry
    pub fn list_registry_images(
        registry_url: &str,
        username: Option<&str>,
        password: Option<&str>,
    ) -> Result<Vec<String>, Box<dyn std::error::Error>> {
        let mut cmd = Command::new("skopeo");
        cmd.args(&["list-tags"]);
        
        // Add credentials if provided
        if let (Some(user), Some(pass)) = (username, password) {
            cmd.args(&["--creds", &format!("{}:{}", user, pass)]);
        }
        
        cmd.arg(&format!("docker://{}", registry_url));
        
        let output = cmd.output()?;
        
        if !output.status.success() {
            return Err(format!(
                "Failed to list registry images: {}",
                String::from_utf8_lossy(&output.stderr)
            ).into());
        }
        
        // Parse output to extract image names
        let output_str = String::from_utf8(output.stdout)?;
        let images: Vec<String> = output_str
            .lines()
            .filter(|line| !line.trim().is_empty())
            .map(|line| line.trim().to_string())
            .collect();
        
        Ok(images)
    }
}

Bootc Compatibility

Bootc Image Generation

apt-ostree can generate bootc-compatible images:

// Bootc integration in src/container.rs
impl OciManager {
    // Generate bootc-compatible image
    pub fn generate_bootc_image(
        commit_checksum: &str,
        image_name: &str,
        image_tag: &str,
    ) -> Result<String, Box<dyn std::error::Error>> {
        // 1. Generate OCI image
        let oci_image_path = Self::generate_oci_image(commit_checksum, image_name, image_tag)?;
        
        // 2. Add bootc-specific metadata
        Self::add_bootc_metadata(&oci_image_path, image_name, image_tag)?;
        
        Ok(oci_image_path)
    }
    
    // Add bootc-specific metadata
    fn add_bootc_metadata(
        image_path: &str,
        image_name: &str,
        image_tag: &str,
    ) -> Result<(), Box<dyn std::error::Error>> {
        // Create bootc metadata
        let bootc_metadata = json!({
            "bootc": {},
            "deployment": {
                "type": "ostree",
                "ref": "ubuntu/24.04/x86_64/desktop"
            }
        });
        
        // Write metadata to image
        Self::write_image_metadata(image_path, &bootc_metadata)
    }
    
    // Write metadata to OCI image
    fn write_image_metadata(
        image_path: &str,
        metadata: &Value,
    ) -> Result<(), Box<dyn std::error::Error>> {
        // Create metadata directory
        let metadata_path = format!("{}/blobs/sha256/metadata", image_path);
        fs::create_dir_all(&metadata_path)?;
        
        // Write metadata file
        let metadata_file = format!("{}/bootc.json", metadata_path);
        fs::write(metadata_file, serde_json::to_string_pretty(metadata)?)?;
        
        Ok(())
    }
}

mmdebstrap Integration

Base Image Creation

apt-ostree uses mmdebstrap for efficient base image creation:

// mmdebstrap integration in src/container.rs
impl OciManager {
    // Create base system image with mmdebstrap
    pub fn create_base_image(
        release: &str,
        arch: &str,
        packages: &[String],
    ) -> Result<String, Box<dyn std::error::Error>> {
        use std::process::Command;
        
        // Create temporary directory for base system
        let base_path = tempfile::tempdir()?.path().to_path_buf();
        
        let mut cmd = Command::new("mmdebstrap");
        cmd.args(&["--arch", arch, "--variant", "minbase", release, base_path.to_str().unwrap()]);
        
        // Add additional packages if specified
        if !packages.is_empty() {
            cmd.arg("--include");
            cmd.arg(&packages.join(","));
        }
        
        let output = cmd.output()?;
        
        if !output.status.success() {
            return Err(format!(
                "Failed to create base system: {}",
                String::from_utf8_lossy(&output.stderr)
            ).into());
        }
        
        // Create OCI image from base system
        let image_path = Self::create_oci_from_filesystem(&base_path, "base", "latest")?;
        
        Ok(image_path)
    }
    
    // Create OCI image from filesystem
    fn create_oci_from_filesystem(
        filesystem_path: &Path,
        image_name: &str,
        image_tag: &str,
    ) -> Result<String, Box<dyn std::error::Error>> {
        // Create OCI image structure
        let image_path = tempfile::tempdir()?.path().to_path_buf();
        let blobs_path = image_path.join("blobs").join("sha256");
        fs::create_dir_all(&blobs_path)?;
        
        // Create layer from filesystem
        let layer_path = Self::create_filesystem_layer_from_path(filesystem_path)?;
        let layer_digest = Self::calculate_layer_digest(&layer_path)?;
        
        // Copy layer to blobs directory
        let blob_path = blobs_path.join(&layer_digest[7..]); // Remove "sha256:" prefix
        fs::copy(&layer_path, &blob_path)?;
        
        // Create manifest
        let manifest = Self::generate_oci_manifest(&[json!({
            "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
            "digest": layer_digest,
            "size": fs::metadata(&blob_path)?.len()
        })], image_name, image_tag)?;
        
        // Write manifest
        let manifest_path = image_path.join("manifest.json");
        fs::write(manifest_path, serde_json::to_string_pretty(&manifest)?)?;
        
        Ok(image_path.to_string_lossy().to_string())
    }
    
    // Create filesystem layer from path
    fn create_filesystem_layer_from_path(
        filesystem_path: &Path,
    ) -> Result<std::path::PathBuf, Box<dyn std::error::Error>> {
        // Create temporary directory for layer
        let layer_path = tempfile::tempdir()?.path().to_path_buf();
        
        // Create tar.gz archive of filesystem
        let archive_path = layer_path.with_extension("tar.gz");
        Self::create_tar_gz(filesystem_path, &archive_path)?;
        
        Ok(archive_path)
    }
}

Future Enhancements

Planned Features

1. Enhanced OCI Support: Full OCI specification compliance
2. Registry Authentication: Advanced registry authentication methods
3. Image Optimization: Layer optimization and compression
4. Multi-Architecture Support: Support for multiple architectures

Integration Roadmap

• Phase 1: Basic OCI integration (🔄 In Progress)
• Phase 2: Registry integration (📋 Planned)
• Phase 3: Bootc compatibility (📋 Planned)
• Phase 4: Advanced features (📋 Planned)

Manual OCI Build Process

Prerequisites

• oci-image-tool (from Ubuntu packages)
• skopeo for image validation
• OSTree repository with test commit

Build Steps

1. Checkout OSTree Commit: Extract filesystem content from OSTree commit
2. Create OCI Image Structure: Set up OCI directory structure with blobs
3. Create Filesystem Layer: Tar and compress filesystem content
4. Update Configuration: Add layer digest to OCI config
5. Create OCI Manifest: Generate manifest with layer and config references
6. Create OCI Index: Generate index for multi-platform support
7. Validate Image: Use skopeo to validate OCI compliance

CLI Commands

# Build OCI image from OSTree commit
apt-ostree oci build --source test/oci/demo --output my-image.oci --format oci --repo /path/to/ostree/repo

# Build Docker image from OSTree commit
apt-ostree oci build --source test/oci/demo --output my-image.tar --format docker --repo /path/to/ostree/repo

# Push to registry
apt-ostree oci push /tmp/demo-image.oci localhost:5000 demo:latest

Image Features

• OCI Specification Compliance: Schema Version 2, SHA256 digests, Gzip compression
• Supported Formats: OCI directory structure, Docker tar archive
• Advanced Configuration: Custom labels, environment variables, exposed ports, volume mounts