apt-ostree/.notes/oci/apt-ostree.md

# apt-ostree OCI Container Integration

## Overview

apt-ostree integrates with OCI (Open Container Initiative) containers to provide container-native deployment capabilities for Debian/Ubuntu systems. This document explains how apt-ostree implements OCI container integration.

## Core OCI Integration

### Container Image Generation

apt-ostree can generate OCI container images from OSTree commits:

```rust
// OCI integration in src/container.rs
use std::fs;
use std::path::Path;
use serde_json::{json, Value};

pub struct OciManager;

impl OciManager {
    // Generate OCI container image from OSTree commit
    pub fn generate_oci_image(
        commit_checksum: &str,
        image_name: &str,
        image_tag: &str,
    ) -> Result<String, Box<dyn std::error::Error>> {
        // 1. Extract OSTree commit to filesystem
        let ostree_manager = OstreeManager::new()?;
        let commit_tree = ostree_manager.read_commit(commit_checksum)?;
        
        // 2. Create OCI image layers
        let layers = Self::create_oci_layers(&commit_tree)?;
        
        // 3. Generate OCI manifest
        let manifest = Self::generate_oci_manifest(&layers, image_name, image_tag)?;
        
        // 4. Create OCI image archive
        Self::create_oci_archive(&manifest, &layers, image_name, image_tag)
    }
    
    // Create OCI layers from filesystem
    fn create_oci_layers(
        filesystem_tree: &OstreeRepoFile,
    ) -> Result<Vec<Value>, Box<dyn std::error::Error>> {
        let mut layers = Vec::new();
        
        // Create layer from filesystem
        let layer_path = Self::create_filesystem_layer(filesystem_tree)?;
        
        // Calculate layer digest
        let layer_digest = Self::calculate_layer_digest(&layer_path)?;
        
        // Create layer descriptor
        let layer_desc = json!({
            "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
            "digest": layer_digest,
            "size": fs::metadata(&layer_path)?.len()
        });
        
        layers.push(layer_desc);
        Ok(layers)
    }
    
    // Generate OCI manifest
    fn generate_oci_manifest(
        layers: &[Value],
        image_name: &str,
        image_tag: &str,
    ) -> Result<Value, Box<dyn std::error::Error>> {
        let manifest = json!({
            "schemaVersion": 2,
            "config": {
                "mediaType": "application/vnd.oci.image.config.v1+json",
                "digest": "sha256:config",
                "size": 0
            },
            "layers": layers
        });
        
        Ok(manifest)
    }
    
    // Create filesystem layer
    fn create_filesystem_layer(
        filesystem_tree: &OstreeRepoFile,
    ) -> Result<std::path::PathBuf, Box<dyn std::error::Error>> {
        // Create temporary directory for layer
        let layer_path = tempfile::tempdir()?.path().to_path_buf();
        
        // Extract filesystem tree to layer
        let ostree_manager = OstreeManager::new()?;
        ostree_manager.extract_tree_to_path(filesystem_tree, &layer_path)?;
        
        // Create tar.gz archive
        let archive_path = layer_path.with_extension("tar.gz");
        Self::create_tar_gz(&layer_path, &archive_path)?;
        
        Ok(archive_path)
    }
    
    // Calculate layer digest
    fn calculate_layer_digest(layer_path: &Path) -> Result<String, Box<dyn std::error::Error>> {
        use std::io::Read;
        use sha2::{Sha256, Digest};
        
        let mut file = fs::File::open(layer_path)?;
        let mut buffer = Vec::new();
        file.read_to_end(&mut buffer)?;
        
        let mut hasher = Sha256::new();
        hasher.update(&buffer);
        let result = hasher.finalize();
        
        Ok(format!("sha256:{:x}", result))
    }
    
    // Create tar.gz archive
    fn create_tar_gz(
        source_path: &Path,
        archive_path: &Path,
    ) -> Result<(), Box<dyn std::error::Error>> {
        use std::process::Command;
        
        let output = Command::new("tar")
            .args(&["-czf", archive_path.to_str().unwrap(), "-C", source_path.to_str().unwrap(), "."])
            .output()?;
        
        if !output.status.success() {
            return Err(format!(
                "Failed to create tar.gz archive: {}",
                String::from_utf8_lossy(&output.stderr)
            ).into());
        }
        
        Ok(())
    }
}
```

### Container Registry Integration

apt-ostree integrates with OCI container registries:

```rust
// Registry integration in src/container.rs
use std::process::Command;

impl OciManager {
    // Push OCI image to registry
    pub fn push_oci_image(
        image_path: &str,
        registry_url: &str,
        image_name: &str,
        image_tag: &str,
        username: Option<&str>,
        password: Option<&str>,
    ) -> Result<(), Box<dyn std::error::Error>> {
        let mut cmd = Command::new("skopeo");
        cmd.arg("copy");
        
        // Add credentials if provided
        if let (Some(user), Some(pass)) = (username, password) {
            cmd.args(&["--dest-creds", &format!("{}:{}", user, pass)]);
        }
        
        cmd.args(&[
            &format!("oci:{}", image_path),
            &format!("docker://{}/{}/{}:{}", registry_url, image_name, image_tag)
        ]);
        
        let output = cmd.output()?;
        
        if !output.status.success() {
            return Err(format!(
                "Failed to push OCI image: {}",
                String::from_utf8_lossy(&output.stderr)
            ).into());
        }
        
        Ok(())
    }
    
    // Pull OCI image from registry
    pub fn pull_oci_image(
        registry_url: &str,
        image_name: &str,
        image_tag: &str,
        local_path: &str,
        username: Option<&str>,
        password: Option<&str>,
    ) -> Result<(), Box<dyn std::error::Error>> {
        let mut cmd = Command::new("skopeo");
        cmd.arg("copy");
        
        // Add credentials if provided
        if let (Some(user), Some(pass)) = (username, password) {
            cmd.args(&["--src-creds", &format!("{}:{}", user, pass)]);
        }
        
        cmd.args(&[
            &format!("docker://{}/{}/{}:{}", registry_url, image_name, image_tag),
            &format!("oci:{}", local_path)
        ]);
        
        let output = cmd.output()?;
        
        if !output.status.success() {
            return Err(format!(
                "Failed to pull OCI image: {}",
                String::from_utf8_lossy(&output.stderr)
            ).into());
        }
        
        Ok(())
    }
    
    // List available images in registry
    pub fn list_registry_images(
        registry_url: &str,
        username: Option<&str>,
        password: Option<&str>,
    ) -> Result<Vec<String>, Box<dyn std::error::Error>> {
        let mut cmd = Command::new("skopeo");
        cmd.args(&["list-tags"]);
        
        // Add credentials if provided
        if let (Some(user), Some(pass)) = (username, password) {
            cmd.args(&["--creds", &format!("{}:{}", user, pass)]);
        }
        
        cmd.arg(&format!("docker://{}", registry_url));
        
        let output = cmd.output()?;
        
        if !output.status.success() {
            return Err(format!(
                "Failed to list registry images: {}",
                String::from_utf8_lossy(&output.stderr)
            ).into());
        }
        
        // Parse output to extract image names
        let output_str = String::from_utf8(output.stdout)?;
        let images: Vec<String> = output_str
            .lines()
            .filter(|line| !line.trim().is_empty())
            .map(|line| line.trim().to_string())
            .collect();
        
        Ok(images)
    }
}
```

## Bootc Compatibility

### Bootc Image Generation

apt-ostree can generate bootc-compatible images:

```rust
// Bootc integration in src/container.rs
impl OciManager {
    // Generate bootc-compatible image
    pub fn generate_bootc_image(
        commit_checksum: &str,
        image_name: &str,
        image_tag: &str,
    ) -> Result<String, Box<dyn std::error::Error>> {
        // 1. Generate OCI image
        let oci_image_path = Self::generate_oci_image(commit_checksum, image_name, image_tag)?;
        
        // 2. Add bootc-specific metadata
        Self::add_bootc_metadata(&oci_image_path, image_name, image_tag)?;
        
        Ok(oci_image_path)
    }
    
    // Add bootc-specific metadata
    fn add_bootc_metadata(
        image_path: &str,
        image_name: &str,
        image_tag: &str,
    ) -> Result<(), Box<dyn std::error::Error>> {
        // Create bootc metadata
        let bootc_metadata = json!({
            "bootc": {},
            "deployment": {
                "type": "ostree",
                "ref": "ubuntu/24.04/x86_64/desktop"
            }
        });
        
        // Write metadata to image
        Self::write_image_metadata(image_path, &bootc_metadata)
    }
    
    // Write metadata to OCI image
    fn write_image_metadata(
        image_path: &str,
        metadata: &Value,
    ) -> Result<(), Box<dyn std::error::Error>> {
        // Create metadata directory
        let metadata_path = format!("{}/blobs/sha256/metadata", image_path);
        fs::create_dir_all(&metadata_path)?;
        
        // Write metadata file
        let metadata_file = format!("{}/bootc.json", metadata_path);
        fs::write(metadata_file, serde_json::to_string_pretty(metadata)?)?;
        
        Ok(())
    }
}
```

## mmdebstrap Integration

### Base Image Creation

apt-ostree uses mmdebstrap for efficient base image creation:

```rust
// mmdebstrap integration in src/container.rs
impl OciManager {
    // Create base system image with mmdebstrap
    pub fn create_base_image(
        release: &str,
        arch: &str,
        packages: &[String],
    ) -> Result<String, Box<dyn std::error::Error>> {
        use std::process::Command;
        
        // Create temporary directory for base system
        let base_path = tempfile::tempdir()?.path().to_path_buf();
        
        let mut cmd = Command::new("mmdebstrap");
        cmd.args(&["--arch", arch, "--variant", "minbase", release, base_path.to_str().unwrap()]);
        
        // Add additional packages if specified
        if !packages.is_empty() {
            cmd.arg("--include");
            cmd.arg(&packages.join(","));
        }
        
        let output = cmd.output()?;
        
        if !output.status.success() {
            return Err(format!(
                "Failed to create base system: {}",
                String::from_utf8_lossy(&output.stderr)
            ).into());
        }
        
        // Create OCI image from base system
        let image_path = Self::create_oci_from_filesystem(&base_path, "base", "latest")?;
        
        Ok(image_path)
    }
    
    // Create OCI image from filesystem
    fn create_oci_from_filesystem(
        filesystem_path: &Path,
        image_name: &str,
        image_tag: &str,
    ) -> Result<String, Box<dyn std::error::Error>> {
        // Create OCI image structure
        let image_path = tempfile::tempdir()?.path().to_path_buf();
        let blobs_path = image_path.join("blobs").join("sha256");
        fs::create_dir_all(&blobs_path)?;
        
        // Create layer from filesystem
        let layer_path = Self::create_filesystem_layer_from_path(filesystem_path)?;
        let layer_digest = Self::calculate_layer_digest(&layer_path)?;
        
        // Copy layer to blobs directory
        let blob_path = blobs_path.join(&layer_digest[7..]); // Remove "sha256:" prefix
        fs::copy(&layer_path, &blob_path)?;
        
        // Create manifest
        let manifest = Self::generate_oci_manifest(&[json!({
            "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
            "digest": layer_digest,
            "size": fs::metadata(&blob_path)?.len()
        })], image_name, image_tag)?;
        
        // Write manifest
        let manifest_path = image_path.join("manifest.json");
        fs::write(manifest_path, serde_json::to_string_pretty(&manifest)?)?;
        
        Ok(image_path.to_string_lossy().to_string())
    }
    
    // Create filesystem layer from path
    fn create_filesystem_layer_from_path(
        filesystem_path: &Path,
    ) -> Result<std::path::PathBuf, Box<dyn std::error::Error>> {
        // Create temporary directory for layer
        let layer_path = tempfile::tempdir()?.path().to_path_buf();
        
        // Create tar.gz archive of filesystem
        let archive_path = layer_path.with_extension("tar.gz");
        Self::create_tar_gz(filesystem_path, &archive_path)?;
        
        Ok(archive_path)
    }
}
```

## Future Enhancements

### Planned Features

1. **Enhanced OCI Support**: Full OCI specification compliance
2. **Registry Authentication**: Advanced registry authentication methods
3. **Image Optimization**: Layer optimization and compression
4. **Multi-Architecture Support**: Support for multiple architectures

### Integration Roadmap

- **Phase 1**: Basic OCI integration (🔄 In Progress)
- **Phase 2**: Registry integration (📋 Planned)
- **Phase 3**: Bootc compatibility (📋 Planned)
- **Phase 4**: Advanced features (📋 Planned)

## Manual OCI Build Process

### Prerequisites
- `oci-image-tool` (from Ubuntu packages)
- `skopeo` for image validation
- OSTree repository with test commit

### Build Steps
1. **Checkout OSTree Commit**: Extract filesystem content from OSTree commit
2. **Create OCI Image Structure**: Set up OCI directory structure with blobs
3. **Create Filesystem Layer**: Tar and compress filesystem content
4. **Update Configuration**: Add layer digest to OCI config
5. **Create OCI Manifest**: Generate manifest with layer and config references
6. **Create OCI Index**: Generate index for multi-platform support
7. **Validate Image**: Use skopeo to validate OCI compliance

### CLI Commands
```bash
# Build OCI image from OSTree commit
apt-ostree oci build --source test/oci/demo --output my-image.oci --format oci --repo /path/to/ostree/repo

# Build Docker image from OSTree commit
apt-ostree oci build --source test/oci/demo --output my-image.tar --format docker --repo /path/to/ostree/repo

# Push to registry
apt-ostree oci push /tmp/demo-image.oci localhost:5000 demo:latest
```

### Image Features
- **OCI Specification Compliance**: Schema Version 2, SHA256 digests, Gzip compression
- **Supported Formats**: OCI directory structure, Docker tar archive
- **Advanced Configuration**: Custom labels, environment variables, exposed ports, volume mounts