particle-os-tools/docs/apt-layer/rpm-ostree/build-chunked-oci.md

rpm-ostree Build Chunked OCI

Overview

rpm-ostree's build chunked OCI feature allows OSTree commits to be exported as OCI container images. This enables seamless integration between rpm-ostree systems and container ecosystems, providing a bridge between immutable OS images and container technology.

OCI Integration Concepts

What is Build Chunked OCI?

Build chunked OCI converts OSTree commits into OCI container images:

• OSTree to OCI: Converts OSTree commits to OCI format
• Chunked distribution: Breaks large images into manageable chunks
• Container compatibility: Enables use in container ecosystems
• Hybrid deployment: Supports both OSTree and container deployments

Benefits

1. Container ecosystem integration: Use in Kubernetes, Docker, etc.
2. Efficient distribution: Chunked transfer reduces bandwidth usage
3. Flexible deployment: Deploy as containers or OSTree commits
4. Standard format: OCI is a widely supported standard

Architecture

Build Process

OSTree Commit
├── Filesystem Tree
├── Metadata
└── Configuration
    ↓
OCI Image
├── Layers
├── Manifest
└── Config
    ↓
Chunked Distribution
├── Chunk 1
├── Chunk 2
└── Chunk N

OCI Image Structure

oci-image/
├── blobs/
│   ├── sha256:abc123... (layer 1)
│   ├── sha256:def456... (layer 2)
│   └── sha256:ghi789... (config)
├── index.json
└── oci-layout

Building OCI Images

Basic Build

# Build OCI image from OSTree commit
rpm-ostree compose build-ostree \
  --repo=/path/to/repo \
  --output-oci=/path/to/oci-image \
  treefile.yaml

Advanced Build Options

# Build with specific options
rpm-ostree compose build-ostree \
  --repo=/path/to/repo \
  --output-oci=/path/to/oci-image \
  --oci-tag=latest \
  --oci-labels="org.example.version=1.0.0" \
  --chunk-size=10M \
  treefile.yaml

Build Configuration

# treefile.yaml with OCI configuration
ref: "fedora/x86_64/coreos/stable"
repos:
  - fedora
packages:
  - systemd
  - kernel
  - bash

# OCI configuration
oci:
  tag: latest
  labels:
    org.example.version: "1.0.0"
    org.example.description: "Fedora CoreOS stable"
  chunk-size: 10M
  compression: gzip

Chunked Distribution

Chunking Strategy

# Configure chunking
rpm-ostree compose build-ostree \
  --repo=/path/to/repo \
  --output-oci=/path/to/oci-image \
  --chunk-size=10M \
  --chunk-algorithm=content-based \
  treefile.yaml

Chunk Management

# List chunks
rpm-ostree oci chunk list /path/to/oci-image

# Verify chunks
rpm-ostree oci chunk verify /path/to/oci-image

# Merge chunks
rpm-ostree oci chunk merge /path/to/chunks /path/to/merged-image

Chunk Transfer

# Upload chunks to registry
rpm-ostree oci chunk upload \
  --registry=quay.io \
  --repository=example/myapp \
  --chunks=/path/to/chunks

# Download chunks from registry
rpm-ostree oci chunk download \
  --registry=quay.io \
  --repository=example/myapp \
  --chunks=/path/to/chunks

OCI Image Configuration

Image Labels

{
  "config": {
    "Labels": {
      "org.example.version": "1.0.0",
      "org.example.description": "Fedora CoreOS stable",
      "org.example.maintainer": "example@example.com",
      "org.example.architecture": "x86_64",
      "org.example.os": "linux"
    }
  }
}

Image History

{
  "history": [
    {
      "created": "2022-01-01T00:00:00Z",
      "created_by": "rpm-ostree compose build-ostree",
      "comment": "Initial build"
    },
    {
      "created": "2022-01-02T00:00:00Z",
      "created_by": "rpm-ostree compose build-ostree",
      "comment": "Security updates"
    }
  ]
}

Layer Configuration

{
  "layers": [
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:abc123...",
      "size": 1048576
    },
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:def456...",
      "size": 2097152
    }
  ]
}

Registry Integration

Pushing to Registry

# Push OCI image to registry
rpm-ostree oci push \
  --registry=quay.io \
  --repository=example/myapp \
  --tag=latest \
  /path/to/oci-image

# Push with authentication
rpm-ostree oci push \
  --registry=quay.io \
  --repository=example/myapp \
  --username=myuser \
  --password=mypassword \
  --tag=latest \
  /path/to/oci-image

Pulling from Registry

# Pull OCI image from registry
rpm-ostree oci pull \
  --registry=quay.io \
  --repository=example/myapp \
  --tag=latest \
  /path/to/oci-image

# Pull specific digest
rpm-ostree oci pull \
  --registry=quay.io \
  --repository=example/myapp \
  --digest=sha256:abc123... \
  /path/to/oci-image

Registry Configuration

# Configure registry
rpm-ostree oci registry config \
  --registry=quay.io \
  --insecure=false \
  --username=myuser \
  --password=mypassword

# List configured registries
rpm-ostree oci registry list

Container Runtime Integration

Docker Integration

# Import OCI image to Docker
rpm-ostree oci import-docker \
  --registry=quay.io \
  --repository=example/myapp \
  --tag=latest

# Run container from OCI image
docker run -it quay.io/example/myapp:latest /bin/bash

Podman Integration

# Import OCI image to Podman
rpm-ostree oci import-podman \
  --registry=quay.io \
  --repository=example/myapp \
  --tag=latest

# Run container from OCI image
podman run -it quay.io/example/myapp:latest /bin/bash

Kubernetes Integration

# Kubernetes deployment using OCI image
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  replicas: 3
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: myapp
        image: quay.io/example/myapp:latest
        ports:
        - containerPort: 8080

Performance Optimization

Compression

# Configure compression
rpm-ostree compose build-ostree \
  --repo=/path/to/repo \
  --output-oci=/path/to/oci-image \
  --compression=gzip \
  --compression-level=6 \
  treefile.yaml

Parallel Processing

# Enable parallel processing
rpm-ostree compose build-ostree \
  --repo=/path/to/repo \
  --output-oci=/path/to/oci-image \
  --parallel-layers=4 \
  --parallel-chunks=8 \
  treefile.yaml

Caching

# Use build cache
rpm-ostree compose build-ostree \
  --repo=/path/to/repo \
  --output-oci=/path/to/oci-image \
  --cache-dir=/path/to/cache \
  --cache-layers=true \
  treefile.yaml

Security Features

Image Signing

# Sign OCI image
rpm-ostree oci sign \
  --key=/path/to/private-key \
  --cert=/path/to/certificate \
  /path/to/oci-image

# Verify image signature
rpm-ostree oci verify \
  --key=/path/to/public-key \
  /path/to/oci-image

Content Verification

# Verify image content
rpm-ostree oci verify-content \
  --expected-digest=sha256:abc123... \
  /path/to/oci-image

# Verify layer integrity
rpm-ostree oci verify-layers \
  /path/to/oci-image

Vulnerability Scanning

# Scan for vulnerabilities
rpm-ostree oci scan \
  --scanner=clair \
  --registry=quay.io \
  --repository=example/myapp \
  --tag=latest

# Generate vulnerability report
rpm-ostree oci scan \
  --output-format=json \
  --output-file=vulnerabilities.json \
  /path/to/oci-image

Examples

Basic OCI Build

#!/bin/bash
# build-oci.sh

# Build OCI image
rpm-ostree compose build-ostree \
  --repo=./repo \
  --output-oci=./oci-image \
  --oci-tag=latest \
  treefile.yaml

# Push to registry
rpm-ostree oci push \
  --registry=quay.io \
  --repository=example/myapp \
  --tag=latest \
  ./oci-image

Multi-Architecture Build

#!/bin/bash
# build-multiarch.sh

ARCHITECTURES=("amd64" "arm64" "ppc64le")

for arch in "${ARCHITECTURES[@]}"; do
    # Build for architecture
    rpm-ostree compose build-ostree \
      --repo=./repo \
      --output-oci=./oci-image-$arch \
      --oci-tag=latest \
      --arch=$arch \
      treefile.yaml
    
    # Push to registry
    rpm-ostree oci push \
      --registry=quay.io \
      --repository=example/myapp \
      --tag=latest-$arch \
      ./oci-image-$arch
done

# Create multi-arch manifest
rpm-ostree oci manifest create \
  --registry=quay.io \
  --repository=example/myapp \
  --tag=latest \
  --architectures=amd64,arm64,ppc64le

CI/CD Integration

# GitHub Actions workflow
name: Build and Push OCI Image
on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      
      - name: Build OCI image
        run: |
          rpm-ostree compose build-ostree \
            --repo=./repo \
            --output-oci=./oci-image \
            --oci-tag=${{ github.sha }} \
            treefile.yaml
      
      - name: Push to registry
        run: |
          rpm-ostree oci push \
            --registry=quay.io \
            --repository=example/myapp \
            --tag=${{ github.sha }} \
            ./oci-image

Troubleshooting

Common Issues

Build Failures

# Debug build process
rpm-ostree compose build-ostree \
  --repo=/path/to/repo \
  --output-oci=/path/to/oci-image \
  --verbose \
  --debug \
  treefile.yaml

Registry Issues

# Test registry connection
rpm-ostree oci registry test \
  --registry=quay.io \
  --repository=example/myapp

# Check registry authentication
rpm-ostree oci registry auth \
  --registry=quay.io \
  --username=myuser \
  --password=mypassword

Performance Issues

# Monitor build performance
rpm-ostree compose build-ostree \
  --repo=/path/to/repo \
  --output-oci=/path/to/oci-image \
  --metrics-file=metrics.json \
  treefile.yaml

Debugging Tools

# Inspect OCI image
rpm-ostree oci inspect /path/to/oci-image

# List image layers
rpm-ostree oci layers /path/to/oci-image

# Extract image content
rpm-ostree oci extract /path/to/oci-image /path/to/extract

Best Practices

Image Design

1. Minimal images: Keep images as small as possible
2. Security: Follow security best practices
3. Documentation: Document image configuration
4. Testing: Test images thoroughly

Build Process

1. Reproducible builds: Ensure reproducible builds
2. Versioning: Use semantic versioning
3. Signing: Sign all images
4. Scanning: Scan for vulnerabilities

Distribution

1. Chunking: Use appropriate chunk sizes
2. Compression: Use efficient compression
3. Caching: Implement proper caching
4. Monitoring: Monitor distribution performance

---

Build chunked OCI provides a powerful bridge between rpm-ostree systems and container ecosystems, enabling flexible deployment and distribution strategies.