From 8b00e29a58ab8461c27d5ce305161099af96b751 Mon Sep 17 00:00:00 2001 From: robojerk Date: Tue, 15 Jul 2025 11:27:42 -0700 Subject: [PATCH] Documentation: Add comprehensive skopeo integration guide mirroring rpm-ostree approach --- docs/apt-layer/skopeo.md | 422 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 422 insertions(+) diff --git a/docs/apt-layer/skopeo.md b/docs/apt-layer/skopeo.md index e69de29..78b6ea5 100644 --- a/docs/apt-layer/skopeo.md +++ b/docs/apt-layer/skopeo.md @@ -0,0 +1,422 @@ +# Skopeo Integration in apt-layer + +## TLDR - Quick Reference + +### Basic Commands + +**Inspect a container image:** +```sh +skopeo inspect docker://ubuntu:24.04 +``` + +**Copy image between registries:** +```sh +skopeo copy docker://ubuntu:24.04 docker://myregistry/ubuntu:24.04 +``` + +**Copy image to local directory:** +```sh +skopeo copy docker://ubuntu:24.04 dir:/path/to/local/directory +``` + +**Copy local directory to registry:** +```sh +skopeo copy dir:/path/to/local/directory docker://myregistry/myimage:latest +``` + +### Quick Example +```sh +# Import OCI image to apt-layer +apt-layer --oci-import ubuntu:24.04 my-base/24.04 + +# Export apt-layer image to OCI +apt-layer --oci-export my-gaming/24.04 myregistry/gaming:latest + +# Inspect image before import +skopeo inspect docker://ubuntu:24.04 +``` + +--- + +## Overview + +apt-layer uses [skopeo](https://github.com/containers/skopeo) for OCI (Open Container Initiative) container image operations, mirroring the approach used by rpm-ostree. Skopeo is a command line utility that performs various operations on container images and image repositories without requiring the user to run a container daemon. + +**Key Role:** Skopeo serves as the primary OCI tool in apt-layer for: +- Container image inspection and validation +- Image copying between registries and local storage +- Image format conversion (OCI ↔ ComposeFS) +- Registry authentication and signature verification + +--- + +## Package Structure + +### Debian/Ubuntu Package + +**Package Name:** `skopeo` +**Purpose:** OCI container image operations +**Contains:** +- `/usr/bin/skopeo` - Main skopeo executable +- `/usr/share/man/man1/skopeo.1.gz` - Manual page +- `/usr/share/doc/skopeo/` - Documentation + +**Dependencies:** +- `libgpgme11` - GPG Made Easy library +- `libostree-1-1` - OSTree library +- `containers-common` - Container utilities + +### Installation + +**Debian/Ubuntu:** +```sh +sudo apt install -y skopeo +``` + +**Fedora/RHEL:** +```sh +sudo dnf install -y skopeo +``` + +--- + +## Skopeo Usage in apt-layer + +### 1. OCI Tool Priority + +apt-layer uses a priority-based approach for OCI operations: + +1. **skopeo** (preferred) - For OCI operations only +2. **podman** (fallback) - For container runtime operations +3. **docker** (alternative) - For container runtime operations + +```bash +# apt-layer automatically detects and uses skopeo when available +if command -v skopeo &> /dev/null; then + OCI_TOOL="skopeo" + log_info "Using skopeo for OCI operations" "apt-layer" +elif command -v podman &> /dev/null; then + OCI_TOOL="podman" + log_info "Using podman for OCI operations" "apt-layer" +else + OCI_TOOL="docker" + log_info "Using docker for OCI operations" "apt-layer" +fi +``` + +### 2. Image Import Operations + +**Import OCI image as ComposeFS:** +```bash +# apt-layer command +apt-layer --oci-import ubuntu:24.04 my-base/24.04 + +# Underlying skopeo operation +skopeo copy docker://ubuntu:24.04 dir:/tmp/oci-import-12345 +``` + +**Process:** +1. `skopeo copy` downloads the OCI image to a local directory +2. apt-layer extracts the filesystem from the OCI structure +3. apt-layer creates a ComposeFS image from the extracted filesystem +4. The ComposeFS image is stored in apt-layer's image workspace + +### 3. Image Export Operations + +**Export ComposeFS image to OCI:** +```bash +# apt-layer command +apt-layer --oci-export my-gaming/24.04 myregistry/gaming:latest + +# Underlying skopeo operation +skopeo copy dir:/tmp/oci-export-12345 docker://myregistry/gaming:latest +``` + +**Process:** +1. apt-layer mounts the ComposeFS image +2. apt-layer creates an OCI directory structure from the mounted filesystem +3. `skopeo copy` uploads the OCI directory to the registry +4. The image is available in the container registry + +### 4. Image Inspection + +**Inspect container images:** +```bash +# Direct skopeo usage +skopeo inspect docker://ubuntu:24.04 + +# apt-layer integration +apt-layer --oci-info ubuntu:24.04 +``` + +**Returns:** +- Image metadata (layers, architecture, OS) +- Labels and annotations +- Creation date and size information +- Digest and signature information + +### 5. Registry Authentication + +**Authentication with registries:** +```bash +# Login to registry (handled by podman/docker) +podman login myregistry.com + +# skopeo uses the same authentication +skopeo copy docker://myregistry.com/image:tag dir:/local/path +``` + +--- + +## Skopeo vs Container Runtimes + +### Skopeo (OCI Operations Only) + +**Use Cases:** +- Image inspection and validation +- Image copying between registries +- Image format conversion +- Signature verification +- Registry operations without running containers + +**Limitations:** +- Cannot run containers +- Cannot build images +- Limited to OCI operations + +### Container Runtimes (podman/docker) + +**Use Cases:** +- Running containers +- Building images +- Container lifecycle management +- Interactive development + +**Integration:** +- apt-layer uses container runtimes for package installation +- skopeo handles OCI operations +- Both work together in the apt-layer ecosystem + +--- + +## OCI Integration Workflow + +### 1. Import Workflow + +```bash +# Step 1: Inspect the image +skopeo inspect docker://ubuntu:24.04 + +# Step 2: Copy image to local directory +skopeo copy docker://ubuntu:24.04 dir:/tmp/oci-import + +# Step 3: Extract filesystem +# apt-layer extracts the root filesystem from the OCI structure + +# Step 4: Create ComposeFS image +mkcomposefs /tmp/extracted-rootfs my-base/24.04 --digest-store=/path/to/objects + +# Step 5: Cleanup +rm -rf /tmp/oci-import +``` + +### 2. Export Workflow + +```bash +# Step 1: Mount ComposeFS image +mount -t composefs -o basedir=/path/to/objects my-gaming/24.04 /mnt/composefs + +# Step 2: Create OCI directory structure +# apt-layer creates manifest.json, config.json, and layer files + +# Step 3: Copy to registry +skopeo copy dir:/tmp/oci-export docker://myregistry/gaming:latest + +# Step 4: Unmount and cleanup +umount /mnt/composefs +rm -rf /tmp/oci-export +``` + +--- + +## Integration with apt-layer Features + +### 1. OSTree Atomic Operations + +```bash +# Import OCI image and create OSTree commit +apt-layer ostree compose install --from-oci ubuntu:24.04 + +# Export OSTree deployment as OCI image +apt-layer ostree compose export my-deployment myregistry/deployment:latest +``` + +### 2. Container-based Package Installation + +```bash +# Use OCI image as base for package installation +apt-layer --container ubuntu:24.04 my-dev/24.04 vscode git + +# Export result back to OCI +apt-layer --oci-export my-dev/24.04 myregistry/dev:latest +``` + +### 3. Live System Integration + +```bash +# Import OCI image for live system base +apt-layer --live-import ubuntu:24.04 + +# Export live system changes as OCI image +apt-layer --live-export myregistry/live-changes:latest +``` + +--- + +## Error Handling and Validation + +### 1. Image Validation + +```bash +# Validate image before import +if ! skopeo inspect "docker://$image_name" >/dev/null 2>&1; then + log_error "Invalid OCI image: $image_name" "apt-layer" + return 1 +fi +``` + +### 2. Registry Connectivity + +```bash +# Test registry connectivity +if ! skopeo inspect "docker://$registry/$image" >/dev/null 2>&1; then + log_error "Cannot access registry: $registry" "apt-layer" + return 1 +fi +``` + +### 3. Authentication Errors + +```bash +# Handle authentication failures +if ! skopeo copy "docker://$source" "docker://$destination"; then + log_error "Authentication failed or insufficient permissions" "apt-layer" + log_info "Try: podman login $registry" "apt-layer" + return 1 +fi +``` + +--- + +## Configuration + +### 1. Registry Configuration + +**File:** `/etc/containers/registries.conf` +**Purpose:** Configure registry mirrors, authentication, and security + +```ini +[[registry]] +prefix = "docker.io" +location = "docker.io" +insecure = false +blocked = false + +[[registry]] +prefix = "myregistry.com" +location = "myregistry.com" +insecure = true +``` + +### 2. Authentication + +**File:** `~/.docker/config.json` (shared with podman/docker) +**Purpose:** Store registry credentials + +```json +{ + "auths": { + "myregistry.com": { + "auth": "base64-encoded-credentials" + } + } +} +``` + +### 3. Policy Configuration + +**File:** `/etc/containers/policy.json` +**Purpose:** Define signature verification policies + +```json +{ + "default": [ + { + "type": "insecureAcceptAnything" + } + ] +} +``` + +--- + +## Troubleshooting + +### Common Issues + +**1. Authentication Errors:** +```bash +# Error: authentication required +# Solution: Login to registry +podman login myregistry.com +``` + +**2. Network Connectivity:** +```bash +# Error: connection refused +# Solution: Check network and firewall +curl -I https://registry-1.docker.io/v2/ +``` + +**3. Image Not Found:** +```bash +# Error: manifest unknown +# Solution: Verify image name and tag +skopeo list-tags docker://ubuntu +``` + +**4. Insufficient Permissions:** +```bash +# Error: permission denied +# Solution: Check registry permissions +skopeo inspect docker://myregistry/private-image +``` + +### Debug Mode + +```bash +# Enable debug output +export SKOPEO_DEBUG=1 +apt-layer --oci-import ubuntu:24.04 my-base/24.04 +``` + +--- + +## Integration Notes + +- **OCI-First Approach:** apt-layer prioritizes skopeo for OCI operations, using container runtimes only when necessary +- **ComposeFS Integration:** Seamless conversion between OCI and ComposeFS formats +- **Registry Support:** Full support for Docker Hub, private registries, and local storage +- **Signature Verification:** Built-in support for image signatures and verification +- **Authentication:** Shared authentication with podman/docker for consistent experience +- **Error Handling:** Comprehensive error handling with helpful diagnostic messages + +--- + +## References + +- [Skopeo GitHub Repository](https://github.com/containers/skopeo) +- [Skopeo Documentation](https://github.com/containers/skopeo/blob/main/README.md) +- [OCI Specification](https://github.com/opencontainers/image-spec) +- [Container Tools Documentation](https://github.com/containers/toolbox) +- [rpm-ostree Skopeo Integration](https://github.com/coreos/rpm-ostree)