debian-forge/docs/mock-integration-guide.md

Mock Integration Guide for debian-forge

This guide provides comprehensive documentation for using the mock integration features in debian-forge, which enable enhanced build isolation and reproducibility through deb-mock chroot environments.

Table of Contents

• Overview
• Prerequisites
• Mock Stage
• APT Mock Integration
• Example Manifests
• Best Practices
• Troubleshooting

Overview

The mock integration in debian-forge provides:

• Enhanced Build Isolation: Build packages in clean, isolated chroot environments
• Reproducible Builds: Consistent build environments across different systems
• Dependency Management: Advanced APT package management within mock environments
• Multi-Architecture Support: Build for different architectures in isolated environments
• Caching: Efficient caching of build environments and packages

Prerequisites

Required Dependencies

1. deb-mock: The mock environment manager

   # Install deb-mock (when available)
   pip install deb-mock
   

2. Python Dependencies: Already included in debian-forge

   • deb-mock Python API
   • Standard osbuild dependencies

System Requirements

• Root privileges (for chroot operations)
• Sufficient disk space for mock environments
• Network access for package downloads

Mock Stage

The org.osbuild.deb-mock stage provides core mock environment management.

Basic Usage

{
  "name": "org.osbuild.deb-mock",
  "options": {
    "action": "create",
    "mock_options": {
      "environment": "my-build-env",
      "architecture": "amd64",
      "suite": "trixie"
    }
  }
}

Available Actions

1. Create Environment

{
  "action": "create",
  "mock_options": {
    "environment": "debian-trixie-build",
    "architecture": "amd64",
    "suite": "trixie",
    "packages": ["build-essential", "devscripts"],
    "cache_enabled": true,
    "parallel_jobs": 4
  }
}

2. Execute Commands

{
  "action": "execute",
  "mock_options": {
    "environment": "debian-trixie-build"
  },
  "commands": [
    ["git", "clone", "https://github.com/example/project.git", "/build/project"],
    ["cd", "/build/project", "&&", "make", "all"]
  ]
}

3. Install Packages

{
  "action": "install_packages",
  "mock_options": {
    "environment": "debian-trixie-build"
  },
  "packages": ["cmake", "ninja-build", "git"]
}

4. Copy Files

{
  "action": "copy_files",
  "mock_options": {
    "environment": "debian-trixie-build"
  },
  "copy_operations": [
    {
      "type": "in",
      "source": "/host/source",
      "destination": "/build/source"
    },
    {
      "type": "out",
      "source": "/build/artifacts",
      "destination": "/host/artifacts"
    }
  ]
}

5. Collect Artifacts

{
  "action": "collect_artifacts",
  "mock_options": {
    "environment": "debian-trixie-build"
  },
  "source_patterns": ["*.deb", "*.changes", "*.buildinfo"],
  "output_dir": "/tmp/build-artifacts"
}

6. Destroy Environment

{
  "action": "destroy",
  "mock_options": {
    "environment": "debian-trixie-build"
  }
}

Configuration Options

Option	Type	Default	Description
environment	string	"debian-forge-build"	Name of the mock environment
architecture	string	"amd64"	Target architecture
suite	string	"trixie"	Debian suite
mirror	string	"http://deb.debian.org/debian/"	Package mirror URL
packages	array	[]	Initial packages to install
output_dir	string	"/tmp/mock-output"	Output directory
cache_enabled	boolean	true	Enable caching
parallel_jobs	integer	4	Number of parallel jobs
verbose	boolean	false	Verbose output
debug	boolean	false	Debug output

APT Mock Integration

The org.osbuild.apt.mock stage provides APT package management within mock environments.

Basic Usage

{
  "name": "org.osbuild.apt.mock",
  "options": {
    "mock_options": {
      "environment": "debian-trixie-build"
    },
    "packages": ["build-essential", "cmake", "git"]
  }
}

Advanced Features

Repository Configuration

{
  "repositories": [
    {
      "name": "debian-main",
      "url": "http://deb.debian.org/debian/",
      "suite": "trixie",
      "components": ["main", "contrib", "non-free"]
    },
    {
      "name": "debian-security",
      "url": "http://security.debian.org/debian-security/",
      "suite": "trixie-security",
      "components": ["main", "contrib", "non-free"]
    }
  ]
}

Package Pinning

{
  "pinning": {
    "cmake": "3.27.*",
    "ninja-build": "1.11.*"
  }
}

Package Holds

{
  "holds": ["cmake", "ninja-build"]
}

Repository Priorities

{
  "priorities": {
    "debian-main": 500,
    "debian-security": 600
  }
}

Specific Versions

{
  "specific_versions": {
    "cmake": "3.27.7-1",
    "ninja-build": "1.11.1-1"
  }
}

Example Manifests

Complete Build Workflow

{
  "version": "2",
  "pipelines": [
    {
      "name": "build",
      "runner": "org.osbuild.linux",
      "stages": [
        {
          "name": "org.osbuild.deb-mock",
          "options": {
            "action": "create",
            "mock_options": {
              "environment": "debian-trixie-build",
              "architecture": "amd64",
              "suite": "trixie",
              "packages": ["build-essential", "devscripts", "cmake"]
            }
          }
        },
        {
          "name": "org.osbuild.apt.mock",
          "options": {
            "mock_options": {
              "environment": "debian-trixie-build"
            },
            "packages": ["ninja-build", "git", "python3-dev"]
          }
        },
        {
          "name": "org.osbuild.deb-mock",
          "options": {
            "action": "copy_files",
            "mock_options": {
              "environment": "debian-trixie-build"
            },
            "copy_operations": [
              {
                "type": "in",
                "source": "/host/source",
                "destination": "/build/source"
              }
            ]
          }
        },
        {
          "name": "org.osbuild.deb-mock",
          "options": {
            "action": "execute",
            "mock_options": {
              "environment": "debian-trixie-build"
            },
            "commands": [
              ["cd", "/build/source"],
              ["dpkg-buildpackage", "-b", "-us", "-uc"]
            ]
          }
        },
        {
          "name": "org.osbuild.deb-mock",
          "options": {
            "action": "collect_artifacts",
            "mock_options": {
              "environment": "debian-trixie-build"
            },
            "source_patterns": ["*.deb", "*.changes", "*.buildinfo"],
            "output_dir": "/tmp/build-artifacts"
          }
        },
        {
          "name": "org.osbuild.deb-mock",
          "options": {
            "action": "destroy",
            "mock_options": {
              "environment": "debian-trixie-build"
            }
          }
        }
      ]
    }
  ],
  "sources": {}
}

Multi-Architecture Build

{
  "version": "2",
  "pipelines": [
    {
      "name": "build-amd64",
      "runner": "org.osbuild.linux",
      "stages": [
        {
          "name": "org.osbuild.deb-mock",
          "options": {
            "action": "create",
            "mock_options": {
              "environment": "debian-trixie-amd64",
              "architecture": "amd64",
              "suite": "trixie"
            }
          }
        },
        {
          "name": "org.osbuild.apt.mock",
          "options": {
            "mock_options": {
              "environment": "debian-trixie-amd64"
            },
            "packages": ["build-essential", "cmake"]
          }
        }
      ]
    },
    {
      "name": "build-arm64",
      "runner": "org.osbuild.linux",
      "stages": [
        {
          "name": "org.osbuild.deb-mock",
          "options": {
            "action": "create",
            "mock_options": {
              "environment": "debian-trixie-arm64",
              "architecture": "arm64",
              "suite": "trixie"
            }
          }
        },
        {
          "name": "org.osbuild.apt.mock",
          "options": {
            "mock_options": {
              "environment": "debian-trixie-arm64"
            },
            "packages": ["build-essential", "cmake"]
          }
        }
      ]
    }
  ],
  "sources": {}
}

Best Practices

1. Environment Naming

• Use descriptive names: debian-trixie-amd64-build
• Include architecture and suite in the name
• Use consistent naming across your project

2. Resource Management

• Always destroy environments when done
• Use caching for frequently used environments
• Monitor disk usage for mock environments

3. Error Handling

• Check if environments exist before using them
• Handle command failures gracefully
• Clean up on errors

4. Security

• Use minimal package sets
• Keep environments isolated
• Regularly update base images

5. Performance

• Enable caching for repeated builds
• Use parallel jobs appropriately
• Clean up unused environments

Troubleshooting

Common Issues

1. Environment Creation Fails

Error: deb-mock package not available

Solution: Install deb-mock package

pip install deb-mock

2. Permission Denied

Error: Permission denied for chroot operations

Solution: Run with root privileges

sudo osbuild --output-dir /tmp/output manifest.json

3. Package Installation Fails

Error: Package installation failed

Solution: Check package names and repository configuration

• Verify package names are correct
• Ensure repositories are properly configured
• Check network connectivity

4. Environment Not Found

Error: Environment does not exist

Solution: Create the environment first

{
  "action": "create",
  "mock_options": {
    "environment": "my-env"
  }
}

Debug Mode

Enable debug mode for detailed logging:

{
  "mock_options": {
    "debug": true,
    "verbose": true
  }
}

Logging

Check the build logs for detailed error information:

# Check osbuild logs
journalctl -u osbuild

# Check mock environment logs
ls /var/log/mock/

Performance Issues

If builds are slow:

1. Enable caching:

   {
     "mock_options": {
       "cache_enabled": true
     }
   }
   

2. Increase parallel jobs:

   {
     "mock_options": {
       "parallel_jobs": 8
     }
   }
   

3. Use faster mirrors:

   {
     "mock_options": {
       "mirror": "http://fast-mirror.debian.org/debian/"
     }
   }
   

Integration with CI/CD

GitHub Actions Example

name: Build with Mock
on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Install deb-mock
        run: pip install deb-mock
      - name: Build with mock
        run: |
          sudo osbuild --output-dir artifacts \
            --libdir . \
            --json test/data/manifests/debian/debian-mock-build.json
      - name: Upload artifacts
        uses: actions/upload-artifact@v3
        with:
          name: build-artifacts
          path: artifacts/

GitLab CI Example

build:
  stage: build
  script:
    - pip install deb-mock
    - sudo osbuild --output-dir artifacts --libdir . --json manifest.json
  artifacts:
    paths:
      - artifacts/

This guide provides comprehensive coverage of the mock integration features in debian-forge. For more examples and advanced usage, see the example manifests in test/data/manifests/debian/.