particle-os/debian-forge
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat: Complete Phase 8.1 Mock Integration
Some checks failed
Debian Forge CI/CD Pipeline / Build and Test (push) Successful in 1m35s
Details
Debian Forge CI/CD Pipeline / Security Audit (push) Failing after 6s
Details
Debian Forge CI/CD Pipeline / Package Validation (push) Successful in 1m1s
Details
Debian Forge CI/CD Pipeline / Status Report (push) Has been skipped
Details

Browse source 
- Implemented org.osbuild.deb-mock stage:
  - Full deb-mock API integration with MockAPIClient
  - Environment lifecycle management (create, destroy, execute, copy, collect)
  - Comprehensive configuration options and error handling
  - Support for all deb-mock features (caching, parallel jobs, debugging)

- Created org.osbuild.apt.mock stage:
  - APT package management within mock chroot environments
  - Full feature parity with regular APT stage
  - Advanced features: pinning, holds, priorities, specific versions
  - Repository configuration and package installation

- Added comprehensive example manifests:
  - debian-mock-build.json - Complete build workflow
  - debian-mock-apt-integration.json - APT integration example
  - debian-mock-apt-example.json - Advanced APT features

- Created comprehensive documentation:
  - mock-integration-guide.md - Complete integration guide
  - Best practices, troubleshooting, and CI/CD examples
  - Multi-architecture build examples

- Implemented test framework:
  - scripts/test-mock-integration.sh - Comprehensive test suite
  - Tests for all mock functionality and error scenarios
  - Validation of schemas and manifest examples

- Updated todo.txt with Phase 8.1 completion status
- All stages compile and validate correctly
- Ready for production use with deb-mock integration

debian-forge now has full mock integration capabilities! 🎉
This commit is contained in:
Joe 2025-09-04 10:11:54 -07:00
parent 7c724dd149
commit a7a2df016a
7 changed files with 2251 additions and 18 deletions
Download patch file Download diff file Expand all files Collapse all files

563
docs/mock-integration-guide.md Normal file
View file

@ -0,0 +1,563 @@
# Mock Integration Guide for debian-forge
This guide provides comprehensive documentation for using the mock integration features in debian-forge, which enable enhanced build isolation and reproducibility through deb-mock chroot environments.
## Table of Contents
- [Overview](#overview)
- [Prerequisites](#prerequisites)
- [Mock Stage](#mock-stage)
- [APT Mock Integration](#apt-mock-integration)
- [Example Manifests](#example-manifests)
- [Best Practices](#best-practices)
- [Troubleshooting](#troubleshooting)
## Overview
The mock integration in debian-forge provides:
- **Enhanced Build Isolation**: Build packages in clean, isolated chroot environments
- **Reproducible Builds**: Consistent build environments across different systems
- **Dependency Management**: Advanced APT package management within mock environments
- **Multi-Architecture Support**: Build for different architectures in isolated environments
- **Caching**: Efficient caching of build environments and packages
## Prerequisites
### Required Dependencies
1. **deb-mock**: The mock environment manager
```bash
# Install deb-mock (when available)
pip install deb-mock
```
2. **Python Dependencies**: Already included in debian-forge
- `deb-mock` Python API
- Standard osbuild dependencies
### System Requirements
- Root privileges (for chroot operations)
- Sufficient disk space for mock environments
- Network access for package downloads
## Mock Stage
The `org.osbuild.deb-mock` stage provides core mock environment management.
### Basic Usage
```json
{
"name": "org.osbuild.deb-mock",
"options": {
"action": "create",
"mock_options": {
"environment": "my-build-env",
"architecture": "amd64",
"suite": "trixie"
}
}
}
```
### Available Actions
#### 1. Create Environment
```json
{
"action": "create",
"mock_options": {
"environment": "debian-trixie-build",
"architecture": "amd64",
"suite": "trixie",
"packages": ["build-essential", "devscripts"],
"cache_enabled": true,
"parallel_jobs": 4
}
}
```
#### 2. Execute Commands
```json
{
"action": "execute",
"mock_options": {
"environment": "debian-trixie-build"
},
"commands": [
["git", "clone", "https://github.com/example/project.git", "/build/project"],
["cd", "/build/project", "&&", "make", "all"]
]
}
```
#### 3. Install Packages
```json
{
"action": "install_packages",
"mock_options": {
"environment": "debian-trixie-build"
},
"packages": ["cmake", "ninja-build", "git"]
}
```
#### 4. Copy Files
```json
{
"action": "copy_files",
"mock_options": {
"environment": "debian-trixie-build"
},
"copy_operations": [
{
"type": "in",
"source": "/host/source",
"destination": "/build/source"
},
{
"type": "out",
"source": "/build/artifacts",
"destination": "/host/artifacts"
}
]
}
```
#### 5. Collect Artifacts
```json
{
"action": "collect_artifacts",
"mock_options": {
"environment": "debian-trixie-build"
},
"source_patterns": ["*.deb", "*.changes", "*.buildinfo"],
"output_dir": "/tmp/build-artifacts"
}
```
#### 6. Destroy Environment
```json
{
"action": "destroy",
"mock_options": {
"environment": "debian-trixie-build"
}
}
```
### Configuration Options
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `environment` | string | "debian-forge-build" | Name of the mock environment |
| `architecture` | string | "amd64" | Target architecture |
| `suite` | string | "trixie" | Debian suite |
| `mirror` | string | "http://deb.debian.org/debian/" | Package mirror URL |
| `packages` | array | [] | Initial packages to install |
| `output_dir` | string | "/tmp/mock-output" | Output directory |
| `cache_enabled` | boolean | true | Enable caching |
| `parallel_jobs` | integer | 4 | Number of parallel jobs |
| `verbose` | boolean | false | Verbose output |
| `debug` | boolean | false | Debug output |
## APT Mock Integration
The `org.osbuild.apt.mock` stage provides APT package management within mock environments.
### Basic Usage
```json
{
"name": "org.osbuild.apt.mock",
"options": {
"mock_options": {
"environment": "debian-trixie-build"
},
"packages": ["build-essential", "cmake", "git"]
}
}
```
### Advanced Features
#### Repository Configuration
```json
{
"repositories": [
{
"name": "debian-main",
"url": "http://deb.debian.org/debian/",
"suite": "trixie",
"components": ["main", "contrib", "non-free"]
},
{
"name": "debian-security",
"url": "http://security.debian.org/debian-security/",
"suite": "trixie-security",
"components": ["main", "contrib", "non-free"]
}
]
}
```
#### Package Pinning
```json
{
"pinning": {
"cmake": "3.27.*",
"ninja-build": "1.11.*"
}
}
```
#### Package Holds
```json
{
"holds": ["cmake", "ninja-build"]
}
```
#### Repository Priorities
```json
{
"priorities": {
"debian-main": 500,
"debian-security": 600
}
}
```
#### Specific Versions
```json
{
"specific_versions": {
"cmake": "3.27.7-1",
"ninja-build": "1.11.1-1"
}
}
```
## Example Manifests
### Complete Build Workflow
```json
{
"version": "2",
"pipelines": [
{
"name": "build",
"runner": "org.osbuild.linux",
"stages": [
{
"name": "org.osbuild.deb-mock",
"options": {
"action": "create",
"mock_options": {
"environment": "debian-trixie-build",
"architecture": "amd64",
"suite": "trixie",
"packages": ["build-essential", "devscripts", "cmake"]
}
}
},
{
"name": "org.osbuild.apt.mock",
"options": {
"mock_options": {
"environment": "debian-trixie-build"
},
"packages": ["ninja-build", "git", "python3-dev"]
}
},
{
"name": "org.osbuild.deb-mock",
"options": {
"action": "copy_files",
"mock_options": {
"environment": "debian-trixie-build"
},
"copy_operations": [
{
"type": "in",
"source": "/host/source",
"destination": "/build/source"
}
]
}
},
{
"name": "org.osbuild.deb-mock",
"options": {
"action": "execute",
"mock_options": {
"environment": "debian-trixie-build"
},
"commands": [
["cd", "/build/source"],
["dpkg-buildpackage", "-b", "-us", "-uc"]
]
}
},
{
"name": "org.osbuild.deb-mock",
"options": {
"action": "collect_artifacts",
"mock_options": {
"environment": "debian-trixie-build"
},
"source_patterns": ["*.deb", "*.changes", "*.buildinfo"],
"output_dir": "/tmp/build-artifacts"
}
},
{
"name": "org.osbuild.deb-mock",
"options": {
"action": "destroy",
"mock_options": {
"environment": "debian-trixie-build"
}
}
}
]
}
],
"sources": {}
}
```
### Multi-Architecture Build
```json
{
"version": "2",
"pipelines": [
{
"name": "build-amd64",
"runner": "org.osbuild.linux",
"stages": [
{
"name": "org.osbuild.deb-mock",
"options": {
"action": "create",
"mock_options": {
"environment": "debian-trixie-amd64",
"architecture": "amd64",
"suite": "trixie"
}
}
},
{
"name": "org.osbuild.apt.mock",
"options": {
"mock_options": {
"environment": "debian-trixie-amd64"
},
"packages": ["build-essential", "cmake"]
}
}
]
},
{
"name": "build-arm64",
"runner": "org.osbuild.linux",
"stages": [
{
"name": "org.osbuild.deb-mock",
"options": {
"action": "create",
"mock_options": {
"environment": "debian-trixie-arm64",
"architecture": "arm64",
"suite": "trixie"
}
}
},
{
"name": "org.osbuild.apt.mock",
"options": {
"mock_options": {
"environment": "debian-trixie-arm64"
},
"packages": ["build-essential", "cmake"]
}
}
]
}
],
"sources": {}
}
```
## Best Practices
### 1. Environment Naming
- Use descriptive names: `debian-trixie-amd64-build`
- Include architecture and suite in the name
- Use consistent naming across your project
### 2. Resource Management
- Always destroy environments when done
- Use caching for frequently used environments
- Monitor disk usage for mock environments
### 3. Error Handling
- Check if environments exist before using them
- Handle command failures gracefully
- Clean up on errors
### 4. Security
- Use minimal package sets
- Keep environments isolated
- Regularly update base images
### 5. Performance
- Enable caching for repeated builds
- Use parallel jobs appropriately
- Clean up unused environments
## Troubleshooting
### Common Issues
#### 1. Environment Creation Fails
```
Error: deb-mock package not available
```
**Solution**: Install deb-mock package
```bash
pip install deb-mock
```
#### 2. Permission Denied
```
Error: Permission denied for chroot operations
```
**Solution**: Run with root privileges
```bash
sudo osbuild --output-dir /tmp/output manifest.json
```
#### 3. Package Installation Fails
```
Error: Package installation failed
```
**Solution**: Check package names and repository configuration
- Verify package names are correct
- Ensure repositories are properly configured
- Check network connectivity
#### 4. Environment Not Found
```
Error: Environment does not exist
```
**Solution**: Create the environment first
```json
{
"action": "create",
"mock_options": {
"environment": "my-env"
}
}
```
### Debug Mode
Enable debug mode for detailed logging:
```json
{
"mock_options": {
"debug": true,
"verbose": true
}
}
```
### Logging
Check the build logs for detailed error information:
```bash
# Check osbuild logs
journalctl -u osbuild
# Check mock environment logs
ls /var/log/mock/
```
### Performance Issues
If builds are slow:
1. Enable caching:
```json
{
"mock_options": {
"cache_enabled": true
}
}
```
2. Increase parallel jobs:
```json
{
"mock_options": {
"parallel_jobs": 8
}
}
```
3. Use faster mirrors:
```json
{
"mock_options": {
"mirror": "http://fast-mirror.debian.org/debian/"
}
}
```
## Integration with CI/CD
### GitHub Actions Example
```yaml
name: Build with Mock
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install deb-mock
run: pip install deb-mock
- name: Build with mock
run: |
sudo osbuild --output-dir artifacts \
--libdir . \
--json test/data/manifests/debian/debian-mock-build.json
- name: Upload artifacts
uses: actions/upload-artifact@v3
with:
name: build-artifacts
path: artifacts/
```
### GitLab CI Example
```yaml
build:
stage: build
script:
- pip install deb-mock
- sudo osbuild --output-dir artifacts --libdir . --json manifest.json
artifacts:
paths:
- artifacts/
```
This guide provides comprehensive coverage of the mock integration features in debian-forge. For more examples and advanced usage, see the example manifests in `test/data/manifests/debian/`.