# Debian Forge User Documentation

## Overview
Debian Forge is a Debian-focused fork of OSBuild that enables building Debian atomic images with OSTree support. It combines OSBuild's pipeline architecture with Debian package management and atomic system capabilities.

## Installation

### Prerequisites
- **System**: Debian 12+ or Ubuntu 22.04+
- **Python**: Python 3.9+
- **Go**: Go 1.21+ (for debos integration)
- **OSTree**: OSTree 2023.1+
- **Build Tools**: sbuild, pbuilder, debootstrap, mmdebstrap

### System Dependencies
```bash
# Install required packages
sudo apt update
sudo apt install -y \
    python3-pip \
    python3-venv \
    ostree \
    debootstrap \
    mmdebstrap \
    sbuild \
    pbuilder \
    bubblewrap \
    qemu-utils \
    curl \
    skopeo \
    git

# Install Go
wget https://go.dev/dl/go1.21.0.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.21.0.linux-amd64.tar.gz
export PATH=$PATH:/usr/local/go/bin
```

### Python Dependencies
```bash
# Create virtual environment
python3 -m venv debian-forge-env
source debian-forge-env/bin/activate

# Install Python packages
pip install -r requirements.txt
```

### Build Environment Setup
```bash
# Run setup script
./setup-build-env.sh

# Configure apt-cacher-ng (optional)
./setup-apt-cacher.sh
```

## Basic Usage

### 1. Simple Debian Base Image
```bash
# Build basic Debian image
python3 -m osbuild --libdir . test-debian-manifest.json
```

**Manifest Example** (`test-debian-manifest.json`):
```json
{
  "pipeline": {
    "build": {
      "pipeline": {
        "stages": [
          {
            "name": "org.osbuild.debootstrap",
            "options": {
              "suite": "bookworm",
              "mirror": "http://deb.debian.org/debian",
              "arch": "amd64",
              "apt_proxy": "http://192.168.1.101:3142"
            }
          },
          {
            "name": "org.osbuild.apt",
            "options": {
              "packages": ["systemd", "linux-image-amd64"],
              "apt_proxy": "http://192.168.1.101:3142"
            }
          }
        ]
      }
    }
  }
}
```

### 2. Debian Atomic Image
```bash
# Build Debian atomic image with OSTree
python3 -m osbuild --libdir . test-debian-atomic-manifest.json
```

**Atomic Manifest Example** (`test-debian-atomic-manifest.json`):
```json
{
  "pipelines": [
    {
      "name": "debian-base",
      "stages": [
        {
          "name": "org.osbuild.debootstrap",
          "options": {
            "suite": "bookworm",
            "mirror": "http://deb.debian.org/debian",
            "arch": "amd64",
            "apt_proxy": "http://192.168.1.101:3142"
          }
        },
        {
          "name": "org.osbuild.apt.config",
          "options": {
            "sources": {
              "main": "deb http://deb.debian.org/debian bookworm main",
              "security": "deb http://security.debian.org/debian-security bookworm-security main"
            }
          }
        },
        {
          "name": "org.osbuild.apt",
          "options": {
            "packages": ["systemd", "linux-image-amd64", "ostree"],
            "apt_proxy": "http://192.168.1.101:3142"
          }
        }
      ]
    },
    {
      "name": "ostree-commit",
      "stages": [
        {
          "name": "org.osbuild.ostree.commit",
          "options": {
            "branch": "debian/bookworm/amd64",
            "subject": "Debian Bookworm Atomic Base"
          }
        }
      ]
    }
  ]
}
```

## Advanced Usage

### 1. Custom Package Building
```bash
# Build custom Debian package
python3 -m osbuild --libdir . custom-package-manifest.json
```

**Package Build Manifest**:
```json
{
  "pipeline": {
    "build": {
      "pipeline": {
        "stages": [
          {
            "name": "org.osbuild.debootstrap",
            "options": {
              "suite": "bookworm",
              "mirror": "http://deb.debian.org/debian",
              "arch": "amd64"
            }
          },
          {
            "name": "org.osbuild.debian.source",
            "options": {
              "package": "my-package",
              "version": "1.0.0"
            }
          },
          {
            "name": "org.osbuild.sbuild",
            "options": {
              "package": "my-package",
              "arch": "amd64"
            }
          }
        ]
      }
    }
  }
}
```

### 2. Multi-Architecture Builds
```bash
# Build for multiple architectures
for arch in amd64 arm64; do
  python3 -m osbuild --libdir . \
    --options '{"arch": "'$arch'"}' \
    multi-arch-manifest.json
done
```

### 3. OSTree Repository Management
```bash
# Initialize OSTree repository
ostree init --mode=archive-z2 /path/to/repo

# List branches
ostree refs --repo=/path/to/repo

# Show commit history
ostree log --repo=/path/to/repo debian/bookworm/amd64
```

## Build Orchestration

### 1. Using Build Orchestrator
```python
from build_orchestrator import BuildOrchestrator

# Initialize orchestrator
orchestrator = BuildOrchestrator()

# Submit build
build_id = orchestrator.submit_build(
    manifest_path="test-debian-atomic-manifest.json",
    priority="high"
)

# Monitor build status
status = orchestrator.get_build_status(build_id)
print(f"Build {build_id}: {status}")

# List all builds
builds = orchestrator.list_builds()
for build in builds:
    print(f"Build {build.id}: {build.status}")
```

### 2. Resource Management
```python
# Check system resources
resources = orchestrator.resource_manager.get_system_resources()
print(f"CPU: {resources.cpu_percent}%")
print(f"Memory: {resources.memory_percent}%")
print(f"Disk: {resources.disk_percent}%")

# Set resource limits
orchestrator.resource_manager.set_build_limits(
    max_cpu=80,
    max_memory=70,
    max_disk=85
)
```

### 3. Build Queue Management
```python
# Submit multiple builds
builds = []
for i in range(5):
    build_id = orchestrator.submit_build(
        manifest_path=f"build-{i}.json",
        priority="normal"
    )
    builds.append(build_id)

# Monitor queue
queue_status = orchestrator.get_queue_status()
print(f"Pending: {queue_status.pending}")
print(f"Running: {queue_status.running}")
print(f"Completed: {queue_status.completed}")

# Cancel build
orchestrator.cancel_build(builds[0])
```

## Configuration

### 1. APT Configuration
```bash
# Configure APT sources
cat > /etc/apt/sources.list.d/debian-forge.list << EOF
deb http://deb.debian.org/debian bookworm main
deb http://security.debian.org/debian-security bookworm-security main
deb http://deb.debian.org/debian bookworm-updates main
EOF

# Configure apt-cacher-ng proxy
cat > /etc/apt/apt.conf.d/99proxy << EOF
Acquire::http::Proxy "http://192.168.1.101:3142";
Acquire::https::Proxy "http://192.168.1.101:3142";
EOF
```

### 2. OSTree Configuration
```bash
# Configure OSTree repository
ostree config set core.min-free-space-size 1GB
ostree config set core.min-free-space-percent 5

# Set up remote for updates
ostree remote add debian-atomic \
  https://your-repo.example.com/debian-atomic \
  --set=gpg-verify=false
```

### 3. Build Environment Configuration
```bash
# Configure sbuild
sudo sbuild-adduser $USER
sudo sbuild-createchroot --include=eatmydata,ccache,distcc bookworm /var/lib/schroot/chroots/bookworm-amd64-sbuild http://deb.debian.org/debian

# Configure pbuilder
pbuilder-dist bookworm create
```

## Troubleshooting

### Common Issues

#### 1. Build Failures
```bash
# Check build logs
tail -f /var/log/osbuild/build-*.log

# Verify manifest syntax
python3 -m osbuild --libdir . --check-only manifest.json

# Test individual stages
python3 test-apt-stage.py
```

#### 2. Resource Issues
```bash
# Check system resources
python3 test-resource-allocation.py

# Monitor build processes
ps aux | grep osbuild
htop

# Check disk space
df -h
du -sh .osbuild/
```

#### 3. Package Issues
```bash
# Verify package availability
apt-cache policy package-name

# Check repository configuration
apt-cache policy

# Test package installation
chroot /path/to/chroot apt update
chroot /path/to/chroot apt install package-name
```

### Debug Mode
```bash
# Enable debug logging
export OSBUILD_DEBUG=1
export OSBUILD_VERBOSE=1

# Run with debug output
python3 -m osbuild --libdir . --verbose manifest.json
```

### Performance Optimization

#### 1. Caching
```bash
# Configure apt-cacher-ng
sudo systemctl enable apt-cacher-ng
sudo systemctl start apt-cacher-ng

# Use local mirror
echo 'deb http://192.168.1.101:3142/deb.debian.org/debian bookworm main' > sources.list
```

#### 2. Parallel Builds
```python
# Configure concurrent builds
orchestrator.resource_manager.set_concurrent_builds(4)

# Monitor performance
python3 test-performance-optimization.py
```

#### 3. Resource Allocation
```python
# Optimize resource allocation
orchestrator.resource_manager.set_build_limits(
    max_cpu=75,      # Leave headroom for system
    max_memory=80,   # Prevent OOM
    max_disk=90      # Maintain free space
)
```

## Integration Examples

### 1. CI/CD Pipeline
```yaml
# .github/workflows/debian-forge.yml
name: Debian Forge Build
on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.9'
      - name: Install dependencies
        run: |
          sudo apt update
          sudo apt install -y ostree debootstrap sbuild
          pip install -r requirements.txt
      - name: Build Debian atomic image
        run: |
          python3 -m osbuild --libdir . test-debian-atomic-manifest.json
```

### 2. Docker Integration
```dockerfile
# Dockerfile
FROM debian:bookworm-slim

# Install Debian Forge
RUN apt update && apt install -y \
    python3-pip \
    ostree \
    debootstrap \
    sbuild

COPY . /debian-forge
WORKDIR /debian-forge

RUN pip install -r requirements.txt

# Build image
CMD ["python3", "-m", "osbuild", "--libdir", ".", "manifest.json"]
```

### 3. API Integration
```python
# REST API client
import requests

class DebianForgeAPI:
    def __init__(self, base_url):
        self.base_url = base_url
    
    def submit_build(self, manifest):
        response = requests.post(
            f"{self.base_url}/api/v1/builds",
            json={"manifest": manifest}
        )
        return response.json()
    
    def get_build_status(self, build_id):
        response = requests.get(
            f"{self.base_url}/api/v1/builds/{build_id}"
        )
        return response.json()

# Usage
api = DebianForgeAPI("http://localhost:8080")
build = api.submit_build(manifest_data)
status = api.get_build_status(build["id"])
```

## Best Practices

### 1. Manifest Design
- Use descriptive pipeline names
- Group related stages logically
- Include proper error handling
- Document complex configurations

### 2. Resource Management
- Monitor system resources during builds
- Set appropriate concurrent build limits
- Use caching for frequently accessed data
- Implement cleanup policies

### 3. Security
- Validate all input manifests
- Use isolated build environments
- Implement proper access controls
- Monitor build processes

### 4. Performance
- Profile build performance regularly
- Optimize stage execution order
- Use parallel processing where possible
- Implement intelligent caching strategies

## Support and Community

### Resources
- **Documentation**: This guide and project README
- **Issues**: GitHub issue tracker
- **Discussions**: GitHub discussions
- **Matrix**: #debian-forge on matrix.org

### Contributing
- Follow OSBuild contribution guidelines
- Maintain 1:1 compatibility where possible
- Test changes thoroughly
- Update documentation for new features

### Getting Help
- Check troubleshooting section first
- Search existing issues
- Create detailed bug reports
- Join community discussions