robojerk/ComposeSync
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ComposeSync/Docs/multi-stack.md

9.3 KiB
Raw Blame History

Multi-Stack Setup

This guide covers how to configure and manage multiple Docker Compose stacks with ComposeSync.

Overview

ComposeSync can manage multiple Docker Compose stacks simultaneously, each with its own configuration, update schedule, and customizations.

TOML Configuration (Recommended)

The TOML format makes multi-stack configuration much cleaner and more readable:

# Global settings for all stacks
[global]
UPDATE_INTERVAL_SECONDS = 3600  # Default: check every hour
KEEP_VERSIONS = 10              # Default: keep 10 versions
DRY_RUN = false                 # Default: apply changes
NOTIFICATION_WEBHOOK_URL = "https://your-webhook-url.com/endpoint"

# Production stacks
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
INTERVAL = 7200  # Check every 2 hours for production

[portainer]
URL = "https://github.com/portainer/portainer-compose.git"
PATH = "/opt/composesync/stacks/portainer"
TOOL = "git"
GIT_SUBPATH = "compose/docker-compose.yml"
GIT_REF = "main"
INTERVAL = 86400  # Check daily for stable tools

# Development stacks
[dev-app]
URL = "https://github.com/user/dev-app.git"
PATH = "/opt/composesync/stacks/dev-app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "develop"
INTERVAL = 1800  # Check every 30 minutes for dev
KEEP_VERSIONS = 5  # Keep fewer versions for dev

# Complex stack with multiple files
[complex-stack]
URL = "https://github.com/user/complex-app.git"
PATH = "/opt/composesync/stacks/complex"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
EXTRA_FILES_1 = "https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml"
EXTRA_FILES_2 = "https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml"
EXTRA_FILES_ORDER = "1,2"

TOML Benefits for Multi-Stack

  • No numbering required - Stack names are used directly
  • Clear organization - Group stacks by purpose with comments
  • Per-stack overrides - Each stack can override global settings
  • Easy maintenance - Add/remove stacks without renumbering

Legacy .env Configuration

The .env format uses numbered variables and is supported for backward compatibility:

# Global settings
UPDATE_INTERVAL_SECONDS=3600
KEEP_VERSIONS=10
DRY_RUN=false
NOTIFICATION_WEBHOOK_URL=https://your-webhook-url.com/endpoint

# Number of stacks
STACKS=4

# Stack 1 - Immich (Production)
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
STACK_1_INTERVAL=7200

# Stack 2 - Portainer (Production)
STACK_2_NAME=portainer
STACK_2_URL=https://github.com/portainer/portainer-compose.git
STACK_2_PATH=/opt/composesync/stacks/portainer
STACK_2_TOOL=git
STACK_2_GIT_SUBPATH=compose/docker-compose.yml
STACK_2_GIT_REF=main
STACK_2_INTERVAL=86400

# Stack 3 - Development App
STACK_3_NAME=dev-app
STACK_3_URL=https://github.com/user/dev-app.git
STACK_3_PATH=/opt/composesync/stacks/dev-app
STACK_3_TOOL=git
STACK_3_GIT_SUBPATH=docker/docker-compose.yml
STACK_3_GIT_REF=develop
STACK_3_INTERVAL=1800
STACK_3_KEEP_VERSIONS=5

# Stack 4 - Complex Stack
STACK_4_NAME=complex-stack
STACK_4_URL=https://github.com/user/complex-app.git
STACK_4_PATH=/opt/composesync/stacks/complex
STACK_4_TOOL=git
STACK_4_GIT_SUBPATH=docker/docker-compose.yml
STACK_4_GIT_REF=main
STACK_4_EXTRA_FILES_1=https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml
STACK_4_EXTRA_FILES_2=https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml
STACK_4_EXTRA_FILES_ORDER=1,2

Creating Stack Directories

For each stack in your configuration, create the corresponding directory:

# Create directories for all stacks
sudo mkdir -p /opt/composesync/stacks/immich
sudo mkdir -p /opt/composesync/stacks/portainer
sudo mkdir -p /opt/composesync/stacks/dev-app
sudo mkdir -p /opt/composesync/stacks/complex

# Set proper ownership
sudo chown -R $USER:docker /opt/composesync/stacks/

Creating Override Files

Each stack can have its own docker-compose.override.yml file for customizations:

Immich Override

sudo nano /opt/composesync/stacks/immich/docker-compose.override.yml

version: '3.8'
services:
  immich-server:
    environment:
      - DATABASE_URL=postgresql://user:pass@localhost:5432/immich
      - REDIS_URL=redis://localhost:6379
    volumes:
      - /mnt/photos:/photos
      - /mnt/backups:/backups
    restart: unless-stopped

Portainer Override

sudo nano /opt/composesync/stacks/portainer/docker-compose.override.yml

version: '3.8'
services:
  portainer:
    environment:
      - PORTAINER_EDGE=0
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /mnt/portainer-data:/data
    restart: unless-stopped

Development App Override

sudo nano /opt/composesync/stacks/dev-app/docker-compose.override.yml

version: '3.8'
services:
  dev-app:
    environment:
      - NODE_ENV=development
      - DEBUG=true
    volumes:
      - ./src:/app/src
    ports:
      - "3000:3000"

Per-Stack Configuration Options

Each stack can override global settings:

Update Intervals

[production-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/production"
TOOL = "wget"
INTERVAL = 7200  # Check every 2 hours

[development-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/development"
TOOL = "git"
INTERVAL = 1800  # Check every 30 minutes

Version Retention

[stable-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/stable"
TOOL = "wget"
KEEP_VERSIONS = 5  # Keep fewer versions

[experimental-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/experimental"
TOOL = "git"
KEEP_VERSIONS = 20  # Keep more versions for testing

Git-Specific Settings

[main-branch]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/main"
TOOL = "git"
GIT_REF = "main"

[feature-branch]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/feature"
TOOL = "git"
GIT_REF = "feature/new-ui"
GIT_SUBPATH = "docker/docker-compose.yml"

Multiple Compose Files

For complex stacks that require multiple compose files:

TOML Configuration

[complex-stack]
URL = "https://github.com/user/complex-app.git"
PATH = "/opt/composesync/stacks/complex"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
EXTRA_FILES_1 = "https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml"
EXTRA_FILES_2 = "https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml"
EXTRA_FILES_3 = "https://raw.githubusercontent.com/user/complex-app/main/docker/monitoring.yml"
EXTRA_FILES_ORDER = "1,2,3"

.env Configuration

STACK_1_NAME=complex-stack
STACK_1_URL=https://github.com/user/complex-app.git
STACK_1_PATH=/opt/composesync/stacks/complex
STACK_1_TOOL=git
STACK_1_GIT_SUBPATH=docker/docker-compose.yml
STACK_1_GIT_REF=main
STACK_1_EXTRA_FILES_1=https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml
STACK_1_EXTRA_FILES_2=https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml
STACK_1_EXTRA_FILES_3=https://raw.githubusercontent.com/user/complex-app/main/docker/monitoring.yml
STACK_1_EXTRA_FILES_ORDER=1,2,3

Monitoring Multiple Stacks

View All Stack Status

# Check status of all stacks
for stack in /opt/composesync/stacks/*/; do
    echo "=== $(basename $stack) ==="
    docker compose -f "$stack/docker-compose.yml" ps
done

View Stack Logs

# View logs for a specific stack
docker compose -f /opt/composesync/stacks/immich/docker-compose.yml logs

# View logs for all stacks
for stack in /opt/composesync/stacks/*/; do
    echo "=== $(basename $stack) ==="
    docker compose -f "$stack/docker-compose.yml" logs --tail=10
done

Check Update Status

# View ComposeSync logs to see update activity
sudo journalctl -u composesync -f

# Filter logs by stack name
sudo journalctl -u composesync | grep "immich"
sudo journalctl -u composesync | grep "portainer"

Best Practices

1. Organize by Purpose

# Production stacks
[immich]
# ...

[portainer]
# ...

# Development stacks
[dev-app]
# ...

[test-app]
# ...

2. Use Appropriate Intervals

  • Production: 2-24 hours (7200-86400 seconds)
  • Development: 15-60 minutes (900-3600 seconds)
  • Stable tools: Daily (86400 seconds)

3. Manage Version Retention

  • Production: 5-10 versions
  • Development: 3-5 versions
  • Experimental: 10-20 versions

4. Group Related Stacks

# Web applications
[web-app]
# ...

[web-api]
# ...

# Infrastructure
[monitoring]
# ...

[backup]
# ...

Troubleshooting Multi-Stack Issues

Stack Not Updating

# Check if stack is configured
grep -A 5 "immich" /opt/composesync/config.toml

# Check stack directory exists
ls -la /opt/composesync/stacks/immich/

# Check ComposeSync logs
sudo journalctl -u composesync | grep "immich"

Configuration Errors

# Test TOML syntax
sudo systemctl restart composesync
sudo journalctl -u composesync -n 20

# Check for missing variables
sudo systemctl status composesync

Permission Issues

# Fix ownership for all stacks
sudo chown -R $USER:docker /opt/composesync/stacks/

# Check specific stack permissions
ls -la /opt/composesync/stacks/immich/