ComposeSync/Docs/multi-stack.md

# Multi-Stack Setup

This guide covers how to configure and manage multiple Docker Compose stacks with ComposeSync.

## Overview

ComposeSync can manage multiple Docker Compose stacks simultaneously, each with its own configuration, update schedule, and customizations.

## TOML Configuration (Recommended)

The TOML format makes multi-stack configuration much cleaner and more readable:

```toml
# Global settings for all stacks
[global]
UPDATE_INTERVAL_SECONDS = 3600  # Default: check every hour
KEEP_VERSIONS = 10              # Default: keep 10 versions
DRY_RUN = false                 # Default: apply changes
NOTIFICATION_WEBHOOK_URL = "https://your-webhook-url.com/endpoint"

# Production stacks
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
INTERVAL = 7200  # Check every 2 hours for production

[portainer]
URL = "https://github.com/portainer/portainer-compose.git"
PATH = "/opt/composesync/stacks/portainer"
TOOL = "git"
GIT_SUBPATH = "compose/docker-compose.yml"
GIT_REF = "main"
INTERVAL = 86400  # Check daily for stable tools

# Development stacks
[dev-app]
URL = "https://github.com/user/dev-app.git"
PATH = "/opt/composesync/stacks/dev-app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "develop"
INTERVAL = 1800  # Check every 30 minutes for dev
KEEP_VERSIONS = 5  # Keep fewer versions for dev

# Complex stack with multiple files
[complex-stack]
URL = "https://github.com/user/complex-app.git"
PATH = "/opt/composesync/stacks/complex"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
EXTRA_FILES_1 = "https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml"
EXTRA_FILES_2 = "https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml"
EXTRA_FILES_ORDER = "1,2"
```

### TOML Benefits for Multi-Stack

- **No numbering required** - Stack names are used directly
- **Clear organization** - Group stacks by purpose with comments
- **Per-stack overrides** - Each stack can override global settings
- **Easy maintenance** - Add/remove stacks without renumbering

## Legacy .env Configuration

The .env format uses numbered variables and is supported for backward compatibility:

```env
# Global settings
UPDATE_INTERVAL_SECONDS=3600
KEEP_VERSIONS=10
DRY_RUN=false
NOTIFICATION_WEBHOOK_URL=https://your-webhook-url.com/endpoint

# Number of stacks
STACKS=4

# Stack 1 - Immich (Production)
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
STACK_1_INTERVAL=7200

# Stack 2 - Portainer (Production)
STACK_2_NAME=portainer
STACK_2_URL=https://github.com/portainer/portainer-compose.git
STACK_2_PATH=/opt/composesync/stacks/portainer
STACK_2_TOOL=git
STACK_2_GIT_SUBPATH=compose/docker-compose.yml
STACK_2_GIT_REF=main
STACK_2_INTERVAL=86400

# Stack 3 - Development App
STACK_3_NAME=dev-app
STACK_3_URL=https://github.com/user/dev-app.git
STACK_3_PATH=/opt/composesync/stacks/dev-app
STACK_3_TOOL=git
STACK_3_GIT_SUBPATH=docker/docker-compose.yml
STACK_3_GIT_REF=develop
STACK_3_INTERVAL=1800
STACK_3_KEEP_VERSIONS=5

# Stack 4 - Complex Stack
STACK_4_NAME=complex-stack
STACK_4_URL=https://github.com/user/complex-app.git
STACK_4_PATH=/opt/composesync/stacks/complex
STACK_4_TOOL=git
STACK_4_GIT_SUBPATH=docker/docker-compose.yml
STACK_4_GIT_REF=main
STACK_4_EXTRA_FILES_1=https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml
STACK_4_EXTRA_FILES_2=https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml
STACK_4_EXTRA_FILES_ORDER=1,2
```

## Creating Stack Directories

For each stack in your configuration, create the corresponding directory:

```bash
# Create directories for all stacks
sudo mkdir -p /opt/composesync/stacks/immich
sudo mkdir -p /opt/composesync/stacks/portainer
sudo mkdir -p /opt/composesync/stacks/dev-app
sudo mkdir -p /opt/composesync/stacks/complex

# Set proper ownership
sudo chown -R $USER:docker /opt/composesync/stacks/
```

## Creating Override Files

Each stack can have its own `docker-compose.override.yml` file for customizations:

### Immich Override
```bash
sudo nano /opt/composesync/stacks/immich/docker-compose.override.yml
```

```yaml
version: '3.8'
services:
  immich-server:
    environment:
      - DATABASE_URL=postgresql://user:pass@localhost:5432/immich
      - REDIS_URL=redis://localhost:6379
    volumes:
      - /mnt/photos:/photos
      - /mnt/backups:/backups
    restart: unless-stopped
```

### Portainer Override
```bash
sudo nano /opt/composesync/stacks/portainer/docker-compose.override.yml
```

```yaml
version: '3.8'
services:
  portainer:
    environment:
      - PORTAINER_EDGE=0
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /mnt/portainer-data:/data
    restart: unless-stopped
```

### Development App Override
```bash
sudo nano /opt/composesync/stacks/dev-app/docker-compose.override.yml
```

```yaml
version: '3.8'
services:
  dev-app:
    environment:
      - NODE_ENV=development
      - DEBUG=true
    volumes:
      - ./src:/app/src
    ports:
      - "3000:3000"
```

## Per-Stack Configuration Options

Each stack can override global settings:

### Update Intervals
```toml
[production-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/production"
TOOL = "wget"
INTERVAL = 7200  # Check every 2 hours

[development-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/development"
TOOL = "git"
INTERVAL = 1800  # Check every 30 minutes
```

### Version Retention
```toml
[stable-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/stable"
TOOL = "wget"
KEEP_VERSIONS = 5  # Keep fewer versions

[experimental-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/experimental"
TOOL = "git"
KEEP_VERSIONS = 20  # Keep more versions for testing
```

### Git-Specific Settings
```toml
[main-branch]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/main"
TOOL = "git"
GIT_REF = "main"

[feature-branch]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/feature"
TOOL = "git"
GIT_REF = "feature/new-ui"
GIT_SUBPATH = "docker/docker-compose.yml"
```

## Multiple Compose Files

For complex stacks that require multiple compose files:

### TOML Configuration
```toml
[complex-stack]
URL = "https://github.com/user/complex-app.git"
PATH = "/opt/composesync/stacks/complex"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
EXTRA_FILES_1 = "https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml"
EXTRA_FILES_2 = "https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml"
EXTRA_FILES_3 = "https://raw.githubusercontent.com/user/complex-app/main/docker/monitoring.yml"
EXTRA_FILES_ORDER = "1,2,3"
```

### .env Configuration
```env
STACK_1_NAME=complex-stack
STACK_1_URL=https://github.com/user/complex-app.git
STACK_1_PATH=/opt/composesync/stacks/complex
STACK_1_TOOL=git
STACK_1_GIT_SUBPATH=docker/docker-compose.yml
STACK_1_GIT_REF=main
STACK_1_EXTRA_FILES_1=https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml
STACK_1_EXTRA_FILES_2=https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml
STACK_1_EXTRA_FILES_3=https://raw.githubusercontent.com/user/complex-app/main/docker/monitoring.yml
STACK_1_EXTRA_FILES_ORDER=1,2,3
```

## Monitoring Multiple Stacks

### View All Stack Status
```bash
# Check status of all stacks
for stack in /opt/composesync/stacks/*/; do
    echo "=== $(basename $stack) ==="
    docker compose -f "$stack/docker-compose.yml" ps
done
```

### View Stack Logs
```bash
# View logs for a specific stack
docker compose -f /opt/composesync/stacks/immich/docker-compose.yml logs

# View logs for all stacks
for stack in /opt/composesync/stacks/*/; do
    echo "=== $(basename $stack) ==="
    docker compose -f "$stack/docker-compose.yml" logs --tail=10
done
```

### Check Update Status
```bash
# View ComposeSync logs to see update activity
sudo journalctl -u composesync -f

# Filter logs by stack name
sudo journalctl -u composesync | grep "immich"
sudo journalctl -u composesync | grep "portainer"
```

## Best Practices

### 1. Organize by Purpose
```toml
# Production stacks
[immich]
# ...

[portainer]
# ...

# Development stacks
[dev-app]
# ...

[test-app]
# ...
```

### 2. Use Appropriate Intervals
- **Production**: 2-24 hours (7200-86400 seconds)
- **Development**: 15-60 minutes (900-3600 seconds)
- **Stable tools**: Daily (86400 seconds)

### 3. Manage Version Retention
- **Production**: 5-10 versions
- **Development**: 3-5 versions
- **Experimental**: 10-20 versions

### 4. Group Related Stacks
```toml
# Web applications
[web-app]
# ...

[web-api]
# ...

# Infrastructure
[monitoring]
# ...

[backup]
# ...
```

## Troubleshooting Multi-Stack Issues

### Stack Not Updating
```bash
# Check if stack is configured
grep -A 5 "immich" /opt/composesync/config.toml

# Check stack directory exists
ls -la /opt/composesync/stacks/immich/

# Check ComposeSync logs
sudo journalctl -u composesync | grep "immich"
```

### Configuration Errors
```bash
# Test TOML syntax
sudo systemctl restart composesync
sudo journalctl -u composesync -n 20

# Check for missing variables
sudo systemctl status composesync
```

### Permission Issues
```bash
# Fix ownership for all stacks
sudo chown -R $USER:docker /opt/composesync/stacks/

# Check specific stack permissions
ls -la /opt/composesync/stacks/immich/
```