Major improvements: flexible install dir, configurable compose file name for git, enhanced webhook notifications, cross-platform lock, robust rollback, and updated docs.\n\n- Install dir is now user-confirmable and dynamic\n- Added COMPOSE_FILENAME for git stacks\n- Webhook payloads now include git context and rollback events\n- Lock file age check is cross-platform\n- Rollback notifications for success/failure\n- Updated TOML example and documentation\n- Many robustness and UX improvements

This commit is contained in:
robojerk 2025-06-25 15:15:40 -07:00
parent f0dba7cc0a
commit 70486907aa
18 changed files with 3788 additions and 1767 deletions

View file

@ -1,254 +1,381 @@
# Multi-Stack Configuration
# Multi-Stack Setup
This guide covers how to configure and manage multiple Docker Compose stacks with ComposeSync.
## Basic Multi-Stack Setup
## Overview
You can manage multiple stacks by adding configurations to your `.env` file. Each stack is configured with a numbered prefix:
ComposeSync can manage multiple Docker Compose stacks simultaneously, each with its own configuration, update schedule, and customizations.
```env
# Number of stacks to manage
STACKS=2
## TOML Configuration (Recommended)
# Stack 1: Immich
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
STACK_1_INTERVAL=86400
The TOML format makes multi-stack configuration much cleaner and more readable:
# Stack 2: Portainer
STACK_2_NAME=portainer
STACK_2_URL=https://github.com/portainer/portainer-compose.git
STACK_2_PATH=/opt/composesync/stacks/portainer
STACK_2_TOOL=git
STACK_2_INTERVAL=43200
```toml
# Global settings for all stacks
[global]
UPDATE_INTERVAL_SECONDS = 3600 # Default: check every hour
KEEP_VERSIONS = 10 # Default: keep 10 versions
DRY_RUN = false # Default: apply changes
NOTIFICATION_WEBHOOK_URL = "https://your-webhook-url.com/endpoint"
# Production stacks
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
INTERVAL = 7200 # Check every 2 hours for production
[portainer]
URL = "https://github.com/portainer/portainer-compose.git"
PATH = "/opt/composesync/stacks/portainer"
TOOL = "git"
GIT_SUBPATH = "compose/docker-compose.yml"
GIT_REF = "main"
INTERVAL = 86400 # Check daily for stable tools
# Development stacks
[dev-app]
URL = "https://github.com/user/dev-app.git"
PATH = "/opt/composesync/stacks/dev-app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "develop"
INTERVAL = 1800 # Check every 30 minutes for dev
KEEP_VERSIONS = 5 # Keep fewer versions for dev
# Complex stack with multiple files
[complex-stack]
URL = "https://github.com/user/complex-app.git"
PATH = "/opt/composesync/stacks/complex"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
EXTRA_FILES_1 = "https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml"
EXTRA_FILES_2 = "https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml"
EXTRA_FILES_ORDER = "1,2"
```
## Multiple Compose Files
### TOML Benefits for Multi-Stack
For stacks that use multiple compose files (like Immich), you can specify them in the configuration:
- **No numbering required** - Stack names are used directly
- **Clear organization** - Group stacks by purpose with comments
- **Per-stack overrides** - Each stack can override global settings
- **Easy maintenance** - Add/remove stacks without renumbering
```env
STACKS=1
## Legacy .env Configuration
# Immich stack with multiple compose files
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
STACK_1_INTERVAL=86400
STACK_1_KEEP_VERSIONS=10
# Additional compose files (numbered)
STACK_1_EXTRA_FILES_1=https://raw.githubusercontent.com/immich-app/immich/refs/heads/main/docker/hwaccel.ml.yml
STACK_1_EXTRA_FILES_2=https://raw.githubusercontent.com/immich-app/immich/refs/heads/main/docker/hwaccel.transcoding.yml
STACK_1_EXTRA_FILES_3=https://raw.githubusercontent.com/immich-app/immich/refs/heads/main/docker/prometheus.yml
# Custom file ordering (optional)
STACK_1_EXTRA_FILES_ORDER=3,1,2
```
If `STACK_1_EXTRA_FILES_ORDER` is set, extra files will be processed in the specified order (e.g., 3,1,2). Otherwise, files are processed in numeric order (1,2,3,...).
## Complete Example: Immich Stack
Here's a complete example of setting up Immich with all its compose files:
### 1. Configure the stack in `.env`
```env
STACKS=1
# Main compose file
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
STACK_1_INTERVAL=86400
STACK_1_KEEP_VERSIONS=10
# Additional compose files (one per line, numbered)
# 1. Hardware acceleration for machine learning
STACK_1_EXTRA_FILES_1=https://raw.githubusercontent.com/immich-app/immich/refs/heads/main/docker/hwaccel.ml.yml
# 2. Hardware acceleration for video transcoding
STACK_1_EXTRA_FILES_2=https://raw.githubusercontent.com/immich-app/immich/refs/heads/main/docker/hwaccel.transcoding.yml
# 3. Prometheus monitoring
STACK_1_EXTRA_FILES_3=https://raw.githubusercontent.com/immich-app/immich/refs/heads/main/docker/prometheus.yml
```
The script will process these files in order (1, 2, 3) when running `docker compose`.
### 2. Create your override file
```bash
sudo mkdir -p /opt/composesync/stacks/immich
sudo nano /opt/composesync/stacks/immich/docker-compose.override.yml
```
**Note:** You must create the `docker-compose.override.yml` file manually. ComposeSync will not create it for you. This file should contain your customizations and will be preserved during updates.
### 3. Add your customizations
```yaml
# docker-compose.override.yml
services:
immich-server:
environment:
- IMMICH_SERVER_URL=http://immich-server:2283
- IMMICH_API_URL_EXTERNAL=https://immich.raines.xyz/api
- IMMICH_WEB_URL=https://immich.raines.xyz
networks:
- npm_network
- immich-backend
devices:
- /dev/dri:/dev/dri
redis:
networks:
- immich-backend
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 30s
timeout: 5s
retries: 3
database:
networks:
- immich-backend
healthcheck:
test: ["CMD-SHELL", "pg_isready -d ${DB_DATABASE_NAME} -U ${DB_USERNAME}"]
interval: 30s
timeout: 5s
retries: 3
start_interval: 30s
immich-machine-learning:
image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-openvino
environment:
- OPENVINO_DEVICE=GPU
- OPENVINO_GPU_DEVICE_ID=0
devices:
- /dev/dri:/dev/dri
device_cgroup_rules:
- 'c 189:* rmw'
volumes:
- /dev/bus/usb:/dev/bus/usb
networks:
- immich-backend
volumes:
cifs_immich:
external: true
networks:
npm_network:
external: true
immich-backend:
name: immich-backend
```
### 4. Set permissions
```bash
sudo chown -R YOUR_USERNAME:docker /opt/composesync/stacks/immich
```
### 5. Restart the service
```bash
sudo systemctl restart composesync
```
## How Multiple Files Work
The service will now:
1. Download the main `docker-compose.yml`
2. Download all additional compose files specified in `STACK_1_EXTRA_FILES`
3. Apply your `docker-compose.override.yml`
4. Use all files when running `docker compose up`
When running commands manually, you'll need to specify all the files:
```bash
docker compose -f docker-compose.yml \
-f hwaccel.ml.yml \
-f hwaccel.transcoding.yml \
-f prometheus.yml \
-f docker-compose.override.yml \
up -d
```
## Stack-Specific Settings
Each stack can have its own settings that override the global configuration:
The .env format uses numbered variables and is supported for backward compatibility:
```env
# Global settings
UPDATE_INTERVAL_SECONDS=3600
KEEP_VERSIONS=10
DRY_RUN=false
NOTIFICATION_WEBHOOK_URL=https://your-webhook-url.com/endpoint
# Stack 1: Check every 12 hours, keep 15 versions
STACK_1_INTERVAL=43200
STACK_1_KEEP_VERSIONS=15
# Number of stacks
STACKS=4
# Stack 2: Check every 6 hours, keep 5 versions
STACK_2_INTERVAL=21600
STACK_2_KEEP_VERSIONS=5
# Stack 1 - Immich (Production)
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
STACK_1_INTERVAL=7200
# Stack 2 - Portainer (Production)
STACK_2_NAME=portainer
STACK_2_URL=https://github.com/portainer/portainer-compose.git
STACK_2_PATH=/opt/composesync/stacks/portainer
STACK_2_TOOL=git
STACK_2_GIT_SUBPATH=compose/docker-compose.yml
STACK_2_GIT_REF=main
STACK_2_INTERVAL=86400
# Stack 3 - Development App
STACK_3_NAME=dev-app
STACK_3_URL=https://github.com/user/dev-app.git
STACK_3_PATH=/opt/composesync/stacks/dev-app
STACK_3_TOOL=git
STACK_3_GIT_SUBPATH=docker/docker-compose.yml
STACK_3_GIT_REF=develop
STACK_3_INTERVAL=1800
STACK_3_KEEP_VERSIONS=5
# Stack 4 - Complex Stack
STACK_4_NAME=complex-stack
STACK_4_URL=https://github.com/user/complex-app.git
STACK_4_PATH=/opt/composesync/stacks/complex
STACK_4_TOOL=git
STACK_4_GIT_SUBPATH=docker/docker-compose.yml
STACK_4_GIT_REF=main
STACK_4_EXTRA_FILES_1=https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml
STACK_4_EXTRA_FILES_2=https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml
STACK_4_EXTRA_FILES_ORDER=1,2
```
## Creating Stack Directories
For each stack in your configuration, create the corresponding directory:
```bash
# Create directories for all stacks
sudo mkdir -p /opt/composesync/stacks/immich
sudo mkdir -p /opt/composesync/stacks/portainer
sudo mkdir -p /opt/composesync/stacks/dev-app
sudo mkdir -p /opt/composesync/stacks/complex
# Set proper ownership
sudo chown -R $USER:docker /opt/composesync/stacks/
```
## Creating Override Files
For each stack, you should create a `docker-compose.override.yml` file in the stack directory. This file contains your customizations and will be preserved during updates.
Each stack can have its own `docker-compose.override.yml` file for customizations:
### Immich Override
```bash
sudo nano /opt/composesync/stacks/immich/docker-compose.override.yml
```
Example override file structure:
```yaml
version: '3.8'
services:
your-service:
immich-server:
environment:
- CUSTOM_VAR=value
networks:
- your-network
- DATABASE_URL=postgresql://user:pass@localhost:5432/immich
- REDIS_URL=redis://localhost:6379
volumes:
- /host/path:/container/path
networks:
your-network:
external: true
- /mnt/photos:/photos
- /mnt/backups:/backups
restart: unless-stopped
```
## Managing Multiple Stacks
### Viewing All Stacks
### Portainer Override
```bash
ls -la /opt/composesync/stacks/
sudo nano /opt/composesync/stacks/portainer/docker-compose.override.yml
```
### Checking Stack Status
```bash
# Check if a specific stack is running
docker compose -f /opt/composesync/stacks/immich/docker-compose.yml ps
```yaml
version: '3.8'
services:
portainer:
environment:
- PORTAINER_EDGE=0
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /mnt/portainer-data:/data
restart: unless-stopped
```
# Check all stacks
### Development App Override
```bash
sudo nano /opt/composesync/stacks/dev-app/docker-compose.override.yml
```
```yaml
version: '3.8'
services:
dev-app:
environment:
- NODE_ENV=development
- DEBUG=true
volumes:
- ./src:/app/src
ports:
- "3000:3000"
```
## Per-Stack Configuration Options
Each stack can override global settings:
### Update Intervals
```toml
[production-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/production"
TOOL = "wget"
INTERVAL = 7200 # Check every 2 hours
[development-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/development"
TOOL = "git"
INTERVAL = 1800 # Check every 30 minutes
```
### Version Retention
```toml
[stable-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/stable"
TOOL = "wget"
KEEP_VERSIONS = 5 # Keep fewer versions
[experimental-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/experimental"
TOOL = "git"
KEEP_VERSIONS = 20 # Keep more versions for testing
```
### Git-Specific Settings
```toml
[main-branch]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/main"
TOOL = "git"
GIT_REF = "main"
[feature-branch]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/feature"
TOOL = "git"
GIT_REF = "feature/new-ui"
GIT_SUBPATH = "docker/docker-compose.yml"
```
## Multiple Compose Files
For complex stacks that require multiple compose files:
### TOML Configuration
```toml
[complex-stack]
URL = "https://github.com/user/complex-app.git"
PATH = "/opt/composesync/stacks/complex"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
EXTRA_FILES_1 = "https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml"
EXTRA_FILES_2 = "https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml"
EXTRA_FILES_3 = "https://raw.githubusercontent.com/user/complex-app/main/docker/monitoring.yml"
EXTRA_FILES_ORDER = "1,2,3"
```
### .env Configuration
```env
STACK_1_NAME=complex-stack
STACK_1_URL=https://github.com/user/complex-app.git
STACK_1_PATH=/opt/composesync/stacks/complex
STACK_1_TOOL=git
STACK_1_GIT_SUBPATH=docker/docker-compose.yml
STACK_1_GIT_REF=main
STACK_1_EXTRA_FILES_1=https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml
STACK_1_EXTRA_FILES_2=https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml
STACK_1_EXTRA_FILES_3=https://raw.githubusercontent.com/user/complex-app/main/docker/monitoring.yml
STACK_1_EXTRA_FILES_ORDER=1,2,3
```
## Monitoring Multiple Stacks
### View All Stack Status
```bash
# Check status of all stacks
for stack in /opt/composesync/stacks/*/; do
echo "=== $(basename $stack) ==="
docker compose -f "$stack/docker-compose.yml" ps
echo "=== $(basename $stack) ==="
docker compose -f "$stack/docker-compose.yml" ps
done
```
### Manual Updates
### View Stack Logs
```bash
# Update a specific stack manually
sudo systemctl restart composesync
# View logs for a specific stack
docker compose -f /opt/composesync/stacks/immich/docker-compose.yml logs
# Or run the update script directly
sudo -u composesync /opt/composesync/update-agent.sh
# View logs for all stacks
for stack in /opt/composesync/stacks/*/; do
echo "=== $(basename $stack) ==="
docker compose -f "$stack/docker-compose.yml" logs --tail=10
done
```
### Check Update Status
```bash
# View ComposeSync logs to see update activity
sudo journalctl -u composesync -f
# Filter logs by stack name
sudo journalctl -u composesync | grep "immich"
sudo journalctl -u composesync | grep "portainer"
```
## Best Practices
1. **Use descriptive stack names** - Make them easy to identify
2. **Group related stacks** - Keep similar applications together
3. **Set appropriate intervals** - More critical stacks can check more frequently
4. **Use stack-specific settings** - Override global settings when needed
5. **Test with dry-run mode** - Verify configurations before applying
6. **Monitor logs** - Keep an eye on update activities
### 1. Organize by Purpose
```toml
# Production stacks
[immich]
# ...
[portainer]
# ...
# Development stacks
[dev-app]
# ...
[test-app]
# ...
```
### 2. Use Appropriate Intervals
- **Production**: 2-24 hours (7200-86400 seconds)
- **Development**: 15-60 minutes (900-3600 seconds)
- **Stable tools**: Daily (86400 seconds)
### 3. Manage Version Retention
- **Production**: 5-10 versions
- **Development**: 3-5 versions
- **Experimental**: 10-20 versions
### 4. Group Related Stacks
```toml
# Web applications
[web-app]
# ...
[web-api]
# ...
# Infrastructure
[monitoring]
# ...
[backup]
# ...
```
## Troubleshooting Multi-Stack Issues
### Stack Not Updating
```bash
# Check if stack is configured
grep -A 5 "immich" /opt/composesync/config.toml
# Check stack directory exists
ls -la /opt/composesync/stacks/immich/
# Check ComposeSync logs
sudo journalctl -u composesync | grep "immich"
```
### Configuration Errors
```bash
# Test TOML syntax
sudo systemctl restart composesync
sudo journalctl -u composesync -n 20
# Check for missing variables
sudo systemctl status composesync
```
### Permission Issues
```bash
# Fix ownership for all stacks
sudo chown -R $USER:docker /opt/composesync/stacks/
# Check specific stack permissions
ls -la /opt/composesync/stacks/immich/
```