robojerk/ComposeSync
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Major improvements: flexible install dir, configurable compose file name for git, enhanced webhook notifications, cross-platform lock, robust rollback, and updated docs.\n\n- Install dir is now user-confirmable and dynamic\n- Added COMPOSE_FILENAME for git stacks\n- Webhook payloads now include git context and rollback events\n- Lock file age check is cross-platform\n- Rollback notifications for success/failure\n- Updated TOML example and documentation\n- Many robustness and UX improvements

Browse source
This commit is contained in:
robojerk 2025-06-25 15:15:40 -07:00
parent f0dba7cc0a
commit 70486907aa
18 changed files with 3788 additions and 1767 deletions
Download patch file Download diff file Expand all files Collapse all files

226
Docs/configuration.md
View file

@ -1,79 +1,191 @@
# Configuration Reference
This guide covers all configuration options available in ComposeSync.
ComposeSync supports two configuration formats: **TOML** (recommended) and **.env** (legacy). The TOML format is more readable and easier to manage for multiple stacks.
## Configuration Files
The service can be configured through:
ComposeSync looks for configuration files in this order:
1. `/opt/composesync/config.toml` (recommended)
2. `/opt/composesync/.env` (legacy format)
1. **Systemd Service File** (`/etc/systemd/system/composesync.service`):
- Basic service configuration
- User and group settings
- Working directory
## TOML Configuration Format (Recommended)
2. **Environment File** (`/opt/composesync/.env`):
- Base directory for stacks
- Global settings
- Stack-specific configurations
The TOML format is more readable and supports both global settings and per-stack configurations:
3. **Stack Directories** (`/opt/composesync/stacks/<stack-name>/`):
- `docker-compose.yml` (managed by agent)
- `docker-compose.override.yml` (your customizations)
- Versioned copies with `.bak` extension
```toml
# Global settings for all stacks
[global]
UPDATE_INTERVAL_SECONDS = 3600 # Check every hour
KEEP_VERSIONS = 10 # Keep 10 versions
DRY_RUN = false # Set to true for testing
NOTIFICATION_WEBHOOK_URL = "https://your-webhook-url.com/endpoint"
## Global Configuration Options
# Stack configurations
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
INTERVAL = 3600 # Check every hour
[dev-app]
URL = "https://github.com/user/dev-app.git"
PATH = "/opt/composesync/stacks/dev-app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "develop"
INTERVAL = 1800 # Check every 30 minutes for dev
```
### TOML Format Benefits
- **More readable** - Clear section-based organization
- **Better comments** - Full comment support
- **No numbering** - Stack names are used directly
- **Type safety** - Better handling of values
- **Easier maintenance** - Add/remove stacks without renumbering
### TOML Configuration Options
#### Global Settings
| Option | Description | Default | Example |
|--------|-------------|---------|---------|
| `UPDATE_INTERVAL_SECONDS` | How often to check for updates (seconds) | `3600` | `1800` |
| `KEEP_VERSIONS` | Number of backup versions to keep | `10` | `5` |
| `DRY_RUN` | Test mode (don't apply changes) | `false` | `true` |
| `NOTIFICATION_WEBHOOK_URL` | Webhook URL for notifications | `""` | `https://...` |
#### Per-Stack Settings
| Option | Description | Required | Example |
|--------|-------------|----------|---------|
| `URL` | Source URL for docker-compose.yml | Yes | `https://...` |
| `PATH` | Local path for the stack | Yes | `/opt/composesync/stacks/name` |
| `TOOL` | Download tool (`wget` or `git`) | Yes | `git` |
| `INTERVAL` | Update interval for this stack (overrides global) | No | `1800` |
| `KEEP_VERSIONS` | Backup versions for this stack (overrides global) | No | `5` |
#### Git-Specific Options
| Option | Description | Required | Example |
|--------|-------------|----------|---------|
| `GIT_SUBPATH` | Path to docker-compose.yml in repository | No | `docker/` |
| `GIT_REF` | Git branch, tag, or commit hash to checkout | No | `main`, `v1.2.3`, `abc1234` |
| `COMPOSE_FILENAME` | Custom compose file name for Git repositories | No | `main.yml`, `docker-compose.prod.yml` |
#### Multiple Compose Files
| Option | Description | Required | Example |
|--------|-------------|----------|---------|
| `EXTRA_FILES_1` | Additional compose file URL (wget only) | No | `https://example.com/network.yml` |
| `EXTRA_FILES_2` | Additional compose file URL (wget only) | No | `https://example.com/volumes.yml` |
| `EXTRA_FILES_ORDER` | Custom order for extra files | No | `2,1,3` |
### TOML Examples
#### Simple Stack
```toml
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
```
#### Git Repository with Subpath
```toml
[my-app]
URL = "https://github.com/user/my-app.git"
PATH = "/opt/composesync/stacks/my-app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
```
#### Complex Stack with Multiple Files
```toml
[complex-stack]
URL = "https://github.com/user/complex-app.git"
PATH = "/opt/composesync/stacks/complex"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
EXTRA_FILES_1 = "https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml"
EXTRA_FILES_2 = "https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml"
EXTRA_FILES_ORDER = "1,2"
```
## Legacy .env Configuration Format
The .env format uses numbered environment variables and is supported for backward compatibility:
```env
# Base directory for all stacks
COMPOSESYNC_BASE_DIR=/opt/composesync/stacks
# Number of versions to keep per stack (default: 10)
KEEP_VERSIONS=10
# Update interval in seconds (default: 3600)
# Global settings
UPDATE_INTERVAL_SECONDS=3600
KEEP_VERSIONS=10
DRY_RUN=false
NOTIFICATION_WEBHOOK_URL=https://your-webhook-url.com/endpoint
# Update mode: notify_only or notify_and_apply (default: notify_and_apply)
UPDATE_MODE=notify_and_apply
# Number of stacks
STACKS=2
# Enable dry-run mode (true/false, default: false)
# Stack 1
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
# Stack 2
STACK_2_NAME=dev-app
STACK_2_URL=https://github.com/user/dev-app.git
STACK_2_PATH=/opt/composesync/stacks/dev-app
STACK_2_TOOL=git
STACK_2_GIT_SUBPATH=docker/docker-compose.yml
STACK_2_GIT_REF=develop
```
## Migration from .env to TOML
To migrate from .env to TOML:
1. **Create a new TOML file:**
```bash
sudo nano /opt/composesync/config.toml
```
2. **Convert your configuration:**
```toml
[global]
UPDATE_INTERVAL_SECONDS = 3600
KEEP_VERSIONS = 10
DRY_RUN = false
# Number of stacks to manage
STACKS=1
# Optional webhook URL for notifications
NOTIFICATION_WEBHOOK_URL=https://your-webhook-url.com/endpoint
[your-stack-name]
URL = "your-url"
PATH = "/opt/composesync/stacks/your-stack"
TOOL = "wget"
```
## Stack Configuration Options
For each stack, you can configure:
```env
# Basic stack configuration
STACK_N_NAME=stack_name # Required: Name of the stack
STACK_N_URL=https://example.com/compose.yml # Required: URL to download from
STACK_N_PATH=/path/to/stack # Required: Local path for the stack
STACK_N_TOOL=wget # Required: Download tool (wget or git)
STACK_N_INTERVAL=86400 # Optional: Stack-specific interval (overrides global)
STACK_N_KEEP_VERSIONS=10 # Optional: Stack-specific version count (overrides global)
# Git-specific options (only when STACK_N_TOOL=git)
STACK_N_GIT_SUBPATH=docker/compose.yml # Optional: Path to compose file in repo
STACK_N_GIT_REF=main # Optional: Branch or tag to checkout
# Multiple compose files (numbered)
STACK_N_EXTRA_FILES_1=https://example.com/file1.yml
STACK_N_EXTRA_FILES_2=https://example.com/file2.yml
# ... continue numbering as needed
# Custom file ordering (optional, comma-separated list of numbers)
STACK_N_EXTRA_FILES_ORDER=2,1,3
3. **Test the configuration:**
```bash
sudo systemctl restart composesync
sudo journalctl -u composesync -f
```
If `STACK_N_EXTRA_FILES_ORDER` is set, extra files will be processed in the specified order (e.g., 2,1,3). Otherwise, files are processed in numeric order (1,2,3,...).
4. **Remove the old .env file:**
```bash
sudo rm /opt/composesync/.env
```
## Update Modes
## Configuration File Location
- **`notify_only`
The configuration files are located at:
- **TOML:** `/opt/composesync/config.toml`
- **ENV:** `/opt/composesync/.env`
Both files can be edited with any text editor:
```bash
sudo nano /opt/composesync/config.toml
# or
sudo nano /opt/composesync/.env
```

481
Docs/dry-run.md
View file

@ -1,204 +1,355 @@
# Dry-Run Mode
This guide covers how to use dry-run mode to test your ComposeSync configuration safely.
Dry-run mode allows you to test ComposeSync configurations without making any actual changes to your Docker stacks. This is perfect for validating your setup before going live.
## Overview
Dry-run mode allows you to test your configuration without actually applying changes. This is useful for:
- Testing new stack configurations
- Verifying download URLs work correctly
- Previewing what changes would be applied
- Debugging configuration issues
- Testing webhook notifications
## Enabling Dry-Run Mode
To enable dry-run mode, add this to your `.env` file:
```env
DRY_RUN=true
```
## What Dry-Run Mode Does
In dry-run mode, ComposeSync will:
- Download and process all files normally
- Show what actions would be taken
- Display a preview of changes (diff output)
- Not create backups or versioned files
- Not apply any changes to your stacks
- Prefix all log messages with `[DRY-RUN]`
When dry-run mode is enabled, ComposeSync will:
- Download and parse configuration files
- Show what changes would be applied
- Log all actions that would be taken
- Send webhook notifications (if configured)
- **Not** actually apply any changes to your stacks
## Example Dry-Run Output
## TOML Configuration
```
[2024-01-15 10:30:00] [DRY-RUN] Processing stack immich
[2024-01-15 10:30:01] [DRY-RUN] Downloading https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml to /opt/composesync/stacks/immich/docker-compose.yml
[2024-01-15 10:30:02] [DRY-RUN] Changes detected in immich
[2024-01-15 10:30:02] [DRY-RUN] DRY-RUN: Would apply changes to immich
[2024-01-15 10:30:02] [DRY-RUN] DRY-RUN: Would run: docker compose -f /opt/composesync/stacks/immich/docker-compose.yml -f /opt/composesync/stacks/immich/hwaccel.ml.yml -f /opt/composesync/stacks/immich/docker-compose.override.yml up -d
[2024-01-15 10:30:02] [DRY-RUN] DRY-RUN: Changes that would be applied:
[2024-01-15 10:30:02] [DRY-RUN] --- compose-20240115103000.yml.bak
[2024-01-15 10:30:02] [DRY-RUN] +++ docker-compose.yml
[2024-01-15 10:30:02] [DRY-RUN] @@ -1,6 +1,6 @@
[2024-01-15 10:30:02] [DRY-RUN] version: '3.8'
[2024-01-15 10:30:02] [DRY-RUN]
[2024-01-15 10:30:02] [DRY-RUN] services:
[2024-01-15 10:30:02] [DRY-RUN] immich-server:
[2024-01-15 10:30:02] [DRY-RUN] - image: ghcr.io/immich-app/immich-server:release
[2024-01-15 10:30:02] [DRY-RUN] + image: ghcr.io/immich-app/immich-server:release-1.91.0
### Global Dry-Run Mode
Enable dry-run mode for all stacks in your TOML configuration:
```toml
# Global settings
[global]
UPDATE_INTERVAL_SECONDS = 3600
KEEP_VERSIONS = 10
DRY_RUN = true # Enable dry-run mode for all stacks
NOTIFICATION_WEBHOOK_URL = "https://your-webhook-url.com/endpoint"
# Stack configurations
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
[dev-app]
URL = "https://github.com/user/dev-app.git"
PATH = "/opt/composesync/stacks/dev-app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "develop"
```
## Testing New Configurations
### Per-Stack Dry-Run Mode
### 1. Test a New Stack
Enable dry-run mode for specific stacks only:
```toml
# Global settings
[global]
UPDATE_INTERVAL_SECONDS = 3600
KEEP_VERSIONS = 10
DRY_RUN = false # Default: apply changes
# Production stack (apply changes)
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
# Development stack (dry-run only)
[dev-app]
URL = "https://github.com/user/dev-app.git"
PATH = "/opt/composesync/stacks/dev-app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "develop"
DRY_RUN = true # Override global setting for this stack
```
## Legacy .env Configuration
### Global Dry-Run Mode
1. Add a new stack configuration to your `.env` file
2. Enable dry-run mode:
```env
# Global settings
UPDATE_INTERVAL_SECONDS=3600
KEEP_VERSIONS=10
DRY_RUN=true
NOTIFICATION_WEBHOOK_URL=https://your-webhook-url.com/endpoint
# Stack configurations
STACKS=2
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
STACK_2_NAME=dev-app
STACK_2_URL=https://github.com/user/dev-app.git
STACK_2_PATH=/opt/composesync/stacks/dev-app
STACK_2_TOOL=git
STACK_2_GIT_SUBPATH=docker/docker-compose.yml
STACK_2_GIT_REF=develop
```
3. Restart the service:
### Per-Stack Dry-Run Mode
```env
# Global settings
UPDATE_INTERVAL_SECONDS=3600
KEEP_VERSIONS=10
DRY_RUN=false
# Stack configurations
STACKS=2
# Production stack (apply changes)
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
# Development stack (dry-run only)
STACK_2_NAME=dev-app
STACK_2_URL=https://github.com/user/dev-app.git
STACK_2_PATH=/opt/composesync/stacks/dev-app
STACK_2_TOOL=git
STACK_2_GIT_SUBPATH=docker/docker-compose.yml
STACK_2_GIT_REF=develop
STACK_2_DRY_RUN=true
```
## Testing Your Configuration
### 1. Enable Dry-Run Mode
Edit your configuration file:
```bash
# For TOML configuration
sudo nano /opt/composesync/config.toml
# For .env configuration
sudo nano /opt/composesync/.env
```
### 2. Restart the Service
```bash
sudo systemctl restart composesync
```
4. Check the logs to see what would happen:
### 3. Monitor the Logs
```bash
# View real-time logs
sudo journalctl -u composesync -f
# View recent logs
sudo journalctl -u composesync -n 50
```
## Dry-Run Output Examples
### Successful Update (Dry-Run)
```
[2024-01-15 10:30:00] Loading TOML configuration from /opt/composesync/config.toml
[2024-01-15 10:30:00] Processing stack: immich
[2024-01-15 10:30:01] Downloading https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml to /opt/composesync/stacks/immich/docker-compose.yml
[2024-01-15 10:30:02] Changes detected for immich
[2024-01-15 10:30:02] DRY-RUN: Would apply changes to immich
[2024-01-15 10:30:02] DRY-RUN: Would run: docker compose -f /opt/composesync/stacks/immich/docker-compose.yml -f /opt/composesync/stacks/immich/docker-compose.override.yml up -d
[2024-01-15 10:30:02] DRY-RUN: Changes that would be applied:
[2024-01-15 10:30:02] --- a/docker-compose.yml
[2024-01-15 10:30:02] +++ b/docker-compose.yml
[2024-01-15 10:30:02] @@ -15,7 +15,7 @@ services:
[2024-01-15 10:30:02] immich-server:
[2024-01-15 10:30:02] - image: ghcr.io/immich-app/immich-server:release
[2024-01-15 10:30:02] + image: ghcr.io/immich-app/immich-server:v1.75.0
[2024-01-15 10:30:02] environment:
[2024-01-15 10:30:02] - NODE_ENV=production
[2024-01-15 10:30:02] [DRY-RUN] Would send webhook notification for immich
```
### No Changes Detected (Dry-Run)
```
[2024-01-15 10:30:00] Processing stack: portainer
[2024-01-15 10:30:01] Downloading https://github.com/portainer/portainer-compose.git to /opt/composesync/stacks/portainer/docker-compose.yml
[2024-01-15 10:30:02] No changes detected for portainer
```
### Error Handling (Dry-Run)
```
[2024-01-15 10:30:00] Processing stack: dev-app
[2024-01-15 10:30:01] ERROR: Failed to download https://invalid-url.com/docker-compose.yml
[2024-01-15 10:30:01] Skipping stack dev-app due to download failure
```
## Webhook Notifications in Dry-Run Mode
When dry-run mode is enabled, webhook notifications will still be sent with a `[DRY-RUN]` prefix:
```json
{
"event": "update_success",
"stack_name": "immich",
"timestamp": "2024-01-15T10:30:02Z",
"message": "[DRY-RUN] Would apply changes to immich",
"version_id": "20240115103002",
"diff": "--- a/docker-compose.yml\n+++ b/docker-compose.yml\n@@ -15,7 +15,7 @@ services:\n immich-server:\n- image: ghcr.io/immich-app/immich-server:release\n+ image: ghcr.io/immich-app/immich-server:v1.75.0\n"
}
```
## Use Cases
### 1. Testing New Configurations
Before adding a new stack to production:
```toml
[new-stack]
URL = "https://github.com/user/new-app.git"
PATH = "/opt/composesync/stacks/new-app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
DRY_RUN = true # Test first
```
### 2. Validating Updates
Test updates before applying them:
```toml
[global]
DRY_RUN = true # Test all updates
[production-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/production"
TOOL = "wget"
```
### 3. Development Environment
Keep development stacks in dry-run mode:
```toml
[development]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/dev"
TOOL = "git"
GIT_REF = "develop"
DRY_RUN = true # Don't auto-update dev environment
```
### 4. Configuration Validation
Validate your configuration without affecting running services:
```bash
# Enable dry-run mode
sudo nano /opt/composesync/config.toml
# Restart service
sudo systemctl restart composesync
# Check logs for any configuration errors
sudo journalctl -u composesync -f
```
### 2. Test URL Changes
1. Modify a stack URL in your `.env` file
2. Enable dry-run mode
3. Restart the service
4. Verify the new URL works and downloads correctly
### 3. Test Multiple Compose Files
1. Add extra compose files to a stack configuration
2. Enable dry-run mode
3. Restart the service
4. Verify all files are downloaded and processed correctly
## Disabling Dry-Run Mode
To disable dry-run mode, set `DRY_RUN=false` or remove the line from your `.env` file:
```env
DRY_RUN=false
```
Then restart the service:
```bash
sudo systemctl restart composesync
```
## Webhook Testing
Dry-run mode is particularly useful for testing webhook notifications because:
- Webhooks are still sent in dry-run mode
- You can verify your webhook configuration works
- No actual changes are applied to your stacks
- You can test different webhook services safely
## Best Practices
### 1. Always Test New Configurations
### 1. Always Test First
Before adding a new stack or changing existing configurations:
1. Enable dry-run mode
2. Test the configuration
3. Verify everything works as expected
4. Disable dry-run mode when ready
```toml
# Test configuration before going live
[global]
DRY_RUN = true
### 2. Use for Debugging
When troubleshooting issues:
1. Enable dry-run mode
2. Check the logs for detailed information
3. Verify URLs and configurations
4. Fix any issues before disabling dry-run mode
### 3. Test Webhook Integration
Use dry-run mode to test webhook notifications:
1. Configure your webhook URL
2. Enable dry-run mode
3. Trigger an update cycle
4. Verify webhook notifications are received
### 4. Preview Changes
Use dry-run mode to preview what changes would be applied:
1. Enable dry-run mode
2. Let the service run a cycle
3. Review the diff output
4. Decide if you want to apply the changes
## Troubleshooting
### No Changes Detected
If dry-run mode shows "No changes detected":
- The remote file is identical to your current file
- This is normal and expected behavior
- No action would be taken in normal mode either
### Download Failures
If you see download errors in dry-run mode:
- Check the URL is correct and accessible
- Verify network connectivity
- Check for authentication requirements
- Fix the URL before disabling dry-run mode
### Configuration Errors
If you see configuration errors:
- Check your `.env` file syntax
- Verify all required fields are present
- Check file permissions
- Fix configuration issues before proceeding
## Example Workflow
Here's a typical workflow for testing a new stack:
1. **Add Configuration**
```env
STACKS=2
STACK_2_NAME=test-app
STACK_2_URL=https://example.com/docker-compose.yml
STACK_2_PATH=/opt/composesync/stacks/test-app
STACK_2_TOOL=wget
[my-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/my-stack"
TOOL = "wget"
```
2. **Enable Dry-Run Mode**
```env
### 2. Use Per-Stack Control
```toml
# Production stacks (apply changes)
[production]
URL = "https://..."
PATH = "/opt/composesync/stacks/production"
TOOL = "wget"
# Development stacks (dry-run only)
[development]
URL = "https://..."
PATH = "/opt/composesync/stacks/development"
TOOL = "git"
DRY_RUN = true
```
3. **Restart Service**
```bash
sudo systemctl restart composesync
### 3. Monitor Webhook Notifications
Even in dry-run mode, webhook notifications help you:
- Verify your webhook configuration
- See what changes would be applied
- Monitor the update process
### 4. Gradual Rollout
```toml
# Phase 1: Test with dry-run
[global]
DRY_RUN = true
# Phase 2: Enable for specific stacks
[stable-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/stable"
TOOL = "wget"
DRY_RUN = false # Override global setting
# Phase 3: Enable for all stacks
[global]
DRY_RUN = false
```
4. **Check Logs**
## Troubleshooting
### Dry-Run Not Working
```bash
# Check if dry-run is enabled
grep -i "dry_run" /opt/composesync/config.toml
# Check service logs
sudo journalctl -u composesync -n 20
# Verify configuration syntax
sudo systemctl status composesync
```
### Unexpected Behavior
```bash
# Check for conflicting settings
grep -A 5 -B 5 "DRY_RUN" /opt/composesync/config.toml
# View detailed logs
sudo journalctl -u composesync -f
```
5. **Verify Results**
- Check that files are downloaded
- Verify no errors occur
- Review any changes that would be applied
### Webhook Issues
6. **Disable Dry-Run Mode**
```env
DRY_RUN=false
sudo systemctl restart composesync
```bash
# Test webhook manually
curl -X POST -H "Content-Type: application/json" \
-d '{"event": "test", "message": "[DRY-RUN] Test notification"}' \
https://your-webhook-url.com/endpoint
```
This workflow ensures your configuration is correct before applying any changes to your running stacks.

434
Docs/git-repositories.md
View file

@ -1,193 +1,339 @@
# Git Repository Setup
This guide covers how to use Git repositories as sources for your Docker Compose files.
This guide covers how to configure ComposeSync to work with Git repositories as sources for Docker Compose files.
## Basic Git Configuration
## Overview
To use a Git repository as your source, set the tool to `git` and provide the repository URL:
ComposeSync can download Docker Compose files from Git repositories, allowing you to:
- Track specific branches, tags, or commits
- Access files in subdirectories
- Maintain version control for your compose configurations
- Use private repositories (with proper authentication)
```env
STACK_1_NAME=myapp
STACK_1_URL=https://github.com/org/repo.git
STACK_1_PATH=/opt/composesync/stacks/myapp
STACK_1_TOOL=git
## TOML Configuration (Recommended)
The TOML format provides clear and readable Git repository configuration:
```toml
# Git repository configuration
[my-app]
URL = "https://github.com/user/my-app.git"
PATH = "/opt/composesync/stacks/my-app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
[development-app]
URL = "https://github.com/user/dev-app.git"
PATH = "/opt/composesync/stacks/dev-app"
TOOL = "git"
GIT_SUBPATH = "deploy/docker-compose.yml"
GIT_REF = "develop"
INTERVAL = 1800 # Check every 30 minutes
[stable-release]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/stable"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "v2.1.0" # Specific tag
INTERVAL = 86400 # Check daily
```
## Git-Specific Options
## Legacy .env Configuration
### Subpath Configuration
If your `docker-compose.yml` file is not in the root of the repository, specify the subpath:
The .env format is supported for backward compatibility:
```env
# Git repository configuration
STACKS=3
STACK_1_NAME=my-app
STACK_1_URL=https://github.com/user/my-app.git
STACK_1_PATH=/opt/composesync/stacks/my-app
STACK_1_TOOL=git
STACK_1_GIT_SUBPATH=docker/docker-compose.yml
```
This is useful for repositories that organize their Docker Compose files in subdirectories.
### Branch and Tag Support
You can specify a specific branch or tag to checkout:
```env
STACK_1_GIT_REF=main
STACK_2_NAME=development-app
STACK_2_URL=https://github.com/user/dev-app.git
STACK_2_PATH=/opt/composesync/stacks/dev-app
STACK_2_TOOL=git
STACK_2_GIT_SUBPATH=deploy/docker-compose.yml
STACK_2_GIT_REF=develop
STACK_2_INTERVAL=1800
STACK_3_NAME=stable-release
STACK_3_URL=https://github.com/user/app.git
STACK_3_PATH=/opt/composesync/stacks/stable
STACK_3_TOOL=git
STACK_3_GIT_SUBPATH=docker/docker-compose.yml
STACK_3_GIT_REF=v2.1.0
STACK_3_INTERVAL=86400
```
Examples:
- `STACK_1_GIT_REF=main` - Use the main branch
- `STACK_1_GIT_REF=v1.0.0` - Use a specific tag
- `STACK_1_GIT_REF=develop` - Use a development branch
## Git Configuration Options
## Complete Example
### Required Options
Here's a complete example of configuring a Git repository:
| Option | Description | Example |
|--------|-------------|---------|
| `URL` | Git repository URL | `https://github.com/user/repo.git` |
| `PATH` | Local path for the stack | `/opt/composesync/stacks/my-app` |
| `TOOL` | Must be set to `"git"` | `git` |
```env
STACK_1_NAME=portainer
STACK_1_URL=https://github.com/portainer/portainer-compose.git
STACK_1_PATH=/opt/composesync/stacks/portainer
STACK_1_TOOL=git
STACK_1_GIT_SUBPATH=docker-compose.yml
STACK_1_GIT_REF=main
STACK_1_INTERVAL=43200
STACK_1_KEEP_VERSIONS=5
### Optional Options
| Option | Description | Default | Example |
|--------|-------------|---------|---------|
| `GIT_SUBPATH` | Path to docker-compose.yml in repo | `docker-compose.yml` (root) | `docker/compose.yml` |
| `GIT_REF` | Branch, tag, or commit hash | `main` | `develop`, `v1.2.3`, `abc123` |
| `INTERVAL` | Update interval (seconds) | Global setting | `1800` |
| `KEEP_VERSIONS` | Number of backup versions | Global setting | `5` |
## Git Reference Types
### Branches
```toml
[main-branch]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/main"
TOOL = "git"
GIT_REF = "main"
[feature-branch]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/feature"
TOOL = "git"
GIT_REF = "feature/new-ui"
```
## How Git Sources Work
### Tags
```toml
[stable-release]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/stable"
TOOL = "git"
GIT_REF = "v2.1.0"
When using Git as a source, ComposeSync:
1. **Clones the repository** (if not already present)
2. **Fetches updates** from the remote repository
3. **Checks out the specified branch/tag** (if configured)
4. **Extracts the compose file** from the specified subpath (or root)
5. **Uses Git commit hash** as the version identifier
## Version Identifiers
For Git sources, ComposeSync uses the Git commit hash as the version identifier:
```
compose-a1b2c3d.yml.bak # Git commit hash
[beta-release]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/beta"
TOOL = "git"
GIT_REF = "beta-v2.2.0"
```
This provides precise version tracking and makes it easy to identify which commit a version came from.
## Repository Management
### Local Repository Storage
Git repositories are stored locally in `.git` directories:
```
/opt/composesync/stacks/myapp/
├── docker-compose.yml
├── docker-compose.override.yml
├── compose-a1b2c3d.yml.bak
└── myapp.git/ # Git repository
### Commit Hashes
```toml
[specific-commit]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/commit"
TOOL = "git"
GIT_REF = "abc123def456789"
```
### Repository Updates
## Subdirectory Configuration
ComposeSync automatically:
- Clones new repositories when first encountered
- Fetches updates from existing repositories
- Handles branch/tag changes
- Maintains local repository state
If your Docker Compose file is not in the repository root, use `GIT_SUBPATH`:
```toml
[app-with-subdir]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
```
Common subdirectory patterns:
- `docker/docker-compose.yml`
- `deploy/docker-compose.yml`
- `compose/docker-compose.yml`
- `infrastructure/docker-compose.yml`
## Multiple Compose Files with Git
For repositories with multiple compose files:
```toml
[complex-app]
URL = "https://github.com/user/complex-app.git"
PATH = "/opt/composesync/stacks/complex"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
EXTRA_FILES_1 = "https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml"
EXTRA_FILES_2 = "https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml"
EXTRA_FILES_ORDER = "1,2"
```
## Private Repositories
### SSH Authentication
For private repositories using SSH:
```toml
[private-app]
URL = "git@github.com:user/private-app.git"
PATH = "/opt/composesync/stacks/private"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
```
**Setup SSH keys:**
```bash
# Generate SSH key (if not exists)
ssh-keygen -t ed25519 -C "composesync@your-server"
# Add to SSH agent
ssh-add ~/.ssh/id_ed25519
# Add public key to GitHub/GitLab
cat ~/.ssh/id_ed25519.pub
```
### HTTPS with Personal Access Token
For private repositories using HTTPS:
```toml
[private-app]
URL = "https://github.com/user/private-app.git"
PATH = "/opt/composesync/stacks/private"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
```
**Setup credentials:**
```bash
# Configure Git credentials
git config --global credential.helper store
git config --global user.name "Your Name"
git config --global user.email "your-email@example.com"
# Test access
git clone https://github.com/user/private-app.git /tmp/test
```
## Best Practices
### 1. Use Specific Branches/Tags
### 1. Use Semantic Versioning
```toml
[production]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/production"
TOOL = "git"
GIT_REF = "v2.1.0" # Specific version
For production environments, pin to specific branches or tags for stability:
```env
STACK_1_GIT_REF=v2.1.0 # Pin to specific version
[development]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/development"
TOOL = "git"
GIT_REF = "develop" # Latest development
```
### 2. Use Subpaths for Organization
### 2. Organize by Environment
```toml
# Production environments
[prod-app]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/prod"
TOOL = "git"
GIT_REF = "main"
INTERVAL = 7200
Organize your repositories with subpaths for better structure:
```
repository/
├── docker/
│ ├── docker-compose.yml
│ └── docker-compose.prod.yml
├── docs/
└── README.md
# Development environments
[dev-app]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/dev"
TOOL = "git"
GIT_REF = "develop"
INTERVAL = 1800
```
### 3. Monitor Repository Changes
### 3. Use Descriptive Branch Names
```toml
[stable]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/stable"
TOOL = "git"
GIT_REF = "stable"
Use webhook notifications to monitor when repositories are updated:
```env
NOTIFICATION_WEBHOOK_URL=https://your-webhook-url.com/endpoint
[experimental]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/experimental"
TOOL = "git"
GIT_REF = "experimental"
```
### 4. Test with Dry-Run Mode
Always test new Git configurations with dry-run mode:
```env
DRY_RUN=true
```
## Troubleshooting Git Issues
## Troubleshooting
### Repository Not Found
```bash
# Test repository access
git clone https://github.com/user/repo.git /tmp/test
If you get "Failed to clone" errors:
- Verify the repository URL is correct
- Ensure the repository is public or you have access
- Check network connectivity
# Check SSH access
ssh -T git@github.com
```
### Branch/Tag Not Found
```bash
# List available branches
git ls-remote --heads https://github.com/user/repo.git
# List available tags
git ls-remote --tags https://github.com/user/repo.git
```
### Subpath Not Found
If you get "Subpath not found" errors:
- Verify the subpath exists in the repository
- Check the branch/tag contains the file
- Use the correct path separator (forward slashes)
### Branch/Tag Issues
If you get "Failed to checkout" errors:
- Verify the branch/tag exists
- Check for typos in the reference name
- Ensure the reference is accessible
## Example Configurations
### Simple Repository (Root Compose File)
```env
STACK_1_NAME=simple-app
STACK_1_URL=https://github.com/user/simple-app.git
STACK_1_PATH=/opt/composesync/stacks/simple-app
STACK_1_TOOL=git
STACK_1_GIT_REF=main
```bash
# Check if file exists in repository
git ls-tree -r --name-only HEAD | grep docker-compose.yml
```
### Complex Repository (Subpath + Tag)
### Authentication Issues
```bash
# Test SSH connection
ssh -T git@github.com
```env
STACK_1_NAME=complex-app
STACK_1_URL=https://github.com/org/complex-app.git
STACK_1_PATH=/opt/composesync/stacks/complex-app
STACK_1_TOOL=git
STACK_1_GIT_SUBPATH=docker/compose/production.yml
STACK_1_GIT_REF=v1.2.3
# Test HTTPS with token
curl -H "Authorization: token YOUR_TOKEN" https://api.github.com/user
```
### Development Branch
## Monitoring Git Operations
```env
STACK_1_NAME=dev-app
STACK_1_URL=https://github.com/user/dev-app.git
STACK_1_PATH=/opt/composesync/stacks/dev-app
STACK_1_TOOL=git
STACK_1_GIT_SUBPATH=docker/docker-compose.yml
STACK_1_GIT_REF=develop
STACK_1_INTERVAL=1800 # Check every 30 minutes for dev
### View Git Operations in Logs
```bash
# View ComposeSync logs
sudo journalctl -u composesync -f
# Filter for Git operations
sudo journalctl -u composesync | grep "git"
```
### Check Repository Status
```bash
# Check if repository is cloned
ls -la /opt/composesync/stacks/my-app/.git
# Check current branch/commit
git -C /opt/composesync/stacks/my-app/.git rev-parse --abbrev-ref HEAD
git -C /opt/composesync/stacks/my-app/.git rev-parse HEAD
```
### Manual Git Operations
```bash
# Manually fetch updates
cd /opt/composesync/stacks/my-app/.git
git fetch origin
# Check available branches
git branch -r
# Switch to different branch
git checkout develop
```

203
Docs/installation.md
View file

@ -20,113 +20,164 @@ Since ComposeSync runs with Docker group privileges, ensure:
## Prerequisites
Make sure these are installed on your host system:
* **Docker Engine** and **Docker Compose Plugin** (ensure `docker` and `docker compose` commands work)
* **Basic Utilities:** `wget`, `git`, `bash`, `curl`, `diffutils`
```bash
# On Debian/Ubuntu:
sudo apt install wget git bash curl diffutils
```
* **User in Docker Group:** The user you specify during installation must be in the `docker` group
```bash
sudo usermod -aG docker YOUR_USERNAME
# Then log out and back in
```
- Linux system with systemd
- Docker and Docker Compose installed
- User with sudo privileges
- Git and wget (for downloading compose files)
## Installation Steps
1. **Clone this repository:**
```bash
git clone <your-repo-url>
cd composesync
```
### 1. Download and Install
2. **Run the installation script as root:**
```bash
# Clone or download ComposeSync
git clone https://github.com/your-repo/ComposeSync.git
cd ComposeSync
# Run the installation script
sudo ./install.sh
```
The script will:
- Ask for the username to run ComposeSync
- Verify the user exists and is in the docker group
- Create necessary directories and set permissions
- Set up the systemd service
- Create a default configuration
3. **Configure your stacks in `/opt/composesync/.env`:**
The installation script will:
- Create the `/opt/composesync` directory
- Copy scripts and configuration files
- Create a systemd service
- Set up proper permissions
### 2. Configure Your Stacks
#### TOML Configuration (Recommended)
Create your configuration file:
```bash
sudo nano /opt/composesync/config.toml
```
Example configuration:
```toml
# Global settings
[global]
UPDATE_INTERVAL_SECONDS = 3600 # Check every hour
KEEP_VERSIONS = 10
DRY_RUN = false
# Stack configurations
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
[dev-app]
URL = "https://github.com/user/dev-app.git"
PATH = "/opt/composesync/stacks/dev-app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "develop"
INTERVAL = 1800 # Check every 30 minutes
```
#### Legacy .env Configuration
If you prefer the legacy .env format:
```bash
sudo nano /opt/composesync/.env
```
Example configuration:
```env
# Base directory for stacks
COMPOSESYNC_BASE_DIR=/opt/composesync/stacks
# Number of versions to keep (default: 10)
KEEP_VERSIONS=10
# Update interval in seconds (default: 3600)
UPDATE_INTERVAL_SECONDS=3600
# Update mode (notify_only or notify_and_apply)
UPDATE_MODE=notify_and_apply
# Stack configurations
STACKS=1
KEEP_VERSIONS=10
DRY_RUN=false
STACKS=2
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
STACK_1_INTERVAL=86400
STACK_1_KEEP_VERSIONS=10
STACK_2_NAME=dev-app
STACK_2_URL=https://github.com/user/dev-app.git
STACK_2_PATH=/opt/composesync/stacks/dev-app
STACK_2_TOOL=git
STACK_2_GIT_SUBPATH=docker/docker-compose.yml
STACK_2_GIT_REF=develop
```
4. **Create stack directories:**
### 3. Create Stack Directories
Create directories for your stacks:
```bash
# For each stack in your configuration
sudo mkdir -p /opt/composesync/stacks/immich
sudo chown YOUR_USERNAME:docker /opt/composesync/stacks/immich
sudo mkdir -p /opt/composesync/stacks/dev-app
# Set proper ownership
sudo chown -R $USER:docker /opt/composesync/stacks/
```
5. **Restart the service to apply changes:**
### 4. Create Override Files (Optional)
Create `docker-compose.override.yml` files for your customizations:
```bash
sudo systemctl restart composesync
sudo nano /opt/composesync/stacks/immich/docker-compose.override.yml
```
## How It Works
ComposeSync runs as a systemd service and monitors your configured stacks for updates. When changes are detected, it:
1. Creates a backup of the current configuration
2. Downloads the new configuration
3. Applies your custom overrides
4. Updates the stack if in `notify_and_apply` mode
5. Maintains versioned copies of configurations
## Directory Structure
Example override:
```yaml
version: '3.8'
services:
immich-server:
environment:
- DATABASE_URL=postgresql://user:pass@localhost:5432/immich
volumes:
- /path/to/photos:/photos
```
/opt/composesync/
├── update-agent.sh
├── .env
└── stacks/
├── immich/
│ ├── docker-compose.yml
│ ├── docker-compose.override.yml
│ ├── compose-*.yml.bak
│ └── backups/
└── portainer/
├── docker-compose.yml
├── docker-compose.override.yml
├── compose-*.yml.bak
└── backups/
### 5. Start the Service
```bash
# Start ComposeSync
sudo systemctl start composesync
# Enable it to start on boot
sudo systemctl enable composesync
# Check the status
sudo systemctl status composesync
```
### 6. Verify Installation
Check the logs to ensure everything is working:
```bash
# View real-time logs
sudo journalctl -u composesync -f
# Check recent logs
sudo journalctl -u composesync -n 50
```
## Configuration Files
After installation, you'll have these files:
- `/opt/composesync/update-agent.sh` - Main update script
- `/opt/composesync/config-parser.sh` - Configuration parser
- `/opt/composesync/config.toml` - TOML configuration (recommended)
- `/opt/composesync/.env` - Legacy .env configuration
- `/opt/composesync/config.toml.example` - Example TOML configuration
## Next Steps
After installation, you should:
1. **[Configure your stacks](configuration.md)** - Set up your Docker Compose stacks
2. **[Create override files](multi-stack.md#creating-override-files)** - Add your customizations
3. **[Test with dry-run mode](dry-run.md)** - Verify your configuration
4. **[Set up monitoring](monitoring.md)** - Monitor the service logs
- **[Configuration Reference](configuration.md)** - All configuration options
- **[Multi-Stack Setup](multi-stack.md)** - Managing multiple stacks
- **[Safety Features](safety-features.md)** - Rollback and error handling
- **[Monitoring](monitoring.md)** - Logs and service management

601
Docs/monitoring.md
View file

@ -1,67 +1,25 @@
# Monitoring Guide
# Monitoring and Logs
This guide covers how to monitor and manage your ComposeSync service.
This guide covers how to monitor ComposeSync, view logs, and troubleshoot issues.
## Viewing Logs
## Overview
### Real-time Log Monitoring
Watch logs in real-time:
```bash
sudo journalctl -u composesync -f
```
### Recent Logs
View recent log entries:
```bash
# Last 50 lines
sudo journalctl -u composesync -n 50
# Last hour
sudo journalctl -u composesync --since "1 hour ago"
# Today's logs
sudo journalctl -u composesync --since "today"
```
### Log Filtering
Filter logs by specific criteria:
```bash
# Only error messages
sudo journalctl -u composesync -p err
# Only messages containing "immich"
sudo journalctl -u composesync | grep immich
# Dry-run messages only
sudo journalctl -u composesync | grep "DRY-RUN"
```
### Log Export
Export logs to a file:
```bash
# Export today's logs
sudo journalctl -u composesync --since "today" > composesync-logs.txt
# Export all logs
sudo journalctl -u composesync > composesync-all-logs.txt
```
ComposeSync provides comprehensive logging and monitoring capabilities to help you track updates, diagnose issues, and ensure your stacks are running smoothly.
## Service Control
### Service Status
### Check Service Status
Check service status:
```bash
# Check if ComposeSync is running
sudo systemctl status composesync
# Check if service is enabled
sudo systemctl is-enabled composesync
```
### Start/Stop/Restart
### Start/Stop/Restart Service
Control the service:
```bash
# Start the service
sudo systemctl start composesync
@ -72,14 +30,6 @@ sudo systemctl stop composesync
# Restart the service
sudo systemctl restart composesync
# Reload configuration (if supported)
sudo systemctl reload composesync
```
### Enable/Disable
Manage service startup:
```bash
# Enable service to start on boot
sudo systemctl enable composesync
@ -87,265 +37,450 @@ sudo systemctl enable composesync
sudo systemctl disable composesync
```
### Service Information
## Viewing Logs
### Real-Time Logs
Get detailed service information:
```bash
# Show service configuration
sudo systemctl show composesync
# Follow logs in real-time
sudo journalctl -u composesync -f
# Show service dependencies
sudo systemctl list-dependencies composesync
# Follow logs with timestamps
sudo journalctl -u composesync -f --output=short-precise
```
### Recent Logs
```bash
# View last 50 log entries
sudo journalctl -u composesync -n 50
# View logs from last hour
sudo journalctl -u composesync --since "1 hour ago"
# View logs from today
sudo journalctl -u composesync --since "today"
# View logs from specific time
sudo journalctl -u composesync --since "2024-01-15 10:00:00"
```
### Filtered Logs
```bash
# View only error messages
sudo journalctl -u composesync | grep "ERROR"
# View logs for specific stack
sudo journalctl -u composesync | grep "immich"
# View successful updates
sudo journalctl -u composesync | grep "Successfully updated"
# View rollback events
sudo journalctl -u composesync | grep "rollback"
# View webhook notifications
sudo journalctl -u composesync | grep "webhook"
```
## Log Format
ComposeSync logs follow this format:
```
[2024-01-15 10:30:00] Loading TOML configuration from /opt/composesync/config.toml
[2024-01-15 10:30:00] Processing stack: immich
[2024-01-15 10:30:01] Downloading https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
[2024-01-15 10:30:02] Changes detected for immich
[2024-01-15 10:30:02] Creating backup: compose-20240115103002.yml.bak
[2024-01-15 10:30:03] Successfully updated stack immich
[2024-01-15 10:30:03] Sending webhook notification for immich
```
### Log Levels
- **INFO**: Normal operations, successful updates
- **WARNING**: Non-critical issues, skipped stacks
- **ERROR**: Failed operations, rollbacks, configuration issues
## Configuration Monitoring
### Check Current Configuration
```bash
# View TOML configuration
sudo cat /opt/composesync/config.toml
# View .env configuration (if using legacy format)
sudo cat /opt/composesync/.env
# Check configuration syntax
sudo systemctl restart composesync
sudo journalctl -u composesync -n 10
```
### Validate Configuration
```bash
# Test TOML syntax
sudo systemctl restart composesync
sudo journalctl -u composesync -n 20 | grep "ERROR"
# Check for missing variables
sudo systemctl status composesync
```
## Stack Monitoring
### Check Stack Status
Monitor individual stacks:
```bash
# Check if a specific stack is running
docker compose -f /opt/composesync/stacks/immich/docker-compose.yml ps
# List all configured stacks
ls -la /opt/composesync/stacks/
# Check all stacks
# Check if stack directories exist
for stack in /opt/composesync/stacks/*/; do
echo "=== $(basename $stack) ==="
docker compose -f "$stack/docker-compose.yml" ps
ls -la "$stack"
done
```
### Stack Health
### Monitor Stack Updates
Check stack health and logs:
```bash
# Check stack logs
docker compose -f /opt/composesync/stacks/immich/docker-compose.yml logs
# View recent stack updates
sudo journalctl -u composesync | grep "Processing stack"
# Check specific service logs
docker compose -f /opt/composesync/stacks/immich/docker-compose.yml logs immich-server
# Check update frequency
sudo journalctl -u composesync | grep "UPDATE_INTERVAL_SECONDS"
# Follow logs in real-time
docker compose -f /opt/composesync/stacks/immich/docker-compose.yml logs -f
# View stack-specific logs
sudo journalctl -u composesync | grep "immich"
```
### Version History
### Check Docker Compose Status
Check versioned files:
```bash
# List versioned files for a stack
ls -la /opt/composesync/stacks/immich/compose-*.yml.bak
# Check backup directories
ls -la /opt/composesync/stacks/immich/backups/
# Check if stacks are running
for stack in /opt/composesync/stacks/*/; do
stack_name=$(basename $stack)
echo "=== $stack_name ==="
if [ -f "$stack/docker-compose.yml" ]; then
docker compose -f "$stack/docker-compose.yml" ps
else
echo "No docker-compose.yml found"
fi
done
```
## Performance Monitoring
### Resource Usage
### Check Resource Usage
Monitor system resources:
```bash
# Check CPU and memory usage
top -p $(pgrep -f update-agent.sh)
# Monitor system resources
htop
# Check disk usage
df -h /opt/composesync/
# Check backup storage
du -sh /opt/composesync/stacks/*/
# Check for large backup directories
find /opt/composesync/stacks/ -name "backups" -type d -exec du -sh {} \;
# Count backup files
find /opt/composesync/stacks -name "*.bak" | wc -l
```
### Network Monitoring
### Monitor Update Intervals
Monitor download activity:
```bash
# Check network connections
netstat -tulpn | grep wget
netstat -tulpn | grep git
# Check update intervals in configuration
grep "UPDATE_INTERVAL_SECONDS" /opt/composesync/config.toml
# Monitor bandwidth usage
iftop -i eth0
# View actual update timing
sudo journalctl -u composesync | grep "Processing stack" | tail -10
```
## Alerting and Notifications
## Webhook Monitoring
### Webhook Monitoring
### Check Webhook Configuration
Test webhook notifications:
```bash
# Check if webhook is configured
grep "NOTIFICATION_WEBHOOK_URL" /opt/composesync/config.toml
# Test webhook manually
curl -X POST -H "Content-Type: application/json" \
-d '{"event": "test", "message": "Test notification"}' \
$NOTIFICATION_WEBHOOK_URL
https://your-webhook-url.com/endpoint
```
### Email Notifications
### Monitor Webhook Delivery
Set up email alerts (if your webhook supports it):
```bash
# Example: Send email via webhook
curl -X POST -H "Content-Type: application/json" \
-d '{"to": "admin@example.com", "subject": "ComposeSync Alert", "body": "Update failed"}' \
https://your-webhook-service.com/email
# View webhook notifications
sudo journalctl -u composesync | grep "webhook"
# Check for webhook errors
sudo journalctl -u composesync | grep "ERROR.*webhook"
# View webhook payloads
sudo journalctl -u composesync | grep "payload"
```
## Monitoring Best Practices
## Error Monitoring
### Regular Health Checks
### Common Error Types
```bash
# Download failures
sudo journalctl -u composesync | grep "Failed to download"
# Docker Compose failures
sudo journalctl -u composesync | grep "docker compose.*failed"
# Permission errors
sudo journalctl -u composesync | grep "Permission denied"
# Configuration errors
sudo journalctl -u composesync | grep "Configuration error"
# Lock file issues
sudo journalctl -u composesync | grep "lock"
```
### Error Analysis
```bash
# View all errors from last hour
sudo journalctl -u composesync --since "1 hour ago" | grep "ERROR"
# Count errors by type
sudo journalctl -u composesync | grep "ERROR" | cut -d' ' -f4- | sort | uniq -c
# View error context
sudo journalctl -u composesync | grep -A 5 -B 5 "ERROR"
```
## Backup Monitoring
### Check Backup Status
```bash
# List all backup files
find /opt/composesync/stacks -name "*.bak" -type f
# Check backup retention
for stack in /opt/composesync/stacks/*/; do
stack_name=$(basename $stack)
backup_count=$(find "$stack" -name "*.bak" | wc -l)
echo "$stack_name: $backup_count backups"
done
# Check backup sizes
find /opt/composesync/stacks -name "*.bak" -exec ls -lh {} \;
```
### Verify Backup Integrity
```bash
# Test backup file syntax
for backup in /opt/composesync/stacks/*/*.bak; do
echo "Testing $backup"
docker compose -f "$backup" config > /dev/null 2>&1
if [ $? -eq 0 ]; then
echo "✓ Valid"
else
echo "✗ Invalid"
fi
done
```
## Health Checks
### Automated Health Check Script
Create a monitoring script:
Set up regular monitoring:
```bash
# Create a monitoring script
cat > /usr/local/bin/composesync-health-check.sh << 'EOF'
#!/bin/bash
# /opt/composesync/health-check.sh
# Check if service is running
if ! systemctl is-active --quiet composesync; then
echo "ComposeSync service is not running!"
exit 1
echo "=== ComposeSync Health Check ==="
echo "Date: $(date)"
echo
# Check service status
echo "Service Status:"
if systemctl is-active --quiet composesync; then
echo "✓ ComposeSync is running"
else
echo "✗ ComposeSync is not running"
fi
# Check for recent errors
if journalctl -u composesync --since "1 hour ago" | grep -q "ERROR"; then
echo "ComposeSync has errors in the last hour"
exit 1
# Check recent errors
echo
echo "Recent Errors (last hour):"
error_count=$(journalctl -u composesync --since "1 hour ago" | grep -c "ERROR")
if [ $error_count -eq 0 ]; then
echo "✓ No errors in the last hour"
else
echo "✗ $error_count errors in the last hour"
fi
# Check stack status
echo
echo "Stack Status:"
for stack in /opt/composesync/stacks/*/; do
stack_name=$(basename $stack)
if [ -f "$stack/docker-compose.yml" ]; then
if docker compose -f "$stack/docker-compose.yml" ps --quiet | grep -q .; then
echo "✓ $stack_name is running"
else
echo "✗ $stack_name is not running"
fi
else
echo "? $stack_name has no docker-compose.yml"
fi
done
# Check disk usage
if [ $(df /opt/composesync | tail -1 | awk '{print $5}' | sed 's/%//') -gt 90 ]; then
echo "ComposeSync disk usage is high"
exit 1
echo
echo "Disk Usage:"
usage=$(df -h /opt/composesync/ | tail -1 | awk '{print $5}' | sed 's/%//')
if [ $usage -lt 80 ]; then
echo "✓ Disk usage: ${usage}%"
else
echo "✗ High disk usage: ${usage}%"
fi
echo "ComposeSync is healthy"
exit 0
EOF
chmod +x /usr/local/bin/composesync-health-check.sh
```
### Automated Monitoring
Make it executable and run:
Set up automated monitoring with cron:
```bash
# Add to crontab
echo "*/15 * * * * /usr/local/bin/composesync-health-check.sh" | sudo crontab -
sudo chmod +x /opt/composesync/health-check.sh
sudo /opt/composesync/health-check.sh
```
### Log Rotation
### Scheduled Health Checks
Add to crontab for regular monitoring:
Configure log rotation to prevent disk space issues:
```bash
# Create logrotate configuration
sudo tee /etc/logrotate.d/composesync << EOF
/var/log/composesync/*.log {
daily
missingok
rotate 7
compress
delaycompress
notifempty
create 644 composesync composesync
}
EOF
# Edit crontab
sudo crontab -e
# Add health check every hour
0 * * * * /opt/composesync/health-check.sh >> /var/log/composesync-health.log 2>&1
```
## Troubleshooting Monitoring
## Alerting
### Service Not Starting
### Set Up Email Alerts
Create an alert script:
If the service won't start:
```bash
# Check for configuration errors
#!/bin/bash
# /opt/composesync/alert.sh
# Check for errors in last 10 minutes
errors=$(journalctl -u composesync --since "10 minutes ago" | grep -c "ERROR")
if [ $errors -gt 0 ]; then
echo "ComposeSync Alert: $errors errors detected" | \
mail -s "ComposeSync Alert" your-email@example.com
fi
```
### Webhook Alerts
Configure webhook notifications in your TOML file:
```toml
[global]
UPDATE_INTERVAL_SECONDS = 3600
KEEP_VERSIONS = 10
DRY_RUN = false
NOTIFICATION_WEBHOOK_URL = "https://your-webhook-url.com/endpoint"
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
```
## Best Practices
### 1. Regular Log Review
```bash
# Daily log review
sudo journalctl -u composesync --since "yesterday" | grep "ERROR"
# Weekly summary
sudo journalctl -u composesync --since "1 week ago" | grep "Successfully updated" | wc -l
```
### 2. Monitor Key Metrics
- **Update success rate**: Track successful vs failed updates
- **Error frequency**: Monitor error patterns
- **Response time**: Track how long updates take
- **Disk usage**: Monitor backup storage
### 3. Set Up Automated Monitoring
```bash
# Create monitoring dashboard
sudo mkdir -p /opt/composesync/monitoring
sudo nano /opt/composesync/monitoring/dashboard.sh
```
### 4. Document Issues
Keep a log of issues and resolutions:
```bash
# Create issue log
sudo nano /opt/composesync/issue-log.md
```
## Troubleshooting Commands
### Quick Diagnostics
```bash
# Check service status
sudo systemctl status composesync
# Check logs for specific errors
# View recent logs
sudo journalctl -u composesync -n 20
# Verify file permissions
# Check configuration
sudo cat /opt/composesync/config.toml
# Test Docker access
docker ps
# Check file permissions
ls -la /opt/composesync/
```
### High Resource Usage
### Detailed Diagnostics
If ComposeSync is using too many resources:
```bash
# Check what's consuming resources
ps aux | grep update-agent
# Full system check
sudo /opt/composesync/health-check.sh
# Check for stuck processes
pgrep -f update-agent
# Restart the service
# Configuration validation
sudo systemctl restart composesync
sudo journalctl -u composesync -n 50
# Stack verification
for stack in /opt/composesync/stacks/*/; do
echo "=== $(basename $stack) ==="
docker compose -f "$stack/docker-compose.yml" config
done
```
### Missing Logs
If logs are missing:
```bash
# Check if journald is working
sudo journalctl --verify
# Check journald status
sudo systemctl status systemd-journald
# Check log storage
sudo journalctl --disk-usage
```
## Integration with External Monitoring
### Prometheus/Grafana
For advanced monitoring, you can integrate with Prometheus:
```bash
# Example: Export metrics via webhook
curl -X POST -H "Content-Type: application/json" \
-d '{"metric": "composesync_updates_total", "value": 1, "labels": {"stack": "immich"}}' \
http://prometheus:9090/api/v1/write
```
### Nagios/Icinga
Create custom checks for monitoring systems:
```bash
# Example Nagios check
#!/bin/bash
if systemctl is-active --quiet composesync; then
echo "OK: ComposeSync is running"
exit 0
else
echo "CRITICAL: ComposeSync is not running"
exit 2
fi
```
## Security Monitoring
### Access Monitoring
Monitor who accesses ComposeSync:
```bash
# Check who modified the configuration
ls -la /opt/composesync/.env
# Check for unauthorized changes
find /opt/composesync -mtime -1 -ls
# Monitor Docker group membership
getent group docker
```
### Audit Logging
Enable audit logging:
```bash
# Monitor file access
auditctl -w /opt/composesync/.env -p wa -k composesync_config
# Monitor Docker socket access
auditctl -w /var/run/docker.sock -p wa -k docker_access
```
This comprehensive monitoring setup will help you keep track of ComposeSync's health and performance.

543
Docs/multi-stack.md
View file

@ -1,254 +1,381 @@
# Multi-Stack Configuration
# Multi-Stack Setup
This guide covers how to configure and manage multiple Docker Compose stacks with ComposeSync.
## Basic Multi-Stack Setup
## Overview
You can manage multiple stacks by adding configurations to your `.env` file. Each stack is configured with a numbered prefix:
ComposeSync can manage multiple Docker Compose stacks simultaneously, each with its own configuration, update schedule, and customizations.
```env
# Number of stacks to manage
STACKS=2
## TOML Configuration (Recommended)
# Stack 1: Immich
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
STACK_1_INTERVAL=86400
The TOML format makes multi-stack configuration much cleaner and more readable:
# Stack 2: Portainer
STACK_2_NAME=portainer
STACK_2_URL=https://github.com/portainer/portainer-compose.git
STACK_2_PATH=/opt/composesync/stacks/portainer
STACK_2_TOOL=git
STACK_2_INTERVAL=43200
```toml
# Global settings for all stacks
[global]
UPDATE_INTERVAL_SECONDS = 3600 # Default: check every hour
KEEP_VERSIONS = 10 # Default: keep 10 versions
DRY_RUN = false # Default: apply changes
NOTIFICATION_WEBHOOK_URL = "https://your-webhook-url.com/endpoint"
# Production stacks
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
INTERVAL = 7200 # Check every 2 hours for production
[portainer]
URL = "https://github.com/portainer/portainer-compose.git"
PATH = "/opt/composesync/stacks/portainer"
TOOL = "git"
GIT_SUBPATH = "compose/docker-compose.yml"
GIT_REF = "main"
INTERVAL = 86400 # Check daily for stable tools
# Development stacks
[dev-app]
URL = "https://github.com/user/dev-app.git"
PATH = "/opt/composesync/stacks/dev-app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "develop"
INTERVAL = 1800 # Check every 30 minutes for dev
KEEP_VERSIONS = 5 # Keep fewer versions for dev
# Complex stack with multiple files
[complex-stack]
URL = "https://github.com/user/complex-app.git"
PATH = "/opt/composesync/stacks/complex"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
EXTRA_FILES_1 = "https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml"
EXTRA_FILES_2 = "https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml"
EXTRA_FILES_ORDER = "1,2"
```
## Multiple Compose Files
### TOML Benefits for Multi-Stack
For stacks that use multiple compose files (like Immich), you can specify them in the configuration:
- **No numbering required** - Stack names are used directly
- **Clear organization** - Group stacks by purpose with comments
- **Per-stack overrides** - Each stack can override global settings
- **Easy maintenance** - Add/remove stacks without renumbering
```env
STACKS=1
## Legacy .env Configuration
# Immich stack with multiple compose files
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
STACK_1_INTERVAL=86400
STACK_1_KEEP_VERSIONS=10
# Additional compose files (numbered)
STACK_1_EXTRA_FILES_1=https://raw.githubusercontent.com/immich-app/immich/refs/heads/main/docker/hwaccel.ml.yml
STACK_1_EXTRA_FILES_2=https://raw.githubusercontent.com/immich-app/immich/refs/heads/main/docker/hwaccel.transcoding.yml
STACK_1_EXTRA_FILES_3=https://raw.githubusercontent.com/immich-app/immich/refs/heads/main/docker/prometheus.yml
# Custom file ordering (optional)
STACK_1_EXTRA_FILES_ORDER=3,1,2
```
If `STACK_1_EXTRA_FILES_ORDER` is set, extra files will be processed in the specified order (e.g., 3,1,2). Otherwise, files are processed in numeric order (1,2,3,...).
## Complete Example: Immich Stack
Here's a complete example of setting up Immich with all its compose files:
### 1. Configure the stack in `.env`
```env
STACKS=1
# Main compose file
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
STACK_1_INTERVAL=86400
STACK_1_KEEP_VERSIONS=10
# Additional compose files (one per line, numbered)
# 1. Hardware acceleration for machine learning
STACK_1_EXTRA_FILES_1=https://raw.githubusercontent.com/immich-app/immich/refs/heads/main/docker/hwaccel.ml.yml
# 2. Hardware acceleration for video transcoding
STACK_1_EXTRA_FILES_2=https://raw.githubusercontent.com/immich-app/immich/refs/heads/main/docker/hwaccel.transcoding.yml
# 3. Prometheus monitoring
STACK_1_EXTRA_FILES_3=https://raw.githubusercontent.com/immich-app/immich/refs/heads/main/docker/prometheus.yml
```
The script will process these files in order (1, 2, 3) when running `docker compose`.
### 2. Create your override file
```bash
sudo mkdir -p /opt/composesync/stacks/immich
sudo nano /opt/composesync/stacks/immich/docker-compose.override.yml
```
**Note:** You must create the `docker-compose.override.yml` file manually. ComposeSync will not create it for you. This file should contain your customizations and will be preserved during updates.
### 3. Add your customizations
```yaml
# docker-compose.override.yml
services:
immich-server:
environment:
- IMMICH_SERVER_URL=http://immich-server:2283
- IMMICH_API_URL_EXTERNAL=https://immich.raines.xyz/api
- IMMICH_WEB_URL=https://immich.raines.xyz
networks:
- npm_network
- immich-backend
devices:
- /dev/dri:/dev/dri
redis:
networks:
- immich-backend
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 30s
timeout: 5s
retries: 3
database:
networks:
- immich-backend
healthcheck:
test: ["CMD-SHELL", "pg_isready -d ${DB_DATABASE_NAME} -U ${DB_USERNAME}"]
interval: 30s
timeout: 5s
retries: 3
start_interval: 30s
immich-machine-learning:
image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-openvino
environment:
- OPENVINO_DEVICE=GPU
- OPENVINO_GPU_DEVICE_ID=0
devices:
- /dev/dri:/dev/dri
device_cgroup_rules:
- 'c 189:* rmw'
volumes:
- /dev/bus/usb:/dev/bus/usb
networks:
- immich-backend
volumes:
cifs_immich:
external: true
networks:
npm_network:
external: true
immich-backend:
name: immich-backend
```
### 4. Set permissions
```bash
sudo chown -R YOUR_USERNAME:docker /opt/composesync/stacks/immich
```
### 5. Restart the service
```bash
sudo systemctl restart composesync
```
## How Multiple Files Work
The service will now:
1. Download the main `docker-compose.yml`
2. Download all additional compose files specified in `STACK_1_EXTRA_FILES`
3. Apply your `docker-compose.override.yml`
4. Use all files when running `docker compose up`
When running commands manually, you'll need to specify all the files:
```bash
docker compose -f docker-compose.yml \
-f hwaccel.ml.yml \
-f hwaccel.transcoding.yml \
-f prometheus.yml \
-f docker-compose.override.yml \
up -d
```
## Stack-Specific Settings
Each stack can have its own settings that override the global configuration:
The .env format uses numbered variables and is supported for backward compatibility:
```env
# Global settings
UPDATE_INTERVAL_SECONDS=3600
KEEP_VERSIONS=10
DRY_RUN=false
NOTIFICATION_WEBHOOK_URL=https://your-webhook-url.com/endpoint
# Stack 1: Check every 12 hours, keep 15 versions
STACK_1_INTERVAL=43200
STACK_1_KEEP_VERSIONS=15
# Number of stacks
STACKS=4
# Stack 2: Check every 6 hours, keep 5 versions
STACK_2_INTERVAL=21600
STACK_2_KEEP_VERSIONS=5
# Stack 1 - Immich (Production)
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
STACK_1_INTERVAL=7200
# Stack 2 - Portainer (Production)
STACK_2_NAME=portainer
STACK_2_URL=https://github.com/portainer/portainer-compose.git
STACK_2_PATH=/opt/composesync/stacks/portainer
STACK_2_TOOL=git
STACK_2_GIT_SUBPATH=compose/docker-compose.yml
STACK_2_GIT_REF=main
STACK_2_INTERVAL=86400
# Stack 3 - Development App
STACK_3_NAME=dev-app
STACK_3_URL=https://github.com/user/dev-app.git
STACK_3_PATH=/opt/composesync/stacks/dev-app
STACK_3_TOOL=git
STACK_3_GIT_SUBPATH=docker/docker-compose.yml
STACK_3_GIT_REF=develop
STACK_3_INTERVAL=1800
STACK_3_KEEP_VERSIONS=5
# Stack 4 - Complex Stack
STACK_4_NAME=complex-stack
STACK_4_URL=https://github.com/user/complex-app.git
STACK_4_PATH=/opt/composesync/stacks/complex
STACK_4_TOOL=git
STACK_4_GIT_SUBPATH=docker/docker-compose.yml
STACK_4_GIT_REF=main
STACK_4_EXTRA_FILES_1=https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml
STACK_4_EXTRA_FILES_2=https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml
STACK_4_EXTRA_FILES_ORDER=1,2
```
## Creating Stack Directories
For each stack in your configuration, create the corresponding directory:
```bash
# Create directories for all stacks
sudo mkdir -p /opt/composesync/stacks/immich
sudo mkdir -p /opt/composesync/stacks/portainer
sudo mkdir -p /opt/composesync/stacks/dev-app
sudo mkdir -p /opt/composesync/stacks/complex
# Set proper ownership
sudo chown -R $USER:docker /opt/composesync/stacks/
```
## Creating Override Files
For each stack, you should create a `docker-compose.override.yml` file in the stack directory. This file contains your customizations and will be preserved during updates.
Each stack can have its own `docker-compose.override.yml` file for customizations:
### Immich Override
```bash
sudo nano /opt/composesync/stacks/immich/docker-compose.override.yml
```
Example override file structure:
```yaml
version: '3.8'
services:
your-service:
immich-server:
environment:
- CUSTOM_VAR=value
networks:
- your-network
- DATABASE_URL=postgresql://user:pass@localhost:5432/immich
- REDIS_URL=redis://localhost:6379
volumes:
- /host/path:/container/path
networks:
your-network:
external: true
- /mnt/photos:/photos
- /mnt/backups:/backups
restart: unless-stopped
```
## Managing Multiple Stacks
### Viewing All Stacks
### Portainer Override
```bash
ls -la /opt/composesync/stacks/
sudo nano /opt/composesync/stacks/portainer/docker-compose.override.yml
```
### Checking Stack Status
```bash
# Check if a specific stack is running
docker compose -f /opt/composesync/stacks/immich/docker-compose.yml ps
```yaml
version: '3.8'
services:
portainer:
environment:
- PORTAINER_EDGE=0
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /mnt/portainer-data:/data
restart: unless-stopped
```
# Check all stacks
### Development App Override
```bash
sudo nano /opt/composesync/stacks/dev-app/docker-compose.override.yml
```
```yaml
version: '3.8'
services:
dev-app:
environment:
- NODE_ENV=development
- DEBUG=true
volumes:
- ./src:/app/src
ports:
- "3000:3000"
```
## Per-Stack Configuration Options
Each stack can override global settings:
### Update Intervals
```toml
[production-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/production"
TOOL = "wget"
INTERVAL = 7200 # Check every 2 hours
[development-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/development"
TOOL = "git"
INTERVAL = 1800 # Check every 30 minutes
```
### Version Retention
```toml
[stable-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/stable"
TOOL = "wget"
KEEP_VERSIONS = 5 # Keep fewer versions
[experimental-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/experimental"
TOOL = "git"
KEEP_VERSIONS = 20 # Keep more versions for testing
```
### Git-Specific Settings
```toml
[main-branch]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/main"
TOOL = "git"
GIT_REF = "main"
[feature-branch]
URL = "https://github.com/user/app.git"
PATH = "/opt/composesync/stacks/feature"
TOOL = "git"
GIT_REF = "feature/new-ui"
GIT_SUBPATH = "docker/docker-compose.yml"
```
## Multiple Compose Files
For complex stacks that require multiple compose files:
### TOML Configuration
```toml
[complex-stack]
URL = "https://github.com/user/complex-app.git"
PATH = "/opt/composesync/stacks/complex"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
EXTRA_FILES_1 = "https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml"
EXTRA_FILES_2 = "https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml"
EXTRA_FILES_3 = "https://raw.githubusercontent.com/user/complex-app/main/docker/monitoring.yml"
EXTRA_FILES_ORDER = "1,2,3"
```
### .env Configuration
```env
STACK_1_NAME=complex-stack
STACK_1_URL=https://github.com/user/complex-app.git
STACK_1_PATH=/opt/composesync/stacks/complex
STACK_1_TOOL=git
STACK_1_GIT_SUBPATH=docker/docker-compose.yml
STACK_1_GIT_REF=main
STACK_1_EXTRA_FILES_1=https://raw.githubusercontent.com/user/complex-app/main/docker/network.yml
STACK_1_EXTRA_FILES_2=https://raw.githubusercontent.com/user/complex-app/main/docker/volumes.yml
STACK_1_EXTRA_FILES_3=https://raw.githubusercontent.com/user/complex-app/main/docker/monitoring.yml
STACK_1_EXTRA_FILES_ORDER=1,2,3
```
## Monitoring Multiple Stacks
### View All Stack Status
```bash
# Check status of all stacks
for stack in /opt/composesync/stacks/*/; do
echo "=== $(basename $stack) ==="
docker compose -f "$stack/docker-compose.yml" ps
done
```
### Manual Updates
### View Stack Logs
```bash
# Update a specific stack manually
sudo systemctl restart composesync
# View logs for a specific stack
docker compose -f /opt/composesync/stacks/immich/docker-compose.yml logs
# Or run the update script directly
sudo -u composesync /opt/composesync/update-agent.sh
# View logs for all stacks
for stack in /opt/composesync/stacks/*/; do
echo "=== $(basename $stack) ==="
docker compose -f "$stack/docker-compose.yml" logs --tail=10
done
```
### Check Update Status
```bash
# View ComposeSync logs to see update activity
sudo journalctl -u composesync -f
# Filter logs by stack name
sudo journalctl -u composesync | grep "immich"
sudo journalctl -u composesync | grep "portainer"
```
## Best Practices
1. **Use descriptive stack names** - Make them easy to identify
2. **Group related stacks** - Keep similar applications together
3. **Set appropriate intervals** - More critical stacks can check more frequently
4. **Use stack-specific settings** - Override global settings when needed
5. **Test with dry-run mode** - Verify configurations before applying
6. **Monitor logs** - Keep an eye on update activities
### 1. Organize by Purpose
```toml
# Production stacks
[immich]
# ...
[portainer]
# ...
# Development stacks
[dev-app]
# ...
[test-app]
# ...
```
### 2. Use Appropriate Intervals
- **Production**: 2-24 hours (7200-86400 seconds)
- **Development**: 15-60 minutes (900-3600 seconds)
- **Stable tools**: Daily (86400 seconds)
### 3. Manage Version Retention
- **Production**: 5-10 versions
- **Development**: 3-5 versions
- **Experimental**: 10-20 versions
### 4. Group Related Stacks
```toml
# Web applications
[web-app]
# ...
[web-api]
# ...
# Infrastructure
[monitoring]
# ...
[backup]
# ...
```
## Troubleshooting Multi-Stack Issues
### Stack Not Updating
```bash
# Check if stack is configured
grep -A 5 "immich" /opt/composesync/config.toml
# Check stack directory exists
ls -la /opt/composesync/stacks/immich/
# Check ComposeSync logs
sudo journalctl -u composesync | grep "immich"
```
### Configuration Errors
```bash
# Test TOML syntax
sudo systemctl restart composesync
sudo journalctl -u composesync -n 20
# Check for missing variables
sudo systemctl status composesync
```
### Permission Issues
```bash
# Fix ownership for all stacks
sudo chown -R $USER:docker /opt/composesync/stacks/
# Check specific stack permissions
ls -la /opt/composesync/stacks/immich/
```

498
Docs/safety-features.md
View file

@ -1,272 +1,396 @@
# Safety Features
This guide covers the safety features built into ComposeSync to protect your Docker Compose stacks.
ComposeSync includes several safety features to protect your Docker stacks and ensure reliable operation.
## Overview
Safety features include:
- **Automatic rollback** on failed updates
- **Versioned backups** for easy recovery
- **Lock file protection** against concurrent updates
- **Override preservation** to maintain customizations
- **Error handling** with detailed logging
- **Dry-run mode** for testing configurations
## Automatic Rollback
ComposeSync includes automatic rollback functionality to handle failed updates:
When a Docker Compose update fails, ComposeSync automatically rolls back to the previous working version.
### How Rollback Works
1. **Pre-Update Backup**: Before applying changes, all current files are backed up
2. **Update Attempt**: Changes are applied using `docker compose up -d`
3. **Failure Detection**: If the Docker Compose command fails, rollback is triggered
4. **Automatic Restoration**: All files are restored from the backup
5. **Stack Restart**: The stack is restarted with the original configuration
1. **Pre-update backup** - Current configuration is saved before applying changes
2. **Update attempt** - New configuration is applied using `docker compose up -d`
3. **Failure detection** - If the update fails, the error is logged
4. **Automatic rollback** - Previous configuration is restored
5. **Notification** - Webhook notification is sent (if configured)
### Rollback Process
When a rollback occurs, ComposeSync will:
```bash
# 1. Restore main compose file
cp /opt/composesync/stacks/myapp/backups/backup-20240115103000/docker-compose.yml /opt/composesync/stacks/myapp/docker-compose.yml
# 2. Restore extra files (if any)
cp /opt/composesync/stacks/myapp/backups/backup-20240115103000/hwaccel.ml.yml /opt/composesync/stacks/myapp/hwaccel.ml.yml
# 3. Restore override file (if it existed)
cp /opt/composesync/stacks/myapp/backups/backup-20240115103000/docker-compose.override.yml /opt/composesync/stacks/myapp/docker-compose.override.yml
# 4. Restart stack with original configuration
docker compose -f /opt/composesync/stacks/myapp/docker-compose.yml up -d
# Example rollback log
[2024-01-15 10:30:00] Processing stack: immich
[2024-01-15 10:30:01] Creating backup: compose-20240115103001.yml.bak
[2024-01-15 10:30:02] Applying new configuration...
[2024-01-15 10:30:03] ERROR: docker compose up -d failed
[2024-01-15 10:30:03] Rolling back to previous version
[2024-01-15 10:30:04] Rollback successful
[2024-01-15 10:30:04] Sending webhook notification
```
### Rollback Benefits
## Versioned Backups
This ensures that:
- Your stack never gets stuck in a broken state
- Failed updates don't leave your services down
- You can quickly recover from problematic updates
- The system remains stable even with network issues or invalid configurations
ComposeSync maintains a history of your Docker Compose configurations for easy recovery.
## Lock File Protection
### Backup Naming
ComposeSync uses lock files to prevent concurrent updates to the same stack.
### How Lock Files Work
Each stack has a `.lock` file in its directory that:
- Is created when an update starts
- Is automatically removed when the update completes (success or failure)
- Has a 5-minute timeout to handle stale locks from crashed processes
- Uses directory-based locking for atomicity
### Lock File Location
Backups are named with timestamps or Git commit hashes:
```
/opt/composesync/stacks/myapp/
├── docker-compose.yml
├── docker-compose.override.yml
├── .lock/ # Lock directory
└── backups/
```
### Stale Lock Detection
If a lock file exists and is older than 5 minutes, ComposeSync will automatically remove it and proceed with the update. This handles cases where:
- The previous update process crashed
- The system was rebooted during an update
- Network issues interrupted the update process
### Lock File Benefits
Lock files prevent:
- Race conditions when multiple instances run simultaneously
- Corruption of compose files during updates
- Multiple update processes running on the same stack
## Versioned History
ComposeSync maintains a versioned history of your compose files for easy rollback and audit trails.
### Version Identifiers
- **For Git sources**: Uses the Git commit hash as the version identifier
- **For other sources**: Uses a timestamp (YYYYMMDDHHMMSS format)
### Versioned File Structure
```
stack/
├── docker-compose.yml # Current active compose file
/opt/composesync/stacks/immich/
├── docker-compose.yml # Current configuration
├── docker-compose.override.yml # Your customizations
├── compose-20240315123456.yml.bak # Versioned copy (timestamp)
├── compose-a1b2c3d.yml.bak # Versioned copy (git hash)
└── backups/ # Backup directory for rollbacks
├── compose-20240115103001.yml.bak # Timestamped backup
├── compose-20240115102001.yml.bak # Previous backup
└── compose-20240115101001.yml.bak # Older backup
```
### TOML Configuration
Configure backup retention in your TOML file:
```toml
# Global settings
[global]
UPDATE_INTERVAL_SECONDS = 3600
KEEP_VERSIONS = 10 # Keep 10 backup versions
DRY_RUN = false
# Stack configurations
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
[dev-app]
URL = "https://github.com/user/dev-app.git"
PATH = "/opt/composesync/stacks/dev-app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "develop"
KEEP_VERSIONS = 5 # Keep fewer versions for dev
```
### Legacy .env Configuration
```env
# Global settings
UPDATE_INTERVAL_SECONDS=3600
KEEP_VERSIONS=10
DRY_RUN=false
# Stack configurations
STACKS=2
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
STACK_2_NAME=dev-app
STACK_2_URL=https://github.com/user/dev-app.git
STACK_2_PATH=/opt/composesync/stacks/dev-app
STACK_2_TOOL=git
STACK_2_GIT_SUBPATH=docker/docker-compose.yml
STACK_2_GIT_REF=develop
STACK_2_KEEP_VERSIONS=5
```
### Manual Rollback
To roll back to a specific version:
To manually rollback to a previous version:
```bash
# Example: Roll back to a specific version
cp /opt/composesync/stacks/immich/compose-20240315123456.yml.bak /opt/composesync/stacks/immich/docker-compose.yml
# List available backups
ls -la /opt/composesync/stacks/immich/compose-*.yml.bak
# Restart the stack with the rolled back configuration
docker compose -f /opt/composesync/stacks/immich/docker-compose.yml \
-f /opt/composesync/stacks/immich/docker-compose.override.yml \
up -d --remove-orphans
# Restore a specific backup
sudo cp /opt/composesync/stacks/immich/compose-20240115102001.yml.bak \
/opt/composesync/stacks/immich/docker-compose.yml
# Apply the rollback
cd /opt/composesync/stacks/immich
docker compose up -d
```
### Version Cleanup
## Lock File Protection
ComposeSync automatically maintains a configurable number of versioned files (default: 10) to prevent disk space issues. You can configure this per stack using the `STACK_N_KEEP_VERSIONS` environment variable.
ComposeSync uses lock files to prevent concurrent updates that could cause conflicts.
## Backup Management
### Lock File Operation
ComposeSync creates comprehensive backups before applying any changes.
1. **Lock acquisition** - Creates a lock file before starting updates
2. **Update process** - Performs the update while lock is held
3. **Lock release** - Removes the lock file when complete
4. **Timeout handling** - Automatically removes stale locks
### Backup Structure
Each update creates a timestamped backup directory:
### Lock File Location
```
/opt/composesync/stacks/myapp/backups/
├── backup-20240115103000/
│ ├── docker-compose.yml
│ ├── docker-compose.override.yml
│ ├── hwaccel.ml.yml
│ └── hwaccel.transcoding.yml
├── backup-20240115100000/
└── backup-20240115093000/
/opt/composesync/
├── update-agent.sh
├── config.toml
├── .env
└── .lock # Global lock file
```
### Backup Contents
### Lock File Behavior
Each backup includes:
- Main `docker-compose.yml` file
- `docker-compose.override.yml` (if it exists)
- All extra compose files
- Complete state before the update
- **Single process** - Only one update process can run at a time
- **Automatic cleanup** - Stale locks are automatically removed
- **Error recovery** - Locks are released even if the process crashes
- **Timeout protection** - Long-running processes don't block indefinitely
### Backup Cleanup
## Override Preservation
Backup directories are automatically cleaned up based on the `KEEP_VERSIONS` setting:
- Default: Keep 10 backup directories
- Configurable per stack with `STACK_N_KEEP_VERSIONS`
- Oldest backups are removed first
Your `docker-compose.override.yml` files are always preserved during updates.
### Override File Structure
```
/opt/composesync/stacks/immich/
├── docker-compose.yml # Updated from remote source
├── docker-compose.override.yml # Your customizations (preserved)
├── compose-*.yml.bak # Backup versions
└── .git/ # Git repository (if using git)
```
### Override File Example
```yaml
# docker-compose.override.yml
version: '3.8'
services:
immich-server:
environment:
- DATABASE_URL=postgresql://user:pass@localhost:5432/immich
- REDIS_URL=redis://localhost:6379
volumes:
- /mnt/photos:/photos
- /mnt/backups:/backups
restart: unless-stopped
networks:
- immich-network
networks:
immich-network:
external: true
```
### How Overrides Work
1. **Download** - New `docker-compose.yml` is downloaded
2. **Preserve** - Your `docker-compose.override.yml` is kept unchanged
3. **Merge** - Docker Compose merges both files automatically
4. **Apply** - Combined configuration is applied
## Error Handling
ComposeSync includes comprehensive error handling to ensure system stability.
ComposeSync includes comprehensive error handling and logging.
### Error Types Handled
### Error Types
1. **Download Failures**: Network issues, invalid URLs, authentication problems
2. **File Validation**: Empty files, corrupted downloads, missing files
3. **Docker Compose Failures**: Invalid configurations, service startup issues
4. **Permission Issues**: File access problems, directory creation failures
5. **Lock File Issues**: Stale locks, concurrent access attempts
### Error Recovery
When errors occur, ComposeSync will:
- Log detailed error messages
- Attempt automatic recovery where possible
- Trigger rollback for critical failures
- Continue processing other stacks
- Send webhook notifications (if configured)
| Error Type | Description | Action Taken |
|------------|-------------|--------------|
| **Download failure** | Cannot download compose file | Skip stack, continue with others |
| **Parse error** | Invalid YAML in compose file | Skip stack, log error |
| **Docker failure** | `docker compose up -d` fails | Rollback to previous version |
| **Permission error** | Cannot write to stack directory | Skip stack, log error |
| **Network error** | Cannot reach remote source | Skip stack, retry later |
### Error Logging
All errors are logged with timestamps and context:
```bash
# View error logs
sudo journalctl -u composesync | grep "ERROR"
```
[2024-01-15 10:30:00] ERROR: Failed to download https://example.com/compose.yml
[2024-01-15 10:30:01] ERROR: Downloaded file is empty
[2024-01-15 10:30:02] ERROR: Failed to update stack myapp, attempting rollback...
[2024-01-15 10:30:03] Successfully rolled back stack myapp
# View recent errors
sudo journalctl -u composesync -n 100 | grep "ERROR"
# View specific stack errors
sudo journalctl -u composesync | grep "immich.*ERROR"
```
## File Validation
### Error Recovery
ComposeSync validates downloaded files before processing them.
ComposeSync automatically recovers from most errors:
### Validation Checks
- **Temporary failures** - Retried on next update cycle
- **Permanent failures** - Logged and skipped until resolved
- **Partial failures** - Other stacks continue to update normally
- **File Existence**: Ensures files were downloaded successfully
- **File Size**: Verifies files are not empty
- **File Format**: Basic YAML validation
- **Content Integrity**: Checks for corrupted downloads
## Dry-Run Mode
### Validation Failures
Test your configuration safely without applying changes.
If validation fails:
- The file is not used
- An error is logged
- Rollback is triggered if necessary
- The process continues with other files
### TOML Configuration
## Update Modes
```toml
# Global dry-run mode
[global]
UPDATE_INTERVAL_SECONDS = 3600
KEEP_VERSIONS = 10
DRY_RUN = true # Enable dry-run mode
ComposeSync supports different update modes for different safety levels.
### Notify Only Mode
```env
UPDATE_MODE=notify_only
# Stack configurations
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
```
In this mode:
- Files are downloaded and processed
- Changes are detected and logged
- No updates are applied to running stacks
- Perfect for testing and monitoring
### Per-Stack Dry-Run Mode
### Notify and Apply Mode
```toml
# Global settings
[global]
DRY_RUN = false # Default: apply changes
```env
UPDATE_MODE=notify_and_apply
# Production stack (apply changes)
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
# Development stack (dry-run only)
[dev-app]
URL = "https://github.com/user/dev-app.git"
PATH = "/opt/composesync/stacks/dev-app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "develop"
DRY_RUN = true # Override global setting
```
In this mode:
- Files are downloaded and processed
- Changes are automatically applied
- Rollback occurs on failures
- Full automation with safety features
### Dry-Run Benefits
- **Safe testing** - No actual changes are applied
- **Configuration validation** - Verify URLs and settings work
- **Change preview** - See what would be updated
- **Webhook testing** - Test notifications without affecting stacks
## Best Practices
### 1. Test with Dry-Run Mode
### 1. Use Appropriate Backup Retention
Always test new configurations with dry-run mode:
```env
```toml
# Production stacks (keep more versions)
[production]
URL = "https://..."
PATH = "/opt/composesync/stacks/production"
TOOL = "wget"
KEEP_VERSIONS = 15
# Development stacks (keep fewer versions)
[development]
URL = "https://..."
PATH = "/opt/composesync/stacks/development"
TOOL = "git"
KEEP_VERSIONS = 5
```
### 2. Test with Dry-Run Mode
```toml
# Always test new configurations
[global]
DRY_RUN = true
[new-stack]
URL = "https://..."
PATH = "/opt/composesync/stacks/new-stack"
TOOL = "wget"
```
### 2. Use Appropriate Update Intervals
### 3. Monitor Error Logs
Set reasonable update intervals based on your needs:
- Production: 6-24 hours
- Development: 30 minutes - 2 hours
- Testing: Use dry-run mode
### 3. Monitor Logs
Regularly check the service logs:
```bash
# Set up log monitoring
sudo journalctl -u composesync -f
# Check for errors regularly
sudo journalctl -u composesync --since "1 hour ago" | grep "ERROR"
```
### 4. Configure Webhook Notifications
### 4. Use Webhook Notifications
Set up webhook notifications to monitor updates:
```env
NOTIFICATION_WEBHOOK_URL=https://your-webhook-url.com/endpoint
```toml
# Get notified of issues
[global]
NOTIFICATION_WEBHOOK_URL = "https://your-webhook-url.com/endpoint"
[immich]
URL = "https://..."
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
```
### 5. Regular Backup Verification
Periodically verify your backups are working:
```bash
ls -la /opt/composesync/stacks/*/backups/
# Verify backups are being created
ls -la /opt/composesync/stacks/*/compose-*.yml.bak
# Test backup restoration
sudo cp /opt/composesync/stacks/immich/compose-*.yml.bak /tmp/test.yml
docker compose -f /tmp/test.yml config
```
### 6. Version Management
## Troubleshooting
Keep an appropriate number of versions:
- More versions = more rollback options
- Fewer versions = less disk space usage
- Balance based on your needs and storage
### Rollback Not Working
```bash
# Check if backups exist
ls -la /opt/composesync/stacks/immich/compose-*.yml.bak
# Check rollback logs
sudo journalctl -u composesync | grep "rollback"
# Verify Docker Compose works
cd /opt/composesync/stacks/immich
docker compose config
```
### Lock File Issues
```bash
# Check for stale lock
ls -la /opt/composesync/.lock
# Remove stale lock (if needed)
sudo rm /opt/composesync/.lock
# Restart service
sudo systemctl restart composesync
```
### Override File Issues
```bash
# Check override file syntax
docker compose -f /opt/composesync/stacks/immich/docker-compose.yml \
-f /opt/composesync/stacks/immich/docker-compose.override.yml config
# Verify override file permissions
ls -la /opt/composesync/stacks/immich/docker-compose.override.yml
```
### Backup Cleanup Issues
```bash
# Check backup retention settings
grep "KEEP_VERSIONS" /opt/composesync/config.toml
# Manually clean old backups
find /opt/composesync/stacks -name "compose-*.yml.bak" -mtime +30 -delete
```

878
Docs/troubleshooting.md
View file

@ -1,342 +1,648 @@
# Troubleshooting Guide
This guide covers common issues and their solutions when using ComposeSync.
This guide helps you diagnose and resolve common issues with ComposeSync.
## Service Issues
## Quick Diagnostics
### Service Status Check
```bash
# Check if ComposeSync is running
sudo systemctl status composesync
# Check recent logs
sudo journalctl -u composesync -n 20
# Check configuration
sudo cat /opt/composesync/config.toml
```
### Basic Health Check
```bash
# Check service status
if systemctl is-active --quiet composesync; then
echo "✓ ComposeSync is running"
else
echo "✗ ComposeSync is not running"
fi
# Check for recent errors
error_count=$(journalctl -u composesync --since "1 hour ago" | grep -c "ERROR")
echo "Errors in last hour: $error_count"
# Check disk usage
usage=$(df -h /opt/composesync/ | tail -1 | awk '{print $5}' | sed 's/%//')
echo "Disk usage: ${usage}%"
```
## Common Issues
### Service Won't Start
**Problem:** The ComposeSync service fails to start.
**Symptoms:**
- `systemctl status composesync` shows failed status
- Service won't start with `systemctl start composesync`
**Diagnosis:**
```bash
# Check service status
sudo systemctl status composesync
# View detailed logs
sudo journalctl -u composesync -n 50
# Check configuration syntax
sudo systemctl restart composesync
sudo journalctl -u composesync -n 10
```
**Common Causes:**
1. **Configuration Error**
```bash
# Check TOML syntax
sudo cat /opt/composesync/config.toml
# Test configuration
sudo systemctl restart composesync
sudo journalctl -u composesync -n 20 | grep "ERROR"
```
2. **Missing Dependencies**
```bash
# Check if required tools are installed
which wget
which git
which docker
which docker-compose
# Install missing tools
sudo apt update
sudo apt install wget git docker.io docker-compose-plugin
```
3. **Permission Issues**
```bash
# Check file permissions
ls -la /opt/composesync/
# Fix permissions
sudo chown -R $USER:docker /opt/composesync/
sudo chmod +x /opt/composesync/update-agent.sh
sudo chmod +x /opt/composesync/config-parser.sh
```
**Solutions:**
1. Check service status:
- Fix configuration syntax errors
- Install missing dependencies
- Correct file permissions
- Check systemd service file
### Configuration Issues
**Symptoms:**
- Service starts but doesn't process stacks
- Stacks are skipped with errors
- Configuration not loaded properly
**Diagnosis:**
```bash
# Check configuration file
sudo cat /opt/composesync/config.toml
# Check for configuration errors
sudo journalctl -u composesync | grep "Configuration"
# Test configuration loading
sudo systemctl restart composesync
sudo journalctl -u composesync -n 20
```
**Common Issues:**
1. **TOML Syntax Error**
```toml
# Incorrect TOML syntax
[global]
UPDATE_INTERVAL_SECONDS = 3600
KEEP_VERSIONS = 10
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
```
**Fix:** Ensure proper TOML syntax with correct indentation and quotes.
2. **Missing Required Fields**
```toml
# Missing required fields
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
# Missing PATH and TOOL
```
**Fix:** Add all required fields (URL, PATH, TOOL).
3. **Invalid Paths**
```toml
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/invalid/path" # Path doesn't exist
TOOL = "wget"
```
**Fix:** Create the directory or use a valid path.
**Solutions:**
- Validate TOML syntax
- Ensure all required fields are present
- Create missing directories
- Check file permissions
### Download Failures
**Symptoms:**
- Stacks are skipped with download errors
- Network connectivity issues
- Invalid URLs
**Diagnosis:**
```bash
# Check network connectivity
ping -c 3 github.com
# Test URL manually
wget -O /tmp/test.yml https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
# Check for download errors
sudo journalctl -u composesync | grep "Failed to download"
```
**Common Issues:**
1. **Network Connectivity**
```bash
# Test basic connectivity
ping -c 3 8.8.8.8
# Test DNS resolution
nslookup github.com
# Check proxy settings
echo $http_proxy
echo $https_proxy
```
2. **Invalid URLs**
```toml
# Incorrect URL format
[immich]
URL = "https://invalid-url.com/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
```
3. **Authentication Required**
```toml
# Private repository without authentication
[private-app]
URL = "https://github.com/user/private-repo.git"
PATH = "/opt/composesync/stacks/private"
TOOL = "git"
```
**Solutions:**
- Check network connectivity
- Verify URLs are correct and accessible
- Configure authentication for private repositories
- Check firewall settings
### Docker Compose Failures
**Symptoms:**
- Updates fail with Docker Compose errors
- Services don't start after updates
- Rollback occurs frequently
**Diagnosis:**
```bash
# Check Docker Compose syntax
cd /opt/composesync/stacks/immich
docker compose config
# Check service status
docker compose ps
# View service logs
docker compose logs
```
**Common Issues:**
1. **Invalid Compose File**
```bash
# Test compose file syntax
docker compose -f /opt/composesync/stacks/immich/docker-compose.yml config
# Check for syntax errors
docker compose -f /opt/composesync/stacks/immich/docker-compose.yml config 2>&1 | grep "ERROR"
```
2. **Port Conflicts**
```bash
# Check for port conflicts
netstat -tulpn | grep :80
netstat -tulpn | grep :443
# Check which services are using ports
sudo lsof -i :80
sudo lsof -i :443
```
3. **Resource Issues**
```bash
# Check available disk space
df -h
# Check available memory
free -h
# Check Docker disk usage
docker system df
```
**Solutions:**
- Fix compose file syntax errors
- Resolve port conflicts
- Free up disk space and memory
- Check Docker daemon status
### Permission Issues
**Symptoms:**
- Cannot write to stack directories
- Cannot access Docker socket
- Permission denied errors
**Diagnosis:**
```bash
# Check file permissions
ls -la /opt/composesync/
# Check user groups
groups $USER
# Check Docker socket permissions
ls -la /var/run/docker.sock
# Test Docker access
docker ps
```
**Common Issues:**
1. **User Not in Docker Group**
```bash
# Check if user is in docker group
groups $USER | grep docker
# Add user to docker group
sudo usermod -aG docker $USER
# Log out and back in, or run:
newgrp docker
```
2. **Incorrect File Ownership**
```bash
# Check ownership
ls -la /opt/composesync/stacks/
# Fix ownership
sudo chown -R $USER:docker /opt/composesync/stacks/
```
3. **Docker Socket Permissions**
```bash
# Check socket permissions
ls -la /var/run/docker.sock
# Fix socket permissions
sudo chmod 666 /var/run/docker.sock
```
**Solutions:**
- Add user to docker group
- Fix file ownership and permissions
- Restart Docker daemon if needed
- Check systemd service user configuration
### Lock File Issues
**Symptoms:**
- Service appears stuck
- Updates not running
- Lock file errors
**Diagnosis:**
```bash
# Check for lock files
ls -la /opt/composesync/.lock
# Check lock file age
stat /opt/composesync/.lock
# Check for stale locks
find /opt/composesync -name "*.lock" -mmin +5
```
**Solutions:**
```bash
# Remove stale lock file
sudo rm /opt/composesync/.lock
# Restart service
sudo systemctl restart composesync
# Check if service starts properly
sudo systemctl status composesync
```
2. Check logs for errors:
### Backup Issues
**Symptoms:**
- No backup files created
- Backup cleanup not working
- Disk space issues
**Diagnosis:**
```bash
sudo journalctl -u composesync -n 50
# Check backup files
find /opt/composesync/stacks -name "*.bak"
# Check backup retention settings
grep "KEEP_VERSIONS" /opt/composesync/config.toml
# Check disk usage
df -h /opt/composesync/
```
3. Verify user permissions:
**Common Issues:**
1. **No Backups Created**
```bash
# Ensure the service user is in the docker group
groups YOUR_USERNAME
# Check if backups are being created
ls -la /opt/composesync/stacks/immich/compose-*.bak
# Check backup creation logs
sudo journalctl -u composesync | grep "backup"
```
4. Check file permissions:
2. **Backup Cleanup Not Working**
```bash
# Ensure the service user owns the ComposeSync directory
ls -la /opt/composesync/
sudo chown -R YOUR_USERNAME:docker /opt/composesync/
# Check backup retention
grep "KEEP_VERSIONS" /opt/composesync/config.toml
# Count backup files
find /opt/composesync/stacks -name "*.bak" | wc -l
# Manual cleanup
find /opt/composesync/stacks -name "*.bak" -mtime +30 -delete
```
### Service Crashes or Stops Unexpectedly
**Problem:** The service runs but crashes or stops unexpectedly.
**Solutions:**
1. Check for configuration errors:
- Check backup creation logs
- Verify backup retention settings
- Manually clean old backups
- Check disk space availability
### Webhook Issues
**Symptoms:**
- Webhook notifications not sent
- Webhook delivery failures
- Missing notifications
**Diagnosis:**
```bash
sudo journalctl -u composesync -f
```
# Check webhook configuration
grep "NOTIFICATION_WEBHOOK_URL" /opt/composesync/config.toml
2. Verify your `.env` file syntax:
```bash
# Check for syntax errors
source /opt/composesync/.env
```
3. Test with dry-run mode:
```env
DRY_RUN=true
```
## Download Issues
### Failed Downloads
**Problem:** ComposeSync fails to download compose files.
**Solutions:**
1. Check network connectivity:
```bash
# Test if the URL is accessible
wget -q --spider https://your-url.com/docker-compose.yml
echo $?
```
2. Verify URLs in your configuration:
```bash
# Check your .env file
grep STACK_.*_URL /opt/composesync/.env
```
3. Check for authentication requirements:
- Some URLs may require authentication
- Consider using Git repositories instead
### Git Repository Issues
**Problem:** Git operations fail.
**Solutions:**
1. Verify repository access:
```bash
# Test git clone manually
git clone --quiet https://github.com/user/repo.git /tmp/test
```
2. Check Git subpath configuration:
```bash
# Ensure the subpath exists in the repository
git ls-tree -r --name-only HEAD | grep docker-compose.yml
```
3. Verify branch/tag exists:
```bash
# List available branches/tags
git ls-remote --heads https://github.com/user/repo.git
git ls-remote --tags https://github.com/user/repo.git
```
## Docker Compose Issues
### Update Failures
**Problem:** Docker Compose updates fail and trigger rollback.
**Solutions:**
1. Check Docker Compose syntax:
```bash
# Validate compose file manually
docker compose -f /path/to/docker-compose.yml config
```
2. Check for port conflicts:
```bash
# Check what's using the ports
netstat -tulpn | grep :80
```
3. Verify override file syntax:
```bash
# Test with override file
docker compose -f docker-compose.yml -f docker-compose.override.yml config
```
### Rollback Failures
**Problem:** Both the update and rollback fail.
**Solutions:**
1. Check backup files:
```bash
# Verify backups exist
ls -la /opt/composesync/stacks/*/backups/
```
2. Manual rollback:
```bash
# Manually restore from backup
cp /opt/composesync/stacks/stackname/backups/backup-*/docker-compose.yml /opt/composesync/stacks/stackname/
```
3. Check Docker daemon:
```bash
# Ensure Docker is running
sudo systemctl status docker
```
## Configuration Issues
### Missing Environment Variables
**Problem:** Required configuration is missing.
**Solutions:**
1. Check your `.env` file:
```bash
# Verify all required variables are set
grep -E "STACK_.*_(NAME|URL|PATH|TOOL)" /opt/composesync/.env
```
2. Check variable syntax:
```bash
# Look for syntax errors
cat -n /opt/composesync/.env
```
### Invalid Paths
**Problem:** Stack paths don't exist or are inaccessible.
**Solutions:**
1. Create missing directories:
```bash
# Create stack directories
sudo mkdir -p /opt/composesync/stacks/stackname
sudo chown YOUR_USERNAME:docker /opt/composesync/stacks/stackname
```
2. Check permissions:
```bash
# Verify directory permissions
ls -la /opt/composesync/stacks/
```
## Webhook Issues
### Webhook Notifications Not Sent
**Problem:** Webhook notifications aren't being sent.
**Solutions:**
1. Check webhook URL:
```bash
# Verify URL is set
grep NOTIFICATION_WEBHOOK_URL /opt/composesync/.env
```
2. Test webhook manually:
```bash
# Test webhook endpoint
# Test webhook manually
curl -X POST -H "Content-Type: application/json" \
-d '{"test": "message"}' \
-d '{"event": "test", "message": "Test notification"}' \
https://your-webhook-url.com/endpoint
# Check webhook logs
sudo journalctl -u composesync | grep "webhook"
```
**Common Issues:**
1. **Invalid Webhook URL**
```toml
# Incorrect webhook URL
[global]
NOTIFICATION_WEBHOOK_URL = "https://invalid-webhook.com/endpoint"
```
2. **Network Issues**
```bash
# Test webhook connectivity
curl -I https://your-webhook-url.com/endpoint
# Check DNS resolution
nslookup your-webhook-url.com
```
3. **Authentication Issues**
```bash
# Test webhook with authentication
curl -X POST -H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_TOKEN" \
-d '{"event": "test"}' \
https://your-webhook-url.com/endpoint
```
3. Check network connectivity:
```bash
# Test if webhook URL is accessible
wget -q --spider https://your-webhook-url.com/endpoint
```
## Performance Issues
### High Resource Usage
**Problem:** ComposeSync uses too much CPU or memory.
**Solutions:**
1. Increase update intervals:
```env
UPDATE_INTERVAL_SECONDS=7200 # Check every 2 hours instead of 1
```
- Verify webhook URL is correct
- Check network connectivity
- Configure authentication if required
- Test webhook endpoint manually
2. Reduce version history:
```env
KEEP_VERSIONS=5 # Keep fewer versions
```
## Advanced Troubleshooting
3. Use dry-run mode for testing:
```env
DRY_RUN=true
```
### Debug Mode
### Slow Downloads
**Problem:** Downloads are taking too long.
**Solutions:**
1. Check network connectivity:
```bash
# Test download speed
wget -O /dev/null https://your-url.com/docker-compose.yml
```
2. Consider using Git instead of wget:
```env
STACK_1_TOOL=git
```
## Lock File Issues
### Stale Lock Files
**Problem:** Lock files prevent updates.
**Solutions:**
1. Check for stale locks:
```bash
# Look for lock files
find /opt/composesync/stacks/ -name ".lock" -type d
```
2. Remove stale locks manually:
```bash
# Remove lock file (be careful!)
rm -rf /opt/composesync/stacks/stackname/.lock
```
3. Restart the service:
```bash
sudo systemctl restart composesync
```
## Debugging Tips
### Enable Verbose Logging
For detailed debugging, you can temporarily modify the log function:
Enable debug logging for detailed troubleshooting:
```bash
# Edit the update-agent.sh file
sudo nano /opt/composesync/update-agent.sh
# Edit service file to enable debug
sudo systemctl edit composesync
# Add more verbose logging
log() {
local prefix=""
if [ "$DRY_RUN" = "true" ]; then
prefix="[DRY-RUN] "
fi
echo "[$(date '+%Y-%m-%d %H:%M:%S')] ${prefix}$1" | tee -a /tmp/composesync-debug.log
}
# Add debug environment variable
[Service]
Environment=DEBUG=true
```
### Test Individual Components
### Manual Testing
Test components manually:
1. **Test download function:**
```bash
# Test wget download
wget -q -O /tmp/test.yml https://your-url.com/docker-compose.yml
# Test configuration parser
sudo -u composesync /opt/composesync/config-parser.sh
# Test update script manually
sudo -u composesync /opt/composesync/update-agent.sh
# Test Docker Compose commands
cd /opt/composesync/stacks/immich
docker compose config
docker compose ps
```
2. **Test Docker Compose:**
### System Information
Gather system information for troubleshooting:
```bash
# Test compose file manually
docker compose -f /path/to/docker-compose.yml config
# System information
uname -a
lsb_release -a
# Docker information
docker version
docker info
# Disk usage
df -h
du -sh /opt/composesync/
# Memory usage
free -h
# Network connectivity
ping -c 3 github.com
curl -I https://github.com
```
3. **Test webhook:**
## Recovery Procedures
### Manual Rollback
If automatic rollback fails:
```bash
# Test webhook manually
curl -X POST -H "Content-Type: application/json" \
-d '{"event": "test"}' \
$NOTIFICATION_WEBHOOK_URL
# List available backups
ls -la /opt/composesync/stacks/immich/compose-*.bak
# Restore from backup
sudo cp /opt/composesync/stacks/immich/compose-20240115102001.yml.bak \
/opt/composesync/stacks/immich/docker-compose.yml
# Apply rollback
cd /opt/composesync/stacks/immich
docker compose up -d
```
### Service Recovery
If the service is completely broken:
```bash
# Stop service
sudo systemctl stop composesync
# Backup configuration
sudo cp /opt/composesync/config.toml /opt/composesync/config.toml.backup
# Reinstall service
sudo ./install.sh
# Restore configuration
sudo cp /opt/composesync/config.toml.backup /opt/composesync/config.toml
# Start service
sudo systemctl start composesync
```
### Complete Reset
As a last resort:
```bash
# Stop service
sudo systemctl stop composesync
sudo systemctl disable composesync
# Backup important data
sudo cp -r /opt/composesync/stacks /tmp/composesync-backup
# Remove installation
sudo rm -rf /opt/composesync
sudo rm /etc/systemd/system/composesync.service
# Reinstall
sudo ./install.sh
# Restore stacks
sudo cp -r /tmp/composesync-backup/* /opt/composesync/stacks/
# Start service
sudo systemctl start composesync
```
## Getting Help
If you're still experiencing issues:
### Log Collection
Collect logs for troubleshooting:
1. **Check the logs:**
```bash
sudo journalctl -u composesync -f
# Create log archive
sudo journalctl -u composesync > /tmp/composesync-logs.txt
sudo cat /opt/composesync/config.toml > /tmp/composesync-config.txt
sudo systemctl status composesync > /tmp/composesync-status.txt
# Archive logs
tar -czf composesync-debug.tar.gz /tmp/composesync-*.txt
```
2. **Enable dry-run mode** to test without making changes:
```env
DRY_RUN=true
```
### Information to Include
3. **Verify your configuration** step by step
When seeking help, include:
4. **Check the documentation** for your specific use case
1. **System Information:**
- OS version and distribution
- Docker version
- ComposeSync version
5. **Submit an issue** with:
- Your configuration (with sensitive data removed)
- Relevant log output
- Steps to reproduce the issue
- Expected vs actual behavior
2. **Configuration:**
- Relevant parts of config.toml (remove sensitive data)
- Service status output
3. **Logs:**
- Recent error logs
- Service status logs
- Configuration loading logs
4. **Steps to Reproduce:**
- What you were trying to do
- What happened
- What you expected to happen
### Common Solutions Summary
| Issue | Quick Fix | Detailed Fix |
|-------|-----------|--------------|
| Service won't start | Check config syntax | Validate TOML, check permissions |
| Download failures | Test URL manually | Check network, verify URLs |
| Docker failures | Check compose syntax | Fix compose file, resolve conflicts |
| Permission issues | Add user to docker group | Fix ownership, check socket permissions |
| Lock file stuck | Remove .lock file | Restart service, check for processes |
| No backups | Check retention settings | Verify backup creation, check disk space |
| Webhook failures | Test URL manually | Check network, verify authentication |

531
Docs/watchtower.md
View file

@ -1,263 +1,404 @@
# Watchtower Integration
This guide covers how to use ComposeSync alongside Watchtower and how to configure them to work together effectively.
This guide explains how ComposeSync works with Watchtower and provides recommendations for using both tools together.
## Understanding the Tools
## Overview
### Watchtower
- Monitors running Docker images and updates them when new versions are available in the registry
- Focuses on updating container images, not compose file configurations
- Works at the container level
ComposeSync and Watchtower are complementary tools that serve different purposes:
### ComposeSync
- Monitors remote sources for changes to `docker-compose.yml` files and applies those changes
- Focuses on updating compose file configurations, not just images
- Works at the compose file level
- **ComposeSync**: Updates Docker Compose configuration files
- **Watchtower**: Updates Docker container images
## Recommended Configuration
## Key Differences
For stacks managed by ComposeSync, it's recommended to disable Watchtower to prevent conflicts and race conditions.
| Feature | ComposeSync | Watchtower |
|---------|-------------|------------|
| **Purpose** | Updates compose files | Updates container images |
| **Scope** | Configuration changes | Image updates |
| **Method** | Downloads new compose files | Pulls new images |
| **Safety** | Preserves overrides, rollback on failure | Automatic image updates |
| **Control** | Version-controlled updates | Real-time updates |
### Disabling Watchtower for ComposeSync Stacks
## When to Use Each Tool
Add the following to your `docker-compose.override.yml`:
### Use ComposeSync When:
- You want to update application configurations
- You need to preserve custom overrides
- You want version-controlled updates
- You're managing complex multi-service stacks
- You need rollback capabilities
```yaml
services:
your-service:
labels:
- "com.centurylinklabs.watchtower.enable=false"
### Use Watchtower When:
- You want automatic image updates
- You're running simple containers
- You don't need configuration changes
- You want real-time updates
- You're using `latest` tags
## Integration Strategies
### Strategy 1: ComposeSync Only (Recommended)
Use ComposeSync for both configuration and image updates:
```toml
# Global settings
[global]
UPDATE_INTERVAL_SECONDS = 3600
KEEP_VERSIONS = 10
DRY_RUN = false
# Stack with specific image versions
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
```
### Example Configuration
**Benefits:**
- Full control over updates
- Configuration and images updated together
- Automatic rollback on failures
- Preserves your customizations
### Strategy 2: Watchtower Only
Use Watchtower for automatic image updates:
```yaml
# docker-compose.override.yml for a ComposeSync-managed stack
# docker-compose.yml
version: '3.8'
services:
immich-server:
labels:
- "com.centurylinklabs.watchtower.enable=false"
immich-microservices:
labels:
- "com.centurylinklabs.watchtower.enable=false"
immich-machine-learning:
labels:
- "com.centurylinklabs.watchtower.enable=false"
watchtower:
image: containrrr/watchtower
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- WATCHTOWER_POLL_INTERVAL=3600
- WATCHTOWER_CLEANUP=true
restart: unless-stopped
```
**Benefits:**
- Real-time image updates
- Simple setup
- Automatic cleanup
### Strategy 3: Hybrid Approach
Use both tools for different purposes:
```toml
# ComposeSync for configuration updates
[global]
UPDATE_INTERVAL_SECONDS = 86400 # Daily config updates
KEEP_VERSIONS = 10
DRY_RUN = false
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
```
```yaml
# Watchtower for image updates
version: '3.8'
services:
watchtower:
image: containrrr/watchtower
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- WATCHTOWER_POLL_INTERVAL=3600 # Hourly image updates
- WATCHTOWER_CLEANUP=true
- WATCHTOWER_LABEL_ENABLE=true
restart: unless-stopped
```
## Best Practices
### 1. Use ComposeSync for:
- Applications where the `docker-compose.yml` configuration evolves
- Complex stacks with multiple services
- Applications that require specific version management
- Stacks with custom configurations and overrides
### 1. Avoid Conflicts
### 2. Use Watchtower for:
- Simple, self-contained applications
- Stacks with static `docker-compose.yml` files
- Applications where only image updates are needed
- Stacks without complex configurations
**Don't use both tools on the same services** - this can cause conflicts and unexpected behavior.
### 3. Notification Integration:
- Configure both tools to send notifications to the same webhook
- This provides a unified view of all Docker updates
- Helps track what's being updated and when
### 2. Use Specific Image Tags
## Configuration Examples
### ComposeSync-Managed Stack (Immich)
When using ComposeSync, prefer specific image tags over `latest`:
```yaml
# docker-compose.override.yml
# Good: Specific version
services:
immich-server:
labels:
- "com.centurylinklabs.watchtower.enable=false"
environment:
- IMMICH_SERVER_URL=http://immich-server:2283
networks:
- npm_network
- immich-backend
image: ghcr.io/immich-app/immich-server:v1.75.0
redis:
labels:
- "com.centurylinklabs.watchtower.enable=false"
networks:
- immich-backend
database:
labels:
- "com.centurylinklabs.watchtower.enable=false"
networks:
- immich-backend
networks:
npm_network:
external: true
immich-backend:
name: immich-backend
# Avoid: Latest tag
services:
immich-server:
image: ghcr.io/immich-app/immich-server:latest
```
### Watchtower-Managed Stack (Simple App)
### 3. Configure Update Intervals
Set appropriate intervals for each tool:
```toml
# ComposeSync: Less frequent config updates
[global]
UPDATE_INTERVAL_SECONDS = 86400 # Daily
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
```
```yaml
# Watchtower: More frequent image updates
services:
watchtower:
environment:
- WATCHTOWER_POLL_INTERVAL=3600 # Hourly
```
### 4. Use Labels for Control
When using Watchtower, use labels to control which containers are updated:
```yaml
# docker-compose.yml
version: '3.8'
services:
simple-app:
image: nginx:latest
ports:
- "8080:80"
immich-server:
image: ghcr.io/immich-app/immich-server:v1.75.0
labels:
- "com.centurylinklabs.watchtower.enable=true"
restart: unless-stopped
database:
image: postgres:15
labels:
- "com.centurylinklabs.watchtower.enable=false" # Don't auto-update
restart: unless-stopped
```
## Potential Conflicts
### Race Conditions
- Both tools might try to update the same stack simultaneously
- Can lead to inconsistent states
- May cause service interruptions
### Configuration Conflicts
- Watchtower might update images while ComposeSync is updating compose files
- Can result in version mismatches
- May break service dependencies
### Resource Contention
- Both tools competing for Docker resources
- Can slow down update processes
- May cause timeouts or failures
## Monitoring Both Tools
### Unified Logging
Monitor both services together:
```bash
# Watch ComposeSync logs
sudo journalctl -u composesync -f
# Watch Watchtower logs (if running as container)
docker logs -f watchtower
# Or if running as systemd service
sudo journalctl -u watchtower -f
```
### Webhook Integration
Configure both tools to use the same webhook:
```env
# ComposeSync webhook
NOTIFICATION_WEBHOOK_URL=https://your-webhook-url.com/endpoint
# Watchtower webhook (in docker-compose.yml)
environment:
- WATCHTOWER_NOTIFICATIONS=webhook
- WATCHTOWER_NOTIFICATION_WEBHOOK_URL=https://your-webhook-url.com/endpoint
```
## Migration Strategy
## Migration Scenarios
### From Watchtower to ComposeSync
1. **Identify stacks to migrate**
- Choose stacks with evolving configurations
- Select stacks that need custom overrides
If you're currently using Watchtower and want to switch to ComposeSync:
2. **Configure ComposeSync**
- Add stack configuration to `.env`
- Set up override files
- Test with dry-run mode
1. **Stop Watchtower:**
```bash
docker compose down watchtower
```
3. **Disable Watchtower**
- Add labels to disable Watchtower for migrated stacks
- Keep Watchtower for simple stacks
2. **Configure ComposeSync:**
```toml
[global]
UPDATE_INTERVAL_SECONDS = 3600
KEEP_VERSIONS = 10
DRY_RUN = false
4. **Monitor and verify**
- Check that updates work correctly
- Verify no conflicts occur
[your-stack]
URL = "https://your-source.com/docker-compose.yml"
PATH = "/opt/composesync/stacks/your-stack"
TOOL = "wget"
```
3. **Create override file:**
```bash
sudo nano /opt/composesync/stacks/your-stack/docker-compose.override.yml
```
4. **Start ComposeSync:**
```bash
sudo systemctl start composesync
```
### From ComposeSync to Watchtower
1. **Identify simple stacks**
- Choose stacks with static configurations
- Select stacks that only need image updates
If you want to switch from ComposeSync to Watchtower:
2. **Remove from ComposeSync**
- Remove stack configuration from `.env`
- Clean up stack directories
1. **Stop ComposeSync:**
```bash
sudo systemctl stop composesync
sudo systemctl disable composesync
```
3. **Enable Watchtower**
- Remove Watchtower disable labels
- Configure Watchtower for the stack
2. **Deploy Watchtower:**
```yaml
version: '3.8'
services:
watchtower:
image: containrrr/watchtower
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- WATCHTOWER_POLL_INTERVAL=3600
- WATCHTOWER_CLEANUP=true
restart: unless-stopped
```
3. **Update your compose files to use `latest` tags:**
```yaml
services:
your-app:
image: your-app:latest
```
## Troubleshooting
### Update Conflicts
### Conflicts Between Tools
If you see conflicts between the tools:
1. Check which tool is managing each stack
2. Ensure proper labels are set
3. Monitor logs for race conditions
4. Consider separating responsibilities more clearly
If you experience conflicts:
### Service Failures
```bash
# Check which tool is managing your containers
docker ps --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"
If services fail after updates:
1. Check which tool performed the update
2. Verify the update was applied correctly
3. Check for configuration conflicts
4. Review logs for error messages
# Check ComposeSync logs
sudo journalctl -u composesync -f
# Check Watchtower logs
docker logs watchtower
```
### Unexpected Updates
If containers are updating unexpectedly:
```bash
# Check ComposeSync configuration
cat /opt/composesync/config.toml
# Check Watchtower configuration
docker inspect watchtower | grep -A 10 "Env"
# Check container labels
docker inspect your-container | grep -A 5 "Labels"
```
### Performance Issues
If updates are slow or failing:
1. Check resource usage during updates
2. Verify network connectivity
3. Consider staggering update intervals
4. Monitor for resource contention
If you experience performance issues:
## Advanced Configuration
```bash
# Check update frequency
grep "UPDATE_INTERVAL_SECONDS" /opt/composesync/config.toml
### Selective Updates
# Check Watchtower interval
docker inspect watchtower | grep "WATCHTOWER_POLL_INTERVAL"
You can configure both tools to update different aspects:
# Monitor resource usage
docker stats
```
## Recommendations
### For Production Environments
**Use ComposeSync** for production environments because:
- Better control over updates
- Automatic rollback on failures
- Preserves your customizations
- Version-controlled updates
- Better monitoring and logging
### For Development Environments
**Use Watchtower** for development environments because:
- Faster iteration
- Real-time updates
- Simpler setup
- Less configuration overhead
### For Mixed Environments
**Use the hybrid approach** with:
- ComposeSync for critical production stacks
- Watchtower for development and testing stacks
- Clear separation of responsibilities
## Example Configurations
### Production Setup (ComposeSync Only)
```toml
# /opt/composesync/config.toml
[global]
UPDATE_INTERVAL_SECONDS = 86400 # Daily updates
KEEP_VERSIONS = 15
DRY_RUN = false
NOTIFICATION_WEBHOOK_URL = "https://your-webhook.com/endpoint"
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
[portainer]
URL = "https://github.com/portainer/portainer-compose.git"
PATH = "/opt/composesync/stacks/portainer"
TOOL = "git"
GIT_SUBPATH = "compose/docker-compose.yml"
GIT_REF = "main"
```
### Development Setup (Watchtower Only)
```yaml
# ComposeSync handles compose file updates
# Watchtower handles image updates for specific services
# docker-compose.yml
version: '3.8'
services:
app:
watchtower:
image: containrrr/watchtower
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- WATCHTOWER_POLL_INTERVAL=1800 # 30 minutes
- WATCHTOWER_CLEANUP=true
- WATCHTOWER_LABEL_ENABLE=true
restart: unless-stopped
dev-app:
image: your-dev-app:latest
labels:
- "com.centurylinklabs.watchtower.enable=true"
- "com.centurylinklabs.watchtower.scope=app"
restart: unless-stopped
```
database:
### Hybrid Setup
```toml
# ComposeSync for production stacks
[global]
UPDATE_INTERVAL_SECONDS = 86400
KEEP_VERSIONS = 10
[production-app]
URL = "https://github.com/user/production-app.git"
PATH = "/opt/composesync/stacks/production"
TOOL = "git"
GIT_REF = "main"
```
```yaml
# Watchtower for development stacks
services:
watchtower:
image: containrrr/watchtower
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- WATCHTOWER_POLL_INTERVAL=3600
- WATCHTOWER_CLEANUP=true
- WATCHTOWER_LABEL_ENABLE=true
restart: unless-stopped
dev-app:
image: your-dev-app:latest
labels:
- "com.centurylinklabs.watchtower.enable=false"
- "com.centurylinklabs.watchtower.enable=true"
restart: unless-stopped
```
### Update Scheduling
Coordinate update schedules to avoid conflicts:
```env
# ComposeSync: Check every 6 hours
UPDATE_INTERVAL_SECONDS=21600
# Watchtower: Check every 12 hours (different schedule)
# Configure in Watchtower settings
```
### Notification Filtering
Use different webhook endpoints for different tools:
```env
# ComposeSync notifications
NOTIFICATION_WEBHOOK_URL=https://webhook.com/composesync
# Watchtower notifications (separate endpoint)
# Configure in Watchtower settings
```
This allows you to handle notifications differently based on the source.

579
Docs/webhooks.md
View file

@ -1,222 +1,465 @@
# Webhook Notifications
This guide covers how to set up and configure webhook notifications in ComposeSync.
## Overview
ComposeSync can send webhook notifications when updates are applied or when errors occur. This is useful for:
- Monitoring update status remotely
- Integrating with notification systems (Discord, Slack, etc.)
- Alerting on failed updates
- Keeping track of when stacks are updated
ComposeSync can send webhook notifications for various events, including enhanced Git context information.
## Configuration
To enable webhook notifications, add this to your `.env` file:
Set the webhook URL in your configuration:
```toml
[global]
NOTIFICATION_WEBHOOK_URL = "https://your-webhook-url.com/endpoint"
```
Or in `.env` format:
```env
NOTIFICATION_WEBHOOK_URL=https://your-webhook-url.com/endpoint
```
## Webhook Events
ComposeSync sends notifications for the following events:
| Event | Description | When Sent |
|-------|-------------|-----------|
| `update_success` | Stack updated successfully | After successful docker compose up |
| `update_failure` | Update failed, rollback initiated | When docker compose up fails |
| `rollback_success` | Rollback completed successfully | After successful rollback |
| `rollback_failure` | Rollback failed | When rollback also fails |
| `dry_run` | Dry-run mode simulation | When DRY_RUN=true |
## Webhook Payload
The webhook will be called with a JSON payload containing:
- `event`: The type of event (`update_success`, `update_failure`, `error`)
- `stack_name`: The name of the stack being updated
- `timestamp`: When the event occurred
- `message`: A human-readable description of what happened
- `version_id`: The version identifier for the update (if applicable)
- `diff`: A unified diff (truncated to 50 lines) showing the changes applied to the main compose file (for update_success and update_failure events; null for errors)
## Event Types
### Update Success
Sent when a stack is successfully updated. Includes a diff of the changes:
Each webhook notification includes:
```json
{
"event": "update_success",
"stack_name": "immich",
"timestamp": "2024-01-15T10:30:00Z",
"message": "Successfully updated stack immich to version a1b2c3d",
"version_id": "a1b2c3d",
"diff": "--- compose-a1b2c3d.yml.bak\n+++ docker-compose.yml\n@@ -1,6 +1,6 @@\n version: '3.8'\n services:\n immich-server:\n- image: ghcr.io/immich-app/immich-server:release\n+ image: ghcr.io/immich-app/immich-server:release-1.91.0\n... (diff truncated, showing first 50 lines)"
"message": "Successfully updated stack immich",
"version_id": "abc1234",
"diff": "- old line\n+ new line\n...",
"git_context": "Commit: abc1234 - Update dependencies (Ref: main) | Repo: immich-app"
}
```
### Update Failure
### Payload Fields
Sent when a stack update fails and rollback occurs. Includes the diff that was attempted:
| Field | Description | Example |
|-------|-------------|---------|
| `event` | Event type | `update_success` |
| `stack_name` | Name of the stack | `immich` |
| `timestamp` | ISO 8601 timestamp | `2024-01-15T10:30:00Z` |
| `message` | Human-readable message | `Successfully updated stack immich` |
| `version_id` | Version identifier | `abc1234` (Git commit) or `20240115103000` (timestamp) |
| `diff` | Changes applied (truncated to 50 lines) | `- old line\n+ new line` |
| `git_context` | Git information (for Git repositories) | `Commit: abc1234 - Update deps (Ref: main) | Repo: immich-app` |
## Git Context Information
For Git repositories, the `git_context` field provides:
- **Commit hash**: Short commit hash
- **Commit message**: First 100 characters of commit message
- **Reference**: Branch, tag, or commit hash being tracked
- **Repository**: Repository name (sanitized)
Example Git context:
```
Commit: abc1234 - Update dependencies and fix security issues (Ref: main) | Repo: immich-app
```
## Testing Webhooks
You can test webhook notifications using services like:
- [webhook.site](https://webhook.site/) - Temporary webhook endpoints
- [ngrok](https://ngrok.com/) - Expose local endpoints
- [Discord webhooks](https://support.discord.com/hc/en-us/articles/228383668-Intro-to-Webhooks)
- [Slack webhooks](https://api.slack.com/messaging/webhooks)
## Example: Discord Integration
```toml
[global]
NOTIFICATION_WEBHOOK_URL = "https://discord.com/api/webhooks/YOUR_WEBHOOK_URL"
```
## Example: Slack Integration
```toml
[global]
NOTIFICATION_WEBHOOK_URL = "https://hooks.slack.com/services/YOUR_WEBHOOK_URL"
```
## Error Handling
- Webhook failures don't stop the update process
- Failed webhook requests are logged but don't cause errors
- Network timeouts are handled gracefully
## Overview
Webhook notifications provide:
- Real-time updates on stack changes
- Error notifications for failed updates
- Integration with monitoring systems
- Diff output showing what changed
- Support for dry-run mode notifications
## TOML Configuration
### Global Webhook Configuration
Configure webhooks for all stacks in your TOML file:
```toml
# Global settings
[global]
UPDATE_INTERVAL_SECONDS = 3600
KEEP_VERSIONS = 10
DRY_RUN = false
NOTIFICATION_WEBHOOK_URL = "https://your-webhook-url.com/endpoint"
# Stack configurations
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
[dev-app]
URL = "https://github.com/user/dev-app.git"
PATH = "/opt/composesync/stacks/dev-app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "develop"
```
### Per-Stack Webhook Configuration
Override webhook settings for specific stacks:
```toml
# Global settings
[global]
UPDATE_INTERVAL_SECONDS = 3600
KEEP_VERSIONS = 10
NOTIFICATION_WEBHOOK_URL = "https://default-webhook.com/endpoint"
# Production stack (use global webhook)
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
# Development stack (different webhook)
[dev-app]
URL = "https://github.com/user/dev-app.git"
PATH = "/opt/composesync/stacks/dev-app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "develop"
NOTIFICATION_WEBHOOK_URL = "https://dev-webhook.com/endpoint"
```
## Legacy .env Configuration
### Global Webhook Configuration
```env
# Global settings
UPDATE_INTERVAL_SECONDS=3600
KEEP_VERSIONS=10
DRY_RUN=false
NOTIFICATION_WEBHOOK_URL=https://your-webhook-url.com/endpoint
# Stack configurations
STACKS=2
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
STACK_2_NAME=dev-app
STACK_2_URL=https://github.com/user/dev-app.git
STACK_2_PATH=/opt/composesync/stacks/dev-app
STACK_2_TOOL=git
STACK_2_GIT_SUBPATH=docker/docker-compose.yml
STACK_2_GIT_REF=develop
```
### Per-Stack Webhook Configuration
```env
# Global settings
UPDATE_INTERVAL_SECONDS=3600
KEEP_VERSIONS=10
NOTIFICATION_WEBHOOK_URL=https://default-webhook.com/endpoint
# Stack configurations
STACKS=2
# Production stack (use global webhook)
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=wget
# Development stack (different webhook)
STACK_2_NAME=dev-app
STACK_2_URL=https://github.com/user/dev-app.git
STACK_2_PATH=/opt/composesync/stacks/dev-app
STACK_2_TOOL=git
STACK_2_GIT_SUBPATH=docker/docker-compose.yml
STACK_2_GIT_REF=develop
STACK_2_NOTIFICATION_WEBHOOK_URL=https://dev-webhook.com/endpoint
```
## Webhook Payload Format
ComposeSync sends JSON payloads with the following structure:
```json
{
"event": "update_success",
"stack_name": "immich",
"timestamp": "2024-01-15T10:30:02Z",
"message": "Successfully updated stack immich",
"version_id": "20240115103002",
"diff": "--- a/docker-compose.yml\n+++ b/docker-compose.yml\n@@ -15,7 +15,7 @@ services:\n immich-server:\n- image: ghcr.io/immich-app/immich-server:release\n+ image: ghcr.io/immich-app/immich-server:v1.75.0\n"
}
```
### Event Types
| Event | Description | When Sent |
|-------|-------------|-----------|
| `update_success` | Stack updated successfully | After successful update |
| `update_failure` | Stack update failed | After failed update (with rollback) |
| `dry_run` | Dry-run mode notification | When dry-run mode is enabled |
### Payload Fields
| Field | Type | Description |
|-------|------|-------------|
| `event` | string | Event type (update_success, update_failure, dry_run) |
| `stack_name` | string | Name of the stack being updated |
| `timestamp` | string | ISO 8601 timestamp |
| `message` | string | Human-readable message |
| `version_id` | string | Version identifier (timestamp or git commit) |
| `diff` | string | Unified diff of changes (truncated to 50 lines) |
## Webhook Service Examples
### Discord Webhook
```toml
[global]
NOTIFICATION_WEBHOOK_URL = "https://discord.com/api/webhooks/YOUR_WEBHOOK_ID/YOUR_TOKEN"
```
Discord will display the notification with:
- Stack name as title
- Message as content
- Diff in a code block
- Timestamp
### Slack Webhook
```toml
[global]
NOTIFICATION_WEBHOOK_URL = "https://hooks.slack.com/services/YOUR_WORKSPACE/YOUR_CHANNEL/YOUR_TOKEN"
```
Slack will display:
- Rich message with stack information
- Diff in a collapsible section
- Color coding for success/failure
### Microsoft Teams Webhook
```toml
[global]
NOTIFICATION_WEBHOOK_URL = "https://your-org.webhook.office.com/webhookb2/YOUR_WEBHOOK_ID/IncomingWebhook/YOUR_TOKEN/YOUR_CHANNEL"
```
Teams will show:
- Adaptive card with stack details
- Diff in a code block
- Action buttons for quick access
### Custom Webhook Service
```toml
[global]
NOTIFICATION_WEBHOOK_URL = "https://your-service.com/webhook"
```
Your service can parse the JSON payload to:
- Store update history
- Trigger automated responses
- Send email notifications
- Update monitoring dashboards
## Dry-Run Mode Notifications
When dry-run mode is enabled, webhooks are still sent with a `[DRY-RUN]` prefix:
```json
{
"event": "update_success",
"stack_name": "immich",
"timestamp": "2024-01-15T10:30:02Z",
"message": "[DRY-RUN] Would apply changes to immich",
"version_id": "20240115103002",
"diff": "--- a/docker-compose.yml\n+++ b/docker-compose.yml\n@@ -15,7 +15,7 @@ services:\n immich-server:\n- image: ghcr.io/immich-app/immich-server:release\n+ image: ghcr.io/immich-app/immich-server:v1.75.0\n"
}
```
This is useful for:
- Testing webhook configurations
- Previewing changes before applying
- Validating webhook integrations
## Error Notifications
When updates fail, ComposeSync sends error notifications:
```json
{
"event": "update_failure",
"stack_name": "immich",
"timestamp": "2024-01-15T10:30:00Z",
"timestamp": "2024-01-15T10:30:02Z",
"message": "Failed to update stack immich, rolled back to previous version",
"version_id": "a1b2c3d",
"diff": "--- compose-a1b2c3d.yml.bak\n+++ docker-compose.yml\n@@ -1,6 +1,6 @@\n ... (diff truncated, showing first 50 lines)"
"version_id": "20240115103002",
"diff": "--- a/docker-compose.yml\n+++ b/docker-compose.yml\n@@ -15,7 +15,7 @@ services:\n immich-server:\n- image: ghcr.io/immich-app/immich-server:release\n+ image: ghcr.io/immich-app/immich-server:v1.75.0\n"
}
```
### Error
Sent when a general error occurs. The diff field is null:
```json
{
"event": "error",
"stack_name": "immich",
"timestamp": "2024-01-15T10:30:00Z",
"message": "Failed to download compose file for stack immich",
"version_id": null,
"diff": null
}
```
## Integration Examples
### Discord
1. Create a Discord webhook in your server settings
2. Configure ComposeSync with the webhook URL:
```env
NOTIFICATION_WEBHOOK_URL=https://discord.com/api/webhooks/YOUR_WEBHOOK_ID/YOUR_WEBHOOK_TOKEN
```
Discord will automatically format the JSON payload into a readable message.
### Slack
1. Create a Slack webhook in your workspace settings
2. Configure ComposeSync with the webhook URL:
```env
NOTIFICATION_WEBHOOK_URL=https://hooks.slack.com/services/YOUR_WEBHOOK_URL
```
Slack will display the notification in your configured channel.
### Custom Webhook Server
You can create your own webhook server to handle notifications:
```python
from flask import Flask, request
import json
app = Flask(__name__)
@app.route('/webhook', methods=['POST'])
def webhook():
data = request.json
if data['event'] == 'update_success':
print(f"✅ {data['stack_name']} updated successfully")
elif data['event'] == 'update_failure':
print(f"❌ {data['stack_name']} update failed")
elif data['event'] == 'error':
print(f"⚠️ Error with {data['stack_name']}: {data['message']}")
return 'OK', 200
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5000)
```
## Dry-Run Mode
**Important:** Webhook notifications are sent regardless of whether you're in dry-run mode. This allows you to test your webhook configuration safely without applying actual changes.
## Testing Webhooks
To test your webhook configuration:
### Manual Testing
1. Enable dry-run mode:
```env
DRY_RUN=true
```
Test your webhook endpoint manually:
2. Restart the service:
```bash
sudo systemctl restart composesync
# Test basic webhook
curl -X POST -H "Content-Type: application/json" \
-d '{"event": "test", "message": "Test notification"}' \
https://your-webhook-url.com/endpoint
# Test with full payload
curl -X POST -H "Content-Type: application/json" \
-d '{
"event": "update_success",
"stack_name": "test-stack",
"timestamp": "2024-01-15T10:30:02Z",
"message": "Test update notification",
"version_id": "test-123",
"diff": "--- a/test.yml\n+++ b/test.yml\n@@ -1,1 +1,1 @@\n- old\n+ new\n"
}' \
https://your-webhook-url.com/endpoint
```
3. Check your webhook endpoint for test notifications
### Enable Dry-Run Mode
Test webhooks safely with dry-run mode:
```toml
[global]
DRY_RUN = true
NOTIFICATION_WEBHOOK_URL = "https://your-webhook-url.com/endpoint"
[test-stack]
URL = "https://github.com/user/test-app.git"
PATH = "/opt/composesync/stacks/test"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
```
## Best Practices
### 1. Use Different Webhooks for Different Environments
```toml
# Production webhook
[global]
NOTIFICATION_WEBHOOK_URL = "https://prod-webhook.com/endpoint"
# Development webhook
[dev-app]
URL = "https://github.com/user/dev-app.git"
PATH = "/opt/composesync/stacks/dev-app"
TOOL = "git"
NOTIFICATION_WEBHOOK_URL = "https://dev-webhook.com/endpoint"
```
### 2. Handle Webhook Failures Gracefully
ComposeSync continues to work even if webhook notifications fail:
- Webhook failures don't affect stack updates
- Failed webhooks are logged but don't stop the process
- You can monitor webhook delivery in the logs
### 3. Secure Your Webhooks
- Use HTTPS endpoints only
- Consider webhook authentication if supported
- Rotate webhook tokens regularly
- Monitor webhook access logs
### 4. Monitor Webhook Delivery
```bash
# Check webhook delivery in logs
sudo journalctl -u composesync | grep "webhook"
# Filter for webhook errors
sudo journalctl -u composesync | grep "ERROR.*webhook"
```
## Troubleshooting
### Webhook Not Sent
### Webhook Not Being Sent
If webhooks aren't being sent:
1. Check the webhook URL is correct
2. Verify the service can reach the webhook endpoint
3. Check the service logs for webhook errors:
```bash
sudo journalctl -u composesync -f
# Check if webhook URL is configured
grep "NOTIFICATION_WEBHOOK_URL" /opt/composesync/config.toml
# Check service logs for webhook errors
sudo journalctl -u composesync | grep "webhook"
```
### Webhook Failures
### Webhook Delivery Failures
If webhook calls are failing:
1. Check the webhook endpoint is accessible
2. Verify the endpoint accepts POST requests
3. Check for authentication requirements
4. Test the webhook URL manually:
```bash
# Test webhook endpoint manually
curl -X POST -H "Content-Type: application/json" \
-d '{"test": "message"}' \
https://your-webhook-url.com/endpoint
# Check network connectivity
wget -q --spider https://your-webhook-url.com/endpoint
echo $?
```
### Webhook Payload Issues
```bash
# Check webhook payload format
sudo journalctl -u composesync | grep "payload"
# Verify JSON syntax
echo '{"test": "payload"}' | jq .
```
### Rate Limiting
Some webhook services have rate limits. If you're hitting limits:
1. Increase the update interval
2. Use a different webhook service
3. Implement your own webhook server with rate limiting
## Best Practices
### 1. Use HTTPS
Always use HTTPS URLs for webhooks to ensure security:
```env
NOTIFICATION_WEBHOOK_URL=https://your-webhook-url.com/endpoint
```
### 2. Test Your Webhook
Always test your webhook configuration with dry-run mode before going live.
### 3. Monitor Webhook Failures
Set up monitoring for webhook failures to ensure you don't miss important notifications.
### 4. Use Descriptive Messages
The webhook messages are designed to be human-readable and informative.
### 5. Handle Different Event Types
Configure your webhook endpoint to handle all event types appropriately.
## Advanced Configuration
### Custom Webhook Headers
If your webhook service requires custom headers, you may need to modify the webhook sending code in the update script.
### Multiple Webhooks
To send to multiple webhook endpoints, you can modify the webhook sending function to iterate through multiple URLs.
### Webhook Authentication
For webhooks requiring authentication, you can include credentials in the URL or modify the webhook sending code to include headers.
If your webhook service has rate limits:
- Consider batching notifications
- Implement retry logic in your webhook service
- Use different webhooks for different stacks

48
README.md
View file

@ -15,36 +15,50 @@ ComposeSync automatically keeps your Docker Compose stacks up to date by:
**Quick Example:**
```bash
# Install
# Install (will ask for confirmation of installation directory)
sudo ./install.sh
# Configure your stack
sudo nano /opt/composesync/.env
# Configure your stack (TOML format - recommended)
sudo nano /path/to/installation/config.toml
```
```toml
# Global settings
[global]
UPDATE_INTERVAL_SECONDS = 3600 # Check every hour
KEEP_VERSIONS = 10
DRY_RUN = false
# Stack configuration
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/path/to/installation/stacks/immich"
TOOL = "wget"
```
**Or for Git repositories:**
```toml
[immich]
URL = "https://github.com/immich-app/immich.git"
PATH = "/path/to/installation/stacks/immich"
TOOL = "git"
GIT_SUBPATH = "deployments/docker"
GIT_REF = "main"
```
**Legacy .env format is also supported:**
```env
STACKS=1
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_PATH=/path/to/installation/stacks/immich
STACK_1_TOOL=wget
```
**Or for Git repositories:**
```env
STACKS=1
STACK_1_NAME=immich
STACK_1_URL=https://github.com/immich-app/immich.git
STACK_1_PATH=/opt/composesync/stacks/immich
STACK_1_TOOL=git
STACK_1_GIT_SUBPATH=deployments/docker
STACK_1_GIT_REF=main
```
```bash
# Create your override file
sudo mkdir -p /opt/composesync/stacks/immich
sudo nano /opt/composesync/stacks/immich/docker-compose.override.yml
sudo mkdir -p /path/to/installation/stacks/immich
sudo nano /path/to/installation/stacks/immich/docker-compose.override.yml
# Start the service
sudo systemctl start composesync

4
composesync.service
View file

@ -13,7 +13,7 @@ EnvironmentFile=/opt/composesync/.env
# Allow the service to access Docker socket
Environment=DOCKER_HOST=unix:///var/run/docker.sock
# Set the base directory for stacks
Environment=COMPOSESYNC_BASE_DIR=/etc/composesync
Environment=COMPOSESYNC_BASE_DIR=/opt/composesync/stacks
# Set the number of versions to keep (default: 10)
Environment=KEEP_VERSIONS=10
# Set update interval in seconds (default: 3600)
@ -23,7 +23,7 @@ Environment=UPDATE_MODE=notify_and_apply
# Optional: Set webhook URL for notifications
# Environment=NOTIFICATION_WEBHOOK_URL=
# The script will be installed in /usr/local/bin
# The script will be installed in the installation directory
ExecStart=/opt/composesync/update-agent.sh
Restart=always
RestartSec=5s

113
config-parser.sh Normal file
View file

@ -0,0 +1,113 @@
#!/bin/bash
# ComposeSync Configuration Parser
# Supports both .env and TOML formats
set -euo pipefail
# Get the directory where this script is located
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# Configuration file paths
CONFIG_DIR="$SCRIPT_DIR"
ENV_FILE="$CONFIG_DIR/.env"
TOML_FILE="$CONFIG_DIR/config.toml"
# Function to parse TOML configuration
parse_toml() {
local toml_file="$1"
if [ ! -f "$toml_file" ]; then
return 1
fi
log "Loading TOML configuration from $toml_file"
# Initialize arrays
declare -g -A STACK_NAMES=()
local current_section=""
local stack_count=0
# Read TOML file and convert to environment variables
while IFS= read -r line; do
# Skip comments and empty lines
[[ "$line" =~ ^[[:space:]]*# ]] && continue
[[ -z "${line// }" ]] && continue
# Parse section headers [section]
if [[ "$line" =~ ^[[:space:]]*\[([^\]]+)\][[:space:]]*$ ]]; then
current_section="${BASH_REMATCH[1]}"
continue
fi
# Parse key-value pairs
if [[ "$line" =~ ^[[:space:]]*([^=]+)[[:space:]]*=[[:space:]]*(.+)$ ]]; then
local key="${BASH_REMATCH[1]}"
local value="${BASH_REMATCH[2]}"
# Remove quotes from value
value=$(echo "$value" | sed 's/^"\(.*\)"$/\1/' | sed "s/^'\(.*\)'$/\1/")
# Convert to environment variable format
if [ -n "$current_section" ]; then
if [[ "$current_section" == "global" ]]; then
# Global settings
export "${key^^}"="$value"
else
# Stack configurations
stack_count=$((stack_count + 1))
STACK_NAMES[$stack_count]="$current_section"
export "STACK_${stack_count}_${key^^}"="$value"
fi
fi
fi
done < "$toml_file"
# Set STACKS count
export STACKS=$stack_count
return 0
}
# Function to load configuration
load_config() {
# Try TOML first, then fall back to .env
if [ -f "$TOML_FILE" ]; then
if parse_toml "$TOML_FILE"; then
log "Successfully loaded TOML configuration"
return 0
else
log "Failed to parse TOML file, falling back to .env"
fi
fi
# Try multiple .env locations
local env_locations=(
"$ENV_FILE" # Default location
"$(dirname "${BASH_SOURCE[0]}")/.env" # Same directory as script
".env" # Current directory
)
for env_file in "${env_locations[@]}"; do
if [ -f "$env_file" ]; then
log "Loading .env configuration from $env_file"
# Use set -a to automatically export variables
set -a
source "$env_file"
set +a
return 0
fi
done
log "ERROR: No configuration file found at $TOML_FILE or any .env locations"
return 1
}
# Function to log messages (if not already defined)
if ! declare -F log >/dev/null; then
log() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1"
}
fi
# Export the function so it can be used by other scripts
export -f load_config
export -f parse_toml

52
config.toml.example Normal file
View file

@ -0,0 +1,52 @@
# ComposeSync TOML Configuration Example
# This file supports both global settings and per-stack configurations
[global]
# Global settings for all stacks
UPDATE_INTERVAL_SECONDS = 3600 # Check every hour
KEEP_VERSIONS = 10 # Keep 10 versions
DRY_RUN = false # Set to true for testing
NOTIFICATION_WEBHOOK_URL = "https://your-webhook-url.com/endpoint"
# Stack configurations
[immich]
URL = "https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml"
PATH = "/opt/composesync/stacks/immich"
TOOL = "wget"
INTERVAL = 3600 # Check every hour
[dev-app]
URL = "https://github.com/user/dev-app.git"
PATH = "/opt/composesync/stacks/dev-app"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "develop"
INTERVAL = 1800 # Check every 30 minutes for dev
[production-app]
URL = "https://github.com/user/production-app.git"
PATH = "/opt/composesync/stacks/production"
TOOL = "git"
GIT_SUBPATH = "deploy/docker-compose.yml"
GIT_REF = "main"
INTERVAL = 7200 # Check every 2 hours for production
KEEP_VERSIONS = 5 # Keep fewer versions for production
# Example with multiple compose files
[complex-stack]
URL = "https://github.com/user/complex-app.git"
PATH = "/opt/composesync/stacks/complex"
TOOL = "git"
GIT_SUBPATH = "docker/docker-compose.yml"
GIT_REF = "main"
EXTRA_FILES_1 = "https://github.com/user/complex-app/raw/main/docker/network.yml"
EXTRA_FILES_2 = "https://github.com/user/complex-app/raw/main/docker/volumes.yml"
EXTRA_FILES_ORDER = "1,2" # Custom order for extra files
# Example with custom compose file name
[custom-compose]
URL = "https://github.com/user/custom-app.git"
PATH = "/opt/composesync/stacks/custom"
TOOL = "git"
GIT_REF = "main"
COMPOSE_FILENAME = "main.yml" # Use main.yml instead of docker-compose.yml

142
install.sh
View file

@ -1,16 +1,80 @@
#!/bin/bash
# ComposeSync Installation Script
#
# For non-interactive SSH installation, set the username:
# export COMPOSESYNC_USER=yourusername
# sudo ./install.sh
#
# For interactive installation:
# sudo ./install.sh
# Exit on error
set -e
# Get the directory where this script is located
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
DEFAULT_INSTALL_DIR="$SCRIPT_DIR"
# Check if running as root
if [ "$EUID" -ne 0 ]; then
echo "Please run as root"
exit 1
fi
# Get username
# Check for required dependencies
check_dependencies() {
local missing_deps=()
for dep in docker systemctl wget git; do
if ! command -v "$dep" &> /dev/null; then
missing_deps+=("$dep")
fi
done
if [ ${#missing_deps[@]} -ne 0 ]; then
echo "Error: Missing required dependencies: ${missing_deps[*]}"
echo "Please install them before running this script."
exit 1
fi
# Check if Docker service is running
if ! systemctl is-active --quiet docker; then
echo "Error: Docker service is not running"
echo "Start Docker with: sudo systemctl start docker"
exit 1
fi
}
check_dependencies
# Get installation directory
if [ -n "${COMPOSESYNC_INSTALL_DIR:-}" ]; then
INSTALL_DIR="$COMPOSESYNC_INSTALL_DIR"
echo "Using installation directory from environment: $INSTALL_DIR"
else
echo "ComposeSync will be installed in: $DEFAULT_INSTALL_DIR"
read -p "Is this correct? (y/N): " confirm
if [[ $confirm =~ ^[Yy]$ ]]; then
INSTALL_DIR="$DEFAULT_INSTALL_DIR"
else
read -p "Enter installation directory: " INSTALL_DIR
fi
fi
# Validate installation directory
if [ ! -d "$INSTALL_DIR" ]; then
echo "Error: Installation directory $INSTALL_DIR does not exist"
exit 1
fi
# Get username (support both interactive and non-interactive modes)
if [ -n "${COMPOSESYNC_USER:-}" ]; then
USERNAME="$COMPOSESYNC_USER"
echo "Using username from environment: $USERNAME"
else
read -p "Enter the username to run ComposeSync (must be in docker group): " USERNAME
fi
# Verify user exists and is in docker group
if ! id "$USERNAME" &>/dev/null; then
@ -21,23 +85,27 @@ fi
if ! groups "$USERNAME" | grep -q docker; then
echo "Error: User $USERNAME is not in the docker group"
echo "Add user to docker group with: sudo usermod -aG docker $USERNAME"
echo "Then log out and back in, or run: newgrp docker"
exit 1
fi
# Create necessary directories
mkdir -p /opt/composesync
chown $USERNAME:docker /opt/composesync
mkdir -p "$INSTALL_DIR"
chown $USERNAME:docker "$INSTALL_DIR"
# Copy files to installation directory
cp update-agent.sh /opt/composesync/
chmod +x /opt/composesync/update-agent.sh
chown $USERNAME:docker /opt/composesync/update-agent.sh
cp update-agent.sh "$INSTALL_DIR/"
cp config-parser.sh "$INSTALL_DIR/"
chmod +x "$INSTALL_DIR/update-agent.sh"
chmod +x "$INSTALL_DIR/config-parser.sh"
chown $USERNAME:docker "$INSTALL_DIR/update-agent.sh"
chown $USERNAME:docker "$INSTALL_DIR/config-parser.sh"
# Create default .env file if it doesn't exist
if [ ! -f /opt/composesync/.env ]; then
cat > /opt/composesync/.env << EOF
if [ ! -f "$INSTALL_DIR/.env" ]; then
cat > "$INSTALL_DIR/.env" << EOF
# Base directory for stacks
COMPOSESYNC_BASE_DIR=/opt/composesync/stacks
COMPOSESYNC_BASE_DIR="$INSTALL_DIR/stacks"
# Number of versions to keep (default: 10)
KEEP_VERSIONS=10
@ -51,11 +119,15 @@ UPDATE_MODE=notify_and_apply
# Optional: Webhook URL for notifications
# NOTIFICATION_WEBHOOK_URL=
EOF
chown $USERNAME:docker /opt/composesync/.env
chown $USERNAME:docker "$INSTALL_DIR/.env"
fi
# Update service file with username
sed "s/YOUR_USERNAME/$USERNAME/" composesync.service > /etc/systemd/system/composesync.service
# Copy example configuration files
cp config.toml.example "$INSTALL_DIR/config.toml"
chown $USERNAME:docker "$INSTALL_DIR/config.toml"
# Update service file with username and installation directory
sed "s|YOUR_USERNAME|$USERNAME|g; s|/opt/composesync|$INSTALL_DIR|g" composesync.service > /etc/systemd/system/composesync.service
# Reload systemd
systemctl daemon-reload
@ -64,10 +136,44 @@ systemctl daemon-reload
systemctl enable composesync
systemctl start composesync
echo "ComposeSync has been installed and started!"
echo "You can check the status with: systemctl status composesync"
echo "View logs with: journalctl -u composesync -f"
# Verify service started successfully
if systemctl is-active --quiet composesync; then
echo "✅ ComposeSync service started successfully!"
else
echo "❌ ComposeSync service failed to start"
echo "Check the logs with: sudo journalctl -u composesync -n 50"
exit 1
fi
echo ""
echo "To manage your stacks, create directories in /opt/composesync/stacks/"
echo "Example: sudo mkdir -p /opt/composesync/stacks/immich"
echo " sudo chown $USERNAME:docker /opt/composesync/stacks/immich"
echo "🎉 ComposeSync has been installed successfully!"
echo ""
echo "📋 Next steps:"
echo "1. Configure your stacks:"
echo " sudo nano $INSTALL_DIR/config.toml # TOML format (recommended)"
echo " sudo nano $INSTALL_DIR/.env # Legacy .env format"
echo ""
echo "2. Create your override files:"
echo " sudo mkdir -p $INSTALL_DIR/stacks/immich"
echo " sudo nano $INSTALL_DIR/stacks/immich/docker-compose.override.yml"
echo ""
echo "3. Check service status:"
echo " sudo systemctl status composesync"
echo ""
echo "4. View logs:"
echo " sudo journalctl -u composesync -f"
echo ""
echo "📖 Example TOML configuration:"
echo " [global]"
echo " UPDATE_INTERVAL_SECONDS = 3600"
echo " "
echo " [immich]"
echo " URL = \"https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml\""
echo " PATH = \"$INSTALL_DIR/stacks/immich\""
echo " TOOL = \"wget\""
echo ""
echo "🔧 Service management:"
echo " sudo systemctl start composesync # Start service"
echo " sudo systemctl stop composesync # Stop service"
echo " sudo systemctl restart composesync # Restart service"
echo " sudo systemctl status composesync # Check status"

73
test-install.sh Normal file
View file

@ -0,0 +1,73 @@
#!/bin/bash
# ComposeSync Installation Test Script
# Run this after installation to verify everything works
set -e
# Get the installation directory from the service file
INSTALL_DIR=$(systemctl show composesync --property=ExecStart | sed 's/.*ExecStart=//' | sed 's|/update-agent.sh.*||')
if [ -z "$INSTALL_DIR" ]; then
echo "❌ Could not determine installation directory from service"
exit 1
fi
echo "🧪 Testing ComposeSync installation in: $INSTALL_DIR"
# Check if service exists
if ! systemctl list-unit-files | grep -q composesync; then
echo "❌ ComposeSync service not found"
exit 1
fi
# Check if service is enabled
if systemctl is-enabled --quiet composesync; then
echo "✅ Service is enabled"
else
echo "❌ Service is not enabled"
fi
# Check if service is running
if systemctl is-active --quiet composesync; then
echo "✅ Service is running"
else
echo "❌ Service is not running"
fi
# Check if files exist
if [ -f "$INSTALL_DIR/update-agent.sh" ]; then
echo "✅ update-agent.sh exists"
else
echo "❌ update-agent.sh missing"
fi
if [ -f "$INSTALL_DIR/config-parser.sh" ]; then
echo "✅ config-parser.sh exists"
else
echo "❌ config-parser.sh missing"
fi
if [ -f "$INSTALL_DIR/.env" ]; then
echo "✅ .env file exists"
else
echo "❌ .env file missing"
fi
# Check permissions
if [ -x "$INSTALL_DIR/update-agent.sh" ]; then
echo "✅ update-agent.sh is executable"
else
echo "❌ update-agent.sh is not executable"
fi
# Check recent logs
echo ""
echo "📋 Recent service logs:"
journalctl -u composesync --no-pager -n 10
echo ""
echo "🎉 Installation test complete!"
echo ""
echo "If you see any ❌ errors above, please check the troubleshooting guide:"
echo " https://github.com/your-repo/ComposeSync/blob/main/Docs/troubleshooting.md"

11
test.env Normal file
View file

@ -0,0 +1,11 @@
# Test .env configuration
UPDATE_INTERVAL_SECONDS=3600
KEEP_VERSIONS=10
DRY_RUN=false
STACKS=1
# Stack 1
STACK_1_NAME=test-stack
STACK_1_URL=https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
STACK_1_PATH=/tmp/test-stack
STACK_1_TOOL=wget

150
update-agent.sh
View file

@ -1,21 +1,65 @@
#!/bin/bash
# Exit on error
set -e
# ComposeSync - Automated Docker Compose Update Agent
# This script downloads and applies updates to docker-compose.yml files from remote sources
# Load environment variables
if [ -f .env ]; then
source .env
set -euo pipefail
# Source the configuration parser (if available)
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
if [ -f "$SCRIPT_DIR/config-parser.sh" ]; then
source "$SCRIPT_DIR/config-parser.sh"
else
# Fallback for .env-only usage
CONFIG_DIR="$SCRIPT_DIR"
ENV_FILE="$CONFIG_DIR/.env"
# Simple load_config function for .env-only
load_config() {
# Try multiple .env locations
local env_locations=(
"$ENV_FILE" # Default location
"$(dirname "${BASH_SOURCE[0]}")/.env" # Same directory as script
".env" # Current directory
)
for env_file in "${env_locations[@]}"; do
if [ -f "$env_file" ]; then
log "Loading .env configuration from $env_file"
# Use set -a to automatically export variables
set -a
source "$env_file"
set +a
return 0
fi
done
log "ERROR: No configuration file found at any .env locations"
return 1
}
# Simple log function if not defined
if ! declare -F log >/dev/null; then
log() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1"
}
fi
fi
# Default values
BASE_DIR=${COMPOSESYNC_BASE_DIR:-"/opt/composesync/stacks"}
BASE_DIR=${COMPOSESYNC_BASE_DIR:-"$SCRIPT_DIR/stacks"}
KEEP_VERSIONS=${KEEP_VERSIONS:-10}
UPDATE_INTERVAL=${UPDATE_INTERVAL_SECONDS:-3600}
UPDATE_MODE=${UPDATE_MODE:-"notify_and_apply"}
DRY_RUN=${DRY_RUN:-"false"}
STACKS=${STACKS:-1}
# Load configuration (supports both .env and TOML)
if ! load_config; then
echo "ERROR: Failed to load configuration"
exit 1
fi
# Function to log messages
log() {
local prefix=""
@ -37,9 +81,26 @@ acquire_lock() {
else
# Check if lock is stale
if [ -d "$lock_file" ]; then
local lock_age=$(($(date +%s) - $(stat -c %Y "$lock_file")))
local lock_age
# Cross-platform lock age checking
if command -v stat >/dev/null 2>&1; then
# Linux: stat -c %Y
if stat -c %Y "$lock_file" >/dev/null 2>&1; then
lock_age=$(($(date +%s) - $(stat -c %Y "$lock_file")))
# BSD/macOS: stat -f %m
elif stat -f %m "$lock_file" >/dev/null 2>&1; then
lock_age=$(($(date +%s) - $(stat -f %m "$lock_file")))
else
# Fallback: use ls -ld
lock_age=$(($(date +%s) - $(ls -ld "$lock_file" | awk '{print $6, $7, $8}' | xargs -I {} date -d "{}" +%s 2>/dev/null || echo 0)))
fi
else
# Fallback: assume lock is stale if we can't determine age
lock_age=$((lock_timeout + 1))
fi
if [ $lock_age -gt $lock_timeout ]; then
log "Found stale lock, removing..."
log "Found stale lock (age: ${lock_age}s), removing..."
rm -rf "$lock_file"
if mkdir "$lock_file" 2>/dev/null; then
return 0
@ -63,6 +124,7 @@ download_file() {
local tool=$3
local subpath=$4
local git_ref=$5
local compose_filename=$6 # New parameter for configurable compose file name
log "Downloading $url to $output"
@ -106,11 +168,13 @@ download_file() {
fi
cp "${output}.git/$subpath" "$output"
else
if [ ! -f "${output}.git/docker-compose.yml" ]; then
log "ERROR: docker-compose.yml not found in repository root"
# Use configurable compose file name, default to docker-compose.yml
local compose_file_name=${compose_filename:-"docker-compose.yml"}
if [ ! -f "${output}.git/$compose_file_name" ]; then
log "ERROR: $compose_file_name not found in repository root"
return 1
fi
cp "${output}.git/docker-compose.yml" "$output"
cp "${output}.git/$compose_file_name" "$output"
fi
;;
*)
@ -155,6 +219,7 @@ send_webhook_notification() {
local message="$3"
local version_id="$4"
local diff="$5"
local git_context="$6" # New parameter for Git context
if [ -z "$NOTIFICATION_WEBHOOK_URL" ]; then
return
@ -168,18 +233,56 @@ send_webhook_notification() {
diff_json=""
fi
# Escape git context for JSON
local git_context_json=""
if [ -n "$git_context" ]; then
git_context_json=$(echo "$git_context" | sed 's/\\/\\\\/g; s/\"/\\\"/g')
fi
local payload="{\n"
payload+=" \"event\": \"$event\",\n"
payload+=" \"stack_name\": \"$stack_name\",\n"
payload+=" \"timestamp\": \"$(date -u +%Y-%m-%dT%H:%M:%SZ)\",\n"
payload+=" \"message\": \"$message\",\n"
payload+=" \"version_id\": \"$version_id\",\n"
payload+=" \"diff\": \"$diff_json\"\n"
payload+=" \"diff\": \"$diff_json\",\n"
payload+=" \"git_context\": \"$git_context_json\"\n"
payload+="}"
curl -s -X POST -H "Content-Type: application/json" -d "$payload" "$NOTIFICATION_WEBHOOK_URL" >/dev/null 2>&1 || true
}
# Function to get Git context for notifications
get_git_context() {
local path=$1
local url=$2
local git_ref=$3
if [ ! -d "${path}.git" ]; then
return
fi
local context=""
# Get commit hash and message
if git -C "${path}.git" rev-parse HEAD >/dev/null 2>&1; then
local commit_hash=$(git -C "${path}.git" rev-parse --short HEAD)
local commit_msg=$(git -C "${path}.git" log -1 --pretty=format:"%s" 2>/dev/null | head -c 100)
context="Commit: $commit_hash - $commit_msg"
# Add branch/tag info if available
if [ -n "$git_ref" ]; then
context="$context (Ref: $git_ref)"
fi
# Add repository URL (sanitized)
local repo_name=$(basename "$url" .git)
context="$context | Repo: $repo_name"
fi
echo "$context"
}
# Function to process a stack
process_stack() {
local stack_num=$1
@ -191,6 +294,7 @@ process_stack() {
local keep_versions_var="STACK_${stack_num}_KEEP_VERSIONS"
local git_subpath_var="STACK_${stack_num}_GIT_SUBPATH"
local git_ref_var="STACK_${stack_num}_GIT_REF"
local compose_filename_var="STACK_${stack_num}_COMPOSE_FILENAME"
local name=${!name_var}
local url=${!url_var}
@ -200,6 +304,7 @@ process_stack() {
local keep_versions=${!keep_versions_var:-$KEEP_VERSIONS}
local git_subpath=${!git_subpath_var}
local git_ref=${!git_ref_var}
local compose_filename=${!compose_filename_var}
if [ -z "$name" ] || [ -z "$url" ] || [ -z "$path" ] || [ -z "$tool" ]; then
log "Error: Missing required configuration for stack $stack_num"
@ -226,7 +331,7 @@ process_stack() {
# Download main compose file
local compose_file="$path/docker-compose.yml"
if ! download_file "$url" "$compose_file" "$tool" "$git_subpath" "$git_ref"; then
if ! download_file "$url" "$compose_file" "$tool" "$git_subpath" "$git_ref" "$compose_filename"; then
log "ERROR: Failed to download main compose file for stack $name, skipping..."
return 1
fi
@ -235,6 +340,15 @@ process_stack() {
local version_id=$(get_version_id "$path" "$tool")
log "Version ID: $version_id"
# Get Git context for notifications
local git_context=""
if [ "$tool" = "git" ]; then
git_context=$(get_git_context "$path" "$url" "$git_ref")
if [ -n "$git_context" ]; then
log "Git context: $git_context"
fi
fi
# Create backup directory for this update
local backup_dir="$path/backups/backup-$(date +%Y%m%d%H%M%S)"
if [ "$DRY_RUN" != "true" ]; then
@ -359,7 +473,7 @@ process_stack() {
log "DRY-RUN: Would run: ${compose_cmd_array[*]}"
log "DRY-RUN: Changes that would be applied:"
echo "$diff_output"
send_webhook_notification "update_success" "$name" "[DRY-RUN] Would apply changes to $name" "$version_id" "$diff_output"
send_webhook_notification "update_success" "$name" "[DRY-RUN] Would apply changes to $name" "$version_id" "$diff_output" "$git_context"
else
log "Applying changes to $name"
@ -376,10 +490,10 @@ process_stack() {
log "Running: ${compose_cmd_array[*]}"
if "${compose_cmd_array[@]}"; then
log "Successfully updated stack $name"
send_webhook_notification "update_success" "$name" "Successfully updated stack $name" "$version_id" "$diff_output"
send_webhook_notification "update_success" "$name" "Successfully updated stack $name" "$version_id" "$diff_output" "$git_context"
else
log "ERROR: Failed to update stack $name, attempting rollback..."
send_webhook_notification "update_failure" "$name" "Failed to update stack $name, rolled back to previous version" "$version_id" "$diff_output"
send_webhook_notification "update_failure" "$name" "Failed to update stack $name, rolled back to previous version" "$version_id" "$diff_output" "$git_context"
# Rollback: restore from backup
if [ -f "$backup_dir/docker-compose.yml" ]; then
@ -406,14 +520,16 @@ process_stack() {
log "Attempting to restart stack with rolled back configuration..."
if "${compose_cmd_array[@]}"; then
log "Successfully rolled back stack $name"
send_webhook_notification "rollback_success" "$name" "Successfully rolled back stack $name to previous version" "$version_id" "" "$git_context"
else
log "CRITICAL: Failed to rollback stack $name - manual intervention required"
send_webhook_notification "rollback_failure" "$name" "CRITICAL: Failed to rollback stack $name - manual intervention required" "$version_id" "" "$git_context"
fi
fi
fi
else
log "Changes detected but not applied (notify_only mode)"
send_webhook_notification "update_success" "$name" "Changes detected in $name (notify_only mode)" "$version_id" "$diff_output"
send_webhook_notification "update_success" "$name" "Changes detected in $name (notify_only mode)" "$version_id" "$diff_output" "$git_context"
fi
else
log "No changes detected in $name"