bootc-docs/upgrade/bootc-upgrade-guide.md

# bootc upgrade - Comprehensive Guide

## Overview

`bootc upgrade` is the primary command for downloading and applying system updates in bootc-managed systems. It implements an A/B upgrade system where updates are staged and applied at the next boot, ensuring system stability and providing rollback capabilities.

## Purpose

The upgrade system serves several critical functions:

1. **Transactional Updates**: Downloads and stages updates without affecting the running system
2. **A/B Deployment**: Maintains two system versions for safe rollback
3. **Registry Integration**: Pulls updates from container registries
4. **Automatic Application**: Can automatically apply updates and reboot
5. **Soft Reboot Support**: Faster restarts when possible

## Command Syntax

```bash
bootc upgrade [OPTIONS...]
```

### Basic Usage

```bash
# Check for available updates
bootc upgrade --check

# Download and stage updates
bootc upgrade

# Download, stage, and apply updates (with reboot)
bootc upgrade --apply

# Upgrade with soft reboot if available
bootc upgrade --apply --soft-reboot=auto

# Quiet mode (no progress output)
bootc upgrade --quiet
```

## Command Options

### Core Options

| Option | Description | Default | Conflicts With |
|--------|-------------|---------|----------------|
| `--check` | Check for updates without downloading | `false` | `--apply` |
| `--apply` | Apply updates and reboot | `false` | `--check` |
| `--quiet` | Suppress progress output | `false` | None |
| `--soft-reboot` | Configure soft reboot behavior | `None` | `--check` |

### Soft Reboot Modes

| Mode | Description | Behavior |
|------|-------------|----------|
| `required` | Fail if soft reboot unavailable | Error if not supported |
| `auto` | Use soft reboot if available | Fallback to regular reboot |

## Upgrade Process

### 1. Check Phase (`--check`)

The check phase only downloads metadata to determine if updates are available:

```bash
bootc upgrade --check
```

**What it does**:
- Downloads container manifest and configuration
- Compares with currently booted image
- Shows available updates without downloading layers
- Displays version information and digest

**Output example**:
```
Update available for: quay.io/myorg/debian-bootc:latest
  Version: 1.2.3
  Digest: sha256:abc123...
```

### 2. Download Phase (default)

Downloads and stages the updated container image:

```bash
bootc upgrade
```

**What it does**:
- Downloads full container image layers
- Stages the update for next boot
- Preserves current running system
- Updates system status

**Process**:
1. **Registry Query**: Check for updated image
2. **Layer Download**: Pull container layers
3. **OSTree Import**: Convert to OSTree format
4. **Staging**: Prepare for next boot
5. **Status Update**: Mark as staged

### 3. Apply Phase (`--apply`)

Immediately applies staged updates and reboots:

```bash
bootc upgrade --apply
```

**What it does**:
- Applies staged updates
- Reboots into new system
- Preserves `/etc` and `/var` state
- Updates bootloader configuration

## Upgrade Modes

### 1. Check-Only Mode

```bash
bootc upgrade --check
```

**Use cases**:
- Monitoring for updates
- CI/CD pipeline checks
- Update availability verification
- Automated update detection

**Benefits**:
- Fast execution (metadata only)
- No bandwidth usage for full download
- Safe to run frequently

### 2. Stage-Only Mode

```bash
bootc upgrade
```

**Use cases**:
- Download updates for later application
- Batch update preparation
- Offline update preparation
- Manual control over application timing

**Benefits**:
- Updates ready for next boot
- No immediate system changes
- Allows review before application

### 3. Immediate Apply Mode

```bash
bootc upgrade --apply
```

**Use cases**:
- Automated update application
- CI/CD deployment
- Immediate update application
- Single-command updates

**Benefits**:
- Complete update cycle
- Automatic reboot
- No manual intervention required

## Soft Reboot Support

### What is Soft Reboot?

Soft reboot is a faster restart mechanism that avoids full hardware reboot when possible, typically used in virtualized environments.

### Soft Reboot Modes

#### Required Mode
```bash
bootc upgrade --apply --soft-reboot=required
```

**Behavior**:
- Fails if soft reboot is not available
- Ensures fastest possible restart
- Use when soft reboot is critical

#### Auto Mode
```bash
bootc upgrade --apply --soft-reboot=auto
```

**Behavior**:
- Uses soft reboot if available
- Falls back to regular reboot if not supported
- Recommended for most use cases

### Soft Reboot Requirements

- Virtualized environment (VMware, QEMU, etc.)
- Kernel support for kexec
- Sufficient memory for kexec
- Compatible hardware

## Integration Patterns

### 1. Manual Updates

```bash
# Check for updates
bootc upgrade --check

# If updates available, apply them
bootc upgrade --apply
```

### 2. Automated Updates

```bash
# Automated update with soft reboot
bootc upgrade --apply --soft-reboot=auto --quiet
```

### 3. CI/CD Integration

```yaml
# GitHub Actions example
- name: Update system
  run: |
    bootc upgrade --check
    if [ $? -eq 0 ]; then
      bootc upgrade --apply --soft-reboot=auto
    fi
```

### 4. Monitoring Integration

```bash
#!/bin/bash
# Update monitoring script
if bootc upgrade --check; then
    echo "Updates available"
    # Send notification
    notify-send "System updates available"
else
    echo "System up to date"
fi
```

## Status and Monitoring

### Check Update Status

```bash
# View current status
bootc status

# Check for updates
bootc upgrade --check
```

### Status Indicators

| Status | Description |
|--------|-------------|
| `booted` | Currently running system |
| `staged` | Update ready for next boot |
| `incompatible` | System has local modifications |

### Update Verification

```bash
# Check what's staged
bootc status

# Verify update details
bootc upgrade --check
```

## Rollback and Recovery

### Rollback Process

```bash
# Rollback to previous version
bootc rollback

# Check rollback status
bootc status
```

### Recovery Scenarios

1. **Failed Update**: Use `bootc rollback`
2. **Corrupted System**: Boot from previous deployment
3. **Incompatible Update**: Check system compatibility

## Best Practices

### 1. Update Strategy

- **Test First**: Test updates in staging environment
- **Gradual Rollout**: Use `bootc switch` for controlled deployments
- **Monitor Status**: Check `bootc status` regularly
- **Backup State**: Ensure `/etc` and `/var` are backed up

### 2. Automation

- **Use `--check`**: For monitoring and CI/CD
- **Use `--apply`**: For automated deployments
- **Use `--soft-reboot`**: For faster restarts when possible
- **Use `--quiet`**: For automated scripts

### 3. Error Handling

- **Check Return Codes**: Handle update failures gracefully
- **Verify Status**: Confirm updates are staged correctly
- **Test Rollback**: Ensure rollback works before applying
- **Monitor Logs**: Check system logs for issues

## Troubleshooting

### Common Issues

#### No Updates Available

```bash
# Check current status
bootc status

# Verify image source
bootc edit

# Check registry connectivity
podman pull quay.io/myorg/debian-bootc:latest
```

#### Update Fails to Apply

```bash
# Check staged status
bootc status

# Verify system compatibility
bootc upgrade --check

# Check system logs
journalctl -u bootc-fetch-apply-updates.service
```

#### Soft Reboot Fails

```bash
# Check soft reboot support
bootc upgrade --apply --soft-reboot=required

# Fall back to regular reboot
bootc upgrade --apply
```

### Debug Commands

```bash
# Enable debug logging
RUST_LOG=debug bootc upgrade --check

# Check system status
bootc status --json

# Verify image integrity
bootc upgrade --check --quiet
```

## Advanced Usage

### 1. Custom Image Sources

```bash
# Switch to different image
bootc switch quay.io/myorg/debian-bootc:v2.0

# Upgrade from new source
bootc upgrade
```

### 2. Offline Updates

```bash
# Download updates offline
bootc upgrade

# Apply when ready
bootc upgrade --apply
```

### 3. Batch Updates

```bash
# Download multiple updates
bootc upgrade
bootc switch quay.io/myorg/debian-bootc:v2.1
bootc upgrade

# Apply all at once
bootc upgrade --apply
```

## Security Considerations

### 1. Image Verification

- **Signature Verification**: Ensure images are signed
- **Registry Security**: Use trusted registries
- **Digest Verification**: Verify image digests

### 2. Update Security

- **Network Security**: Use secure connections
- **Authentication**: Authenticate with registries
- **Authorization**: Control update permissions

### 3. Rollback Security

- **State Preservation**: Ensure `/etc` and `/var` are secure
- **Access Control**: Limit rollback permissions
- **Audit Logging**: Log all update activities

## Performance Optimization

### 1. Update Speed

- **Use `--check`**: For quick availability checks
- **Use `--soft-reboot`**: For faster restarts
- **Use `--quiet`**: For automated scripts

### 2. Bandwidth Usage

- **Check First**: Use `--check` before downloading
- **Incremental Updates**: Only download changed layers
- **Local Caching**: Use local registry mirrors

### 3. System Impact

- **Staged Updates**: No impact on running system
- **Atomic Application**: All-or-nothing updates
- **Rollback Capability**: Quick recovery from issues

## Future Enhancements

### 1. Planned Features

- **Userspace Restart**: For kernel-unchanged updates
- **Delta Updates**: More efficient layer downloads
- **Rollback Automation**: Automatic rollback on failure
- **Update Scheduling**: Time-based update application

### 2. Integration Improvements

- **API Support**: REST API for update management
- **Webhook Integration**: Event notifications
- **Metrics Collection**: Update performance metrics
- **Dashboard Integration**: Web-based management

---

This comprehensive guide provides everything needed to understand and use `bootc upgrade` effectively for system updates and maintenance.