bootc-docs/switch/bootc-switch-technical-guide.md

# bootc switch - Technical Guide

## Overview

`bootc switch` is a command for changing the container image reference that a bootc-managed system will boot from. It operates similarly to `bootc upgrade` but changes the image source rather than updating the current image. This enables blue/green deployments, A/B testing, and controlled rollouts across different image versions.

## Purpose

The switch command serves several critical functions:

1. **Image Source Management**: Change the container image reference for future boots
2. **Blue/Green Deployments**: Switch between different image versions
3. **A/B Testing**: Test different image versions on different systems
4. **Controlled Rollouts**: Gradually migrate systems to new image versions
5. **Rollback Capability**: Switch back to previous image versions

## Command Syntax

```bash
bootc switch [OPTIONS...] <TARGET>
```

### Basic Usage

```bash
# Switch to different image version
bootc switch quay.io/myorg/debian-bootc:v2.0

# Switch and apply immediately
bootc switch --apply quay.io/myorg/debian-bootc:v2.0

# Switch with soft reboot
bootc switch --apply --soft-reboot=auto quay.io/myorg/debian-bootc:v2.0

# Switch with specific transport
bootc switch --transport=oci-archive quay.io/myorg/debian-bootc:v2.0
```

## Command Options

### Core Options

| Option | Description | Default | Required |
|--------|-------------|---------|----------|
| `TARGET` | Target image to use for next boot | `None` | Yes |
| `--apply` | Restart/reboot into new target image | `false` | No |
| `--quiet` | Don't display progress | `false` | No |
| `--soft-reboot` | Configure soft reboot behavior | `None` | No |

### Transport Options

| Option | Description | Default | Values |
|--------|-------------|---------|--------|
| `--transport` | Container transport type | `registry` | `registry`, `oci`, `oci-archive`, `containers-storage` |

### Security Options

| Option | Description | Default |
|--------|-------------|---------|
| `--enforce-container-sigpolicy` | Enforce container signature policy | `false` |

### Advanced Options

| Option | Description | Default |
|--------|-------------|---------|
| `--retain` | Retain reference to currently booted image | `false` |

## Architecture Overview

### 1. Switch Command Structure

```rust
pub(crate) struct SwitchOpts {
    pub(crate) quiet: bool,                           // Suppress progress output
    pub(crate) apply: bool,                           // Apply and reboot
    pub(crate) soft_reboot: Option<SoftRebootMode>,  // Soft reboot behavior
    pub(crate) transport: String,                     // Container transport
    pub(crate) enforce_container_sigpolicy: bool,    // Signature enforcement
    pub(crate) retain: bool,                          // Retain current image ref
    pub(crate) mutate_in_place: bool,                // In-place mutation (hidden)
    pub(crate) progress: ProgressOptions,            // Progress reporting
}
```

### 2. Switch Process Flow

```rust
async fn switch(opts: SwitchOpts) -> Result<()> {
    // 1. Parse and validate target image
    let transport = ostree_container::Transport::try_from(opts.transport.as_str())?;
    let imgref = ostree_container::ImageReference {
        transport,
        name: opts.target.to_string(),
    };
    let sigverify = sigpolicy_from_opt(opts.enforce_container_sigpolicy);
    let target = ostree_container::OstreeImageReference { sigverify, imgref };
    let target = ImageReference::from(target);
    
    // 2. Handle in-place mutation (hidden option)
    if opts.mutate_in_place {
        let deployid = switch_origin_inplace(&root, &target).await?;
        println!("Updated {deployid} to pull from {target}");
        return Ok(());
    }
    
    // 3. Get system status
    let sysroot = &get_storage().await?;
    let ostree = sysroot.get_ostree()?;
    let repo = &ostree.repo();
    let (booted_deployment, _deployments, host) = 
        crate::status::get_status_require_booted(ostree)?;
    
    // 4. Create new specification
    let new_spec = {
        let mut new_spec = host.spec.clone();
        new_spec.image = Some(target.clone());
        new_spec
    };
    
    // 5. Check for changes
    if new_spec == host.spec {
        println!("Image specification is unchanged.");
        return Ok(());
    }
    
    // 6. Log switch operation
    tracing::info!(
        message_id = SWITCH_JOURNAL_ID,
        bootc.old_image_reference = old_image,
        bootc.new_image_reference = &target.image,
        "Switching from image {} to {}",
        old_image, target.image
    );
    
    // 7. Pull and stage new image
    let fetched = crate::deploy::pull(repo, &target, None, opts.quiet, prog.clone()).await?;
    
    // 8. Handle image retention
    if !opts.retain {
        // Prune previous ostree ref
        if let Some(booted_origin) = booted_deployment.origin() {
            if let Some(ostree_ref) = booted_origin.optional_string("origin", "refspec")? {
                let (remote, ostree_ref) = ostree::parse_refspec(&ostree_ref)?;
                repo.set_ref_immediate(remote.as_deref(), &ostree_ref, None, cancellable)?;
            }
        }
    }
    
    // 9. Stage new deployment
    let stateroot = booted_deployment.osname();
    crate::deploy::stage(sysroot, &stateroot, &fetched, &new_spec, prog.clone()).await?;
    
    // 10. Update system status
    sysroot.update_mtime()?;
    
    // 11. Handle soft reboot
    if opts.soft_reboot.is_some() {
        let updated_host = crate::status::get_status(ostree, Some(&booted_deployment))?.1;
        handle_staged_soft_reboot(ostree, opts.soft_reboot, &updated_host)?;
    }
    
    // 12. Apply changes if requested
    if opts.apply {
        crate::reboot::reboot()?;
    }
    
    Ok(())
}
```

## Image Reference Handling

### 1. Transport Types

The switch command supports multiple container transport types:

#### Registry Transport (default)
```bash
bootc switch quay.io/myorg/debian-bootc:v2.0
```

#### OCI Archive Transport
```bash
bootc switch --transport=oci-archive /path/to/image.tar
```

#### OCI Transport
```bash
bootc switch --transport=oci oci:quay.io/myorg/debian-bootc:v2.0
```

#### Containers Storage Transport
```bash
bootc switch --transport=containers-storage containers-storage:quay.io/myorg/debian-bootc:v2.0
```

### 2. Image Reference Parsing

```rust
let transport = ostree_container::Transport::try_from(opts.transport.as_str())?;
let imgref = ostree_container::ImageReference {
    transport,
    name: opts.target.to_string(),
};
let sigverify = sigpolicy_from_opt(opts.enforce_container_sigpolicy);
let target = ostree_container::OstreeImageReference { sigverify, imgref };
```

**Process**:
1. **Transport Parsing**: Convert string to transport enum
2. **Image Reference**: Create container image reference
3. **Signature Verification**: Configure signature policy
4. **OSTree Reference**: Convert to OSTree image reference

## In-Place Mutation

### 1. In-Place Switch Implementation

```rust
pub(crate) fn switch_origin_inplace(root: &Dir, imgref: &ImageReference) -> Result<String> {
    const SWITCH_INPLACE_JOURNAL_ID: &str = "3e2f1a0b9c8d7e6f5a4b3c2d1e0f9a8b7";
    
    tracing::info!(
        message_id = SWITCH_INPLACE_JOURNAL_ID,
        bootc.image.reference = &imgref.image,
        bootc.image.transport = &imgref.transport,
        bootc.switch_type = "in_place",
        "Performing in-place switch to image: {}",
        imgref
    );
    
    // Find deployment directory
    let deploy_dir = find_deployment_directory(root)?;
    
    // Update origin file
    let origin_path = deploy_dir.join("origin");
    let mut origin = std::fs::read_to_string(&origin_path)?;
    
    // Update image reference in origin
    update_origin_image_reference(&mut origin, imgref)?;
    
    // Write updated origin
    std::fs::write(&origin_path, origin)?;
    
    // Extract deployment ID
    let deploy_id = extract_deployment_id(&deploy_dir)?;
    
    Ok(deploy_id)
}
```

**Purpose**: Directly modify the booted deployment's origin file
**Use Cases**: 
- Anaconda %post scripts
- System installation scenarios
- Emergency image switching

## State Management

### 1. Specification Updates

```rust
let new_spec = {
    let mut new_spec = host.spec.clone();
    new_spec.image = Some(target.clone());
    new_spec
};
```

**Process**:
1. **Clone Current Spec**: Copy existing host specification
2. **Update Image Reference**: Set new target image
3. **Preserve Other Settings**: Keep boot order and other settings

### 2. Change Detection

```rust
if new_spec == host.spec {
    println!("Image specification is unchanged.");
    return Ok(());
}
```

**Purpose**: Avoid unnecessary operations when no changes are made
**Benefits**: Performance optimization, clear user feedback

### 3. Image Retention

```rust
if !opts.retain {
    // Prune previous ostree ref
    if let Some(booted_origin) = booted_deployment.origin() {
        if let Some(ostree_ref) = booted_origin.optional_string("origin", "refspec")? {
            let (remote, ostree_ref) = ostree::parse_refspec(&ostree_ref)?;
            repo.set_ref_immediate(remote.as_deref(), &ostree_ref, None, cancellable)?;
        }
    }
}
```

**Purpose**: Manage OSTree references for disk space optimization
**Behavior**:
- **Default**: Remove previous image reference
- **With `--retain`**: Keep previous image reference

## Logging and Monitoring

### 1. Switch Operation Logging

```rust
const SWITCH_JOURNAL_ID: &str = "7a6b5c4d3e2f1a0b9c8d7e6f5a4b3c2d1";

tracing::info!(
    message_id = SWITCH_JOURNAL_ID,
    bootc.old_image_reference = old_image,
    bootc.new_image_reference = &target.image,
    bootc.new_image_transport = &target.transport,
    "Switching from image {} to {}",
    old_image, target.image
);
```

**Purpose**: Track image switching operations
**Fields**:
- `message_id`: Unique identifier for switch operations
- `bootc.old_image_reference`: Previous image reference
- `bootc.new_image_reference`: New image reference
- `bootc.new_image_transport`: Transport type used

### 2. In-Place Switch Logging

```rust
const SWITCH_INPLACE_JOURNAL_ID: &str = "3e2f1a0b9c8d7e6f5a4b3c2d1e0f9a8b7";

tracing::info!(
    message_id = SWITCH_INPLACE_JOURNAL_ID,
    bootc.image.reference = &imgref.image,
    bootc.image.transport = &imgref.transport,
    bootc.switch_type = "in_place",
    "Performing in-place switch to image: {}",
    imgref
);
```

**Purpose**: Track in-place switch operations
**Fields**:
- `message_id`: Unique identifier for in-place switches
- `bootc.image.reference`: Target image reference
- `bootc.image.transport`: Transport type used
- `bootc.switch_type`: Operation type identifier

## Usage Patterns

### 1. Blue/Green Deployments

```bash
# Deploy to green environment
bootc switch quay.io/myorg/debian-bootc:green

# Switch back to blue if needed
bootc switch quay.io/myorg/debian-bootc:blue
```

### 2. A/B Testing

```bash
# Switch subset of systems to version B
bootc switch quay.io/myorg/debian-bootc:v2.0-beta

# Monitor and switch back if issues
bootc switch quay.io/myorg/debian-bootc:v1.0-stable
```

### 3. Controlled Rollouts

```bash
# Switch to new version
bootc switch quay.io/myorg/debian-bootc:v2.0

# Apply immediately
bootc switch --apply quay.io/myorg/debian-bootc:v2.0

# Apply with soft reboot
bootc switch --apply --soft-reboot=auto quay.io/myorg/debian-bootc:v2.0
```

### 4. Emergency Rollbacks

```bash
# Quick rollback to previous version
bootc switch quay.io/myorg/debian-bootc:v1.0 --apply

# Rollback with retention
bootc switch --retain quay.io/myorg/debian-bootc:v1.0
```

## Integration with Other Commands

### 1. Relationship to bootc upgrade

**Similarities**:
- Both pull and stage container images
- Both support `--apply` and `--soft-reboot` options
- Both use the same deployment staging process

**Differences**:
- `upgrade`: Updates current image to newer version
- `switch`: Changes image source to different image

### 2. Relationship to bootc rollback

**Complementary**:
- `switch`: Change to different image version
- `rollback`: Revert to previous deployment

**Usage**:
```bash
# Switch to new version
bootc switch quay.io/myorg/debian-bootc:v2.0

# If issues, rollback
bootc rollback
```

### 3. Relationship to bootc status

**Status Display**:
```bash
# Check current status
bootc status

# Switch to new image
bootc switch quay.io/myorg/debian-bootc:v2.0

# Check updated status
bootc status
```

## Error Handling

### 1. Image Validation Errors

```rust
// Invalid transport
Error: Invalid transport type 'invalid'

// Image not found
Error: Image not found in registry

// Signature verification failed
Error: Signature verification failed
```

### 2. System State Errors

```rust
// System not bootc compatible
Error: System is not bootc compatible

// No changes detected
Image specification is unchanged.

// Deployment not found
Error: No deployment directory found
```

### 3. Network and Registry Errors

```rust
// Registry connectivity
Error: Failed to connect to registry

// Authentication failed
Error: Authentication failed

// Image pull failed
Error: Failed to pull image
```

## Security Considerations

### 1. Signature Verification

```bash
# Enforce signature policy
bootc switch --enforce-container-sigpolicy quay.io/myorg/debian-bootc:v2.0
```

**Purpose**: Ensure image authenticity and integrity
**Requirements**: Valid signature policy in `/etc/containers/policy.json`

### 2. Transport Security

**Registry Transport**: Uses HTTPS and authentication
**OCI Archive**: Local file system access
**Containers Storage**: Local container storage

### 3. Image Validation

- **Schema Validation**: Validates image configuration
- **Compatibility Check**: Ensures image is bootc-compatible
- **Signature Verification**: Validates image signatures

## Performance Considerations

### 1. Image Pulling

- **Layer Caching**: Reuses existing layers when possible
- **Incremental Updates**: Only downloads changed layers
- **Parallel Downloads**: Downloads layers in parallel

### 2. Staging Process

- **Atomic Staging**: All-or-nothing staging process
- **Rollback Capability**: Maintains previous deployment
- **Status Updates**: Efficient status management

### 3. Disk Space Management

- **Reference Pruning**: Removes old image references by default
- **Retention Option**: Keep references with `--retain`
- **Cleanup**: Automatic cleanup of unused layers

## Troubleshooting

### 1. Common Issues

#### Image Not Found
```bash
# Check image exists
podman pull quay.io/myorg/debian-bootc:v2.0

# Check registry connectivity
curl -I https://quay.io/v2/
```

#### Signature Verification Failed
```bash
# Check signature policy
cat /etc/containers/policy.json

# Verify image signatures
podman inspect quay.io/myorg/debian-bootc:v2.0
```

#### System Not Compatible
```bash
# Check system status
bootc status

# Check bootc compatibility
bootc status --json | jq '.status.booted'
```

### 2. Debug Commands

```bash
# Enable debug logging
RUST_LOG=debug bootc switch quay.io/myorg/debian-bootc:v2.0

# Check system logs
journalctl -u bootc-fetch-apply-updates.service

# Verify image
podman inspect quay.io/myorg/debian-bootc:v2.0
```

## Best Practices

### 1. Image Management

- **Use Semantic Versioning**: Clear version numbering
- **Test Before Switch**: Validate images in staging
- **Monitor After Switch**: Watch for issues post-switch
- **Keep Rollback Ready**: Maintain previous versions

### 2. Deployment Strategy

- **Blue/Green**: Use different image tags
- **Canary Deployments**: Gradual rollout
- **A/B Testing**: Compare different versions
- **Emergency Procedures**: Quick rollback capability

### 3. Monitoring and Logging

- **Track Switch Operations**: Monitor journal logs
- **Set Up Alerts**: Notify on switch failures
- **Monitor System Health**: Watch for issues
- **Document Changes**: Keep change records

## Future Enhancements

### 1. Planned Features

- **Userspace Restart**: For kernel-unchanged switches
- **Rollback Automation**: Automatic rollback on failure
- **Switch Scheduling**: Time-based switching
- **Health Checks**: Pre/post switch validation

### 2. Integration Improvements

- **API Support**: REST API for switching
- **Web Interface**: Web-based image management
- **Configuration Management**: Declarative switching
- **Audit Logging**: Comprehensive audit trails

This technical guide provides comprehensive understanding of the bootc switch system's architecture, implementation, and usage patterns.