apt-ostree/docs/.old/apt-ostree-daemon-plan/architecture/transaction-management-detailed.md

Transaction Management - Detailed Architecture

Overview

This document provides a comprehensive overview of the transaction management system in apt-ostree. The transaction system ensures atomic operations, provides rollback capabilities, and maintains system consistency during package operations and system updates.

Transaction System Architecture

Core Components

The transaction management system consists of several key components:

1. Transaction Manager - Coordinates all transaction operations
2. Transaction Store - Persistent storage for transaction metadata
3. Transaction Executor - Executes transaction steps
4. Rollback Engine - Handles transaction rollbacks
5. Progress Tracker - Monitors transaction progress

Transaction Lifecycle

Transaction Lifecycle:
1. Creation → 2. Preparation → 3. Execution → 4. Completion/Rollback
    ↓              ↓              ↓              ↓
  Created      Prepared      Executing      Completed/Failed

Transaction Types

Package Management Transactions

Package Installation

• Purpose: Install new packages
• Components: Package download, dependency resolution, OSTree commit creation
• Rollback: Remove packages, restore previous OSTree commit

Package Removal

• Purpose: Remove installed packages
• Components: Package removal, dependency cleanup, OSTree commit creation
• Rollback: Reinstall packages, restore previous state

Package Update

• Purpose: Update existing packages
• Components: Package download, replacement, OSTree commit creation
• Rollback: Restore previous package versions

System Management Transactions

System Upgrade

• Purpose: Update entire system
• Components: Package updates, system updates, OSTree deployment
• Rollback: Restore previous system state

System Rollback

• Purpose: Rollback to previous system state
• Components: OSTree deployment switch, configuration restoration
• Rollback: Return to current state

Configuration Changes

• Purpose: Modify system configuration
• Components: File modifications, service restarts
• Rollback: Restore previous configuration

Transaction States

State Machine

Transaction States:
┌─────────────┐    ┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│  CREATED   │───▶│  PREPARED  │───▶│ EXECUTING  │───▶│ COMPLETED  │
└─────────────┘    └─────────────┘    └─────────────┘    └─────────────┘
       │                   │                   │                   │
       ▼                   ▼                   ▼                   ▼
┌─────────────┐    ┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   FAILED    │    │   FAILED    │    │   FAILED    │    │   FAILED    │
└─────────────┘    └─────────────┘    └─────────────┘    └─────────────┘

State Descriptions

CREATED

• Description: Transaction has been created but not yet prepared
• Actions Allowed: Cancel, modify
• Next States: PREPARED, FAILED

PREPARED

• Description: Transaction has been validated and prepared for execution
• Actions Allowed: Execute, cancel
• Next States: EXECUTING, FAILED

EXECUTING

• Description: Transaction is currently being executed
• Actions Allowed: Monitor, cancel (with limitations)
• Next States: COMPLETED, FAILED

COMPLETED

• Description: Transaction has completed successfully
• Actions Allowed: Cleanup, archive
• Next States: None (terminal state)

FAILED

• Description: Transaction has failed and may be rolled back
• Actions Allowed: Rollback, cleanup, retry
• Next States: CREATED (if retried)

Transaction Operations

Core Operations

Create Transaction

pub async fn create_transaction(
    &self,
    transaction_type: TransactionType,
    description: String,
    metadata: HashMap<String, String>,
) -> AptOstreeResult<String>

Parameters:

• transaction_type: Type of transaction to create
• description: Human-readable description
• metadata: Additional transaction metadata

Returns:

• Transaction ID (UUID string)

Example:

let transaction_id = transaction_manager
    .create_transaction(
        TransactionType::PkgChange,
        "Install vim and git".to_string(),
        HashMap::new(),
    )
    .await?;

Execute Transaction

pub async fn execute_transaction(
    &self,
    transaction_id: &str,
) -> AptOstreeResult<TransactionResult>

Parameters:

• transaction_id: ID of transaction to execute

Returns:

• TransactionResult with execution status and metadata

Example:

let result = transaction_manager
    .execute_transaction(&transaction_id)
    .await?;

Rollback Transaction

pub async fn rollback_transaction(
    &self,
    transaction_id: &str,
) -> AptOstreeResult<TransactionResult>

Parameters:

• transaction_id: ID of transaction to rollback

Returns:

• TransactionResult with rollback status

Example:

let result = transaction_manager
    .rollback_transaction(&transaction_id)
    .await?;

Query Operations

Get Transaction Status

pub async fn get_transaction(
    &self,
    transaction_id: &str,
) -> AptOstreeResult<Option<Transaction>>

Parameters:

• transaction_id: ID of transaction to retrieve

Returns:

• Option<Transaction> - Transaction if found, None otherwise

List Transactions

pub async fn list_transactions(
    &self,
    filter: Option<TransactionState>,
) -> AptOstreeResult<Vec<Transaction>>

Parameters:

• filter: Optional state filter

Returns:

• Vector of matching transactions

List Active Transactions

pub async fn list_active_transactions(&self) -> AptOstreeResult<Vec<Transaction>>

Returns:

• Vector of active (non-completed, non-failed) transactions

Management Operations

Cancel Transaction

pub async fn cancel_transaction(
    &self,
    transaction_id: &str,
) -> AptOstreeResult<TransactionResult>

Parameters:

• transaction_id: ID of transaction to cancel

Returns:

• TransactionResult with cancellation status

Update Transaction

pub async fn update_transaction(
    &self,
    transaction_id: &str,
    metadata: HashMap<String, String>,
) -> AptOstreeResult<TransactionResult>

Parameters:

• transaction_id: ID of transaction to update
• metadata: New metadata to set

Returns:

• TransactionResult with update status

Cleanup Completed Transactions

pub async fn cleanup_completed_transactions(
    &self,
    max_age: Duration,
) -> AptOstreeResult<usize>

Parameters:

• max_age: Maximum age of transactions to keep

Returns:

• Number of transactions cleaned up

Transaction Metadata

Standard Metadata Fields

Required Fields

• transaction_id: Unique transaction identifier
• transaction_type: Type of transaction
• state: Current transaction state
• created_at: Transaction creation timestamp
• description: Human-readable description

Optional Fields

• user_id: User who created the transaction
• priority: Transaction priority level
• timeout: Transaction timeout duration
• tags: User-defined tags for categorization

Custom Metadata

Transactions can store custom metadata as key-value pairs:

let mut metadata = HashMap::new();
metadata.insert("packages".to_string(), "vim,git".to_string());
metadata.insert("source".to_string(), "cli".to_string());
metadata.insert("environment".to_string(), "production".to_string());

Transaction Persistence

Storage Backend

Transactions are persisted using:

• File-based storage: JSON files in /var/lib/apt-ostree/transactions/
• Database storage: SQLite database for complex queries
• Memory cache: In-memory cache for active transactions

Data Structure

{
  "transaction_id": "550e8400-e29b-41d4-a716-446655440000",
  "transaction_type": "pkg_change",
  "state": "executing",
  "created_at": "2024-01-15T10:30:00Z",
  "updated_at": "2024-01-15T10:35:00Z",
  "description": "Install vim and git",
  "metadata": {
    "packages": "vim,git",
    "user_id": "1000",
    "priority": "normal"
  },
  "steps": [
    {
      "step_id": "1",
      "description": "Download packages",
      "status": "completed",
      "started_at": "2024-01-15T10:30:00Z",
      "completed_at": "2024-01-15T10:32:00Z"
    }
  ]
}

Transaction Steps

Step Structure

Each transaction consists of multiple steps:

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TransactionStep {
    pub step_id: String,
    pub description: String,
    pub status: StepStatus,
    pub started_at: Option<DateTime<Utc>>,
    pub completed_at: Option<DateTime<Utc>>,
    pub error_message: Option<String>,
    pub metadata: HashMap<String, String>,
}

Step States

• PENDING: Step has not yet started
• RUNNING: Step is currently executing
• COMPLETED: Step has completed successfully
• FAILED: Step has failed
• SKIPPED: Step was skipped (e.g., already completed)

Step Execution

Steps are executed sequentially by default:

1. Validation: Verify step prerequisites
2. Execution: Run step logic
3. Verification: Confirm step completion
4. Progress Update: Update transaction progress

Rollback Mechanism

Automatic Rollback

Failed transactions automatically trigger rollback:

1. Failure Detection: Identify failed step
2. State Assessment: Determine rollback scope
3. Rollback Execution: Execute rollback steps
4. State Restoration: Restore previous system state

Manual Rollback

Users can manually trigger rollback:

# Rollback specific transaction
apt-ostree transaction rollback <transaction-id>

# Rollback last failed transaction
apt-ostree transaction rollback --last-failed

Rollback Steps

Rollback operations execute in reverse order:

1. Undo Changes: Reverse completed steps
2. State Restoration: Restore previous state
3. Cleanup: Remove temporary files
4. Verification: Confirm rollback success

Progress Tracking

Progress Reporting

Transactions report progress through:

• DBus signals: Real-time progress updates
• Log files: Detailed execution logs
• Status files: Human-readable status files
• CLI output: Command-line progress display

Progress Metrics

Tracked metrics include:

• Completion percentage: Overall progress
• Step progress: Individual step progress
• Time estimates: Remaining time estimates
• Resource usage: CPU, memory, disk usage

Error Handling

Error Classification

Errors are classified by severity:

• WARNING: Non-critical issues
• ERROR: Critical issues requiring attention
• FATAL: Unrecoverable errors

Error Recovery

Error recovery strategies:

1. Retry: Automatic retry with backoff
2. Skip: Skip problematic steps
3. Rollback: Automatic rollback on failure
4. Manual Intervention: Require user action

Error Reporting

Errors are reported through:

• Log files: Detailed error logs
• DBus signals: Error notifications
• Exit codes: Process exit codes
• User messages: Human-readable error messages

Security Considerations

Access Control

Transaction operations require appropriate permissions:

• Create: User can create transactions
• Execute: User can execute transactions
• Cancel: User can cancel their own transactions
• Rollback: Admin privileges required

Audit Logging

All transaction operations are audited:

• Operation logging: Log all operations
• User tracking: Track user actions
• Change logging: Log all state changes
• Security events: Log security-related events

Performance Optimization

Caching Strategy

Implement intelligent caching:

• Transaction cache: Cache active transactions
• Metadata cache: Cache transaction metadata
• Result cache: Cache operation results
• Progress cache: Cache progress information

Parallel Processing

Enable parallel operations where possible:

• Independent steps: Execute independent steps in parallel
• Resource operations: Parallel resource operations
• Validation: Parallel validation checks
• Cleanup: Parallel cleanup operations

Integration Points

DBus Integration

Transactions integrate with DBus:

• Progress signals: Real-time progress updates
• State changes: State change notifications
• Error signals: Error notifications
• Completion signals: Completion notifications

CLI Integration

CLI commands integrate with transactions:

• Transaction creation: Create transactions for operations
• Progress display: Show transaction progress
• Status queries: Query transaction status
• Rollback operations: Execute rollbacks

System Integration

System-level integration:

• Service management: Manage system services
• File operations: Handle file operations
• Package management: Manage package operations
• Configuration management: Handle configuration changes

Future Enhancements

Planned Features

1. Transaction Scheduling

   • Schedule transactions for later execution
   • Batch multiple transactions
   • Priority-based scheduling

2. Advanced Rollback

   • Partial rollback support
   • Rollback to arbitrary points
   • Rollback history tracking

3. Transaction Templates

   • Predefined transaction templates
   • Template customization
   • Template sharing

4. Performance Monitoring

   • Transaction performance metrics
   • Bottleneck identification
   • Optimization recommendations

Implementation Notes

Current Status

• Basic transaction system implemented
• Core operations functional
• State management working
• Rollback support operational

Next Steps

1. Advanced Features

   • Transaction scheduling
   • Advanced rollback
   • Performance monitoring

2. Integration

   • Enhanced DBus integration
   • CLI improvements
   • System integration

3. Testing and Validation

   • Comprehensive testing
   • Performance testing
   • Security testing