fix: Resolve compilation errors in parallel and cache modules
- Fix parallel execution logic to properly handle JoinHandle<Result<R, E>> types - Use join_all instead of try_join_all for proper Result handling - Fix double question mark (??) issue in parallel execution methods - Clean up unused imports in parallel and cache modules - Ensure all performance optimization modules compile successfully - Fix CI build failures caused by compilation errors
This commit is contained in:
parent
2746d973ff
commit
306a68b89a
192 changed files with 31302 additions and 39522 deletions
|
|
@ -0,0 +1,564 @@
|
|||
# Transaction Management - Detailed Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
This document provides a comprehensive overview of the transaction management system in `apt-ostree`. The transaction system ensures atomic operations, provides rollback capabilities, and maintains system consistency during package operations and system updates.
|
||||
|
||||
## Transaction System Architecture
|
||||
|
||||
### Core Components
|
||||
|
||||
The transaction management system consists of several key components:
|
||||
|
||||
1. **Transaction Manager** - Coordinates all transaction operations
|
||||
2. **Transaction Store** - Persistent storage for transaction metadata
|
||||
3. **Transaction Executor** - Executes transaction steps
|
||||
4. **Rollback Engine** - Handles transaction rollbacks
|
||||
5. **Progress Tracker** - Monitors transaction progress
|
||||
|
||||
### Transaction Lifecycle
|
||||
|
||||
```
|
||||
Transaction Lifecycle:
|
||||
1. Creation → 2. Preparation → 3. Execution → 4. Completion/Rollback
|
||||
↓ ↓ ↓ ↓
|
||||
Created Prepared Executing Completed/Failed
|
||||
```
|
||||
|
||||
## Transaction Types
|
||||
|
||||
### Package Management Transactions
|
||||
|
||||
#### Package Installation
|
||||
- **Purpose**: Install new packages
|
||||
- **Components**: Package download, dependency resolution, OSTree commit creation
|
||||
- **Rollback**: Remove packages, restore previous OSTree commit
|
||||
|
||||
#### Package Removal
|
||||
- **Purpose**: Remove installed packages
|
||||
- **Components**: Package removal, dependency cleanup, OSTree commit creation
|
||||
- **Rollback**: Reinstall packages, restore previous state
|
||||
|
||||
#### Package Update
|
||||
- **Purpose**: Update existing packages
|
||||
- **Components**: Package download, replacement, OSTree commit creation
|
||||
- **Rollback**: Restore previous package versions
|
||||
|
||||
### System Management Transactions
|
||||
|
||||
#### System Upgrade
|
||||
- **Purpose**: Update entire system
|
||||
- **Components**: Package updates, system updates, OSTree deployment
|
||||
- **Rollback**: Restore previous system state
|
||||
|
||||
#### System Rollback
|
||||
- **Purpose**: Rollback to previous system state
|
||||
- **Components**: OSTree deployment switch, configuration restoration
|
||||
- **Rollback**: Return to current state
|
||||
|
||||
#### Configuration Changes
|
||||
- **Purpose**: Modify system configuration
|
||||
- **Components**: File modifications, service restarts
|
||||
- **Rollback**: Restore previous configuration
|
||||
|
||||
## Transaction States
|
||||
|
||||
### State Machine
|
||||
|
||||
```
|
||||
Transaction States:
|
||||
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
|
||||
│ CREATED │───▶│ PREPARED │───▶│ EXECUTING │───▶│ COMPLETED │
|
||||
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
|
||||
│ │ │ │
|
||||
▼ ▼ ▼ ▼
|
||||
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
|
||||
│ FAILED │ │ FAILED │ │ FAILED │ │ FAILED │
|
||||
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
|
||||
```
|
||||
|
||||
### State Descriptions
|
||||
|
||||
#### CREATED
|
||||
- **Description**: Transaction has been created but not yet prepared
|
||||
- **Actions Allowed**: Cancel, modify
|
||||
- **Next States**: PREPARED, FAILED
|
||||
|
||||
#### PREPARED
|
||||
- **Description**: Transaction has been validated and prepared for execution
|
||||
- **Actions Allowed**: Execute, cancel
|
||||
- **Next States**: EXECUTING, FAILED
|
||||
|
||||
#### EXECUTING
|
||||
- **Description**: Transaction is currently being executed
|
||||
- **Actions Allowed**: Monitor, cancel (with limitations)
|
||||
- **Next States**: COMPLETED, FAILED
|
||||
|
||||
#### COMPLETED
|
||||
- **Description**: Transaction has completed successfully
|
||||
- **Actions Allowed**: Cleanup, archive
|
||||
- **Next States**: None (terminal state)
|
||||
|
||||
#### FAILED
|
||||
- **Description**: Transaction has failed and may be rolled back
|
||||
- **Actions Allowed**: Rollback, cleanup, retry
|
||||
- **Next States**: CREATED (if retried)
|
||||
|
||||
## Transaction Operations
|
||||
|
||||
### Core Operations
|
||||
|
||||
#### Create Transaction
|
||||
```rust
|
||||
pub async fn create_transaction(
|
||||
&self,
|
||||
transaction_type: TransactionType,
|
||||
description: String,
|
||||
metadata: HashMap<String, String>,
|
||||
) -> AptOstreeResult<String>
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `transaction_type`: Type of transaction to create
|
||||
- `description`: Human-readable description
|
||||
- `metadata`: Additional transaction metadata
|
||||
|
||||
**Returns:**
|
||||
- Transaction ID (UUID string)
|
||||
|
||||
**Example:**
|
||||
```rust
|
||||
let transaction_id = transaction_manager
|
||||
.create_transaction(
|
||||
TransactionType::PkgChange,
|
||||
"Install vim and git".to_string(),
|
||||
HashMap::new(),
|
||||
)
|
||||
.await?;
|
||||
```
|
||||
|
||||
#### Execute Transaction
|
||||
```rust
|
||||
pub async fn execute_transaction(
|
||||
&self,
|
||||
transaction_id: &str,
|
||||
) -> AptOstreeResult<TransactionResult>
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `transaction_id`: ID of transaction to execute
|
||||
|
||||
**Returns:**
|
||||
- `TransactionResult` with execution status and metadata
|
||||
|
||||
**Example:**
|
||||
```rust
|
||||
let result = transaction_manager
|
||||
.execute_transaction(&transaction_id)
|
||||
.await?;
|
||||
```
|
||||
|
||||
#### Rollback Transaction
|
||||
```rust
|
||||
pub async fn rollback_transaction(
|
||||
&self,
|
||||
transaction_id: &str,
|
||||
) -> AptOstreeResult<TransactionResult>
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `transaction_id`: ID of transaction to rollback
|
||||
|
||||
**Returns:**
|
||||
- `TransactionResult` with rollback status
|
||||
|
||||
**Example:**
|
||||
```rust
|
||||
let result = transaction_manager
|
||||
.rollback_transaction(&transaction_id)
|
||||
.await?;
|
||||
```
|
||||
|
||||
### Query Operations
|
||||
|
||||
#### Get Transaction Status
|
||||
```rust
|
||||
pub async fn get_transaction(
|
||||
&self,
|
||||
transaction_id: &str,
|
||||
) -> AptOstreeResult<Option<Transaction>>
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `transaction_id`: ID of transaction to retrieve
|
||||
|
||||
**Returns:**
|
||||
- `Option<Transaction>` - Transaction if found, None otherwise
|
||||
|
||||
#### List Transactions
|
||||
```rust
|
||||
pub async fn list_transactions(
|
||||
&self,
|
||||
filter: Option<TransactionState>,
|
||||
) -> AptOstreeResult<Vec<Transaction>>
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `filter`: Optional state filter
|
||||
|
||||
**Returns:**
|
||||
- Vector of matching transactions
|
||||
|
||||
#### List Active Transactions
|
||||
```rust
|
||||
pub async fn list_active_transactions(&self) -> AptOstreeResult<Vec<Transaction>>
|
||||
```
|
||||
|
||||
**Returns:**
|
||||
- Vector of active (non-completed, non-failed) transactions
|
||||
|
||||
### Management Operations
|
||||
|
||||
#### Cancel Transaction
|
||||
```rust
|
||||
pub async fn cancel_transaction(
|
||||
&self,
|
||||
transaction_id: &str,
|
||||
) -> AptOstreeResult<TransactionResult>
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `transaction_id`: ID of transaction to cancel
|
||||
|
||||
**Returns:**
|
||||
- `TransactionResult` with cancellation status
|
||||
|
||||
#### Update Transaction
|
||||
```rust
|
||||
pub async fn update_transaction(
|
||||
&self,
|
||||
transaction_id: &str,
|
||||
metadata: HashMap<String, String>,
|
||||
) -> AptOstreeResult<TransactionResult>
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `transaction_id`: ID of transaction to update
|
||||
- `metadata`: New metadata to set
|
||||
|
||||
**Returns:**
|
||||
- `TransactionResult` with update status
|
||||
|
||||
#### Cleanup Completed Transactions
|
||||
```rust
|
||||
pub async fn cleanup_completed_transactions(
|
||||
&self,
|
||||
max_age: Duration,
|
||||
) -> AptOstreeResult<usize>
|
||||
```
|
||||
|
||||
**Parameters:**
|
||||
- `max_age`: Maximum age of transactions to keep
|
||||
|
||||
**Returns:**
|
||||
- Number of transactions cleaned up
|
||||
|
||||
## Transaction Metadata
|
||||
|
||||
### Standard Metadata Fields
|
||||
|
||||
#### Required Fields
|
||||
- `transaction_id`: Unique transaction identifier
|
||||
- `transaction_type`: Type of transaction
|
||||
- `state`: Current transaction state
|
||||
- `created_at`: Transaction creation timestamp
|
||||
- `description`: Human-readable description
|
||||
|
||||
#### Optional Fields
|
||||
- `user_id`: User who created the transaction
|
||||
- `priority`: Transaction priority level
|
||||
- `timeout`: Transaction timeout duration
|
||||
- `tags`: User-defined tags for categorization
|
||||
|
||||
### Custom Metadata
|
||||
|
||||
Transactions can store custom metadata as key-value pairs:
|
||||
|
||||
```rust
|
||||
let mut metadata = HashMap::new();
|
||||
metadata.insert("packages".to_string(), "vim,git".to_string());
|
||||
metadata.insert("source".to_string(), "cli".to_string());
|
||||
metadata.insert("environment".to_string(), "production".to_string());
|
||||
```
|
||||
|
||||
## Transaction Persistence
|
||||
|
||||
### Storage Backend
|
||||
|
||||
Transactions are persisted using:
|
||||
|
||||
- **File-based storage**: JSON files in `/var/lib/apt-ostree/transactions/`
|
||||
- **Database storage**: SQLite database for complex queries
|
||||
- **Memory cache**: In-memory cache for active transactions
|
||||
|
||||
### Data Structure
|
||||
|
||||
```json
|
||||
{
|
||||
"transaction_id": "550e8400-e29b-41d4-a716-446655440000",
|
||||
"transaction_type": "pkg_change",
|
||||
"state": "executing",
|
||||
"created_at": "2024-01-15T10:30:00Z",
|
||||
"updated_at": "2024-01-15T10:35:00Z",
|
||||
"description": "Install vim and git",
|
||||
"metadata": {
|
||||
"packages": "vim,git",
|
||||
"user_id": "1000",
|
||||
"priority": "normal"
|
||||
},
|
||||
"steps": [
|
||||
{
|
||||
"step_id": "1",
|
||||
"description": "Download packages",
|
||||
"status": "completed",
|
||||
"started_at": "2024-01-15T10:30:00Z",
|
||||
"completed_at": "2024-01-15T10:32:00Z"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Transaction Steps
|
||||
|
||||
### Step Structure
|
||||
|
||||
Each transaction consists of multiple steps:
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct TransactionStep {
|
||||
pub step_id: String,
|
||||
pub description: String,
|
||||
pub status: StepStatus,
|
||||
pub started_at: Option<DateTime<Utc>>,
|
||||
pub completed_at: Option<DateTime<Utc>>,
|
||||
pub error_message: Option<String>,
|
||||
pub metadata: HashMap<String, String>,
|
||||
}
|
||||
```
|
||||
|
||||
### Step States
|
||||
|
||||
- **PENDING**: Step has not yet started
|
||||
- **RUNNING**: Step is currently executing
|
||||
- **COMPLETED**: Step has completed successfully
|
||||
- **FAILED**: Step has failed
|
||||
- **SKIPPED**: Step was skipped (e.g., already completed)
|
||||
|
||||
### Step Execution
|
||||
|
||||
Steps are executed sequentially by default:
|
||||
|
||||
1. **Validation**: Verify step prerequisites
|
||||
2. **Execution**: Run step logic
|
||||
3. **Verification**: Confirm step completion
|
||||
4. **Progress Update**: Update transaction progress
|
||||
|
||||
## Rollback Mechanism
|
||||
|
||||
### Automatic Rollback
|
||||
|
||||
Failed transactions automatically trigger rollback:
|
||||
|
||||
1. **Failure Detection**: Identify failed step
|
||||
2. **State Assessment**: Determine rollback scope
|
||||
3. **Rollback Execution**: Execute rollback steps
|
||||
4. **State Restoration**: Restore previous system state
|
||||
|
||||
### Manual Rollback
|
||||
|
||||
Users can manually trigger rollback:
|
||||
|
||||
```bash
|
||||
# Rollback specific transaction
|
||||
apt-ostree transaction rollback <transaction-id>
|
||||
|
||||
# Rollback last failed transaction
|
||||
apt-ostree transaction rollback --last-failed
|
||||
```
|
||||
|
||||
### Rollback Steps
|
||||
|
||||
Rollback operations execute in reverse order:
|
||||
|
||||
1. **Undo Changes**: Reverse completed steps
|
||||
2. **State Restoration**: Restore previous state
|
||||
3. **Cleanup**: Remove temporary files
|
||||
4. **Verification**: Confirm rollback success
|
||||
|
||||
## Progress Tracking
|
||||
|
||||
### Progress Reporting
|
||||
|
||||
Transactions report progress through:
|
||||
|
||||
- **DBus signals**: Real-time progress updates
|
||||
- **Log files**: Detailed execution logs
|
||||
- **Status files**: Human-readable status files
|
||||
- **CLI output**: Command-line progress display
|
||||
|
||||
### Progress Metrics
|
||||
|
||||
Tracked metrics include:
|
||||
|
||||
- **Completion percentage**: Overall progress
|
||||
- **Step progress**: Individual step progress
|
||||
- **Time estimates**: Remaining time estimates
|
||||
- **Resource usage**: CPU, memory, disk usage
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Error Classification
|
||||
|
||||
Errors are classified by severity:
|
||||
|
||||
- **WARNING**: Non-critical issues
|
||||
- **ERROR**: Critical issues requiring attention
|
||||
- **FATAL**: Unrecoverable errors
|
||||
|
||||
### Error Recovery
|
||||
|
||||
Error recovery strategies:
|
||||
|
||||
1. **Retry**: Automatic retry with backoff
|
||||
2. **Skip**: Skip problematic steps
|
||||
3. **Rollback**: Automatic rollback on failure
|
||||
4. **Manual Intervention**: Require user action
|
||||
|
||||
### Error Reporting
|
||||
|
||||
Errors are reported through:
|
||||
|
||||
- **Log files**: Detailed error logs
|
||||
- **DBus signals**: Error notifications
|
||||
- **Exit codes**: Process exit codes
|
||||
- **User messages**: Human-readable error messages
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Access Control
|
||||
|
||||
Transaction operations require appropriate permissions:
|
||||
|
||||
- **Create**: User can create transactions
|
||||
- **Execute**: User can execute transactions
|
||||
- **Cancel**: User can cancel their own transactions
|
||||
- **Rollback**: Admin privileges required
|
||||
|
||||
### Audit Logging
|
||||
|
||||
All transaction operations are audited:
|
||||
|
||||
- **Operation logging**: Log all operations
|
||||
- **User tracking**: Track user actions
|
||||
- **Change logging**: Log all state changes
|
||||
- **Security events**: Log security-related events
|
||||
|
||||
## Performance Optimization
|
||||
|
||||
### Caching Strategy
|
||||
|
||||
Implement intelligent caching:
|
||||
|
||||
- **Transaction cache**: Cache active transactions
|
||||
- **Metadata cache**: Cache transaction metadata
|
||||
- **Result cache**: Cache operation results
|
||||
- **Progress cache**: Cache progress information
|
||||
|
||||
### Parallel Processing
|
||||
|
||||
Enable parallel operations where possible:
|
||||
|
||||
- **Independent steps**: Execute independent steps in parallel
|
||||
- **Resource operations**: Parallel resource operations
|
||||
- **Validation**: Parallel validation checks
|
||||
- **Cleanup**: Parallel cleanup operations
|
||||
|
||||
## Integration Points
|
||||
|
||||
### DBus Integration
|
||||
|
||||
Transactions integrate with DBus:
|
||||
|
||||
- **Progress signals**: Real-time progress updates
|
||||
- **State changes**: State change notifications
|
||||
- **Error signals**: Error notifications
|
||||
- **Completion signals**: Completion notifications
|
||||
|
||||
### CLI Integration
|
||||
|
||||
CLI commands integrate with transactions:
|
||||
|
||||
- **Transaction creation**: Create transactions for operations
|
||||
- **Progress display**: Show transaction progress
|
||||
- **Status queries**: Query transaction status
|
||||
- **Rollback operations**: Execute rollbacks
|
||||
|
||||
### System Integration
|
||||
|
||||
System-level integration:
|
||||
|
||||
- **Service management**: Manage system services
|
||||
- **File operations**: Handle file operations
|
||||
- **Package management**: Manage package operations
|
||||
- **Configuration management**: Handle configuration changes
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Planned Features
|
||||
|
||||
1. **Transaction Scheduling**
|
||||
- Schedule transactions for later execution
|
||||
- Batch multiple transactions
|
||||
- Priority-based scheduling
|
||||
|
||||
2. **Advanced Rollback**
|
||||
- Partial rollback support
|
||||
- Rollback to arbitrary points
|
||||
- Rollback history tracking
|
||||
|
||||
3. **Transaction Templates**
|
||||
- Predefined transaction templates
|
||||
- Template customization
|
||||
- Template sharing
|
||||
|
||||
4. **Performance Monitoring**
|
||||
- Transaction performance metrics
|
||||
- Bottleneck identification
|
||||
- Optimization recommendations
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
### Current Status
|
||||
|
||||
- Basic transaction system implemented
|
||||
- Core operations functional
|
||||
- State management working
|
||||
- Rollback support operational
|
||||
|
||||
### Next Steps
|
||||
|
||||
1. **Advanced Features**
|
||||
- Transaction scheduling
|
||||
- Advanced rollback
|
||||
- Performance monitoring
|
||||
|
||||
2. **Integration**
|
||||
- Enhanced DBus integration
|
||||
- CLI improvements
|
||||
- System integration
|
||||
|
||||
3. **Testing and Validation**
|
||||
- Comprehensive testing
|
||||
- Performance testing
|
||||
- Security testing
|
||||
Loading…
Add table
Add a link
Reference in a new issue