Major improvements: rollbacks, testing, docs, and code quality

- Fixed rollback implementation with two-phase approach (remove new, downgrade upgraded)
- Added explicit package tracking (newly_installed vs upgraded)
- Implemented graceful error handling with fail-fast approach
- Added comprehensive test suite (20 tests across 3 test files)
- Created centralized APT command execution module (apt_commands.rs)
- Added configuration system with dry-run, quiet, and APT options
- Reduced code duplication and improved maintainability
- Added extensive documentation (rollbacks.md, rollbacks-not-featured.md, ffi-bridge.md)
- Created configuration usage example
- Updated README with crate usage instructions
- All tests passing, clean compilation, production-ready
This commit is contained in:
robojerk 2025-09-13 11:21:35 -07:00
parent 534c0e87a8
commit 2daad2837d
15 changed files with 1412 additions and 139 deletions

View file

@ -0,0 +1,446 @@
# Rollback Implementation Discussion
** This file was used to keep track of ideas for doing rollbacks**
It is not an official document. It's kept here in case we want to relitigate our process
## Current Issues
### Problem 1: Tries to install specific versions that may no longer be available
```rust
// Current problematic code in rollback():
packages_to_restore.push(format!("{}={}", package, before_version));
```
**Issue**: If the old version is no longer in repositories, this will fail.
### Problem 2: Doesn't properly handle newly installed packages
```rust
// Current code treats newly installed packages the same as upgraded ones
if let Some(after_version) = self.after_versions.get(package) {
// Only rollback if version changed
if before_version != after_version {
packages_to_restore.push(format!("{}={}", package, before_version));
}
} else {
// Package was newly installed, remove it
packages_to_restore.push(format!("{}", package));
}
```
**Issue**: Newly installed packages should be removed with `apt remove`, not downgraded.
## Proposed Solutions
### Solution 1: Separate handling for new vs upgraded packages
```rust
// Track what was newly installed vs upgraded
let mut packages_to_remove = Vec::new();
let mut packages_to_downgrade = Vec::new();
for (package, before_version) in &self.before_versions {
if let Some(after_version) = self.after_versions.get(package) {
if before_version != after_version {
packages_to_downgrade.push(format!("{}={}", package, before_version));
}
} else {
// Package was newly installed
packages_to_remove.push(package.clone());
}
}
```
### Solution 2: Use apt remove for newly installed packages
```rust
// Remove newly installed packages
if !packages_to_remove.is_empty() {
let output = Command::new("apt")
.args(&["remove", "-y"])
.args(&packages_to_remove)
.output()
.context("Failed to execute APT remove command")?;
if !output.status.success() {
let error = String::from_utf8_lossy(&output.stderr);
return Err(anyhow!("APT removal failed: {}", error)
.context(format!("Failed to remove packages: {:?}", packages_to_remove)));
}
}
```
### Solution 3: Fallback for unavailable versions
```rust
// Try to downgrade, with fallback to removal
for package_downgrade in packages_to_downgrade {
let output = Command::new("apt")
.args(&["install", "-y", &package_downgrade])
.output()
.context("Failed to execute APT downgrade command")?;
if !output.status.success() {
// Fallback: remove the package entirely
let package_name = package_downgrade.split('=').next().unwrap();
let remove_output = Command::new("apt")
.args(&["remove", "-y", package_name])
.output()
.context("Failed to execute APT remove fallback")?;
if !remove_output.status.success() {
// Log warning but continue
eprintln!("Warning: Could not rollback package {}", package_name);
}
}
}
```
## Questions for Discussion
1. **Should we track package installation state more explicitly?**
- Currently we infer it from before/after version comparison
- Could track "newly_installed" packages separately
2. **How to handle partial rollback failures?**
- Continue with other packages?
- Stop and report what failed?
3. **Should we use apt-mark for version pinning?**
- Pin versions before changes to prevent upgrades
- More complex but potentially more reliable
4. **What about dependency conflicts during rollback?**
- Some packages might have been installed as dependencies
- Removing them might break other packages
## Discussion & Decision
### **Scope: "Good Enough" vs. "Complex"**
**Decision: Aim for "good enough" (80/20 rule)**
- Vast majority of package updates are simple and don't involve complex dependency trees
- Complex dependency graph analysis would push code well beyond line count goals
- Acknowledging apt's inherent limitations is better than fragile workarounds
### **Failure Mode: "Fail Fast"**
**Decision: Stop completely and let user fix manually**
- Most honest approach - communicates exactly what failed
- Prevents further damage from partial rollback
- Gives user control to make informed decisions
- Aligns with "fail with loud, clear error" principle
### **Future Direction: Stepping Stone**
**Decision: This is a stepping stone toward more robust solutions**
- Document limitations clearly in README
- Demonstrate value of rollback feature
- Highlight problems that snapshot-based solutions would solve
- Create foundation for more advanced approaches
## **Final Approach: Hybrid with Explicit Tracking**
```rust
pub struct AptTransaction {
packages: Vec<String>,
newly_installed: HashSet<String>, // Explicit tracking
upgraded: HashMap<String, String>, // package -> old_version
}
```
**Benefits:**
- Handles 80% of common cases correctly
- Clear failure modes
- Maintains simplicity goals
- Sets stage for future improvements
## **Refined Scope: "Good Enough + Safety"**
**Decision: Aim for common cases with robust safety measures**
- Handle 80% of rollbacks correctly
- Add essential safety features (locking, critical package detection)
- Avoid complex dependency graph analysis
- Stop short of filesystem snapshots or full atomicity
**Key Design Principle**: The tracked state represents the explicitly managed packages in the transaction. Due to cascading dependencies, the actual system changes may be larger. The `apt-clone` safety net provides complete system state capture for these edge cases.
## Additional Considerations
### **Package Locking & Concurrency**
- **Lock apt during operations** to prevent concurrent package management
- Use `dpkg --configure -a` before operations, unlock after
- Prevents race conditions with other package managers
### **Critical Package Handling**
- **Use apt's built-in essential package detection**: `dpkg -l | grep 'ii' | grep -E '^E'`
- **Prevent rollback of essential packages** to avoid system breakage
- **No hardcoded whitelist** - rely on apt's own package classification
### **State Persistence & Retry**
- **Simple JSON file**: `/var/lib/apt-wrapper/rollback-state.json`
- **Store before each operation**: Package lists, versions, operation type
- **Enable retry**: Read state file to resume from last successful operation
- **Crash recovery**: Detect incomplete rollback and offer retry option
- **Corruption handling**: Simple checksum validation, fallback to clean state if corrupted
### **Enhanced Error Handling**
- **Fail fast on critical errors**: Package not found, version unavailable, permission denied
- **Continue with warnings on soft errors**: Package already removed, non-critical dependency warnings
- **Clear distinction**: Critical = system-breaking, Soft = recoverable or ignorable
- **Accumulate warnings**: Collect all soft errors and display summary at end
- **Informative fallback messages**: Explain why package removal occurred (version unavailable)
### **Soft Error Definition**
**Soft errors are non-critical issues that don't break the rollback:**
- Package already removed (no action needed)
- Non-essential dependency warnings (system still functional)
- Version downgrade warnings (package still works)
**Critical errors stop the entire rollback:**
- Package not found in repository
- Permission denied
- Essential package conflicts
### **Exit Code Strategy**
- **0**: Rollback successful, maybe with warnings
- **1**: Rollback failed on critical error
- **2**: Rollback succeeded but with unresolved soft issues
## **Rollback Command Strategy**
**Decision: Separate commands for remove vs downgrade (non-atomic)**
- **Phase 1**: `apt remove -y package1 package2` (newly installed packages)
- **Phase 2**: `apt install -y package1=oldver package2=oldver` (upgraded packages)
- **Acknowledge**: This is not atomic, but simpler and more reliable than complex single commands
## **Implementation Details**
### **Warning Accumulation**
```rust
let mut warnings = Vec::new();
// During rollback operations
if some_error_condition {
warnings.push(format!("Warning: Could not remove package {}", package));
}
// After the rollback
if !warnings.is_empty() {
eprintln!("Warnings during rollback:");
for warning in warnings {
eprintln!("{}", warning);
}
}
```
### **Informative Fallback Messages**
```rust
// This is the "continue with warnings" path for a non-critical error
if !output.status.success() {
let package_name = package_downgrade.split('=').next().unwrap();
eprintln!("Warning: Could not downgrade {} to version {} (version no longer available in repository). Removing instead.", package_name, before_version);
let remove_output = Command::new("apt")
.args(&["remove", "-y", package_name])
.output()
.context("Failed to execute APT remove fallback")?;
}
```
### **Lightweight Dependency Checking**
```rust
// Check what packages depend on the package being removed
let output = Command::new("apt-cache")
.arg("rdepends")
.arg(removed_package)
.output()
.expect("Failed to check reverse dependencies");
if !output.stdout.is_empty() {
eprintln!("Warning: Removing {} may affect the following packages:", removed_package);
println!("{}", String::from_utf8_lossy(&output.stdout));
}
```
### **User-Installed Package Awareness**
```rust
// Quick check for user-installed rdepends before removal
let output = Command::new("apt-mark")
.args(&["showmanual"])
.output()?;
let manual_packages: HashSet<_> = String::from_utf8_lossy(&output.stdout)
.lines()
.collect();
for pkg in &packages_to_remove {
let rdepends = get_reverse_dependencies(pkg)?;
// Filter out automatically-installed packages to reduce noise
let user_rdepends: Vec<_> = rdepends
.into_iter()
.filter(|dep| manual_packages.contains(dep))
.collect();
if !user_rdepends.is_empty() {
eprintln!("Warning: Removing {} may break manually installed packages: {:?}", pkg, user_rdepends);
}
}
```
### **Meta-Package Detection**
```rust
// Detect meta-packages and warn about cascade removals
for pkg in &packages_to_remove {
let output = Command::new("apt-cache")
.args(&["show", pkg])
.output()?;
if output.status.success() {
let info = String::from_utf8_lossy(&output.stdout);
if info.contains("Meta package") || info.contains("Virtual package") {
eprintln!("Warning: {} is a meta-package, removal may trigger cascade removals", pkg);
}
}
}
```
### **Dry-Run JSON Output**
```json
{
"would_remove": ["pkg1", "pkg2"],
"would_downgrade": ["pkg3=1.2.3"],
"warnings": ["pkg1 has user-installed reverse dependencies: foo, bar"]
}
```
## **Strategic Improvements**
### **Pre-Transaction Safety Net**
- **Use `apt-clone` (preferred) or `dpkg --get-selections`** to create full system snapshot before rollback
- **apt-clone**: Creates complete package state backup including sources and keyrings
- **dpkg --get-selections**: Simpler but only captures package selections
- **Complete restore point** if rollback fails catastrophically
- **Better than partial rollback** - can restore entire system state
### **Enhanced Dependency Awareness**
- **Warn on non-auto-installed reverse dependencies** (user-installed packages that depend on removed package)
- **Prevent breaking user workflows** by highlighting critical dependencies
- **Interactive confirmation** for potentially destructive operations
### **Version Handling Sophistication**
- **Find closest available older version** instead of exact match or remove
- **Interactive fallback**: "Version X not found. Install nearest available version Y? [Y/n]"
- **User-configurable fallback strategy**:
- `--strict`: fail if exact version not found
- `--nearest`: choose closest available lower version
- `--remove`: remove if version missing (default)
- **Less destructive** than complete package removal
### **Concurrency Safety Improvements**
- **Wait for lock with timeout** instead of immediate failure
- **Check `/var/lib/dpkg/lock-frontend`** and wait up to 30 seconds
- **Clear error message** if timeout expires: "APT is locked by another process, aborting rollback"
- **Optional `--wait-indefinitely` flag** for CI/automation use cases
- **Better user experience** when apt is busy with other operations
### **Enhanced State Management**
- **Schema versioning** in rollback-state.json for future compatibility
- **Deterministic checksums** using canonical JSON serialization with stable key ordering
- **Atomic persistence** after each operation step to enable safe resume
- **State file security**: `/var/lib/apt-wrapper/rollback-state.json` owned by `root:root` with mode `0600`
- **Manual state clearing**: `apt-wrapper rollback --abort`
- **Dry-run mode**: `apt-wrapper rollback --dry-run` to preview changes
- **Machine-readable dry-run output** (JSON format) for automation/CI integration
## **Future Considerations**
### **Auditing & Troubleshooting**
- Track rollback metadata (timestamps, user IDs) for auditing
- Integration with larger system management tools
- Logging for troubleshooting and system analysis
### **Scalability Planning**
- Handle multiple system states simultaneously
- Related package update rollbacks
- Service-level rollback coordination
## **Implementation Phases**
The following phased plan will incrementally implement the features outlined in the 'Final Approach' and 'Strategic Improvements' sections.
### **Phase 1: Core Rollback Logic**
1. Implement separate tracking for new vs upgraded packages
2. Use `apt remove` for newly installed packages
3. **Fail fast on critical errors**, continue with warnings on soft errors
### **Phase 2: Robustness Enhancements**
4. **Add warning accumulation** for better user experience
5. **Add informative fallback messages** explaining why operations occurred
6. **Add lightweight dependency checking** for removed packages
7. Add proper error handling and logging
8. **Add package locking with timeout** to prevent concurrent operations
9. **Implement critical package detection** using apt's essential package list
10. **Add enhanced state persistence** with schema versioning and deterministic checksums
11. **Add dry-run mode** for previewing rollback operations
12. **Add CLI ergonomics**: `--resume`, `--abort`, `--dry-run --json` vs `--dry-run --pretty`
13. **Add exit code strategy** for automation/CI integration
### **Phase 2.5: Advanced Safety Features**
14. **Add pre-transaction snapshot** using apt-clone (preferred) or dpkg --get-selections
15. **Enhanced dependency awareness** for non-auto-installed packages
16. **Sophisticated version handling** with closest available version fallback
17. **Interactive confirmation** for potentially destructive operations
18. **Add meta-package detection** and cascade removal warnings
### **Phase 3: Testing and Documentation**
19. Test with various scenarios
20. Document limitations clearly
21. **Add JSON schema documentation** for rollback-state.json
## **Rollback State JSON Schema**
```json
{
"schema_version": "1.0",
"checksum": "sha256:abc123...",
"timestamp": "2024-01-15T10:30:00Z",
"transaction_id": "tx_12345",
"operation": "rollback",
"state": {
"newly_installed": ["package1", "package2"],
"upgraded": {
"package3": "1.0.0",
"package4": "2.1.0"
},
"before_versions": {
"package3": "0.9.0",
"package4": "2.0.0"
},
"after_versions": {
"package3": "1.0.0",
"package4": "2.1.0"
}
},
"operations_completed": [
{
"phase": "remove",
"packages": ["package1", "package2"],
"status": "success",
"timestamp": "2024-01-15T10:31:00Z"
}
],
"operations_pending": [
{
"phase": "downgrade",
"packages": ["package3=0.9.0", "package4=2.0.0"],
"status": "pending"
}
]
}
```
**Key Features:**
- **Schema versioning**: Future compatibility
- **Checksum**: Corruption detection
- **Transaction tracking**: Unique ID for each rollback
- **Phase tracking**: What's completed vs pending
- **State preservation**: Complete before/after package states

51
docs/ffi-bridge.md Normal file
View file

@ -0,0 +1,51 @@
# FFI Bridge Documentation
## Overview
The APT Wrapper provides a C++ Foreign Function Interface (FFI) bridge for integration with C++ applications, particularly designed for apt-ostree.
## C++ Integration
### Header File
The bridge requires a C++ header file `apt-wrapper/bridge.h` that defines the C++ side of the interface.
### AptPackage FFI
The `AptPackage` struct can be used across the FFI boundary:
```cpp
// C++ side
#include "apt-wrapper/bridge.h"
// Create package from Rust
AptPackage package = new_ffi("vim", "2:8.1.2269-1ubuntu5", "Vi IMproved", true);
// Access package properties
std::string name = package.name();
std::string version = package.version();
std::string description = package.description();
bool installed = package.is_installed();
```
### Rust Side Functions
```rust
// Rust side - these functions are exposed to C++
pub fn new_ffi(name: String, version: String, description: String, installed: bool) -> AptPackage;
pub fn name_ffi(package: &AptPackage) -> &str;
pub fn version_ffi(package: &AptPackage) -> &str;
pub fn description_ffi(package: &AptPackage) -> &str;
pub fn is_installed_ffi(package: &AptPackage) -> bool;
```
## Usage in C++ Projects
1. Include the generated header file
2. Link against the Rust library
3. Use the FFI functions to interact with APT packages
## Dependencies
- `cxx` crate for FFI generation
- C++ compiler with C++17 support

View file

@ -0,0 +1,149 @@
# Rollback Features Not Implemented
This document explains advanced rollback features that were considered but not implemented, and the reasons why.
## Why These Features Were Not Added
The goal was to create a **clean, simple, and maintainable** rollback system that handles 90% of real-world cases without over-engineering. These features were deemed too complex for the current scope.
## Advanced Features Considered
### 1. State Persistence
**What it would do:**
- Save rollback state to `/var/lib/apt-wrapper/rollback-state.json`
- Allow rollback even after the program restarts
- Include checksums and schema versioning
**Why not implemented:**
- Adds file I/O complexity
- Requires error handling for disk operations
- The current in-memory approach is sufficient for most use cases
### 2. Pre-Transaction Snapshots
**What it would do:**
- Use `apt-clone` or `dpkg --get-selections` to create system snapshots
- Store complete package state before transactions
- Enable more comprehensive rollbacks
**Why not implemented:**
- `apt-clone` may not be available on all systems
- Snapshot management adds significant complexity
- Current approach handles the most common scenarios
### 3. Enhanced Dependency Awareness
**What it would do:**
- Track reverse dependencies with `apt-cache rdepends`
- Distinguish between user-installed and auto-installed packages
- Handle meta-package dependencies
**Why not implemented:**
- APT already handles most dependency resolution
- Adds parsing complexity for dependency graphs
- Current simple approach works for most cases
### 4. Sophisticated Version Handling
**What it would do:**
- Find closest available version if exact version not found
- User-configurable fallback strategies (`--strict`, `--nearest`, `--remove`)
- Handle version conflicts gracefully
**Why not implemented:**
- Version resolution is complex and error-prone
- Current "remove if can't downgrade" approach is simple and safe
- Most users don't need fine-grained version control
### 5. Concurrency Safety
**What it would do:**
- Use APT locking mechanisms
- Handle concurrent package operations
- Prevent rollback conflicts
**Why not implemented:**
- APT already provides basic locking
- Concurrent operations are rare in practice
- Adds complexity without clear benefit
### 6. Critical Package Detection
**What it would do:**
- Identify essential packages using `dpkg -l`
- Prevent rollback of critical system packages
- Add safety checks for system stability
**Why not implemented:**
- Most users don't try to rollback critical packages
- APT already provides some protection
- Adds complexity for edge cases
### 7. Dry-Run Mode
**What it would do:**
- Show what would be rolled back without doing it
- Machine-readable JSON output
- Preview rollback operations
**Why not implemented:**
- Current `changed_packages()` method provides similar functionality
- Dry-run for rollback is less critical than for installation
- Adds output formatting complexity
### 8. Resume/Abort Operations
**What it would do:**
- Allow resuming interrupted rollbacks
- Provide `--abort` to cancel rollback operations
- Handle partial rollback states
**Why not implemented:**
- Rollbacks are typically fast operations
- Current fail-fast approach is simpler
- Resume logic adds significant complexity
### 9. Enhanced Error Handling
**What it would do:**
- Accumulate warnings instead of failing immediately
- Provide detailed error context
- Suggest recovery actions
**Why not implemented:**
- Current error handling is clear and actionable
- Warning accumulation can hide critical issues
- Simple approach is easier to debug
### 10. Configuration Management
**What it would do:**
- Track configuration file changes
- Restore previous configurations
- Handle package configuration conflicts
**Why not implemented:**
- Configuration management is extremely complex
- APT already handles most configuration issues
- Beyond the scope of a simple package manager wrapper
## Future Considerations
If the simple rollback system proves insufficient, these features could be added in future versions:
1. **State persistence** would be the first addition for production use
2. **Enhanced error handling** would improve user experience
3. **Dry-run mode** would be useful for automation
4. **Critical package detection** would add safety for system packages
## Design Philosophy
The current implementation follows these principles:
- **Simplicity over completeness**: Handle common cases well
- **Fail fast**: Clear error messages when things go wrong
- **Maintainability**: Easy to understand and modify
- **Reliability**: Works consistently in real-world scenarios
This approach ensures the rollback system is robust, understandable, and maintainable while handling the vast majority of actual use cases.

89
docs/rollbacks.md Normal file
View file

@ -0,0 +1,89 @@
# Rollback Implementation
This document explains how the rollback functionality works in the APT wrapper.
## Overview
The rollback system provides a simple way to undo package installations and upgrades. It tracks what packages were changed during a transaction and can restore the system to its previous state.
## How It Works
### Package Tracking
The `AptTransaction` struct tracks two types of package changes:
- **`newly_installed`**: Packages that weren't installed before the transaction
- **`upgraded`**: Packages that were upgraded, with their previous versions stored
### Two-Phase Rollback
When `rollback()` is called, it performs two phases:
1. **Phase 1 - Remove New Packages**: Uses `apt remove -y` to remove all packages in `newly_installed`
2. **Phase 2 - Downgrade Upgraded Packages**: Uses `apt install -y package=oldversion` to restore previous versions
### Example Usage
```rust
use apt_wrapper::AptTransaction;
// Create and commit a transaction
let mut tx = AptTransaction::new()?;
tx.add_package("vim")?;
tx.add_package("curl")?;
tx.commit()?;
// Later, rollback the changes
tx.rollback()?;
```
### Error Handling
The rollback system uses a "fail fast" approach:
- **Critical errors** (package not found, permission denied) cause immediate failure
- **Soft errors** (package already removed) generate warnings but continue
- If a specific version can't be installed, the package is removed instead
### Limitations
This implementation is intentionally simple and handles the most common rollback scenarios:
- ✅ New package installations
- ✅ Package upgrades with available previous versions
- ✅ Mixed transactions (some new, some upgrades)
It does **not** handle:
- Complex dependency changes
- Configuration file modifications
- System state beyond package versions
- Concurrent package operations
For more advanced rollback scenarios, see [rollbacks-not-featured.md](rollbacks-not-featured.md).
## Implementation Details
### Data Structures
```rust
pub struct AptTransaction {
packages: Vec<String>,
newly_installed: HashSet<String>, // packages that weren't installed before
upgraded: HashMap<String, String>, // package -> old_version for upgrades
}
```
### Key Methods
- `add_package()`: Adds a package to the transaction
- `commit()`: Installs packages and categorizes them as new vs upgraded
- `rollback()`: Performs the two-phase rollback process
- `changed_packages()`: Returns a list of what was changed
### State Management
The tracking fields (`newly_installed`, `upgraded`) are only populated after a successful `commit()`. This ensures the rollback data reflects what actually happened during the transaction.
## Testing
The rollback functionality is tested in the test suite, but actual package installation/removal is not performed in tests to avoid side effects on the test environment.