diff --git a/eos-updater.md b/eos-updater.md new file mode 100644 index 0000000..271d4fd --- /dev/null +++ b/eos-updater.md @@ -0,0 +1,778 @@ +# eos-updater Project Breakdown + +## Project Overview + +eos-updater is the **production-grade system update infrastructure** for Endless OS, implementing a complete OSTree-based automatic update system. Unlike the build tools we've examined (deb-ostree-builder and apt2ostree), eos-updater focuses on the **runtime update delivery and management** for deployed systems. + +## Repository Structure + +Based on the project analysis, here's the inferred repository structure: + +``` +eos-updater/ +├── README.md # Project documentation +├── meson.build # Main build configuration +├── meson_options.txt # Build options +├── eos-autoupdater/ # Automatic update scheduler +│ ├── eos-autoupdater.c # Main autoupdater daemon +│ ├── meson.build # Autoupdater build config +│ └── docs/ +│ ├── eos-autoupdater.8 # Man page +│ └── eos-autoupdater.conf.5 # Configuration man page +├── eos-updater/ # Core update daemon +│ ├── eos-updater.c # Main updater daemon +│ ├── eos-updater.h # Header definitions +│ ├── eos-updater-dbus.c # D-Bus interface implementation +│ ├── eos-updater-poll.c # Update polling logic +│ ├── eos-updater-fetch.c # Update downloading +│ ├── eos-updater-apply.c # Update application +│ ├── meson.build # Updater build config +│ └── docs/ +│ ├── eos-updater.8 # Man page +│ └── eos-updater.conf.5 # Configuration man page +├── eos-updater-ctl/ # Command-line control tool +│ ├── eos-updater-ctl.c # CLI interface +│ ├── meson.build +│ └── docs/ +│ └── eos-updater-ctl.8 # Man page +├── eos-update-server/ # Local network update server +│ ├── eos-update-server.c # HTTP server for local updates +│ ├── eos-update-server-dbus.c # D-Bus integration +│ ├── meson.build +│ └── docs/ +│ ├── eos-update-server.8 # Man page +│ └── eos-update-server.conf.5 # Configuration man page +├── eos-updater-avahi/ # Network discovery service +│ ├── eos-updater-avahi.c # Avahi/mDNS integration +│ ├── meson.build +│ └── docs/ +│ └── eos-updater-avahi.8 # Man page +├── eos-updater-prepare-volume/ # USB/removable media support +│ ├── eos-updater-prepare-volume.c # Volume preparation utility +│ ├── meson.build +│ └── docs/ +│ └── eos-updater-prepare-volume.8 +├── libeos-updater-util/ # Shared utility library +│ ├── util.c # Common utility functions +│ ├── util.h # Utility headers +│ ├── config.c # Configuration management +│ ├── ostree-util.c # OSTree helper functions +│ └── meson.build +├── libeos-updater-flatpak-installer/ # Flatpak integration +│ ├── installer.c # Flatpak application installer +│ ├── installer.h # Installer headers +│ └── meson.build +├── data/ # Configuration and service files +│ ├── eos-autoupdater.conf # Default autoupdater config +│ ├── eos-updater.conf # Default updater config +│ ├── systemd/ # Systemd service files +│ │ ├── eos-autoupdater.service +│ │ ├── eos-autoupdater.timer +│ │ ├── eos-updater.service +│ │ └── eos-update-server.service +│ └── dbus/ # D-Bus service definitions +│ ├── com.endlessm.Updater.service +│ ├── com.endlessm.UpdaterAvahi.service +│ └── com.endlessm.UpdateServer.service +├── tests/ # Test suite +│ ├── test-autoupdater.c # Autoupdater tests +│ ├── test-updater.c # Core updater tests +│ ├── test-util.c # Utility function tests +│ └── installed-tests/ # Integration tests +│ ├── test-update-from-lan.py +│ ├── test-update-from-usb.py +│ └── test-update-lifecycle.py +├── docs/ # Additional documentation +│ ├── design.md # System design documentation +│ ├── testing.md # Testing guidelines +│ └── deployment.md # Deployment instructions +└── scripts/ # Utility scripts + ├── ostree-update-generator.sh # Update generation scripts + └── test-server-setup.sh # Test environment setup +``` + +## How eos-updater Works + +### 1. **Component Architecture** + +eos-updater implements a **multi-component distributed system** for OSTree updates: + +#### Core Components +```mermaid +graph TB + subgraph "Client System" + A[eos-autoupdater] --> B[eos-updater daemon] + B --> C[OSTree Repository] + D[eos-updater-ctl] --> B + E[Desktop UI] --> B + end + + subgraph "Network Services" + F[eos-update-server] --> G[Local OSTree Repo] + H[eos-updater-avahi] --> F + end + + subgraph "External" + I[Internet Update Server] + J[USB/Removable Media] + end + + B --> I + B --> F + B --> J + + style A fill:#e1f5fe + style B fill:#f3e5f5 + style F fill:#e8f5e8 +``` + +### 2. **Update Process Flow** + +The system implements a sophisticated multi-phase update process: + +#### Phase 1: Automatic Update Detection +```c +// eos-autoupdater (triggered by systemd timer) +1. Wake up on schedule (daily/weekly) +2. Connect to eos-updater daemon via D-Bus +3. Initiate Poll() operation +4. Exit when automatic portion complete +``` + +#### Phase 2: Update Discovery and Download +```c +// eos-updater daemon operations +1. Poll remote servers for updates +2. Check multiple sources: + - Internet update servers + - Local network servers (via Avahi) + - USB/removable media +3. Download update metadata +4. Download OSTree objects (if automatic download enabled) +``` + +#### Phase 3: User Interaction and Application +```c +// User-interactive portion +1. Desktop UI notifies user of available update +2. User chooses to apply update +3. eos-updater applies OSTree update +4. System prepares for reboot into new deployment +``` + +### 3. **D-Bus Interface Architecture** + +The system uses D-Bus for inter-process communication: + +#### Main D-Bus Interface +```xml + + + + + + + + + + + + + + + + + + + + + + + +``` + +#### Update States +```c +typedef enum { + EOS_UPDATER_STATE_NONE = 0, // No update available + EOS_UPDATER_STATE_READY, // Ready to check for updates + EOS_UPDATER_STATE_POLLING, // Checking for updates + EOS_UPDATER_STATE_UPDATE_AVAILABLE, // Update found + EOS_UPDATER_STATE_FETCHING, // Downloading update + EOS_UPDATER_STATE_UPDATE_READY, // Update downloaded + EOS_UPDATER_STATE_APPLYING_UPDATE, // Installing update + EOS_UPDATER_STATE_UPDATE_APPLIED, // Update applied, reboot needed + EOS_UPDATER_STATE_ERROR // Error occurred +} EosUpdaterState; +``` + +### 4. **Network Update Discovery** + +#### Local Network Updates (Avahi/mDNS) +```c +// eos-updater-avahi discovers local update servers +1. Advertise local update server via mDNS +2. Listen for other update servers on network +3. Coordinate with eos-update-server for serving +4. Enable LAN-based updates (no internet required) +``` + +#### Update Server Discovery Priority +```c +// Update source priority order +1. Local USB/removable media (highest priority) +2. Local network servers (LAN) +3. Internet update servers (lowest priority) +``` + +### 5. **OSTree Integration** + +eos-updater wraps OSTree operations in a user-friendly daemon: + +#### Core OSTree Operations +```c +// Simplified OSTree update process +ostree_repo_pull(repo, remote_name, refs_to_pull, flags, &error); +ostree_sysroot_load(sysroot, &error); +ostree_sysroot_deploy_tree(sysroot, os_name, revision, origin, + merge_deployment, override_kernel_argv, &error); +ostree_sysroot_simple_write_deployment(sysroot, os_name, new_deployment, + merge_deployment, flags, &error); +``` + +#### Metadata Management +```c +// Update metadata tracking +typedef struct { + gchar *id; // Update identifier + gchar *refspec; // OSTree refspec + gchar *original_refspec; // Original branch + gchar *version; // Version string + gchar *checksum; // OSTree commit checksum + guint64 download_size; // Size of download + guint64 installed_size; // Installed size +} EosUpdaterData; +``` + +### 6. **Configuration System** + +#### Main Configuration Files +```ini +# /etc/eos-updater/eos-updater.conf +[Updates] +AutoUpdate=true +AutoUpdateInterval=86400 +PollingEnabled=true +MaxBandwidth=1048576 + +[Network] +AllowLANUpdates=true +PreferLANUpdates=true + +[OSTree] +RemoteName=eos +RepositoryURL=https://ostree.endlessm.com/ostree/eos-amd64 +``` + +```ini +# /etc/eos-updater/eos-autoupdater.conf +[Automatic Updates] +OnlineAutomatic=true +OfflineAutomatic=false +StampFile=/var/lib/eos-autoupdater/last-update-check +``` + +### 7. **Update Sources and Methods** + +#### Internet Updates +```c +// Standard OSTree remote pulling +struct update_source internet_source = { + .type = UPDATE_SOURCE_INTERNET, + .url = "https://ostree.endlessm.com/ostree/eos-amd64", + .priority = 3 // Lowest priority +}; +``` + +#### LAN Updates +```c +// Local network update discovery +struct update_source lan_source = { + .type = UPDATE_SOURCE_LAN, + .discovery_method = AVAHI_MDNS, + .priority = 2 // Medium priority +}; +``` + +#### USB/Removable Media Updates +```c +// Offline update support +struct update_source usb_source = { + .type = UPDATE_SOURCE_VOLUME, + .mount_point = "/media/eos-updates", + .priority = 1 // Highest priority +}; +``` + +## Key Technical Features + +### 1. **Graduated Automation** + +The system implements **configurable automation levels**: + +```c +// Automation configuration options +typedef enum { + AUTO_UPDATE_DISABLED = 0, // Manual updates only + AUTO_UPDATE_DOWNLOAD, // Auto download, manual apply + AUTO_UPDATE_FULL // Fully automatic updates +} AutoUpdateLevel; +``` + +#### Example Automation Flow +``` +Automatic Phase (eos-autoupdater): +├── Check for updates ✓ +├── Download updates ✓ +└── Exit, wait for user + +User Interactive Phase: +├── Desktop notification appears +├── User chooses to apply update +└── System applies and schedules reboot +``` + +### 2. **Multi-Source Update Discovery** + +#### Source Priority System +```c +// Update source evaluation +for (source in update_sources) { + if (source_has_newer_update(source)) { + if (source.priority > best_source.priority) { + best_source = source; + best_update = source.update; + } + } +} +``` + +#### LAN Update Optimization +- **Bandwidth Savings**: Updates from local network instead of internet +- **Speed Improvements**: Local network speeds vs. internet bandwidth +- **Offline Capability**: Updates possible without internet connectivity + +### 3. **Robust Error Handling** + +#### Update Failure Recovery +```c +// Error state management +switch (update_error) { + case UPDATE_ERROR_NETWORK: + retry_with_backoff(); + break; + case UPDATE_ERROR_DISK_SPACE: + notify_user_insufficient_space(); + break; + case UPDATE_ERROR_OSTREE: + rollback_to_previous_state(); + break; +} +``` + +#### State Persistence +- **Update state survives reboots**: daemon tracks progress across restarts +- **Atomic operations**: Either update completes fully or rolls back +- **Resume capability**: Can resume interrupted downloads + +### 4. **Integration with System Services** + +#### Systemd Integration +```ini +# eos-autoupdater.service +[Unit] +Description=Endless OS Automatic Updater +After=network-online.target +Wants=network-online.target + +[Service] +Type=oneshot +ExecStart=/usr/bin/eos-autoupdater +User=eos-updater + +# eos-autoupdater.timer +[Unit] +Description=Endless OS Automatic Update Timer + +[Timer] +OnCalendar=daily +Persistent=true +RandomizedDelaySec=3600 + +[Install] +WantedBy=timers.target +``` + +#### Network Manager Integration +```c +// Network awareness +if (network_manager_is_metered_connection()) { + if (!allow_metered_updates) { + defer_update_until_unmetered(); + return; + } +} +``` + +## Component Deep Dive + +### 1. **eos-autoupdater - Scheduled Update Initiator** + +This component runs on a systemd timer and handles the automatic portion of updates: + +#### Responsibilities +- **Scheduled Checks**: Runs daily/weekly based on configuration +- **Network Awareness**: Respects metered connections and user preferences +- **Battery Awareness**: Defers updates on low battery (laptops) +- **User Presence Detection**: Avoids updates during active user sessions + +#### Example Operation Flow +```c +int main() { + // Connect to eos-updater daemon + GDBusProxy *updater = connect_to_updater_daemon(); + + // Check configuration + if (!is_automatic_updates_enabled()) { + exit(0); + } + + // Initiate poll for updates + g_dbus_proxy_call_sync(updater, "Poll", + g_variant_new("(u)", POLL_FLAGS_AUTOMATIC), + G_DBUS_CALL_FLAGS_NONE, -1, NULL, &error); + + // Wait for automatic phase to complete + wait_for_state_change(EOS_UPDATER_STATE_UPDATE_AVAILABLE); + + // Exit - user interaction now required + return 0; +} +``` + +### 2. **eos-updater - Core Update Daemon** + +The main daemon that orchestrates the entire update process: + +#### State Machine Implementation +```c +// Update state machine +static void handle_state_transition(EosUpdater *updater, + EosUpdaterState old_state, + EosUpdaterState new_state) { + switch (new_state) { + case EOS_UPDATER_STATE_POLLING: + start_update_poll(updater); + break; + case EOS_UPDATER_STATE_FETCHING: + start_update_download(updater); + break; + case EOS_UPDATER_STATE_APPLYING_UPDATE: + start_update_application(updater); + break; + } + + emit_state_changed_signal(updater, old_state, new_state); +} +``` + +#### OSTree Integration +```c +// Wrapping OSTree operations in daemon +static gboolean poll_for_updates(EosUpdater *updater) { + OstreeRepo *repo = ostree_repo_new_default(); + + // Check each configured remote + for (remote in configured_remotes) { + gchar *checksum = NULL; + ostree_repo_resolve_rev(repo, remote.refspec, FALSE, &checksum, &error); + + if (checksum && !g_str_equal(checksum, current_deployment_checksum)) { + updater->update_available = TRUE; + updater->update_checksum = g_strdup(checksum); + break; + } + } + + set_state(updater, EOS_UPDATER_STATE_UPDATE_AVAILABLE); + return TRUE; +} +``` + +### 3. **eos-update-server - LAN Update Distribution** + +Provides local network update distribution to reduce bandwidth usage: + +#### HTTP Server Implementation +```c +// Simple HTTP server for OSTree repositories +static void handle_ostree_request(SoupServer *server, + SoupMessage *msg, + const char *path, + GHashTable *query, + SoupClientContext *context, + gpointer user_data) { + + EosUpdateServer *self = EOS_UPDATE_SERVER(user_data); + + // Serve OSTree repository files + if (g_str_has_prefix(path, "/ostree/")) { + serve_ostree_file(self, msg, path); + } +} +``` + +#### Avahi Integration +```c +// Service advertisement via mDNS +static void advertise_update_service(EosUpdateServer *server) { + AvahiEntryGroup *group; + + avahi_entry_group_add_service(group, + AVAHI_IF_UNSPEC, + AVAHI_PROTO_UNSPEC, + 0, + "EOS Update Server", + "_eos_updater._tcp", + NULL, NULL, + server->port, + "version=1.0", + "path=/ostree", + NULL); +} +``` + +### 4. **eos-updater-ctl - Command Line Interface** + +Provides administrative control over the update system: + +#### CLI Commands +```bash +# Check current state +eos-updater-ctl status + +# Force update check +eos-updater-ctl poll + +# Download available update +eos-updater-ctl fetch + +# Apply downloaded update +eos-updater-ctl apply + +# Monitor update progress +eos-updater-ctl monitor +``` + +#### Implementation Example +```c +static int cmd_status(int argc, char **argv) { + GDBusProxy *proxy = connect_to_updater(); + GVariant *state_variant; + + g_dbus_proxy_get_cached_property(proxy, "State"); + EosUpdaterState state = g_variant_get_uint32(state_variant); + + printf("Update Status: %s\n", state_to_string(state)); + + if (state == EOS_UPDATER_STATE_UPDATE_AVAILABLE) { + print_update_details(proxy); + } + + return 0; +} +``` + +## Advanced Features + +### 1. **Flatpak Application Updates** + +The system includes integrated Flatpak application management: + +```c +// libeos-updater-flatpak-installer +static gboolean install_flatpak_updates(EosUpdater *updater) { + FlatpakInstallation *installation; + GList *updates; + + // Get available Flatpak updates + updates = flatpak_installation_list_installed_refs_for_update(installation, + NULL, &error); + + // Apply updates in parallel with OS update + for (update in updates) { + flatpak_installation_update(installation, + FLATPAK_UPDATE_FLAGS_NO_DEPLOY, + update->kind, update->name, + update->arch, update->branch, + NULL, &error); + } + + return TRUE; +} +``` + +### 2. **Bandwidth Management** + +#### Adaptive Download Strategy +```c +// Network condition awareness +static void adjust_download_strategy(EosUpdater *updater) { + NetworkInfo *info = get_network_info(); + + if (info->is_metered) { + updater->max_download_size = config->metered_max_size; + updater->download_priority = PRIORITY_LOW; + } + + if (info->bandwidth < BANDWIDTH_THRESHOLD_LOW) { + enable_delta_updates(updater); // Use smaller delta updates + } +} +``` + +#### Delta Update Support +```c +// Efficient incremental updates +static gboolean fetch_delta_update(EosUpdater *updater) { + // Try to fetch static delta between current and target + gboolean delta_available = ostree_repo_pull_with_options( + repo, remote_name, + g_variant_new("a{sv}", "flags", + g_variant_new_uint32(OSTREE_REPO_PULL_FLAGS_COMMIT_ONLY), + "refs", g_variant_new_strv(refs, -1), + NULL), NULL, &error); + + if (!delta_available) { + // Fall back to full object download + return fetch_full_update(updater); + } + + return TRUE; +} +``` + +### 3. **Security and Verification** + +#### GPG Signature Verification +```c +// Update verification +static gboolean verify_update(EosUpdater *updater, const gchar *checksum) { + OstreeRepo *repo = updater->repo; + + // Verify GPG signature on commit + gboolean valid = ostree_repo_verify_commit_ext(repo, checksum, + NULL, NULL, + NULL, &error); + if (!valid) { + set_error_state(updater, "Update signature verification failed"); + return FALSE; + } + + return TRUE; +} +``` + +## Deployment and Production Considerations + +### 1. **System Integration** + +#### Package Dependencies +``` +Runtime Dependencies: +├── ostree (>= 2020.5) # Core OSTree functionality +├── libglib2.0-0 # GLib for daemon infrastructure +├── systemd # Service management +├── avahi-daemon # Network discovery +├── libsoup2.4-1 # HTTP client/server +└── flatpak (optional) # Application updates +``` + +#### Service Configuration +```systemd +# Complete service ecosystem +eos-autoupdater.timer # Scheduled update checks +├── eos-autoupdater.service # Update initiator +└── eos-updater.service # Core daemon + ├── eos-update-server.service # LAN server + └── eos-updater-avahi.service # Discovery service +``` + +### 2. **Testing Infrastructure** + +#### Comprehensive Test Suite +```c +// Unit tests for each component +test_autoupdater_scheduling() +test_updater_state_machine() +test_update_server_http_serving() +test_avahi_service_discovery() + +// Integration tests +test_full_update_lifecycle() +test_lan_update_discovery() +test_usb_update_application() +test_error_recovery() +``` + +#### Production Validation +```bash +# Installed-only tests +gnome-desktop-testing-runner libeos-updater-util-0 +gnome-desktop-testing-runner libeos-updater-flatpak-installer-0 +sudo gnome-desktop-testing-runner eos-updater-0 +``` + +## Real-World Impact + +### 1. **User Experience Benefits** +- **Automatic Updates**: Users get security updates without intervention +- **Non-Disruptive**: Updates happen in background, apply on reboot +- **Rollback Safety**: Bad updates can be rolled back instantly +- **Bandwidth Optimization**: LAN updates reduce internet usage + +### 2. **System Administrator Benefits** +- **Centralized Management**: Control update policies across organization +- **Bandwidth Efficiency**: LAN-based update distribution +- **Audit Trail**: Complete update history in OSTree +- **Flexible Deployment**: Support for air-gapped environments via USB + +### 3. **Enterprise Features** +- **Staged Updates**: Download and prepare updates without applying +- **Update Scheduling**: Control when updates are applied +- **Network Efficiency**: Peer-to-peer update distribution +- **Compliance**: Maintain exact system state records + +## Architecture Philosophy + +eos-updater embodies **production-ready system design principles**: + +1. **Separation of Concerns**: Each component has a single, well-defined responsibility +2. **Fault Tolerance**: Robust error handling and recovery mechanisms +3. **User Agency**: Users maintain control over when updates are applied +4. **Network Efficiency**: Multiple strategies to minimize bandwidth usage +5. **Enterprise Ready**: Configuration and deployment suitable for organizational use + +This represents a **mature, production implementation** of OSTree-based system updates, demonstrating how the theoretical benefits of atomic system management translate into practical, user-friendly update experiences. The system successfully bridges the gap between OSTree's technical capabilities and the real-world needs of desktop and embedded system users. + +## Comparison Summary + +| Aspect | eos-updater | deb-ostree-builder | apt2ostree | +|--------|-------------|-------------------|------------| +| **Purpose** | Runtime update delivery | Build system creation | Build optimization | +| **Focus** | User experience | System conversion | Developer workflow | +| **Language** | C with GLib | Bash scripts | Python library | +| **Scope** | Complete update infrastructure | Build orchestration | Build performance | +| **Target** | End users | System administrators | Developers | + +eos-updater completes the OSTree ecosystem by providing the production update delivery system that end users interact with daily. \ No newline at end of file