particle-os/roadmap.md

This is an exciting and ambitious project! Based on your goals and chosen tools, here is a detailed roadmap to guide you through building a Debian Atomic Desktop, mirroring the success of ublue-os while using the strengths of the Debian ecosystem.

The roadmap is broken down into four distinct phases, from the foundational build to a polished, distributable product.

---

Phase 1: Foundation & Core Build (The "Hello, World" Image) 🔄 IN PROGRESS

Goal: Create a minimal, bootable Debian OSTree image and automate its build. This is your Minimum Viable Product.

Tools: bootc, just, podman/docker

⚠️ CRITICAL REQUIREMENT: Successful deployment using bootc install to-disk requires specific disk utilities that are fundamental to the partitioning process. This requirement affects all phases and must be addressed in deployment environments.

Essential Disk Utilities for bootc Deployment:

• sfdisk (from util-linux) - CRITICAL for automated partitioning
• parted - Alternative partitioning tool
• mkfs.ext4 (from e2fsprogs) - Filesystem creation
• mkfs.fat (from dosfstools) - FAT32 filesystem for EFI
• grub-install - Bootloader installation
• efibootmgr - UEFI boot manager

Current Status: ✅ Completed:

• Project scaffolding and Git repository setup
• Containerfile with essential packages including disk utilities
• Automated build with justfile
• Fixed critical PATH issues for sfdisk in Debian environments
• Resolved UTF-8 encoding issues with locale configuration
• Added proper OSTree labels (ostree.bootable=true)
• Installed Linux kernel and created kernel module symlinks
• Set up /usr/lib/ostree-boot directory with kernel files
• Successfully tested bootc install to-disk partitioning and filesystem creation

🔄 In Progress:

• Kernel Detection Issue: Failed to find kernel in /usr/lib/modules, /usr/lib/ostree-boot or /boot
• Investigating bootc's kernel detection logic
• Testing different kernel file locations and symlinks
• Verifying kernel module paths and dependencies

Tasks:

1. Project Scaffolding: ✅ COMPLETE

   • ✅ Create a new Git repository for your project (e.g., my-debian-atomic-desktop).
   • ✅ Create the foundational files: Containerfile and justfile.

2. Define the Base Image (Containerfile): ✅ COMPLETE

   • ✅ Start with a minimal Debian image.
   • ✅ Example Containerfile snippet:

     FROM debian:trixie
     
     # Install essential packages
     RUN apt-get update && apt-get install -y \
         systemd \
         dbus \
         sudo \
         util-linux \  # CRITICAL: Provides sfdisk
         parted \
         e2fsprogs \   # CRITICAL: Provides mkfs.ext4
         dosfstools \  # CRITICAL: Provides mkfs.fat
         grub-efi-amd64 \
         efibootmgr \
         linux-image-amd64 \
         linux-headers-amd64 \
         locales \
         ...
     

   • ✅ Focus on only the bare minimum for now. Don't add a desktop yet. The goal is to get a working, bootable command line.
   • ✅ Critical: Explicitly create the /home -> /var/home symbolic link with RUN ln -sf ../var/home /home.
   • ✅ Critical: Add OSTree labels and kernel setup.

3. Automated Build (justfile): ✅ COMPLETE

   • ✅ Create a simple justfile with a recipe to build the container image.
   • ✅ Example justfile snippet:

     build-image:
         podman build -t my-debian-atomic:latest .
     
     # Command to clean up
     clean:
         podman rmi my-debian-atomic:latest
     

4. Initial Boot Test: 🔄 IN PROGRESS

   • ✅ Create a just recipe (just test-base-image-vm) to deploy and boot this minimal image in a VM using bootc.
   • ✅ Critical Prerequisites: Ensure the deployment environment has all required disk utilities:

     # Verify disk utilities are available
     which sfdisk parted mkfs.ext4 mkfs.fat grub-install efibootmgr
     sfdisk --version
     

   • ✅ Example just recipe for testing:

     install-vm:
         # Verify prerequisites first
         @echo "Verifying disk utilities..."
         which sfdisk parted mkfs.ext4 mkfs.fat grub-install efibootmgr
         sudo env PATH="/usr/sbin:/sbin:/usr/local/bin:/usr/bin:/bin" \
           podman run --rm --privileged --pid=host --volume /dev:/dev \
           localhost/debian-atomic:latest \
           /usr/bin/bootc install to-disk /dev/loop0 --filesystem ext4
     

   • 🔄 Current Issue: Kernel detection in final deployment step
   • Next Steps: Resolve kernel detection issue to complete Phase 1

Deliverable: A minimal, bootable Debian bootc image and a justfile to build and test it.

Key Challenge: This phase validates that apt-ostree and bootc work reliably with Debian, establishing the foundation for all subsequent work. The critical requirement for disk utilities (sfdisk, etc.) has been successfully addressed.

---

Phase 2: Calamares Installer Integration

Goal: Create a bootable ISO with a Calamares installer that can deploy your atomic image.

Tools: live-build, calamares

⚠️ CRITICAL: The live environment must include all disk utilities required for bootc deployment.

Current Status: ✅ Completed:

• Terminal installer approach (02-installer-bootc-tui/) - Fully functional
• Basic bootc approach (02-installer-bootc/) - Core functionality working
• Traditional live-build approach (02-installer/) - Available but complex

Tasks:

1. Build a Live ISO Environment:

   • Use live-build to create a minimal live environment.
   • Configure live-build to include the calamares package and all its dependencies.
   • Critical: Ensure the live environment includes all required disk utilities:

     # In live-build configuration, include these packages:
     util-linux parted e2fsprogs dosfstools grub-efi-amd64 efibootmgr
     

   • The live environment will also need access to your bootc image, either by embedding it in the ISO or pointing to a container registry.

2. Configure Calamares:

   • Create a custom Calamares configuration (a set of .yml files).
   • The Partitioning Module: Configure it to create the necessary partitions (e.g., /boot/efi, /, and a separate /boot for bootc).
   • The post-install Module (Crucial Step): Write a script or configure this module to:
     • Verify disk utilities: Ensure sfdisk, mkfs.ext4, etc. are available
     • Run the command bootc install to-disk --device /dev/sda --replace-os --image ghcr.io/your-project/your-image:latest.
     • Handle the bootloader installation, which bootc can assist with.

3. Integrate the Installer Build with just:

   • Add a new recipe to your justfile to orchestrate the live-build process.
   • Example justfile recipe:

     build-iso:
         ./build_live_iso.sh
         # The script would use live-build to create the .iso
     
     test-iso:
         qemu-system-x86_64 -cdrom my-debian-installer.iso -m 2G
     

Deliverable: A bootable .iso that presents a Calamares installer, which successfully installs your minimal atomic image.

Key Challenge: The live environment must include all disk utilities required for bootc deployment, and the Calamares configuration must properly handle the immutable system deployment process.

---

Phase 3: Advanced Features (The ublue-os Mimicry)

Goal: Add a full desktop environment and a robust solution for building kernel modules like the NVIDIA driver.

Tools: Multi-stage Containerfile builds, podman/docker

⚠️ CRITICAL: All deployment scenarios must maintain the disk utility requirements established in Phase 1.

Tasks:

1. Add a Desktop Environment:

   • Update your Containerfile from Phase 1 to include a full desktop environment. For example, for KDE Plasma:

     # Inside the Containerfile
     RUN apt-get install -y sddm task-kde-desktop
     

   • Critical: Maintain all disk utilities from Phase 1 in the desktop image.

2. Create the Kernel Module Pipeline:

   • Separate Repository: Create a new repository, for example, my-debian-atomic-kmods.
   • Build Containerfile: In this new repo, create a Containerfile to build the NVIDIA driver from source for a specific Debian kernel version.

     # Inside the kmods Containerfile
     FROM debian:trixie
     RUN apt-get update && apt-get install -y build-essential linux-headers-$(uname -r) ...
     RUN cd /path/to/nvidia-source && make KSRC=/usr/src/linux-headers-$(uname -r)
     # Copy the compiled .ko file to a known location
     

   • Build Automation (justfile): Add a just recipe to build and push this new kmods container image to a registry.

3. Integrate the Pre-built Module:

   • Go back to your main Containerfile from Phase 1.
   • Use a multi-stage build. The first stage pulls from your kmods image. The second stage copies the pre-compiled .ko file into the main image's /lib/modules/ directory.
   • Example multi-stage Containerfile snippet:

     # Stage 1: Build or get the kernel module
     FROM ghcr.io/your-project/my-debian-atomic-kmods:latest AS kmods-builder
     
     # Stage 2: Build the final image
     FROM debian:trixie
     # ... (rest of your desktop setup) ...
     # CRITICAL: Maintain all disk utilities from Phase 1
     RUN apt-get install -y util-linux parted e2fsprogs dosfstools grub-efi-amd64 efibootmgr
     
     # Copy the pre-compiled kernel module
     COPY --from=kmods-builder /path/to/nvidia.ko /lib/modules/$(uname -r)/updates/nvidia.ko
     RUN depmod -a $(uname -r)
     

   • This mimics the ublue-os approach: the complex build is isolated and the final product simply integrates the finished artifacts.

Deliverable: A fully-featured desktop image with an integrated, pre-compiled NVIDIA driver, built using a clean, automated pipeline.

Key Challenge: Managing the complex build dependencies and timing between kernel updates, driver compilation, and system image composition while maintaining system stability and disk utility requirements.

---

Phase 4: Polish & Distribution

Goal: Make the project ready for others to use and contribute to.

Tools: GitHub Actions, Git

⚠️ CRITICAL: All deployment and testing scenarios must validate disk utility availability.

Tasks:

1. Public Repositories: Ensure your my-debian-atomic-desktop and my-debian-atomic-kmods repositories are public on a platform like GitHub.

2. Set up CI/CD (GitHub Actions):

   • Create workflows in both repositories to automatically build and push new container images whenever you push code.
   • Trigger an automatic build of the kmods repository whenever a new Debian kernel is released.
   • Trigger an automatic build of the main desktop image after the kmods image has been successfully built and pushed.
   • Critical: Include validation steps to ensure all disk utilities are present in built images:

     # In GitHub Actions workflow
     - name: Verify disk utilities
       run: |
         podman run --rm ${{ steps.image.outputs.image }} which sfdisk parted mkfs.ext4 mkfs.fat grub-install efibootmgr
     

3. Write Comprehensive Documentation:

   • Create a README.md that explains the project's goals.
   • Write a guide for users on how to install your desktop using the Calamares ISO.
   • Document the build process for contributors.
   • Explain any custom ujust commands you include.
   • Critical: Document disk utility requirements and provide troubleshooting guides for common deployment failures.

4. Finalize the User Experience:

   • Add custom desktop branding, wallpapers, and default application choices.
   • Add a ujustfile inside your main Containerfile to provide a user-friendly command line interface for updates and system maintenance.

Deliverable: A stable, automated, and well-documented project with a polished user experience, ready for public consumption.

Key Challenge: Establishing reliable continuous delivery for immutable systems requires sophisticated automation and testing to handle complex update dependencies while ensuring all deployment requirements are met.

---

Critical Success Factors

Technical Requirements:

• All deployment environments must include complete disk utility sets
• Atomic images must contain all necessary partitioning and filesystem creation tools
• Testing must validate that deployment environments have required utilities available
• Documentation must clearly communicate disk utility requirements to users and contributors

Common Failure Points:

• error: Installing to disk: Creating rootfs: Failed to run sfdisk: No such file or directory
• Missing filesystem creation tools in deployment environments
• Incomplete bootloader installation utilities
• Live environments lacking essential disk utilities

Implementation Solutions:

• Include util-linux, e2fsprogs, dosfstools, grub-efi-amd64, efibootmgr in all atomic images
• Verify disk utility availability in all deployment scenarios
• Provide clear documentation and troubleshooting guides for disk utility requirements
• Implement automated validation of disk utility availability in CI/CD pipelines

Current Progress: ✅ Phase 1 Foundation: Disk utility requirements successfully addressed 🔄 Phase 1 Final Step: Resolving kernel detection issue to complete minimal bootable image 📋 Phase 2 Preparation: Terminal installer approach completed as alternative to Calamares

This roadmap acknowledges that building an immutable desktop system involves not just understanding the architecture, but successfully implementing complex toolchain integration and handling the practical challenges that theory alone cannot address. The critical requirement for disk utilities represents a fundamental implementation challenge that has been successfully addressed in Phase 1.