deb-bootc-image-builder/docs/todo

# TODO - particle-os: Debian-Native OS Image Builder

## 🎯 **Current Status: REPOSITORY CLEANUP & STRUCTURE ALIGNMENT**

**Date**: August 17, 2025  
**Phase**: Phase 5 - Repository Cleanup & Structure Alignment  
**Status**: 🧹 **REPOSITORY CLEANUP - Aligning Structure with Original osbuild/bootc-image-builder**

**Summary**: We have a working prototype that demonstrates container-to-bootable-image conversion. Now we need to clean up the repository structure to match the original [osbuild/bootc-image-builder](https://github.com/osbuild/bootc-image-builder) repository layout for better maintainability and contributor experience.

**Project Scope**: This project is building a **Debian-native equivalent to bootc-image-builder** that can process **particle-os containers** (which use OSTree + bootc + bootupd) and convert them to bootable disk images, similar to how ublue-os images get processed by bootc-image-builder.

---

## ✅ **COMPLETED MILESTONES**

### **Phase 1: Analysis & Architecture** ✅ COMPLETE
- [x] **Analyze bootc-image-builder + osbuild relationship**
  - ✅ Deep dive into osbuild source code completed
  - ✅ Understanding of declarative stage system achieved
  - ✅ Knowledge of bootupd integration patterns gained
- [x] **Analyze debos for reusable components** (ABANDONED - hanging issues)
  - ✅ Deep dive into debos source code completed
  - ✅ Identified fundamental mismatch with container-first approach
  - ✅ Decision to abandon debos due to hanging and "build from scratch" philosophy
- [x] **Create integration roadmap**
  - ✅ Strategic plan for hybrid approach developed
  - ✅ Phases defined and progress tracked
- [x] **Design hybrid architecture**
  - ✅ Custom Go-based pipeline designed
  - ✅ Container extraction + manual image creation approach planned

### **Phase 2: Core Integration** ✅ COMPLETE
- [x] **Implement debos integration framework** (ABANDONED - hanging issues)
  - ✅ Initial debos integration attempted
  - ✅ Discovered hanging issues during package installation
  - ✅ Identified fundamental incompatibility with container-first workflow
  - ✅ Decision to abandon debos approach
- [x] **Create manifest generation system** (ABANDONED - debos approach)
  - ✅ Dynamic manifest generation implemented
  - ✅ OS detection and architecture detection working
  - ✅ Abandoned due to debos integration failure
- [x] **Build container processing pipeline** (ABANDONED - debos approach)
  - ✅ Container extraction pipeline implemented
  - ✅ Filesystem analysis and processing working
  - ✅ Abandoned due to debos integration failure
- [x] **Implement end-to-end testing framework** (ABANDONED - debos approach)
  - ✅ Comprehensive testing framework created
  - ✅ Validation of container extraction and processing
  - ✅ Abandoned due to debos integration failure

### **Phase 3: Strategic Pivot** ✅ **COMPLETED!**
- [x] **Purge all debos elements** from project ✅ **COMPLETED!**
  - ✅ All debos-related code removed
  - ✅ All debos-related documentation removed
  - ✅ All debos-related test files removed
  - ✅ Clean project structure achieved
- [x] **Analyze Universal Blue approach** using bootc-image-builder ✅ **COMPLETED!**
  - ✅ Deep dive into ublue-os ecosystem completed
  - ✅ Understanding of BlueBuild recipe system achieved
  - ✅ Knowledge of bootc + bootupd + OSTree integration gained
  - ✅ **Key insight**: We need to build a Debian-native equivalent to bootc-image-builder
- [x] **Identify proven container-to-bootable workflow** ✅ **COMPLETED!**
  - ✅ Container-first approach identified as proven pattern
  - ✅ Declarative recipe system identified as best practice
  - ✅ OSTree + bootupd integration identified as modern approach
  - ✅ **Corrected understanding**: We're building a tool to process particle-os containers (OSTree + bootc + bootupd) into bootable images

### **Phase 4: particle-os Implementation** ✅ **COMPLETED!**
- [x] **Create particle-os recipe system** ✅ **COMPLETED!**
  - ✅ YAML recipe parser implemented
  - ✅ Recipe validation working
  - ✅ Stage-based execution framework created
  - ✅ Recipe templates for common use cases created
- [x] **Implement real container extraction** ✅ **COMPLETED!**
  - ✅ Docker/Podman integration working
  - ✅ Container image pulling and inspection working
  - ✅ Filesystem extraction to target directory working
  - ✅ Container metadata analysis working
- [x] **Build real package management** ✅ **COMPLETED!**
  - ✅ apt package installation working
  - ✅ debootstrap system creation working
  - ✅ chroot environment setup working
  - ✅ Package cache management working
- [x] **Create real system configuration** ✅ **COMPLETED!**
  - ✅ **Locale stage**: Sudo fixes implemented for file operations
  - ✅ **Timezone stage**: Sudo fixes implemented for file operations
  - ✅ **Users stage**: Sudo fixes implemented for file operations
  - ✅ **Helper functions**: writeFileWithSudo, removeFileWithSudo, createSymlinkWithSudo
  - ✅ **Binary status**: All fixes implemented and working
- [x] **Basic image creation** ✅ **COMPLETED!**
  - ✅ Raw disk image creation working
  - ✅ GPT partitioning working
  - ✅ Basic bootloader installation working
  - ✅ Images recognized as bootable by QEMU

### **Phase 5: Repository Cleanup & Structure Alignment** 🧹 **IN PROGRESS**
- [ ] **Analyze original repository structure** 🔄 **IN PROGRESS**
  - [x] Review [osbuild/bootc-image-builder](https://github.com/osbuild/bootc-image-builder) structure
  - [x] Identify standard directory layout
  - [x] Compare with current structure
  - [ ] Plan cleanup actions
- [ ] **Clean up file organization**
  - [ ] Move scattered test files to proper locations
  - [ ] Organize scripts and utilities
  - [ ] Consolidate documentation
  - [ ] Remove duplicate or obsolete files
- [ ] **Align with original structure**
  - [ ] Create `.fmf/` directory for testing
  - [ ] Organize `.github/` workflows
  - [ ] Standardize `bib/` directory structure
  - [ ] Clean up `devel/` directory
- [ ] **Update documentation**
  - [ ] Update README.md to reflect new structure
  - [ ] Create CONTRIBUTING.md for contributors
  - [ ] Standardize documentation layout

---

## 🚧 **CURRENT PRIORITIES**

### **Immediate (This Session)**
1. **Repository Structure Analysis** - Complete analysis of original repository
2. **Cleanup Planning** - Create detailed cleanup plan
3. **File Organization** - Begin moving files to proper locations

### **Short Term (Next 1-2 Sessions)**
1. **Complete File Reorganization** - Move all files to proper locations
2. **Structure Alignment** - Match original repository layout
3. **Documentation Updates** - Update all documentation to reflect new structure

### **Medium Term (Next Week)**
1. **Testing Infrastructure** - Set up proper `.fmf/` testing
2. **CI/CD Integration** - Standardize GitHub Actions
3. **Contributor Experience** - Create clear contribution guidelines

---

## 🔍 **REPOSITORY CLEANUP ANALYSIS**

### **Current Structure Issues**
- **Mixed naming**: `bib/` vs `devel/` directories
- **Scattered tests**: Test files in multiple locations
- **Inconsistent docs**: Documentation spread across directories
- **Missing standards**: No `.fmf/` directory for testing

### **Target Structure (Based on Original)**
```
.
├── .fmf/                    # FMF testing framework
├── .github/                 # GitHub workflows and templates
├── .tekton/                 # Tekton CI/CD pipelines
├── bib/                     # Main Go application
├── devel/                   # Development tools
├── plans/                   # Test plans
├── test/                    # Test files
├── Containerfile            # Main container
├── Makefile                 # Build tasks
└── README.md                # Project documentation
```

### **Cleanup Actions Needed**
1. **Create `.fmf/` directory** for standardized testing
2. **Organize test files** into proper `test/` directory
3. **Consolidate scripts** into logical groups
4. **Standardize documentation** layout
5. **Remove obsolete files** and duplicates

---

## 📋 **NEXT STEPS**

1. **Complete repository structure analysis**
2. **Create detailed cleanup plan**
3. **Begin file reorganization**
4. **Update documentation**
5. **Validate new structure**

---

## 🎯 **SUCCESS CRITERIA**

- [ ] Repository structure matches original osbuild/bootc-image-builder
- [ ] All files properly organized in standard locations
- [ ] Documentation reflects new structure
- [ ] Contributor experience improved
- [ ] Ready for community contributions