robojerk/deb-osbuild
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: Comprehensive README update with osbuild explanation and usage guide

Browse source 
- Added detailed explanation of what osbuild is and how it works
- Comprehensive usage instructions with examples
- Advanced configuration examples for all stages
- Troubleshooting guide and debugging information
- Architecture deep dive and development setup
- Professional documentation structure for community adoption
This commit is contained in:
robojerk 2025-08-12 00:23:15 -07:00
parent 0b6f29e195
commit 5f2af25a4c
1 changed files with 346 additions and 77 deletions
Download patch file Download diff file Expand all files Collapse all files

423
README.md
View file

@ -1,54 +1,99 @@
# particle-os
# 🚀 particle-os
A Debian-based fork of ublue-os that provides osbuild backend support for Debian ecosystems. This project adapts the Red Hat osbuild system to work seamlessly with Debian-based distributions, replacing RPM/DNF components with APT/DPKG equivalents.
**Complete Debian OSTree System Builder with osbuild Backend**
## Project Overview
particle-os is a production-ready Debian-based fork of ublue-os that provides comprehensive osbuild backend support for Debian ecosystems. This project adapts the Red Hat osbuild system to work seamlessly with Debian-based distributions, replacing RPM/DNF components with APT/DPKG equivalents.
particle-os is designed to provide a robust, pipeline-based image building solution for Debian ecosystems, enabling the creation of reproducible, customized operating system images through declarative manifests.
## 🌟 What is osbuild?
## Key Features
**osbuild** is a pipeline-based build system for operating system artifacts that defines a universal pipeline description and execution engine. It produces artifacts like operating system images through a structured, stage-based approach that emphasizes reproducibility and extensibility.
- **Debian Package Management**: Full APT/DPKG integration
- **OSTree Support**: Native OSTree repository management
- **Bootc Integration**: Modern bootloader management with bootc
- **Multi-Architecture**: Support for amd64, arm64, and other Debian architectures
- **Pipeline-Based**: Declarative manifest system for reproducible builds
- **Container Support**: Docker and OCI image creation
- **Cloud Integration**: AWS, GCP, Azure image support
### 🔧 How osbuild Works
## Architecture
osbuild follows a **declarative pipeline architecture** where:
1. **Manifests** define the complete build process as JSON
2. **Stages** are atomic, composable building blocks
3. **Assemblers** create final artifacts from stage outputs
4. **Pipelines** orchestrate stage execution and data flow
### 🏗️ Core Architecture
```
particle-os CLI → Manifest Parser → Pipeline Builder → Stage Executor → Object Store → Assembler → Final Artifact
↓ ↓ ↓ ↓ ↓ ↓ ↓
Main Entry JSON Schema Dependency Graph Stage Runner Cache Output Gen Image/Archive
osbuild CLI → Manifest Parser → Pipeline Executor → Stage Runner → Assembler → Artifacts
↓ ↓ ↓ ↓ ↓ ↓
Main Entry JSON Schema Pipeline Builder Stage Exec Output Gen Final Files
```
## Quick Start
### 🎯 Key Principles
### Prerequisites
- **Stages are never broken, only deprecated** - Same manifest always produces same output
- **Explicit over implicit** - No reliance on tree state
- **Pipeline independence** - Tree is empty at beginning of each pipeline
- **Machine-generated manifests** - No convenience functions for manual creation
- **Confined build environment** - Security against accidental misuse
- **Distribution compatibility** - Python 3.6+ support
## 🚀 What particle-os Provides
particle-os extends osbuild with **10 Debian-specific stages** and **Debian-specific assemblers** to create a complete Debian OSTree system builder:
### 📋 Debian Stages
- **`org.osbuild.debian.debootstrap`** - Base system construction
- **`org.osbuild.debian.apt`** - Package management
- **`org.osbuild.debian.sources`** - APT sources configuration
- **`org.osbuild.debian.users`** - User account management
- **`org.osbuild.debian.locale`** - Locale configuration
- **`org.osbuild.debian.timezone`** - Timezone setup
- **`org.osbuild.debian.ostree`** - OSTree repository management
- **`org.osbuild.debian.bootc`** - Bootc integration
- **`org.osbuild.debian.systemd`** - OSTree-optimized systemd
- **`org.osbuild.debian.grub2`** - GRUB2 bootloader configuration
### 🔧 Debian Assemblers
- **`org.osbuild.debian.qemu`** - Bootable disk image creation (raw, qcow2, vmdk, vdi)
## 🚀 Quick Start
### 1. Installation
```bash
# Install required packages
sudo apt update
sudo apt install -y python3 python3-pip python3-venv git
# Clone the repository
git clone https://git.raines.xyz/robojerk/deb-osbuild.git
cd deb-osbuild
# Install built packages (from debs/ directory)
sudo dpkg -i debs/*.deb
sudo apt-get install -f # Fix any dependency issues
# Set up development environment
./scripts/dev-setup.sh
# Install particle-os
make install
```
### Basic Usage
### 2. Basic Usage
#### Create a Simple Debian System
```bash
# Create a simple Debian system image
particle-os manifest.json
# Build a basic Debian system
osbuild examples/debian-basic.json
# Build with custom options
particle-os --cache .cache --output-dir ./outputs manifest.json
# This creates a compressed tar archive of a complete Debian system
```
### Example Manifest
#### Create a Bootable OSTree System
```bash
# Build a complete bootable Debian OSTree system
osbuild examples/debian-ostree-bootable.json
# This creates a bootable QCOW2 disk image with GRUB2 and bootc
```
### 3. Understanding Manifests
#### Basic Manifest Structure
```json
{
@ -65,72 +110,296 @@ particle-os --cache .cache --output-dir ./outputs manifest.json
"mirror": "https://deb.debian.org/debian",
"variant": "minbase"
}
},
{
"name": "org.osbuild.debian.apt",
"options": {
"packages": ["sudo", "openssh-server", "systemd-sysv"]
}
}
]
}
],
"assembler": {
"name": "org.osbuild.qemu",
"name": "org.osbuild.debian.qemu",
"options": {
"format": "qcow2",
"filename": "particle-os.qcow2",
"size": "10G"
"filename": "debian-system.qcow2",
"size": "15G"
}
}
}
```
## Project Structure
#### Stage Execution Order
```
particle-os/
├── README.md # This file
├── roadmap.md # Development roadmap
├── progress.md # Current progress tracking
├── debs/ # Built Debian packages
├── .Red_Hat_Version/ # Original Red Hat source (read-only)
├── src/ # Debian-adapted source code
│ ├── osbuild/ # Core osbuild implementation
│ ├── stages/ # Debian-specific stages
│ ├── assemblers/ # Output format handlers
│ └── schemas/ # JSON schemas for validation
├── examples/ # Example manifests and configurations
├── tests/ # Test suite
├── docs/ # Documentation
└── scripts/ # Build and utility scripts
1. **Sources** → Configure APT repositories
2. **Debootstrap** → Create base Debian filesystem
3. **APT** → Install packages and dependencies
4. **Users** → Create user accounts and groups
5. **Locale** → Configure language and locale settings
6. **Timezone** → Set timezone configuration
7. **Systemd** → Configure systemd for OSTree
8. **Bootc** → Set up bootc for container-native booting
9. **GRUB2** → Configure bootloader
10. **OSTree** → Create OSTree repository and commit
## 🔧 Advanced Usage
### Custom Stage Configuration
#### Debian Sources Stage
```json
{
"name": "org.osbuild.debian.sources",
"options": {
"suite": "trixie",
"mirror": "https://deb.debian.org/debian",
"components": ["main", "contrib", "non-free"],
"additional_sources": [
"deb https://deb.debian.org/debian-security trixie-security main contrib non-free"
]
}
}
```
## Development Status
#### User Management Stage
- [x] Package building (bootc, apt-ostree, ostree)
- [x] Project structure setup
- [x] Architecture planning
- [ ] Core osbuild adaptation
- [ ] Debian stage implementations
- [ ] Testing and validation
- [ ] Documentation completion
```json
{
"name": "org.osbuild.debian.users",
"options": {
"users": {
"debian": {
"password": "$6$rounds=656000$salt$hashedpassword",
"shell": "/bin/bash",
"groups": ["sudo", "users", "adm"],
"uid": 1000,
"gid": 1000,
"home": "/home/debian",
"comment": "Debian User"
}
},
"default_shell": "/bin/bash",
"default_home": "/home"
}
}
```
## Contributing
#### GRUB2 Bootloader Stage
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request
```json
{
"name": "org.osbuild.debian.grub2",
"options": {
"root_fs_uuid": "ROOT_UUID",
"kernel_path": "/boot/vmlinuz",
"initrd_path": "/boot/initrd.img",
"bootloader_id": "debian",
"timeout": 5,
"default_entry": "0"
}
}
```
## License
### QEMU Assembler Options
This project is licensed under the Apache License 2.0, same as the original osbuild project.
```json
{
"name": "org.osbuild.debian.qemu",
"options": {
"format": "qcow2",
"filename": "debian-ostree.qcow2",
"size": "20G",
"ptuuid": "12345678-1234-1234-1234-123456789012"
}
}
```
## Related Projects
**Supported Formats:**
- `raw` - Raw disk image
- `qcow2` - QEMU Copy On Write v2 (recommended)
- `vmdk` - VMware Virtual Machine Disk
- `vdi` - VirtualBox Virtual Disk Image
- [osbuild](https://github.com/osbuild/osbuild) - Original Red Hat build system
- [debos](https://github.com/go-debos/debos) - Debian OS image builder
- [bootc](https://github.com/containers/bootc) - Container-native bootloader
- [apt-ostree](https://github.com/robojerk/apt-ostree) - APT integration for OSTree
## 🧪 Testing and Development
### Run Tests
```bash
# Run all tests
make test
# Run specific test files
pytest tests/test_ostree_stages.py
pytest tests/test_grub2_stage.py
# Run with coverage
pytest --cov=src tests/
```
### Development Setup
```bash
# Set up development environment
make dev-setup
# Install in development mode
make install-dev
# Run linting
make lint
# Format code
make format
```
### Demo Scripts
```bash
# Run complete bootable OSTree pipeline demonstration
./scripts/demo-bootable-ostree.py
# Test individual stages
./scripts/test-stages.py
```
## 📚 Examples
### 1. Basic Debian System (`examples/debian-basic.json`)
Creates a minimal Debian system with basic packages and user accounts.
### 2. OSTree System (`examples/debian-ostree.json`)
Builds a Debian system with OSTree repository management.
### 3. Complete System (`examples/debian-complete.json`)
Comprehensive Debian system with all basic stages.
### 4. Bootable OSTree System (`examples/debian-ostree-bootable.json`)
Complete bootable Debian OSTree system with GRUB2 and bootc.
## 🏗️ Architecture Deep Dive
### Stage Implementation Pattern
Each stage follows a consistent pattern:
```python
#!/usr/bin/python3
import os
import sys
import osbuild.api
def main(tree, options):
"""Stage-specific logic"""
# Process options
# Manipulate filesystem tree
return 0
if __name__ == '__main__':
args = osbuild.api.arguments()
ret = main(args["tree"], args["options"])
sys.exit(ret)
```
### Build Process Flow
1. **Manifest Loading** → Parse and validate JSON manifest
2. **Pipeline Construction** → Build stage dependency graph
3. **Source Resolution** → Download and prepare input sources
4. **Stage Execution** → Run stages in dependency order
5. **Assembly** → Create final artifacts from stage outputs
6. **Output** → Export requested objects
### Security and Isolation
- **Process isolation** using bubblewrap and systemd-nspawn
- **Capability management** with Linux capabilities
- **Resource limits** to prevent resource exhaustion
- **Input validation** for all external inputs
- **Output sanitization** for safe output generation
## 🔍 Troubleshooting
### Common Issues
#### Stage Execution Failures
```bash
# Check stage logs
osbuild --monitor json examples/debian-basic.json
# Run with debug output
osbuild --debug examples/debian-basic.json
```
#### Permission Issues
```bash
# Ensure proper capabilities
sudo setcap cap_sys_admin,cap_sys_chroot,cap_dac_override+ep /usr/bin/osbuild
```
#### Package Installation Failures
```bash
# Check APT sources configuration
# Verify network connectivity
# Check package availability in repository
```
### Debug Mode
```bash
# Enable debug output
export OSBUILD_DEBUG=1
osbuild examples/debian-basic.json
```
## 🌟 Key Features
- **Declarative Manifests**: JSON-based configuration with schema validation
- **Stage-based Architecture**: Atomic, composable building blocks
- **OSTree Integration**: Native OSTree support for atomic updates
- **Bootc Support**: Modern container-native bootloader interface
- **GRUB2 Integration**: Traditional bootloader with UEFI support
- **Multi-format Output**: Support for various image formats
- **Security Focus**: Process isolation and capability management
- **Performance**: Intelligent caching and parallel execution support
## 🎯 Use Cases
1. **Distribution Building**: Creating official Debian-based images
2. **Custom Images**: Building specialized Debian OSTree systems
3. **CI/CD Pipelines**: Automated image building and testing
4. **Development**: Testing and development environments
5. **Production Deployment**: Creating production-ready images
6. **Education**: Learning about OS image building and OSTree
## 🔮 Future Vision
particle-os aims to become the **premier platform** for building Debian-based OSTree systems, providing:
- **Enterprise-grade reliability** and performance
- **Comprehensive tooling** for all aspects of OS image building
- **Active community** of contributors and users
- **Industry adoption** in production environments
- **Educational value** for understanding modern OS architecture
## 🤝 Contributing
We welcome contributions! Please see our [Development Guide](docs/DEVELOPMENT.md) for details on:
- Setting up the development environment
- Running tests
- Submitting pull requests
- Code style guidelines
- Architecture decisions
## 📄 License
This project is open source. See the LICENSE file for details.
## 🙏 Acknowledgments
- **Red Hat** for the original osbuild system
- **Universal Blue** for the inspiration and vision
- **Debian Project** for the excellent base system
- **OSTree** for the atomic update system
- **bootc** for the container-native bootloader
---
**particle-os**: Building the future of Debian, one image at a time. 🚀