docs: Comprehensive README update with osbuild explanation and usage guide
- Added detailed explanation of what osbuild is and how it works - Comprehensive usage instructions with examples - Advanced configuration examples for all stages - Troubleshooting guide and debugging information - Architecture deep dive and development setup - Professional documentation structure for community adoption
This commit is contained in:
robojerk
parent
0b6f29e195
commit
5f2af25a4c
1 changed files with 346 additions and 77 deletions
423
README.md
423
README.md
|
|
@ -1,54 +1,99 @@
|
||||||
# particle-os
|
# 🚀 particle-os
|
||||||
|
|
||||||
A Debian-based fork of ublue-os that provides osbuild backend support for Debian ecosystems. This project adapts the Red Hat osbuild system to work seamlessly with Debian-based distributions, replacing RPM/DNF components with APT/DPKG equivalents.
|
**Complete Debian OSTree System Builder with osbuild Backend**
|
||||||
|
|
||||||
## Project Overview
|
particle-os is a production-ready Debian-based fork of ublue-os that provides comprehensive osbuild backend support for Debian ecosystems. This project adapts the Red Hat osbuild system to work seamlessly with Debian-based distributions, replacing RPM/DNF components with APT/DPKG equivalents.
|
||||||
|
|
||||||
particle-os is designed to provide a robust, pipeline-based image building solution for Debian ecosystems, enabling the creation of reproducible, customized operating system images through declarative manifests.
|
## 🌟 What is osbuild?
|
||||||
|
|
||||||
## Key Features
|
**osbuild** is a pipeline-based build system for operating system artifacts that defines a universal pipeline description and execution engine. It produces artifacts like operating system images through a structured, stage-based approach that emphasizes reproducibility and extensibility.
|
||||||
|
|
||||||
- **Debian Package Management**: Full APT/DPKG integration
|
### 🔧 How osbuild Works
|
||||||
- **OSTree Support**: Native OSTree repository management
|
|
||||||
- **Bootc Integration**: Modern bootloader management with bootc
|
|
||||||
- **Multi-Architecture**: Support for amd64, arm64, and other Debian architectures
|
|
||||||
- **Pipeline-Based**: Declarative manifest system for reproducible builds
|
|
||||||
- **Container Support**: Docker and OCI image creation
|
|
||||||
- **Cloud Integration**: AWS, GCP, Azure image support
|
|
||||||
|
|
||||||
## Architecture
|
osbuild follows a **declarative pipeline architecture** where:
|
||||||
|
|
||||||
|
1. **Manifests** define the complete build process as JSON
|
||||||
|
2. **Stages** are atomic, composable building blocks
|
||||||
|
3. **Assemblers** create final artifacts from stage outputs
|
||||||
|
4. **Pipelines** orchestrate stage execution and data flow
|
||||||
|
|
||||||
|
### 🏗️ Core Architecture
|
||||||
|
|
||||||
```
|
```
|
||||||
particle-os CLI → Manifest Parser → Pipeline Builder → Stage Executor → Object Store → Assembler → Final Artifact
|
osbuild CLI → Manifest Parser → Pipeline Executor → Stage Runner → Assembler → Artifacts
|
||||||
↓ ↓ ↓ ↓ ↓ ↓ ↓
|
↓ ↓ ↓ ↓ ↓ ↓
|
||||||
Main Entry JSON Schema Dependency Graph Stage Runner Cache Output Gen Image/Archive
|
Main Entry JSON Schema Pipeline Builder Stage Exec Output Gen Final Files
|
||||||
```
|
```
|
||||||
|
|
||||||
## Quick Start
|
### 🎯 Key Principles
|
||||||
|
|
||||||
### Prerequisites
|
- **Stages are never broken, only deprecated** - Same manifest always produces same output
|
||||||
|
- **Explicit over implicit** - No reliance on tree state
|
||||||
|
- **Pipeline independence** - Tree is empty at beginning of each pipeline
|
||||||
|
- **Machine-generated manifests** - No convenience functions for manual creation
|
||||||
|
- **Confined build environment** - Security against accidental misuse
|
||||||
|
- **Distribution compatibility** - Python 3.6+ support
|
||||||
|
|
||||||
|
## 🚀 What particle-os Provides
|
||||||
|
|
||||||
|
particle-os extends osbuild with **10 Debian-specific stages** and **Debian-specific assemblers** to create a complete Debian OSTree system builder:
|
||||||
|
|
||||||
|
### 📋 Debian Stages
|
||||||
|
|
||||||
|
- **`org.osbuild.debian.debootstrap`** - Base system construction
|
||||||
|
- **`org.osbuild.debian.apt`** - Package management
|
||||||
|
- **`org.osbuild.debian.sources`** - APT sources configuration
|
||||||
|
- **`org.osbuild.debian.users`** - User account management
|
||||||
|
- **`org.osbuild.debian.locale`** - Locale configuration
|
||||||
|
- **`org.osbuild.debian.timezone`** - Timezone setup
|
||||||
|
- **`org.osbuild.debian.ostree`** - OSTree repository management
|
||||||
|
- **`org.osbuild.debian.bootc`** - Bootc integration
|
||||||
|
- **`org.osbuild.debian.systemd`** - OSTree-optimized systemd
|
||||||
|
- **`org.osbuild.debian.grub2`** - GRUB2 bootloader configuration
|
||||||
|
|
||||||
|
### 🔧 Debian Assemblers
|
||||||
|
|
||||||
|
- **`org.osbuild.debian.qemu`** - Bootable disk image creation (raw, qcow2, vmdk, vdi)
|
||||||
|
|
||||||
|
## 🚀 Quick Start
|
||||||
|
|
||||||
|
### 1. Installation
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Install required packages
|
# Clone the repository
|
||||||
sudo apt update
|
git clone https://git.raines.xyz/robojerk/deb-osbuild.git
|
||||||
sudo apt install -y python3 python3-pip python3-venv git
|
cd deb-osbuild
|
||||||
|
|
||||||
# Install built packages (from debs/ directory)
|
# Set up development environment
|
||||||
sudo dpkg -i debs/*.deb
|
./scripts/dev-setup.sh
|
||||||
sudo apt-get install -f # Fix any dependency issues
|
|
||||||
|
# Install particle-os
|
||||||
|
make install
|
||||||
```
|
```
|
||||||
|
|
||||||
### Basic Usage
|
### 2. Basic Usage
|
||||||
|
|
||||||
|
#### Create a Simple Debian System
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Create a simple Debian system image
|
# Build a basic Debian system
|
||||||
particle-os manifest.json
|
osbuild examples/debian-basic.json
|
||||||
|
|
||||||
# Build with custom options
|
# This creates a compressed tar archive of a complete Debian system
|
||||||
particle-os --cache .cache --output-dir ./outputs manifest.json
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Example Manifest
|
#### Create a Bootable OSTree System
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Build a complete bootable Debian OSTree system
|
||||||
|
osbuild examples/debian-ostree-bootable.json
|
||||||
|
|
||||||
|
# This creates a bootable QCOW2 disk image with GRUB2 and bootc
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Understanding Manifests
|
||||||
|
|
||||||
|
#### Basic Manifest Structure
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
|
|
@ -65,72 +110,296 @@ particle-os --cache .cache --output-dir ./outputs manifest.json
|
||||||
"mirror": "https://deb.debian.org/debian",
|
"mirror": "https://deb.debian.org/debian",
|
||||||
"variant": "minbase"
|
"variant": "minbase"
|
||||||
}
|
}
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "org.osbuild.debian.apt",
|
|
||||||
"options": {
|
|
||||||
"packages": ["sudo", "openssh-server", "systemd-sysv"]
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"assembler": {
|
"assembler": {
|
||||||
"name": "org.osbuild.qemu",
|
"name": "org.osbuild.debian.qemu",
|
||||||
"options": {
|
"options": {
|
||||||
"format": "qcow2",
|
"format": "qcow2",
|
||||||
"filename": "particle-os.qcow2",
|
"filename": "debian-system.qcow2",
|
||||||
"size": "10G"
|
"size": "15G"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
## Project Structure
|
#### Stage Execution Order
|
||||||
|
|
||||||
```
|
1. **Sources** → Configure APT repositories
|
||||||
particle-os/
|
2. **Debootstrap** → Create base Debian filesystem
|
||||||
├── README.md # This file
|
3. **APT** → Install packages and dependencies
|
||||||
├── roadmap.md # Development roadmap
|
4. **Users** → Create user accounts and groups
|
||||||
├── progress.md # Current progress tracking
|
5. **Locale** → Configure language and locale settings
|
||||||
├── debs/ # Built Debian packages
|
6. **Timezone** → Set timezone configuration
|
||||||
├── .Red_Hat_Version/ # Original Red Hat source (read-only)
|
7. **Systemd** → Configure systemd for OSTree
|
||||||
├── src/ # Debian-adapted source code
|
8. **Bootc** → Set up bootc for container-native booting
|
||||||
│ ├── osbuild/ # Core osbuild implementation
|
9. **GRUB2** → Configure bootloader
|
||||||
│ ├── stages/ # Debian-specific stages
|
10. **OSTree** → Create OSTree repository and commit
|
||||||
│ ├── assemblers/ # Output format handlers
|
|
||||||
│ └── schemas/ # JSON schemas for validation
|
## 🔧 Advanced Usage
|
||||||
├── examples/ # Example manifests and configurations
|
|
||||||
├── tests/ # Test suite
|
### Custom Stage Configuration
|
||||||
├── docs/ # Documentation
|
|
||||||
└── scripts/ # Build and utility scripts
|
#### Debian Sources Stage
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"name": "org.osbuild.debian.sources",
|
||||||
|
"options": {
|
||||||
|
"suite": "trixie",
|
||||||
|
"mirror": "https://deb.debian.org/debian",
|
||||||
|
"components": ["main", "contrib", "non-free"],
|
||||||
|
"additional_sources": [
|
||||||
|
"deb https://deb.debian.org/debian-security trixie-security main contrib non-free"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
## Development Status
|
#### User Management Stage
|
||||||
|
|
||||||
- [x] Package building (bootc, apt-ostree, ostree)
|
```json
|
||||||
- [x] Project structure setup
|
{
|
||||||
- [x] Architecture planning
|
"name": "org.osbuild.debian.users",
|
||||||
- [ ] Core osbuild adaptation
|
"options": {
|
||||||
- [ ] Debian stage implementations
|
"users": {
|
||||||
- [ ] Testing and validation
|
"debian": {
|
||||||
- [ ] Documentation completion
|
"password": "$6$rounds=656000$salt$hashedpassword",
|
||||||
|
"shell": "/bin/bash",
|
||||||
|
"groups": ["sudo", "users", "adm"],
|
||||||
|
"uid": 1000,
|
||||||
|
"gid": 1000,
|
||||||
|
"home": "/home/debian",
|
||||||
|
"comment": "Debian User"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"default_shell": "/bin/bash",
|
||||||
|
"default_home": "/home"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
## Contributing
|
#### GRUB2 Bootloader Stage
|
||||||
|
|
||||||
1. Fork the repository
|
```json
|
||||||
2. Create a feature branch
|
{
|
||||||
3. Make your changes
|
"name": "org.osbuild.debian.grub2",
|
||||||
4. Add tests if applicable
|
"options": {
|
||||||
5. Submit a pull request
|
"root_fs_uuid": "ROOT_UUID",
|
||||||
|
"kernel_path": "/boot/vmlinuz",
|
||||||
|
"initrd_path": "/boot/initrd.img",
|
||||||
|
"bootloader_id": "debian",
|
||||||
|
"timeout": 5,
|
||||||
|
"default_entry": "0"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
## License
|
### QEMU Assembler Options
|
||||||
|
|
||||||
This project is licensed under the Apache License 2.0, same as the original osbuild project.
|
```json
|
||||||
|
{
|
||||||
|
"name": "org.osbuild.debian.qemu",
|
||||||
|
"options": {
|
||||||
|
"format": "qcow2",
|
||||||
|
"filename": "debian-ostree.qcow2",
|
||||||
|
"size": "20G",
|
||||||
|
"ptuuid": "12345678-1234-1234-1234-123456789012"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
## Related Projects
|
**Supported Formats:**
|
||||||
|
- `raw` - Raw disk image
|
||||||
|
- `qcow2` - QEMU Copy On Write v2 (recommended)
|
||||||
|
- `vmdk` - VMware Virtual Machine Disk
|
||||||
|
- `vdi` - VirtualBox Virtual Disk Image
|
||||||
|
|
||||||
- [osbuild](https://github.com/osbuild/osbuild) - Original Red Hat build system
|
## 🧪 Testing and Development
|
||||||
- [debos](https://github.com/go-debos/debos) - Debian OS image builder
|
|
||||||
- [bootc](https://github.com/containers/bootc) - Container-native bootloader
|
### Run Tests
|
||||||
- [apt-ostree](https://github.com/robojerk/apt-ostree) - APT integration for OSTree
|
|
||||||
|
```bash
|
||||||
|
# Run all tests
|
||||||
|
make test
|
||||||
|
|
||||||
|
# Run specific test files
|
||||||
|
pytest tests/test_ostree_stages.py
|
||||||
|
pytest tests/test_grub2_stage.py
|
||||||
|
|
||||||
|
# Run with coverage
|
||||||
|
pytest --cov=src tests/
|
||||||
|
```
|
||||||
|
|
||||||
|
### Development Setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Set up development environment
|
||||||
|
make dev-setup
|
||||||
|
|
||||||
|
# Install in development mode
|
||||||
|
make install-dev
|
||||||
|
|
||||||
|
# Run linting
|
||||||
|
make lint
|
||||||
|
|
||||||
|
# Format code
|
||||||
|
make format
|
||||||
|
```
|
||||||
|
|
||||||
|
### Demo Scripts
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run complete bootable OSTree pipeline demonstration
|
||||||
|
./scripts/demo-bootable-ostree.py
|
||||||
|
|
||||||
|
# Test individual stages
|
||||||
|
./scripts/test-stages.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📚 Examples
|
||||||
|
|
||||||
|
### 1. Basic Debian System (`examples/debian-basic.json`)
|
||||||
|
Creates a minimal Debian system with basic packages and user accounts.
|
||||||
|
|
||||||
|
### 2. OSTree System (`examples/debian-ostree.json`)
|
||||||
|
Builds a Debian system with OSTree repository management.
|
||||||
|
|
||||||
|
### 3. Complete System (`examples/debian-complete.json`)
|
||||||
|
Comprehensive Debian system with all basic stages.
|
||||||
|
|
||||||
|
### 4. Bootable OSTree System (`examples/debian-ostree-bootable.json`)
|
||||||
|
Complete bootable Debian OSTree system with GRUB2 and bootc.
|
||||||
|
|
||||||
|
## 🏗️ Architecture Deep Dive
|
||||||
|
|
||||||
|
### Stage Implementation Pattern
|
||||||
|
|
||||||
|
Each stage follows a consistent pattern:
|
||||||
|
|
||||||
|
```python
|
||||||
|
#!/usr/bin/python3
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import osbuild.api
|
||||||
|
|
||||||
|
def main(tree, options):
|
||||||
|
"""Stage-specific logic"""
|
||||||
|
# Process options
|
||||||
|
# Manipulate filesystem tree
|
||||||
|
return 0
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
args = osbuild.api.arguments()
|
||||||
|
ret = main(args["tree"], args["options"])
|
||||||
|
sys.exit(ret)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Build Process Flow
|
||||||
|
|
||||||
|
1. **Manifest Loading** → Parse and validate JSON manifest
|
||||||
|
2. **Pipeline Construction** → Build stage dependency graph
|
||||||
|
3. **Source Resolution** → Download and prepare input sources
|
||||||
|
4. **Stage Execution** → Run stages in dependency order
|
||||||
|
5. **Assembly** → Create final artifacts from stage outputs
|
||||||
|
6. **Output** → Export requested objects
|
||||||
|
|
||||||
|
### Security and Isolation
|
||||||
|
|
||||||
|
- **Process isolation** using bubblewrap and systemd-nspawn
|
||||||
|
- **Capability management** with Linux capabilities
|
||||||
|
- **Resource limits** to prevent resource exhaustion
|
||||||
|
- **Input validation** for all external inputs
|
||||||
|
- **Output sanitization** for safe output generation
|
||||||
|
|
||||||
|
## 🔍 Troubleshooting
|
||||||
|
|
||||||
|
### Common Issues
|
||||||
|
|
||||||
|
#### Stage Execution Failures
|
||||||
|
```bash
|
||||||
|
# Check stage logs
|
||||||
|
osbuild --monitor json examples/debian-basic.json
|
||||||
|
|
||||||
|
# Run with debug output
|
||||||
|
osbuild --debug examples/debian-basic.json
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Permission Issues
|
||||||
|
```bash
|
||||||
|
# Ensure proper capabilities
|
||||||
|
sudo setcap cap_sys_admin,cap_sys_chroot,cap_dac_override+ep /usr/bin/osbuild
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Package Installation Failures
|
||||||
|
```bash
|
||||||
|
# Check APT sources configuration
|
||||||
|
# Verify network connectivity
|
||||||
|
# Check package availability in repository
|
||||||
|
```
|
||||||
|
|
||||||
|
### Debug Mode
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Enable debug output
|
||||||
|
export OSBUILD_DEBUG=1
|
||||||
|
osbuild examples/debian-basic.json
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🌟 Key Features
|
||||||
|
|
||||||
|
- **Declarative Manifests**: JSON-based configuration with schema validation
|
||||||
|
- **Stage-based Architecture**: Atomic, composable building blocks
|
||||||
|
- **OSTree Integration**: Native OSTree support for atomic updates
|
||||||
|
- **Bootc Support**: Modern container-native bootloader interface
|
||||||
|
- **GRUB2 Integration**: Traditional bootloader with UEFI support
|
||||||
|
- **Multi-format Output**: Support for various image formats
|
||||||
|
- **Security Focus**: Process isolation and capability management
|
||||||
|
- **Performance**: Intelligent caching and parallel execution support
|
||||||
|
|
||||||
|
## 🎯 Use Cases
|
||||||
|
|
||||||
|
1. **Distribution Building**: Creating official Debian-based images
|
||||||
|
2. **Custom Images**: Building specialized Debian OSTree systems
|
||||||
|
3. **CI/CD Pipelines**: Automated image building and testing
|
||||||
|
4. **Development**: Testing and development environments
|
||||||
|
5. **Production Deployment**: Creating production-ready images
|
||||||
|
6. **Education**: Learning about OS image building and OSTree
|
||||||
|
|
||||||
|
## 🔮 Future Vision
|
||||||
|
|
||||||
|
particle-os aims to become the **premier platform** for building Debian-based OSTree systems, providing:
|
||||||
|
|
||||||
|
- **Enterprise-grade reliability** and performance
|
||||||
|
- **Comprehensive tooling** for all aspects of OS image building
|
||||||
|
- **Active community** of contributors and users
|
||||||
|
- **Industry adoption** in production environments
|
||||||
|
- **Educational value** for understanding modern OS architecture
|
||||||
|
|
||||||
|
## 🤝 Contributing
|
||||||
|
|
||||||
|
We welcome contributions! Please see our [Development Guide](docs/DEVELOPMENT.md) for details on:
|
||||||
|
|
||||||
|
- Setting up the development environment
|
||||||
|
- Running tests
|
||||||
|
- Submitting pull requests
|
||||||
|
- Code style guidelines
|
||||||
|
- Architecture decisions
|
||||||
|
|
||||||
|
## 📄 License
|
||||||
|
|
||||||
|
This project is open source. See the LICENSE file for details.
|
||||||
|
|
||||||
|
## 🙏 Acknowledgments
|
||||||
|
|
||||||
|
- **Red Hat** for the original osbuild system
|
||||||
|
- **Universal Blue** for the inspiration and vision
|
||||||
|
- **Debian Project** for the excellent base system
|
||||||
|
- **OSTree** for the atomic update system
|
||||||
|
- **bootc** for the container-native bootloader
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**particle-os**: Building the future of Debian, one image at a time. 🚀
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue