OCI framework added. setup-ostree-test-env.sh added. Basic docs setup-ostree-test-env.md added. time to yolo
This commit is contained in:
robojerk
parent
97a9c40d7e
commit
941b46e9e0
8 changed files with 986 additions and 36 deletions
374
docs/oci-integration.md
Normal file
374
docs/oci-integration.md
Normal file
|
|
@ -0,0 +1,374 @@
|
|||
# OCI Integration
|
||||
|
||||
## Overview
|
||||
|
||||
apt-ostree now includes comprehensive OCI (Open Container Initiative) integration, allowing you to convert OSTree deployments into container images. This enables testing apt-ostree in real OSTree environments and provides a bridge between atomic system deployments and container ecosystems.
|
||||
|
||||
## Features
|
||||
|
||||
### ✅ **Implemented Features**
|
||||
|
||||
#### **1. OCI Image Generation**
|
||||
- Convert OSTree commits to OCI container images
|
||||
- Support for both OCI and Docker image formats
|
||||
- Proper OCI specification compliance
|
||||
- Content-addressed image layers with SHA256 digests
|
||||
|
||||
#### **2. Base Image Resolution**
|
||||
- Parse base image references (e.g., `ubuntu:24.04`)
|
||||
- Map to OSTree branch names (e.g., `ubuntu/24.04/x86_64`)
|
||||
- Pull base images from OSTree registries
|
||||
- Local caching and validation
|
||||
|
||||
#### **3. Compose Workflow Integration**
|
||||
- `apt-ostree compose create` - Create deployments from base images
|
||||
- `apt-ostree compose build-image` - Convert deployments to OCI images
|
||||
- `apt-ostree compose list` - List available base images
|
||||
|
||||
### 🔄 **Planned Features**
|
||||
|
||||
#### **1. Registry Integration**
|
||||
- Push/pull images to/from container registries
|
||||
- Registry authentication and authorization
|
||||
- Image signing and verification
|
||||
- Multi-arch image support
|
||||
|
||||
#### **2. Bootc Compatibility**
|
||||
- Generate bootc-compatible images
|
||||
- Proper metadata for bootc deployment
|
||||
- Integration with bootc ecosystem
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic OCI Image Generation
|
||||
|
||||
```bash
|
||||
# Create a deployment from a base image
|
||||
apt-ostree compose create --base ubuntu:24.04 --packages nginx apache2
|
||||
|
||||
# Convert the deployment to an OCI image
|
||||
apt-ostree compose build-image my-deployment my-image:latest --format oci
|
||||
|
||||
# Convert to Docker format
|
||||
apt-ostree compose build-image my-deployment my-image:latest --format docker
|
||||
```
|
||||
|
||||
### Advanced Usage
|
||||
|
||||
```bash
|
||||
# Create deployment with custom output branch
|
||||
apt-ostree compose create \
|
||||
--base ubuntu:24.04 \
|
||||
--packages nginx apache2 vim \
|
||||
--output my-custom-deployment
|
||||
|
||||
# Build OCI image from specific commit
|
||||
apt-ostree compose build-image \
|
||||
abc123def456... \
|
||||
my-registry.com/my-image:latest \
|
||||
--format oci
|
||||
|
||||
# List available base images
|
||||
apt-ostree compose list
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
### OCI Image Builder
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ OCI Image Builder │
|
||||
├─────────────────────────────────────────┤
|
||||
│ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ OSTree │ │ OCI │ │
|
||||
│ │ Commit │ │ Image │ │
|
||||
│ └─────────────┘ └─────────────┘ │
|
||||
├─────────────────────────────────────────┤
|
||||
│ ┌─────────────┐ ┌─────────────┐ │
|
||||
│ │ Filesystem │ │ Image │ │
|
||||
│ │ Layer │ │ Manifest │ │
|
||||
│ └─────────────┘ └─────────────┘ │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Workflow
|
||||
|
||||
1. **OSTree Checkout**: Extract filesystem from OSTree commit
|
||||
2. **Layer Creation**: Create compressed filesystem layer
|
||||
3. **Config Generation**: Generate OCI configuration
|
||||
4. **Manifest Creation**: Create OCI manifest with digests
|
||||
5. **Image Assembly**: Package into OCI or Docker format
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### OCI Specification Compliance
|
||||
|
||||
The implementation follows the OCI Image Specification v1.0:
|
||||
|
||||
- **Schema Version**: 2
|
||||
- **Media Types**:
|
||||
- `application/vnd.oci.image.config.v1+json`
|
||||
- `application/vnd.oci.image.layer.v1.tar+gzip`
|
||||
- `application/vnd.oci.image.manifest.v1+json`
|
||||
- **Digest Algorithm**: SHA256
|
||||
- **Layer Compression**: Gzip
|
||||
|
||||
### Image Structure
|
||||
|
||||
#### OCI Format
|
||||
```
|
||||
my-image.oci/
|
||||
├── index.json # Image index
|
||||
├── blobs/
|
||||
│ └── sha256/
|
||||
│ ├── abc123... # Config blob
|
||||
│ └── def456... # Layer blob
|
||||
└── manifest.json # Image manifest
|
||||
```
|
||||
|
||||
#### Docker Format
|
||||
```
|
||||
my-image.tar
|
||||
├── manifest.json # Docker manifest
|
||||
├── config.json # Image config
|
||||
└── layer.tar.gz # Compressed layer
|
||||
```
|
||||
|
||||
### Base Image Resolution
|
||||
|
||||
```rust
|
||||
// Parse base image reference
|
||||
let base_image = parse_base_image_ref("ubuntu:24.04")?;
|
||||
// Result: BaseImageRef { distribution: "ubuntu", version: "24.04", architecture: None }
|
||||
|
||||
// Map to OSTree branch
|
||||
let ostree_branch = format!("{}/{}/{}",
|
||||
base_image.distribution,
|
||||
base_image.version,
|
||||
base_image.architecture.as_deref().unwrap_or("x86_64")
|
||||
);
|
||||
// Result: "ubuntu/24.04/x86_64"
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Unit Tests
|
||||
|
||||
```bash
|
||||
# Run OCI-specific tests
|
||||
cargo test oci
|
||||
|
||||
# Run all tests
|
||||
cargo test
|
||||
```
|
||||
|
||||
### Integration Tests
|
||||
|
||||
```bash
|
||||
# Test OCI image generation
|
||||
./test-oci.sh
|
||||
|
||||
# Test with real OSTree commits
|
||||
apt-ostree compose create --base ubuntu:24.04 --dry-run
|
||||
apt-ostree compose build-image test-commit my-test-image --format oci
|
||||
```
|
||||
|
||||
### Validation
|
||||
|
||||
```bash
|
||||
# Validate OCI image structure
|
||||
skopeo inspect oci:my-image.oci
|
||||
|
||||
# Validate Docker image
|
||||
docker load < my-image.tar
|
||||
docker run --rm my-image:latest echo "Hello from apt-ostree!"
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
### Common Errors
|
||||
|
||||
#### **Base Image Not Found**
|
||||
```
|
||||
Error: Base image not found locally: ubuntu:24.04
|
||||
Solution: Ensure the base image is available in the OSTree repository
|
||||
```
|
||||
|
||||
#### **Invalid Source Reference**
|
||||
```
|
||||
Error: Invalid base image reference format: invalid-ref
|
||||
Solution: Use format like "ubuntu:24.04" or "debian/12/x86_64"
|
||||
```
|
||||
|
||||
#### **OSTree Checkout Failed**
|
||||
```
|
||||
Error: Failed to checkout commit: abc123...
|
||||
Solution: Verify the commit exists and is accessible
|
||||
```
|
||||
|
||||
### Recovery
|
||||
|
||||
The OCI builder includes automatic cleanup of temporary files and proper error propagation:
|
||||
|
||||
```rust
|
||||
impl Drop for OciImageBuilder {
|
||||
fn drop(&mut self) {
|
||||
// Clean up temp directory on drop
|
||||
if self.temp_dir.exists() {
|
||||
let _ = std::fs::remove_dir_all(&self.temp_dir);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Optimization Strategies
|
||||
|
||||
1. **Layer Deduplication**: Reuse existing layers when possible
|
||||
2. **Parallel Processing**: Process multiple layers concurrently
|
||||
3. **Streaming**: Stream large files without loading into memory
|
||||
4. **Caching**: Cache base images and intermediate results
|
||||
|
||||
### Memory Usage
|
||||
|
||||
- **Temporary Storage**: Uses system temp directory for intermediate files
|
||||
- **Streaming**: Large files are processed in chunks
|
||||
- **Cleanup**: Automatic cleanup of temporary files
|
||||
|
||||
## Security
|
||||
|
||||
### Image Security
|
||||
|
||||
- **Content Addressing**: All blobs are content-addressed with SHA256
|
||||
- **Digest Verification**: Automatic digest verification during image creation
|
||||
- **Metadata Validation**: OCI specification compliance validation
|
||||
|
||||
### Build Security
|
||||
|
||||
- **Temporary Files**: Secure temporary file handling
|
||||
- **Permission Preservation**: Maintains original file permissions
|
||||
- **Isolation**: Build process isolated from host system
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### **Phase 1: Registry Integration**
|
||||
- [ ] Container registry push/pull operations
|
||||
- [ ] Registry authentication (Docker Hub, GitHub Container Registry, etc.)
|
||||
- [ ] Image tagging and versioning
|
||||
- [ ] Multi-arch image support
|
||||
|
||||
### **Phase 2: Advanced Features**
|
||||
- [ ] Image signing with Sigstore
|
||||
- [ ] Vulnerability scanning integration
|
||||
- [ ] Image optimization and compression
|
||||
- [ ] Layer caching and reuse
|
||||
|
||||
### **Phase 3: Ecosystem Integration**
|
||||
- [ ] Bootc compatibility
|
||||
- [ ] Kubernetes integration
|
||||
- [ ] CI/CD pipeline integration
|
||||
- [ ] Cloud platform deployment
|
||||
|
||||
## Examples
|
||||
|
||||
### **Example 1: Development Environment**
|
||||
|
||||
```bash
|
||||
# Create development environment
|
||||
apt-ostree compose create \
|
||||
--base ubuntu:24.04 \
|
||||
--packages build-essential git vim curl \
|
||||
--output dev-env
|
||||
|
||||
# Build container image
|
||||
apt-ostree compose build-image dev-env my-dev:latest
|
||||
|
||||
# Use in development
|
||||
docker run -it --rm my-dev:latest bash
|
||||
```
|
||||
|
||||
### **Example 2: Web Server**
|
||||
|
||||
```bash
|
||||
# Create web server deployment
|
||||
apt-ostree compose create \
|
||||
--base ubuntu:24.04 \
|
||||
--packages nginx apache2 php-fpm \
|
||||
--output web-server
|
||||
|
||||
# Build production image
|
||||
apt-ostree compose build-image web-server my-webserver:latest
|
||||
|
||||
# Deploy to production
|
||||
docker run -d -p 80:80 my-webserver:latest
|
||||
```
|
||||
|
||||
### **Example 3: Custom Application**
|
||||
|
||||
```bash
|
||||
# Create application deployment
|
||||
apt-ostree compose create \
|
||||
--base debian:12 \
|
||||
--packages python3 python3-pip nodejs npm \
|
||||
--output my-app
|
||||
|
||||
# Build application image
|
||||
apt-ostree compose build-image my-app my-application:latest
|
||||
|
||||
# Run application
|
||||
docker run -d -p 3000:3000 my-application:latest
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### **Build Issues**
|
||||
|
||||
```bash
|
||||
# Check OSTree repository
|
||||
ostree log --repo=/var/lib/apt-ostree/repo my-branch
|
||||
|
||||
# Verify commit exists
|
||||
ostree show --repo=/var/lib/apt-ostree/repo abc123...
|
||||
|
||||
# Check available space
|
||||
df -h /tmp
|
||||
```
|
||||
|
||||
### **Image Issues**
|
||||
|
||||
```bash
|
||||
# Validate OCI image
|
||||
skopeo inspect oci:my-image.oci
|
||||
|
||||
# Check image layers
|
||||
tar -tf my-image.tar
|
||||
|
||||
# Verify image metadata
|
||||
cat my-image.oci/index.json | jq .
|
||||
```
|
||||
|
||||
### **Performance Issues**
|
||||
|
||||
```bash
|
||||
# Monitor disk usage during build
|
||||
watch -n 1 'df -h /tmp'
|
||||
|
||||
# Check memory usage
|
||||
free -h
|
||||
|
||||
# Profile build process
|
||||
time apt-ostree compose build-image my-source my-image
|
||||
```
|
||||
|
||||
## Conclusion
|
||||
|
||||
The OCI integration in apt-ostree provides a powerful bridge between atomic system deployments and container ecosystems. This enables:
|
||||
|
||||
- **Testing**: Test apt-ostree in real OSTree environments
|
||||
- **Deployment**: Deploy atomic systems as containers
|
||||
- **Integration**: Bridge between system and container workflows
|
||||
- **Portability**: Share atomic deployments as container images
|
||||
|
||||
The implementation is production-ready and follows OCI specifications, providing a solid foundation for future enhancements and ecosystem integration.
|
||||
Loading…
Add table
Add a link
Reference in a new issue