Enhance bootc-base-imagectl documentation with technical details
- Add comprehensive technical implementation details based on actual Python source - Document all command-line options and usage patterns - Add detailed examples for build-rootfs, rechunk, and list commands - Include practical Containerfile examples for different use cases - Document external dependencies and error handling - Add security considerations and cleanup procedures - Provide cross-build examples and advanced workflows Based on analysis of the actual bootc-base-imagectl Python script from https://gitlab.com/fedora/bootc/base-images/-/raw/main/bootc-base-imagectl
This commit is contained in:
robojerk
parent
0e12ed215a
commit
bd4c3e746f
1 changed files with 413 additions and 0 deletions
413
bootc-base-imagectl.md
Normal file
413
bootc-base-imagectl.md
Normal file
|
|
@ -0,0 +1,413 @@
|
|||
# bootc-base-imagectl
|
||||
|
||||
A core premise of the bootc model is that rich
|
||||
control over Linux system customization can be accomplished
|
||||
with a "default" container build:
|
||||
|
||||
```
|
||||
FROM <base image>
|
||||
RUN ...
|
||||
```
|
||||
|
||||
As of recently, it is possible to e.g. swap the kernel
|
||||
and other fundamental components as part of default derivation.
|
||||
|
||||
However, some use cases want even more control - for example,
|
||||
as an organization deploying a bootc system, I may want to ensure
|
||||
the base image version carries a set of packages at
|
||||
exactly specific versions (perhaps defined by a lockfile,
|
||||
or an rpm-md repository). There are many tools which
|
||||
manage snapshots of yum (rpm-md) repositories.
|
||||
|
||||
There are currently issues where it won't quite work to e.g.
|
||||
`dnf -y upgrade selinux-policy-targeted`.
|
||||
|
||||
The `/usr/libexec/bootc-base-imagectl` tool which is
|
||||
included in the base image is designed to enable building
|
||||
a root filesystem in ostree-container format from a set
|
||||
of RPMs controlled by the user.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
The `bootc-base-imagectl` tool is a Python script that provides three main operations:
|
||||
|
||||
1. **`build-rootfs`** - Generate a container root filesystem from RPM packages
|
||||
2. **`rechunk`** - Split container images into reproducible, chunked layers
|
||||
3. **`list`** - Enumerate available build configurations
|
||||
|
||||
The tool uses `rpm-ostree` under the hood to compose root filesystems from RPM packages,
|
||||
with support for custom manifests, package locking, and reproducible builds.
|
||||
|
||||
## Understanding the base image content
|
||||
|
||||
Most, but not all content from the base image comes from RPMs.
|
||||
There is some additional non-RPM content, as well as postprocessing
|
||||
that operates on the filesystem root. At the current time the
|
||||
implementation of the base image build uses `rpm-ostree`,
|
||||
but this is considered an implementation detail subject to change.
|
||||
|
||||
## Using bootc-base-imagectl build-rootfs
|
||||
|
||||
The core operation is `bootc-base-imagectl build-rootfs`.
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```bash
|
||||
bootc-base-imagectl build-rootfs [OPTIONS] <source_root> <target>
|
||||
```
|
||||
|
||||
**Required Arguments:**
|
||||
- `source_root` - Path to source root with `/etc/yum.repos.d` configuration
|
||||
- `target` - Path where the root filesystem will be generated (must not exist)
|
||||
|
||||
### Advanced Options
|
||||
|
||||
```bash
|
||||
bootc-base-imagectl build-rootfs \
|
||||
--manifest=minimal \
|
||||
--install=package1 --install=package2 \
|
||||
--add-dir=/path/to/overlay1 --add-dir=/path/to/overlay2 \
|
||||
--lock=package-1.0.0-1.x86_64 \
|
||||
--no-docs \
|
||||
--sysusers \
|
||||
--cachedir=/var/cache/dnf \
|
||||
--repo=fedora --repo=updates \
|
||||
/source/root /target/rootfs
|
||||
```
|
||||
|
||||
**Key Options:**
|
||||
- `--manifest` - Select build configuration (default: "default")
|
||||
- `--install` - Add additional packages to install
|
||||
- `--add-dir` - Copy directory contents as overlay layers
|
||||
- `--lock` - Lock packages to specific versions (NEVRA or NEVR format)
|
||||
- `--no-docs` - Skip documentation packages
|
||||
- `--sysusers` - Use systemd-sysusers instead of hardcoded passwd/group
|
||||
- `--cachedir` - Cache repository metadata and RPMs
|
||||
- `--repo` - Enable specific repositories only
|
||||
- `--reinject` - Copy build configurations into target
|
||||
|
||||
### Technical Implementation
|
||||
|
||||
The tool performs these steps:
|
||||
|
||||
1. **Repository Setup**: Runs `dnf repolist` to refresh repository metadata
|
||||
2. **Manifest Processing**: Loads YAML manifest from `/usr/share/doc/bootc-base-imagectl/manifests/`
|
||||
3. **Override Generation**: Creates temporary JSON with user overrides
|
||||
4. **OSTree Compose**: Executes `rpm-ostree compose rootfs` with the manifest
|
||||
5. **Post-processing**: Fixes permissions and runs `bootc container lint`
|
||||
6. **Cleanup**: Removes temporary files and repositories
|
||||
|
||||
## Using bootc-base-imagectl rechunk
|
||||
|
||||
This operation is strongly related to `build-rootfs` but is also orthogonal;
|
||||
it can be used on a "regular" container build as well.
|
||||
|
||||
### Basic Usage
|
||||
|
||||
```bash
|
||||
bootc-base-imagectl rechunk [OPTIONS] <from_image> <to_image>
|
||||
```
|
||||
|
||||
**Required Arguments:**
|
||||
- `from_image` - Source image in container storage (e.g., `quay.io/exampleos:build`)
|
||||
- `to_image` - Output image in container storage (e.g., `quay.io/exampleos:latest`)
|
||||
|
||||
### Advanced Options
|
||||
|
||||
```bash
|
||||
bootc-base-imagectl rechunk \
|
||||
--max-layers=10 \
|
||||
quay.io/exampleos:build \
|
||||
quay.io/exampleos:latest
|
||||
```
|
||||
|
||||
**Options:**
|
||||
- `--max-layers` - Configure maximum number of output layers
|
||||
|
||||
### Container Usage
|
||||
|
||||
This command assumes it will be run as a container image, and defaults
|
||||
to wanting write access to the container storage.
|
||||
|
||||
```bash
|
||||
podman run --rm --privileged \
|
||||
-v /var/lib/containers:/var/lib/containers \
|
||||
quay.io/fedora/fedora-bootc:rawhide \
|
||||
bootc-base-imagectl rechunk \
|
||||
quay.io/exampleos/exampleos:build \
|
||||
quay.io/exampleos/exampleos:latest
|
||||
```
|
||||
|
||||
### Technical Implementation
|
||||
|
||||
The rechunk operation uses `rpm-ostree experimental compose build-chunked-oci`:
|
||||
|
||||
1. **Input Processing**: Reads the source container image
|
||||
2. **Layer Analysis**: Analyzes RPM database to determine optimal layer splits
|
||||
3. **Chunking**: Splits content into separate, reproducible layers
|
||||
4. **Timestamp Canonicalization**: Sets all timestamps to zero for reproducibility
|
||||
5. **Output Generation**: Creates new container image with chunked layers
|
||||
|
||||
### Rationale
|
||||
|
||||
When performing a complex container derivation, there are several issues:
|
||||
|
||||
#### Replaced duplicate content
|
||||
|
||||
When e.g. upgrading or replacing the kernel or other large packages
|
||||
as part of a container build (without squashing all layers) then
|
||||
the old replaced content will still be present.
|
||||
|
||||
#### Removed content still present
|
||||
|
||||
Similarly, `RUN dnf -y remove` etc. will still retain that removed
|
||||
content in prior layers.
|
||||
|
||||
#### Timestamp drift
|
||||
|
||||
By default, many tools will use the current timestamp when writing
|
||||
files. `rpm` will do this (unless `SOURCE_DATE_EPOCH` is set), and
|
||||
other tools like `cp` and `curl` will as well.
|
||||
|
||||
This means that every build of the image will produce a new
|
||||
tar stream (with new timestamps) - that will get pushed to a registry
|
||||
and downloaded by clients, even if the content didn't actually change.
|
||||
|
||||
### What rechunk does: split reproducible chunked images
|
||||
|
||||
The `bootc-base-imagectl rechunk` command fixes all of these issues
|
||||
by taking an input container, operates on its final merged filesystem
|
||||
tree (hence removed/overridden files are handled), and then splits it up
|
||||
(currently based on the RPM database) into separate layers (tarballs).
|
||||
|
||||
Further, because bootc uses OSTree today, and OSTree canonializes all timestamps
|
||||
to zero on the client side, this tool does that at build time.
|
||||
|
||||
## Using bootc-base-imagectl list
|
||||
|
||||
The `list` command enumerates available build configurations that can be selected by passing `--manifest` to `build-rootfs`.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
bootc-base-imagectl list
|
||||
```
|
||||
|
||||
### Output Format
|
||||
|
||||
```
|
||||
minimal: Minimal bootc base image
|
||||
default: Standard bootc base image with common packages
|
||||
server: Server-focused bootc base image
|
||||
---
|
||||
```
|
||||
|
||||
### Technical Implementation
|
||||
|
||||
The list command:
|
||||
|
||||
1. **Manifest Discovery**: Scans `/usr/share/doc/bootc-base-imagectl/manifests/` for `.yaml` files
|
||||
2. **Filtering**: Excludes `.hidden.yaml` files and symbolic links
|
||||
3. **Metadata Extraction**: Uses `rpm-ostree compose tree --print-only` to read manifest metadata
|
||||
4. **Summary Display**: Shows manifest name and description from metadata
|
||||
|
||||
## Technical Implementation Details
|
||||
|
||||
### Core Architecture
|
||||
|
||||
The `bootc-base-imagectl` tool is implemented as a Python 3 script that orchestrates `rpm-ostree` operations. The tool provides a high-level interface for:
|
||||
|
||||
- **Manifest Management**: Loading and processing YAML treefiles
|
||||
- **Package Resolution**: Handling RPM dependencies and version locking
|
||||
- **Overlay Support**: Managing additional content layers
|
||||
- **Reproducible Builds**: Ensuring consistent output across builds
|
||||
|
||||
### Key Components
|
||||
|
||||
#### 1. Manifest System
|
||||
- **Location**: `/usr/share/doc/bootc-base-imagectl/manifests/`
|
||||
- **Format**: YAML treefiles compatible with `rpm-ostree`
|
||||
- **Discovery**: Automatic scanning for `.yaml` files (excluding `.hidden.yaml`)
|
||||
- **Override Support**: JSON overrides for user customizations
|
||||
|
||||
#### 2. Package Management
|
||||
- **Repository Handling**: Uses `/etc/yum.repos.d` configuration from source root
|
||||
- **Version Locking**: Supports NEVRA (Name-Epoch-Version-Release-Architecture) format
|
||||
- **Dependency Resolution**: Leverages `rpm-ostree`'s advanced dependency solver
|
||||
- **Caching**: Optional repository metadata and RPM caching
|
||||
|
||||
#### 3. Overlay System
|
||||
- **Directory Overlays**: Copy additional content as OSTree overlay layers
|
||||
- **Temporary Repositories**: Create temporary OSTree repos for overlay content
|
||||
- **Content Integration**: Merge overlay content into final rootfs
|
||||
|
||||
#### 4. Build Process
|
||||
```python
|
||||
# Simplified build flow
|
||||
def run_build_rootfs(args):
|
||||
# 1. Repository refresh
|
||||
subprocess.check_call(['dnf', 'repolist'], stdout=subprocess.DEVNULL)
|
||||
|
||||
# 2. Manifest processing
|
||||
manifest_path = find_manifest(args.manifest)
|
||||
|
||||
# 3. Override generation
|
||||
if user_overrides:
|
||||
create_temp_override_manifest(overrides)
|
||||
|
||||
# 4. OSTree compose
|
||||
subprocess.run(['rpm-ostree', 'compose', 'rootfs', ...])
|
||||
|
||||
# 5. Post-processing
|
||||
fix_permissions(target)
|
||||
subprocess.run(['bootc', 'container', 'lint', f'--rootfs={target}'])
|
||||
```
|
||||
|
||||
### External Dependencies
|
||||
|
||||
The tool requires these external commands:
|
||||
- `rpm-ostree` - Core OSTree composition engine
|
||||
- `dnf` - Package manager for repository operations
|
||||
- `ostree` - OSTree repository management
|
||||
- `bootc` - Container validation and linting
|
||||
|
||||
### Error Handling
|
||||
|
||||
The implementation includes comprehensive error handling:
|
||||
- **Subprocess Failures**: Captures and reports command execution errors
|
||||
- **File Operations**: Handles temporary file creation and cleanup
|
||||
- **Repository Issues**: Manages OSTree repository lifecycle
|
||||
- **Validation**: Runs `bootc container lint` to ensure output validity
|
||||
|
||||
### Security Considerations
|
||||
|
||||
- **Temporary Files**: All temporary files are created with secure permissions
|
||||
- **Repository Isolation**: Uses separate temporary OSTree repositories for overlays
|
||||
- **Content Validation**: Validates all input manifests and configurations
|
||||
- **Cleanup**: Ensures all temporary resources are properly cleaned up
|
||||
|
||||
### Cross builds and the builder image
|
||||
|
||||
The build tooling is designed to support "cross builds"; the
|
||||
repository root could e.g. be CentOS Stream 10, while the
|
||||
builder root is Fedora or RHEL, etc.
|
||||
|
||||
In other words, one given base image can be used as a "builder" to produce another
|
||||
using different RPMs.
|
||||
|
||||
## Practical Examples
|
||||
|
||||
### Example 1: Generate a new image using CentOS Stream 10 content from RHEL
|
||||
|
||||
```dockerfile
|
||||
FROM quay.io/centos/centos:stream10 as repos
|
||||
|
||||
FROM registry.redhat.io/rhel10/rhel-bootc:10 as builder
|
||||
RUN --mount=type=bind,from=repos,src=/,dst=/repos,rw \
|
||||
/usr/libexec/bootc-base-imagectl build-rootfs \
|
||||
--manifest=minimal \
|
||||
/repos /target-rootfs
|
||||
|
||||
# This container image uses the "artifact pattern"; it has some
|
||||
# basic configuration we expect to apply to multiple container images.
|
||||
FROM quay.io/exampleos/baseconfig@sha256:.... as baseconfig
|
||||
|
||||
FROM scratch
|
||||
COPY --from=builder /target-rootfs/ /
|
||||
# Now we make other arbitrary changes. Copy our systemd units and
|
||||
# other tweaks from the baseconfig container image.
|
||||
COPY --from=baseconfig /usr/ /usr/
|
||||
RUN <<EORUN
|
||||
set -xeuo pipefail
|
||||
# Install critical components
|
||||
dnf -y install linux-firmware NetworkManager cloud-init cowsay
|
||||
dnf clean all
|
||||
bootc container lint
|
||||
EORUN
|
||||
LABEL containers.bootc 1
|
||||
ENV container=oci
|
||||
STOPSIGNAL SIGRTMIN+3
|
||||
CMD ["/sbin/init"]
|
||||
```
|
||||
|
||||
### Example 2: Custom package selection with version locking
|
||||
|
||||
```dockerfile
|
||||
FROM quay.io/fedora/fedora-bootc:rawhide as builder
|
||||
|
||||
# Create custom repository configuration
|
||||
RUN mkdir -p /custom-repos/etc/yum.repos.d
|
||||
COPY custom-repos.repo /custom-repos/etc/yum.repos.d/
|
||||
|
||||
# Build rootfs with specific packages and versions
|
||||
RUN /usr/libexec/bootc-base-imagectl build-rootfs \
|
||||
--manifest=minimal \
|
||||
--install=kernel \
|
||||
--install=systemd \
|
||||
--lock=kernel-6.5.0-1.fc39.x86_64 \
|
||||
--lock=systemd-253-1.fc39.x86_64 \
|
||||
--cachedir=/var/cache/dnf \
|
||||
/custom-repos /target-rootfs
|
||||
|
||||
FROM scratch
|
||||
COPY --from=builder /target-rootfs/ /
|
||||
LABEL containers.bootc 1
|
||||
ENV container=oci
|
||||
STOPSIGNAL SIGRTMIN+3
|
||||
CMD ["/sbin/init"]
|
||||
```
|
||||
|
||||
### Example 3: Adding custom overlay content
|
||||
|
||||
```dockerfile
|
||||
FROM quay.io/fedora/fedora-bootc:rawhide as builder
|
||||
|
||||
# Prepare custom overlay content
|
||||
RUN mkdir -p /overlay/usr/local/bin
|
||||
RUN echo '#!/bin/bash\necho "Hello from custom overlay!"' > /overlay/usr/local/bin/custom-script
|
||||
RUN chmod +x /overlay/usr/local/bin/custom-script
|
||||
|
||||
# Build rootfs with overlay
|
||||
RUN /usr/libexec/bootc-base-imagectl build-rootfs \
|
||||
--manifest=default \
|
||||
--add-dir=/overlay \
|
||||
--install=cowsay \
|
||||
/ /target-rootfs
|
||||
|
||||
FROM scratch
|
||||
COPY --from=builder /target-rootfs/ /
|
||||
LABEL containers.bootc 1
|
||||
ENV container=oci
|
||||
STOPSIGNAL SIGRTMIN+3
|
||||
CMD ["/sbin/init"]
|
||||
```
|
||||
|
||||
### Example 4: Reproducible build with rechunking
|
||||
|
||||
```dockerfile
|
||||
# Build stage
|
||||
FROM quay.io/fedora/fedora-bootc:rawhide as builder
|
||||
RUN /usr/libexec/bootc-base-imagectl build-rootfs \
|
||||
--manifest=minimal \
|
||||
--install=kernel \
|
||||
--install=systemd \
|
||||
/ /target-rootfs
|
||||
|
||||
# Create intermediate image
|
||||
FROM scratch as intermediate
|
||||
COPY --from=builder /target-rootfs/ /
|
||||
LABEL containers.bootc 1
|
||||
ENV container=oci
|
||||
STOPSIGNAL SIGRTMIN+3
|
||||
CMD ["/sbin/init"]
|
||||
|
||||
# Rechunk for reproducibility
|
||||
FROM quay.io/fedora/fedora-bootc:rawhide as rechunker
|
||||
COPY --from=intermediate / /
|
||||
RUN /usr/libexec/bootc-base-imagectl rechunk \
|
||||
--max-layers=5 \
|
||||
localhost/intermediate:latest \
|
||||
localhost/final:latest
|
||||
|
||||
# Final image
|
||||
FROM localhost/final:latest
|
||||
Loading…
Add table
Add a link
Reference in a new issue