bootc-docs/building/guidance.md

# Generic guidance for building Debian bootc images

The bootc project intends to be operating system and distribution independent as possible, similar to its related projects podman and systemd, etc.

The recommendations for creating bootc-compatible images for Debian will in general need to be owned by the Debian project - in particular those who create the default bootc base image(s). However, some guidance is very generic to most Linux systems (and bootc only supports Linux).

Let's however restate a base goal of this project:

> The original Docker container model of using "layers" to model applications has been extremely successful. This project aims to apply the same technique for bootable host systems - using standard OCI/Docker containers as a transport and delivery format for base operating system updates.

Every tool and technique for creating application base images should apply to the host Linux OS as much as possible.

## Understanding mutability

When run as a container (particularly as part of a build), bootc-compatible images have all parts of the filesystem (e.g. `/usr` in particular) as fully mutable state, and writing there is encouraged (see below).

When "deployed" to a physical or virtual machine, the container image files are read-only by default; for more, see filesystem.

**Important**: Do not manually create `/usr/etc` directories or copy files there. The `/usr/etc` tree is generated client-side by bootc/ostree and contains the default container image's view of `/etc`. Manually putting files into this location can create undefined behavior and will cause `bootc container lint` to fail.

## Installing software

For Debian package management using `apt`, it is very much expected that the pattern of

```dockerfile
RUN apt update && apt install -y somepackage && apt clean && rm -rf /var/lib/apt/lists/*
```

type flow Just Works here - the same way as it does for "application" container images. This pattern is really how Docker got started.

There's not much special to this that doesn't also apply to application containers; but see below.

### Nesting OCI containers in bootc containers

The OCI format uses "whiteouts" represented in the tar stream as special `.wh` files, and typically consumed by the Linux kernel `overlayfs` driver as special `0:0` character devices. Without special work, whiteouts cannot be nested.

Hence, an invocation like

```dockerfile
RUN podman pull quay.io/exampleimage/someimage
```

will create problems, as the `podman` runtime will create whiteout files inside the container image filesystem itself.

Special care and code changes will need to be made to container runtimes to support such nesting. Some more discussion in this tracker issue.

## systemd units

The model that is most popular with the Docker/OCI world is "microservice" style containers with the application as pid 1, isolating the applications from each other and from the host system - as opposed to "system containers" which run an init system like systemd, typically also SSH and often multiple logical "application" components as part of the same container.

The bootc project generally expects systemd as pid 1, and if you embed software in your derived image, the default would then be that that software is initially launched via a systemd unit.

```dockerfile
RUN apt update && apt install -y postgresql && apt clean && rm -rf /var/lib/apt/lists/*
```

Would typically also carry a systemd unit, and that service will be launched the same way as it would on a package-based system.

## Users and groups

Note that the above `postgresql` today will allocate a user; this leads to the topic of users, groups and SSH keys.

## Configuration

A key aspect of choosing a bootc-based operating system model is that _code_ and _configuration_ can be strictly "lifecycle bound" together in exactly the same way.

(Today, that's by including the configuration into the base container image; however a future enhancement for bootc will also support dynamically-injected ConfigMaps, similar to kubelet)

You can add configuration files to the same places they're expected by typical package systems on Debian - in `/usr` (preferred where possible) or `/etc`. systemd has long advocated and supported a model where `/usr` (e.g. `/usr/lib/systemd/system`) contains content owned by the operating system image.

`/etc` is machine-local state. However, per filesystem.md it's important to note that the underlying OSTree system performs a 3-way merge of `/etc`, so changes you make in the container image to e.g. `/etc/postgresql/postgresql.conf` will be applied on update, assuming it is not modified locally.

**Important**: When building bootc images, work with `/etc` normally during the build process. Do not manually create or manipulate `/usr/etc` - this is handled automatically by bootc/ostree during deployment.

### Prefer using drop-in directories

These "locally modified" files can be a source of state drift. The best pattern to use is "drop-in" directories that are merged dynamically by the relevant software. systemd supports this comprehensively; see drop-ins for example in units.

And instead of modifying `/etc/sudoers`, it's best practice to add a file into `/etc/sudoers.d` for example.

Not all software supports this, however; and this is why there is generic support for `/etc`.

### Configuration in /usr vs /etc

Some software supports generic configuration both `/usr` and `/etc` - systemd, among others. Because bootc supports _derivation_ (the way OCI containers work) - it is supported and encouraged to put configuration files in `/usr` (instead of `/etc`) where possible, because then the state is consistently immutable.

One pattern is to replace a configuration file like `/etc/postgresql/postgresql.conf` with a symlink to e.g. `/usr/postgres/etc/postgresql.conf` for example, although this can run afoul of SELinux labeling.

### Secrets

There is a dedicated document for secrets, which is a special case of configuration.

## Handling read-only vs writable locations

The high level pattern for bootc systems is summarized again this way:

* Put read-only data and executables in `/usr`
* Put configuration files in `/usr` (if they're static), or `/etc` if they need to be machine-local
* Put "data" (log files, databases, etc.) underneath `/var`

However, some software installs to `/opt/examplepkg` or another location outside of `/usr`, and may include all three types of data underneath its single toplevel directory. For example, it may write log files to `/opt/examplepkg/logs`. A simple way to handle this is to change the directories that need to be writable to symbolic links to `/var`:

```dockerfile
RUN apt update && apt install -y examplepkg && apt clean && rm -rf /var/lib/apt/lists/* && \
    mv /opt/examplepkg/logs /var/log/examplepkg && \
    ln -sr /opt/examplepkg/logs /var/log/examplepkg
```

The Debian bootc puppet example is one instance of this.

Another option is to configure the systemd unit launching the service to do these mounts dynamically via e.g.

```
BindPaths=/var/log/exampleapp:/opt/exampleapp/logs
```

## Building bootc Images Correctly

### The Right Way to Build bootc Images

When building bootc images, follow these principles:

1. **Work with `/etc` normally**: Configure files in `/etc` during the build process as you would in any container
2. **Don't manually create `/usr/etc`**: This directory is managed automatically by bootc/ostree
3. **Use `bootc container lint`**: Always run this to validate your image before finalizing
4. **Let bootc handle the transformation**: The `/etc` to `/usr/etc` conversion happens during deployment, not during build

### Common Mistakes to Avoid

- ❌ **Don't do this**: Manually creating `/usr/etc` and copying files there
- ❌ **Don't do this**: Removing `/etc` and trying to recreate it manually
- ❌ **Don't do this**: Assuming you need to handle `/etc` normalization yourself

### Correct Build Pattern

```dockerfile
FROM debian:bookworm-slim

# Install packages normally
RUN apt update && apt install -y your-packages && apt clean

# Configure /etc files normally
RUN echo "config" > /etc/myapp/config.conf
RUN systemctl enable myapp

# Let bootc validate everything
RUN bootc container lint

LABEL containers.bootc 1
LABEL ostree.bootable 1
```

## Debian-Specific Considerations

### Package Management Integration

When building Debian bootc images, consider:

- **apt repositories**: Ensure your base image includes the necessary Debian repositories
- **Package selection**: Choose packages that are compatible with the bootc model
- **Dependencies**: Handle Debian package dependencies properly in your Dockerfile
- **Security updates**: Plan for how security updates will be handled through the bootc model

### Debian Configuration Patterns

- **dpkg configuration**: Use `debconf` for non-interactive package configuration
- **Service management**: Leverage Debian's systemd integration
- **User management**: Follow Debian conventions for user and group creation
- **Logging**: Use Debian's standard logging locations in `/var/log`

### Example Dockerfile for Debian bootc

```dockerfile
FROM debian:bookworm-slim

# Install bootc and dependencies
RUN apt update && \
    apt install -y bootc ostree podman systemd && \
    apt clean && \
    rm -rf /var/lib/apt/lists/*

# Install your application packages
RUN apt update && \
    apt install -y nginx postgresql && \
    apt clean && \
    rm -rf /var/lib/apt/lists/*

# Configure services normally - bootc handles /etc automatically
RUN systemctl enable nginx postgresql

# Configure /etc files normally - don't manually create /usr/etc
RUN echo "server_name example.com;" > /etc/nginx/conf.d/default.conf

# Set up proper permissions and links
RUN ln -sf /var/log/nginx /usr/share/nginx/logs

# Let bootc lint check everything is correct
RUN bootc container lint

LABEL containers.bootc 1
LABEL ostree.bootable 1
```

---

The Linux Foundation® (TLF) has registered trademarks and uses trademarks. For a list of TLF trademarks, see Trademark Usage.