Wrap all calls to the various setup functions in the new exception
handler provided by `osbuild.api`. This will make sure that any
exception is properly printed to stderr, as well as communicated
to osbuild in a structured and machine readable way.
Now that the BuildRoot is capable of capturing the output of the
runner and modules (stages, assemblers), there is no need for
using `api.setup_stdio`. Therefore, drop it from all runners and
replace `api.output` with `BuildRoot.output`, which will contain
the output if `api.setup_stdio` is not called from the runners.
Runner are invoked to prepare the execution of stages and assemblers
inside the container. The setup tasks are specific to the distribution
and maybe the version of it, therefore specific runners are used for
each distribution+version combination.
The build the first (most nested) build root, `/usr` is taken from the
host to bootstrap the container. On RHEL, the python interpreter to be
used for software that belongs to the platform is platform-python, as
it provides a stable API. Therefore the RHEL runners should use that
instead of relying on the presence of /usr/bin/python3.6, which might
not be installed and is indeed not installed by default.
Drop the `osbuild -> ../osbuild` symlink from all module directories.
We now properly initialize the PYTHONPATH to provide the imported
osbuild module from the host environment. Therefore, these links are no
longer needed.
The sources run from the host environment, so they should just pick them
up from the environment the same way osbuild itself does.
We want to run stages and other scripts inside of the nspawn containers
we use to build pipelines. Since our pipelines are meant to be
self-contained, this should imply that the build-root must have osbuild
installed. However, this has not been the case so far for several
reasons including:
1. OSBuild is not packaged for all the build-roots we want to support
and thus we have the chicken-and-egg problem.
2. During testing and development, we want to support using a local
`libdir`.
3. We already provide an API to the container. Importing scripts from
the outside just makes this API bigger, but does not change the
fact that build-roots are not self-contained. Same is true for the
running kernel, and probably much more..
With all this in mind, our strategy probably still is to eventually
package osbuild for the build-root. This would significantly reduce our
API exposure, points-of-failure, and host-reliance. However, this switch
might still be some weeks out.
With this in mind, though, we can expect the ideal setup to have a full
osbuild available in the build-root. Hence, any script we import so far
should be able to access the entire `libdir`. This commit unifies the
libdir handling by installing the symlinks into `libdir` and providing
a single bind-mount of the module-path into `libdir`.
We can always decide to scratch that in the future when we scratch the
libdir-import from the host-root. Until then, I believe this commit
nicely unifies the way we import the module both in a local checkout as
well as in the container.
Now that stages no longer access the network, drop CA certificate
setup.
In the future, we may want to restrict all network access to the
container, but that requires more work.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This adds a new runner for Arch Linux. For now this simply links to the
blank linux runner, which works perfectly fine to bootstrap more
complex build pipelines.
Note that if we ever end up with more complex pipelines native to Arch
Linux, we might have to update this runner as well, since even on Arch
/etc must be pre-populated. Regardless, the blank linux runner serves
as a nice base and allows us to easily bootstrap osbuild on foreign
distros.
Now with `os-release` using `linux` as default ID+VERSION string, we
have a proper fallback name for our blank runner. Rename the blank
runner to `org.osbuild.linux`. It now serves as default fallback for
anything not further specified.
This adds a new runner called `org.osbuild.blank`, which assumes /usr
is pre-populated and ready to go. It does not perform any OS setup. It
only initializes the environment and executes the stage.
This runner allows easy bootstrapping of new systems. It assumes our
ideal setup where `/usr` describes a host system in its entirety,
without any local policy applied. Thus, this runner is also what we
ultimately want to work towards as a default. This might not happen
anytime soon, though, given how `passwd`, `ldconfig`, `nss`, etc. still
depend on prepopulated caches in `/etc`.
We've been using a generic `osbuild-run`, which sets up the build
environment (and works around bugs) for all build roots. It is already
getting unwieldy, because it tries to detect the OS for some things it
configures. It's also about to cause problems for RHEL, which doesn't
currently support a python3 shebang without having /etc around.
This patch changes the `build` key in a pipeline to not be a pipeline
itself, but an object with `runner` and `pipeline` keys. `pipeline` is
the build pipeline, as before. `runner` is the name of the runner to
use. Runners are programs in the `runners` subdirectory.
Three runners are included in this patch. They're copies of osbuild-run
for now (except some additions for rhel82). The idea is that each of
them only contains the minimal setup code necessary for an OS, and that
we can review what's needed when updating a build root.
Also modify the `--build-pipeline` command line switch to accept such a
build object (instead of a pipeline) and rename it accordingly, to
`--build-env`.
Correspondingly, `OSBUILD_TEST_BUILD_PIPELINE` → `OSBUILD_TEST_BUILD_ENV`.
