If a pipeline has an assembler and that assembler failed, the
overall status of the build also needs to be marked as failed.
This used to be the case, but a bug got introduced when the
format abstraction code was added.
This is, like the stage with the same name, an assembler that
will exit with an error code (default 255, but can be specified
via the assembler options). It is mostly useful for testing.
Set the "GRUB_CMDLINE_LINUX" variable in /etc/default/grub to the
kernel command line options. This is used by `grub2-mkconfig` to
assemble the full kernel command line when generating the menu
entires. NB: `GRUB_CMDLINE_LINUX` does NOT include the root fs
bits (`root=...`), since that is generated by grub2-mkconfig
itself.
Do the check if there is a /etc/machine-id before moving /etc to
/usr/etc, because otherwise /etc/machine-id will obviously not
exist and thus the detection is broken.
Instead of including SELinux labels for the content layers via the
`--selinux` tar option, make sure selinux labels are not included by
using the `--no-selinux` option.
The inclusion of the labels was a mistake, since they should be
determined by the target system because selinux labels are not
namespaced. On RHEL/Fedora the SELinux label used is something like
`system_u:object_r:container_ro_file_t:s0` for all the files in the
container.
Including the label was leading to permission problems because
the files had a different label on the host and programs inside
the container get `EACCES`, i.e. Permission denied, errors when
accessing files with the different label.
Interestingly this does not happen on Fedora 33 but only on RHEL.
One possibility is that the overlayfs kernel driver in RHEL is
behaving differently on RHEL than on Fedora.
The pylorax implementation of the template running code supports
globbing, as well as `--exclude` and `--optional` commands. These
are handled independently for each `installpkg` command, so that
requesting the installation of firmware packages in one command
together with an exclude of `*alsa` does indeed only exclude any
alsa firmware packages but not any other alsa packages. The
previous version of this script would just build an global list
for excludes which has a drastically different result because
a global exclude of "*alsa" would result in a global exclusion of
all matching packages and probably a dependency error.
Therefore, add support for dnf based filtering of packages on a
per `installpkg` command bases, very much like pylorax does.
Add a simple helper that is meant to gather the list of packages
to be installed via a lorax template that uses the "installpkg"
directives. A prominent example is the 'runtime-install.tmpl'
script from lorax-templates-generic, used to create boot isos.
This builds osbuild in F33aarch64 as well as RHEL8.4 and CentOS8. No tests are currently run.
With the mockbuilds in place we will be able to run composer CI against osbuild master, when necessary.
Extend the current ostree container test case to use the new
bootiso manifest to build a bootable iso. This is done in the
existing container test case in order to share the build root
and ostree commit results and speed up testing. In the future,
the test infrastructure should be extended so that the cache
can be optionally shared between test cases.
Port the org.osbuild.tar assembler to a new assembler like stage,
that takes a tree input. The only real change is that instead of
having a compression argument, the compression is now based on
the file ending.
Add support for the 'liveimg' kickstart command, which can be used
with tar payloads included in the installer image, to install
pre-built image archives.
This new manifests creates an boot iso which contains an ostree
commit that is build in the same manifest. The boot iso can then
be installed via the ISO.
New stage that uses the implantisomd5(1) to implant MD5 checksums
into an ISO. This is then used by a dracut module in the installer
ISO to check the installation medium.
Add a new stage that uses the `xorrisofs`(1) command line utility
to assemble a. The iso can be made bootable by specifying a
combination of the `boot` and `efi` options.
Add a new stage that prepares a bootable file system tree suitable
for writing to an ISO file system. It currently only supports
EFI and PC-BIOS boot. It takes a tree input which will be wrapped
into a ext4 file-system wrapped into a squashfs image.
Add a new stage that uses the recently added lorax template
helpers to execute such a template. The template itself will
be search in the build root, but the command of the script
will operate on the tree.
A new module that can parse and execute Lorax script templates,
which are mako template based files that support a limited set
of commands, like "install", "remove" and such.
The module provides helper functions to parse such templates
and execute them by providing a re-implementation of a subset
of the commands. All commands needed for running the post
installationtemplates were implemented.
Add a stage to create a buildstamp file, which is required by
anaconda to properly function, since it configures varies
aspects of the installation target (product, build arch, ...).
Add a stage to configure anaconda. For now only the enabled
kickstart modules can be configured. This is done by dropping
a file "90-osbuild.conf" in `/etc/anaconda/conf.d`.
There was a bug in mke2fs (fixed in versionv 1.45.7, with commit
6fa8edd0) where mkfs.ext4 would fail because the default config,
created on the fly, would contain a syntax error. The program
would abort with:
Syntax error in mke2fs config file (<default>, line #22)
Unknown code prof 17
To avoid this error, we try to bind mount the config from the build
root.
Since pipelines can now be uniquely addressed via their names,
add the ability to checkpoint via the pipeline name. This will
effectively checkpoint the last stage of a pipeline.
For format v1 manifests, the build pipeline is called "build",
the main pipeline is called "tree" and the pipeline for the
assembler is called "assembler".
This is to test the functionality of the new `saved_entry` grub2
config file. Ideally an integration test would install a new non-
default kernel and check that it does not get selected. Something
for the future.
The current grub configuration has no logic to select the default
boot entry, which means the next boot entry is chosen by grub2
by the sort order of available entries. This will break tooling
like grub2-set-default and grub2-reboot that set on a specific
variable `saved_entry` in the grub environment file `grubenv` to
to point to the select boot entry.
Change our grub configuration file to use that `saved_entry` var
to select the next boot entry to unbreak the tooling.
In addition to the aforementioned tools, the saved_entry is also
updated by the kernel install scripts when new kernel is installed,
but only if the newly installed kernel is the selected default.
Thus not respecting the `saved_entry` variable might cause a bug
where a newly installed non-default kernel gets selected by grub2.
Use the new immutable image infrastructure from `osbuild/containers`.
While at it, also switch over to the new github-actions helper, now that
we no longer run `systemd-nspawn` in our tests.
The old image was renamed from `ghci-osbuild` to `osbuild-ci` to avoid
accidentally replacing old images. The new infrastructure uses immutable
images, so downstream will no longer get automatic updates, unless the
`latest` tags are used.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
Although MPP supported having multiple repo entries for each
depsolve block, in reality that didn't actually work because
the same (global) baseurl was used for all packages.
Now the basurl can still be (otionally) specified globally,
but also overridden by the repo block and the package paths
will be relative to that.
In addition to the required base layer, provided via the the input
of the same name, the oci-archive stage now accepts up to nine
additional layers that get added on top of each other, sorted in
ascending order, i.e. `layer.1` to `layer.9`.
Adapt the `fedora-ostree-container` example manifest so that the
ostree commit is now in a separate layer, which makes it possible
to share the base layer between different commits container.
Add a new basic test for the dracut stage. It uses a osbuild
pipeline to build an initrd and inspects it via the initrd.py
module. The content is compared to a reference located in the
same directory as the pipeline (test/data/stages/dracut/).
Add a new module that contains a utilities to inspect an initrd.
It contains a `Initrd` class that loads a given initrd and can
then be used get the list of files via Initrd.filelist, the dracut
modules via `Initrd.modules` and the kernel modules via the
`Initrd.kmods` properties. Another top level function `read_initrd`
returns a dict that contains the name of the initrd as key and as
value another dictionary with `modules`, `kmods` as well as the
`compression` & `early_cpio` metadata.
The initrd.py can also use as a script that will print the dict
from `read_initrd` as JSON.
The implementation is a heavily based on dracut's `lsinitrd` bash
script and additionally contains a python port of the skipcpio.c
utility.
Should enable greater flexibility of how the initial ram disk is
created. Most importantly it runs dracut by default in no-host-
mode, which is the mode where dracut only includes the drivers
its needs for the currently running system. This mode obviously
does not make much sense in the installer.
The naming of the options follows the dracut nomenclature.
If the run of dracut has been disabled via installing a link to
/dev/null in /etc/kernel/install.d and after the installation of
the packages the /etc/kernel directory is not owned by any newly
installed packages, remove it again.
Add the ability to prevent `dracut` from running via the post
install scripts. This is done by installing symlinks named
like the dracut scripts in `/usr/lib/kernel/install.d/` in
`/etc/kernel/install.d`. They symlinks point to `/dev/null`
and since the ones in `/etc` takes precedent over the system
ones they effectively disable dracut from running.
After the run is completed the symlinks are removed and the
original files thus unmasked.
Change the test manifests that use containers to not include
docs when installing. Also don't install docs in the build root
for those manifests. Since the fedora-ostree-container.mpp is
being built in CI, this also tests that the new option.
The `org.osbuild.files` source provides files, but might in the
future not be the only one that does. Therefore rename it to
match the internal tool that is being used to fetch the files.
This is done for most other osbuild modules that target tools.
The format v1 loader is adapted to make this change transparent
for users of the v1 format, so we are backwards compatible.
Change the MPP depsolve preprocessor so that for format v2 based
manifest `org.osbuild.curl` source is used. Also rename the
corresponding source test. Adapt the format v2 mod test to use
the curl source.
