Rewrite image pre-processor to single tool so that it is easier to use.
Now also supports `ignore-weak-deps` when dep-solving and supports
relative paths for local files.
Also create a symlink to the osbuild package, so that the tools can be
run from the source checkout and have access to the osbuild package.
In the function `treeid_from_manifest`, use dynamic format detection
instead of hard-coding the use of the version 1 format. Additionally,
directly access the `tree` pipeline and its `id` instead of going
via the `get_ids` helper function, which is only present in v1.py.
Add a new `org.osbuild.authselect` stage for configuring system identity
and authentication sources using `authselect`. The stage runs
`authselect select` command from the image in a chroot to set the
desired 'profile_id' profile. Optionally, a list of specific profile
features to enable can be passed using the 'features' option.
Add a test case for the new stage.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add a new option 'x11-keymap' for configuring the X11 keyboard settings.
The value of the option is a dictionary with keys representing settings
which can be configured. Currently, only the 'layouts' setting can be
configured. Its value is a list of strings, representing the specific
layouts, which should be configured for the X11 keyboard.
Update the stage test case with the new option 'x11-keymap'.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Checks if one path is a child of a second one. Useful for checking if
paths defined in a manifest exist inside the tree.
Optionally checks if the target path exists.
This will prevent any modification of anything in `/sys`. It will
also prevent `udevadm tigger` to run, which needs /sys writeable.
This is a desired effect, since uevents are not delivered to the
contained environment, so `udevadm trigger` might hang.
The `org.osbuild.chrony` stage currently supports only a single option
'timeservers' which does not allow specifying additional options for the
configured timeservers. The option can not be easily extended to allow
specifying additional options and at the same time keep the backward
compatibility with old manifests.
The need for a lower-level stage option allowing to configure additional
options of the 'server' directive is required by RHEL AMI images, which
use 'maxpoll', 'minpoll' and 'prefered' options.
Extend the `org.osbuild.chrony` stage with two additional options. The
'servers' option accepts a list of dictionaries specifying timeservers
to be configured using the 'server' directive, including a subset of the
directive options. The 'servers' option can not be used at the same time
in the stage options as the 'timeservers' option.
The second added option is 'leapsectz' corresponding with a directive of
the same name. The value of the option is a string. If the provided
string is empty, then all occurrences of the 'leapsectz' directive are
removed from the `chrony.conf`. Otherwise, the 'leapsectz' directive is
added to the `chrony.conf` with the provided value, while all original
occurrences of the option are removed.
Add a new stage test case under `test/data/stages/chrony-servers` to
test the new variant of the stage options. The reason is that the
'timeservers' option conflicts with 'servers' option, which makes it
impossible to test both of them in the same test case.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add new `org.osbuild.cloud-init` stage, which currently allows to create
configuration files for cloud-init under `/etc/cloud/cloud.cfg.d`. The
stage supports only a very limited subset of cloud-init configuration
options, which is covering needs of RHEL AMI images.
The schema mandates that if the 'configuration_files' option is
specified, then at least one configuration file must be defined. In
addition each section of the configuration must contain at least one
property (section or configuration option).
Add `python3-pyyaml` package to the `F34-build` testing manifest,
because it is required for running and testing the new stage.
Regenerate all affected manifests.
Add test for the new stage.
Update the `osbuild-ci` container image used for testing to a new tag,
which includes python3-pyyaml, the dependency of the new stage.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add new `org.osbuild.dracut.conf` stage. The option
'configuration_files' allows to create dracut configuration files under
`/usr/lib/dracut/dracut.conf.d` and thus make the
configuration persistent. The stage supports only a subset of all
configuration options allowed in dracut configuration. The intention is
to provide almost functional parity with the options supported by
`org.osbuild.dracut` stage.
The schema mandates that at least one configuration file must be defined
in the stage options. In addition, each configuration file must contain
at least one configuration option.
Add test for the new stage.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
In the object store, temporary bind mounts are used when accessing the
content, i.e. the individual trees. Their unmount is currently done
with the `--lazy` flag. The use of this flag goes way back to commit
da121beda1, which sadly does not mention
why the flag was introduced. Since the tree and files in the tree will
be used by consequent stages it seems reasonable to do the un-mounting
eagerly and thus this reverts back to that behavior.
The logic to sort urls was added globally in `mpp-import-pipeline`
but only the in the v1 code path was the `state.manifest_urls`
variable set and thus for v2 the actual sorting did not happen.
Fix this and set the `manifest_urls` to the `org.osbuild.curl`
items, which makes sense because we only know how to sort those.
For `setfiles` in the `org.osbuild.selinux` to work properly it
needs itself have the correct labels. This is true for setfiles
on the host, but also needs to be true for the `setfiles` that
is located in the build root. Therefore we need to label the
build root via `org.osbuild.selinux`. Additionally re-label
the `cp` and `tar` binary to have the `install_exec_t` which is
needed to read and write unknown labels (to the host). Since
`cp` and `tar` are used to read (and write) files inside stages
and assemblers they need to have the special label.
The commit that added the ostree tarball to manifest version 2
went in after the PR to sort the urls and thus the source urls
for that manifest were not sorted. This of course no breaks ci
which makes sure that the test data up to date. I blame the
ci model used by github but I am also sorry.
In the ostree assembler, `var`, `usr` and `boot` are copied from
the built tree to a newly initialized and ostree-conforming root
filesystem. The way in which `cp` was called resulted in the
source being created inside the target, if the latter existed.
This was the case for `var` resulting in `var/var`.
Use `cp ${source}/. {target}` to fix that.
Reported-by: Luca Bruno <luca.bruno@coreos.com>
Convert the test to use `pytest` and split out the individual
tests. The temp-directory fixture has the session scope so
that checkpoints can be shared between the individual tests.
Remove the dependency on unittest for the `OSBuild` class which
used the `unittest` instance only for `assertEqual`, which can
easily also be done via a plain `assert`.
In both mpp-depsolve and mpp-import-pipeline, sort the packages to
url dictionary before writing the JSON. This makes it easier to
look for packages but more importantly ensures that the resulting
set of packages has the same ordering in the sources section
independently of how it was assembled.
Use the `--no-tell-kernel` option to avoid invoking the `ioctl` call to
tell the kernel to reload the partition table. Since we are not using
the kernel to access the partitions this introduces unnecessary i/o and
might also lead to spurious warnings when a partition without dos label
is written, like:
Re-reading the partition table failed.: Invalid argument
Instead of operating directly on a file, which was previously specified
by `filename`, operate on a device. This is more flexible since a file
can be accessed via a loop back device; but the inverse is obviously
not true, like other devices can not be accessed via a plain file.
Therefore, re-factor the stage to use a device and adapt the existing
test (`fedora-ostree-image`).
This was left over from the old qemu assembler and is not really
needed anymore. Also removes some defaults that are not valid
according to the new schema of the stage.
Add new stage `org.osbuild.systemd-logind` allowing to create
systemd-logind configuration drop-ins in `/usr/lib/systemd/logind.conf.d`.
Currently only the `NAutoVTs` option in the `Login` section can be
configured.
The schema mandates that:
- There must be at least one configuration file defined.
- The 'Login' section is required, as it is the only one in the
systemd-logind configuration.
- At least one option must be configured in the 'Login' section.
Add test for the new stage.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add new stage `org.osbuild.modprobe` allowing to create modprobe
configuration files in `/usr/lib/modprobe.d`. Currently only the
`blacklist` command can be used in the configuration files.
The schema mandates, that at least one configuration file must be
defined.
Add test for the new stage.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The `OSBUILD_QEMU_IMG_COROUTINES` was introduced to allow specifying
the number of coroutines used in `qemu-img convert` by the runner,
or osbuild directly. This can be useful in various scenarios, but
is specifically used by the rhel 8.2+ runner to limit the number of
coroutines used for Aarch64 to one, since a bug in `qemu-img` leads
to random hangs on that platform.
Extend the `org.osbuild.sysconfig` stage to create `ifcfg-*` files
under `network-scripts` subdirectory. It is possible to set only values
currently set in RHEL AMI images, specifically:
- BOOTPROTO
- DEVICE
- IPV6INIT
- ONBOOT
- PEERDNS
- TYPE
- USERCTL
Change all `configure_*` functions to raise ValueError exception,
instead of returning values. As a follow up change, remove all checks of
the returned value from these functions.
Update the `org.osbuild.sysconfig` stage test case to create ifcfg
configuration files for two interfaces.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Extend the `org.osbuild.systemd` stage to create drop-in configuration
files for Systemd `.service` units under `/usr/lib/systemd/system`.
Currently only the `Environment` option in the `Service` section can be
configured.
Update the `org.osbuild.systemd` stage test case to create drop-in
configuration `10-rh-enable-for-ec2.conf` for `nm-cloud-setup.service`
unit, as used in RHEL AMI images.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
