The `org.osbuild.chrony` stage currently supports only a single option
'timeservers' which does not allow specifying additional options for the
configured timeservers. The option can not be easily extended to allow
specifying additional options and at the same time keep the backward
compatibility with old manifests.
The need for a lower-level stage option allowing to configure additional
options of the 'server' directive is required by RHEL AMI images, which
use 'maxpoll', 'minpoll' and 'prefered' options.
Extend the `org.osbuild.chrony` stage with two additional options. The
'servers' option accepts a list of dictionaries specifying timeservers
to be configured using the 'server' directive, including a subset of the
directive options. The 'servers' option can not be used at the same time
in the stage options as the 'timeservers' option.
The second added option is 'leapsectz' corresponding with a directive of
the same name. The value of the option is a string. If the provided
string is empty, then all occurrences of the 'leapsectz' directive are
removed from the `chrony.conf`. Otherwise, the 'leapsectz' directive is
added to the `chrony.conf` with the provided value, while all original
occurrences of the option are removed.
Add a new stage test case under `test/data/stages/chrony-servers` to
test the new variant of the stage options. The reason is that the
'timeservers' option conflicts with 'servers' option, which makes it
impossible to test both of them in the same test case.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add new `org.osbuild.cloud-init` stage, which currently allows to create
configuration files for cloud-init under `/etc/cloud/cloud.cfg.d`. The
stage supports only a very limited subset of cloud-init configuration
options, which is covering needs of RHEL AMI images.
The schema mandates that if the 'configuration_files' option is
specified, then at least one configuration file must be defined. In
addition each section of the configuration must contain at least one
property (section or configuration option).
Add `python3-pyyaml` package to the `F34-build` testing manifest,
because it is required for running and testing the new stage.
Regenerate all affected manifests.
Add test for the new stage.
Update the `osbuild-ci` container image used for testing to a new tag,
which includes python3-pyyaml, the dependency of the new stage.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add new `org.osbuild.dracut.conf` stage. The option
'configuration_files' allows to create dracut configuration files under
`/usr/lib/dracut/dracut.conf.d` and thus make the
configuration persistent. The stage supports only a subset of all
configuration options allowed in dracut configuration. The intention is
to provide almost functional parity with the options supported by
`org.osbuild.dracut` stage.
The schema mandates that at least one configuration file must be defined
in the stage options. In addition, each configuration file must contain
at least one configuration option.
Add test for the new stage.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Use the `--no-tell-kernel` option to avoid invoking the `ioctl` call to
tell the kernel to reload the partition table. Since we are not using
the kernel to access the partitions this introduces unnecessary i/o and
might also lead to spurious warnings when a partition without dos label
is written, like:
Re-reading the partition table failed.: Invalid argument
Instead of operating directly on a file, which was previously specified
by `filename`, operate on a device. This is more flexible since a file
can be accessed via a loop back device; but the inverse is obviously
not true, like other devices can not be accessed via a plain file.
Therefore, re-factor the stage to use a device and adapt the existing
test (`fedora-ostree-image`).
This was left over from the old qemu assembler and is not really
needed anymore. Also removes some defaults that are not valid
according to the new schema of the stage.
Add new stage `org.osbuild.systemd-logind` allowing to create
systemd-logind configuration drop-ins in `/usr/lib/systemd/logind.conf.d`.
Currently only the `NAutoVTs` option in the `Login` section can be
configured.
The schema mandates that:
- There must be at least one configuration file defined.
- The 'Login' section is required, as it is the only one in the
systemd-logind configuration.
- At least one option must be configured in the 'Login' section.
Add test for the new stage.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add new stage `org.osbuild.modprobe` allowing to create modprobe
configuration files in `/usr/lib/modprobe.d`. Currently only the
`blacklist` command can be used in the configuration files.
The schema mandates, that at least one configuration file must be
defined.
Add test for the new stage.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The `OSBUILD_QEMU_IMG_COROUTINES` was introduced to allow specifying
the number of coroutines used in `qemu-img convert` by the runner,
or osbuild directly. This can be useful in various scenarios, but
is specifically used by the rhel 8.2+ runner to limit the number of
coroutines used for Aarch64 to one, since a bug in `qemu-img` leads
to random hangs on that platform.
Extend the `org.osbuild.sysconfig` stage to create `ifcfg-*` files
under `network-scripts` subdirectory. It is possible to set only values
currently set in RHEL AMI images, specifically:
- BOOTPROTO
- DEVICE
- IPV6INIT
- ONBOOT
- PEERDNS
- TYPE
- USERCTL
Change all `configure_*` functions to raise ValueError exception,
instead of returning values. As a follow up change, remove all checks of
the returned value from these functions.
Update the `org.osbuild.sysconfig` stage test case to create ifcfg
configuration files for two interfaces.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Extend the `org.osbuild.systemd` stage to create drop-in configuration
files for Systemd `.service` units under `/usr/lib/systemd/system`.
Currently only the `Environment` option in the `Service` section can be
configured.
Update the `org.osbuild.systemd` stage test case to create drop-in
configuration `10-rh-enable-for-ec2.conf` for `nm-cloud-setup.service`
unit, as used in RHEL AMI images.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Extend the `org.osbuild.rhsm` stage to configure selected options in the
subscription-manager configuration (in `/etc/rhsm/rhsm.conf`). It is
possible to set only values currently set in RHEL AMI images,
specifically:
- `manage_repos` option in `rhsm` section
- `auto_registration` option in `rhsmcertd` section
Ensure that the stage does not "touch" any configuration files, unless
it actually changes them. This prevents changing the file modification
time.
Update the `org.osbuild.rhsm` stage test case to set the additional
configuration options.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Use `patternProperties` instead of `propertyNames` and `pattern`,
which is not in draft 4 and so did not work (but also did not
throw an error).
Co-Developed-by: Achilleas Koutsou <achilleas@koutsou.net>
The stage was converted to use inputs, but its schema was not, which
means that although the stage requires inputs, they could not be
specified. Doh. Change the expected input to `commit`.
NB: This stage should be broken up, so *SHOULD NOT* be used in newly
created pipelines.
Fix a small whitespace change as well.
Based on that part of the qemu assembler that converts the raw image
into different virtualization formats, like qcow2 and such. Supports
all the formats the old qemu assembler also supported.
The `org.osbuild.files` input provides individual files to a stage.
Change the `refs` key in the returned dict to `files` to better
reflect that fact. Also adapt the documentation to indicate that
the keys actually paths and not necessarily checksums. This prepares
for future extension of the `files` input to pipeline origins.
This stage is the part of the qemu assembler that generates and
installs the grub2 core image on non-uefi or hybrid systems,
like x86 legacy and PPC64LE (Open Firmware).
This sage can be used to copy items, such as files or trees, from one
location to another. The only supported location for reading currently
is currently `input`. Supported locations for writing are `mount` and
`tree`.
Disable host-only mode when running dracut and generate reproducible
images by default.
Suggested-by: gicmo
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Define a set of pre-defined ostree related annotations that can
and should be used to indicate that a container image contains
an OSTree commit. This can be used by other tools to inspect and
extract the commit more easily.
Add support for arbitrary manifest annotations: allow anything
with the exception of the `org.osbuild` and `org.opencontainer`
prefixes. The former is reserved by us, the latter by the OCI
image specification. The latter specifies a set of pre defined
keys, which are not yet supported by osbuild but will be in the
future, partly via more generic options (creation time).
According to the OCI Image Format Specification[1] history entries
for layers in the container are optional; but when trying to push
a container quay.io via skopeo (copy oci-archive:… docker://quay)
it will fail with "Cannot convert an image with 0 history entries".
This seems to come from the containers/image[2] library when the
container is converted back from the docker distribution format
to oci-archive on quay.io. Thus it seems that when skopeo converts
the image to the docker format for the distribution it does not
fill any the history entries, which are then assumed and required
to be there when converting back.
To fix this, insert history entries for each layer that is created.
[1] https://github.com/opencontainers/image-spec/blob/master/config.md
[2] https://github.com/containers/image/
Only include a very specific set of extended attributes:
- user.*: user specified extended attributes
- security.ima: Integrity Measurement Architecture (IMA)
- security.capability: Linux capabilities(7)
This follows what containers/storage[1] and containers/buildah[2]
are doing. It is important to note that we DO NOT want selinux
related extended attributes (`security.selinux`) in there, which
seems to be pulled in by some versions of `tar` even if that was
seemingly excluded via `--no-selinux`. Therefore we also exclude
selinux and xattrs explicitly from the wrapping container to
make sure they are never included.
[1] 35ebda8ae2/pkg/archive/archive.go (L399)
[2] 214e4c9335/copier/xattrs.go (L19)
This reverts commit 59184b23a2.
This change breaks current testing and is not critical.
We will reintroduce it later when there is time to adapt the tests.
Instead of deleting and re-creating /etc/machine-id, just truncate
it to an empty file. This should let the mode be 0444, which is
the mode that systemd also creates it with.
In order to have a more stable package metadata representation,
sort the generated metadata by name. Adapt the tests' metadata
file to reflect that change.
Since `/home` will not end up in the commit¹ move the home
directories to `/var/home`. This is done after the new root
file system has been initialized, and only if `/home` is not
empty.
¹ it is neither copied back in the preptree stage itself, nor
would it be picked up by rpm-ostree compose tree postprocess
were it copied back.
Set the "GRUB_CMDLINE_LINUX" variable in /etc/default/grub to the
kernel command line options. This is used by `grub2-mkconfig` to
assemble the full kernel command line when generating the menu
entires. NB: `GRUB_CMDLINE_LINUX` does NOT include the root fs
bits (`root=...`), since that is generated by grub2-mkconfig
itself.
Do the check if there is a /etc/machine-id before moving /etc to
/usr/etc, because otherwise /etc/machine-id will obviously not
exist and thus the detection is broken.
Instead of including SELinux labels for the content layers via the
`--selinux` tar option, make sure selinux labels are not included by
using the `--no-selinux` option.
The inclusion of the labels was a mistake, since they should be
determined by the target system because selinux labels are not
namespaced. On RHEL/Fedora the SELinux label used is something like
`system_u:object_r:container_ro_file_t:s0` for all the files in the
container.
Including the label was leading to permission problems because
the files had a different label on the host and programs inside
the container get `EACCES`, i.e. Permission denied, errors when
accessing files with the different label.
Interestingly this does not happen on Fedora 33 but only on RHEL.
One possibility is that the overlayfs kernel driver in RHEL is
behaving differently on RHEL than on Fedora.
Port the org.osbuild.tar assembler to a new assembler like stage,
that takes a tree input. The only real change is that instead of
having a compression argument, the compression is now based on
the file ending.
Add support for the 'liveimg' kickstart command, which can be used
with tar payloads included in the installer image, to install
pre-built image archives.
New stage that uses the implantisomd5(1) to implant MD5 checksums
into an ISO. This is then used by a dracut module in the installer
ISO to check the installation medium.
Add a new stage that uses the `xorrisofs`(1) command line utility
to assemble a. The iso can be made bootable by specifying a
combination of the `boot` and `efi` options.
Add a new stage that prepares a bootable file system tree suitable
for writing to an ISO file system. It currently only supports
EFI and PC-BIOS boot. It takes a tree input which will be wrapped
into a ext4 file-system wrapped into a squashfs image.
Add a new stage that uses the recently added lorax template
helpers to execute such a template. The template itself will
be search in the build root, but the command of the script
will operate on the tree.
