Commit graph

457 commits

Author SHA1 Message Date
Christian Kellner
662fe0feb9 test/host: checks for invalid fd handling
Check that we properly handle `dispatch` returning invalid fds and
that all fds are properly closed.
2021-09-24 12:14:04 +01:00
Christian Kellner
fe3bb30f4c test/host: add check for call with fds
Create a new test that checks method calls that pass file
descriptors in both directions.
2021-09-24 08:27:19 +01:00
Christian Kellner
c902a7a754 sources: port to host services
Port sources to also use the host services infrastructure that is
used by inputs, devices and mounts. Sources are a bit different
from the other services that they don't run for the duration of
the stage but are run before anything is built. By using the same
infrastructure we re-use the process management and inter process
communcation. Additionally, this will forward all messages from
sources to the existing monitoring framework.
Adapt all existing sources and tests.
2021-09-22 00:00:20 +02:00
Christian Kellner
99dd623383 stages: pam_limits.conf → pam.limits.conf
Mostly my personal preference, but we do not have any stage
with an underscore in the name, so let not start now.
2021-09-14 15:31:12 +02:00
Tomas Hozza
46ff53d5f4 Add a new stage for configuring pam_limits module
Add a new stage `org.osbuild.pam_limits.conf`, which created
configuration files for `pam_limits` module in /etc/security/limits.d.

Add unit test for the new stage.

Fix #788

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-09 15:00:30 +02:00
Christian Kellner
7ec305a343 ostree.config: add bootloader config option
Ability to set the bootloader backend that OSTree should use. NB:
normally this should be set to `none` since in modern distros and
bootloaders the BLS is used and the BLS snippets are generated on
`none` but none of the of the specific bootloader tools are run,
like `grub2-mkconfig` for grub.

Update the fedora image manifest to use that config setting.
2021-09-09 12:29:54 +02:00
Tomas Hozza
187681f17d Add a new stage for setting kernel parameters via sysctl.d
Add a new stage `org.osbuild.sysctld` for setting kernel parameters at
boot by creating a configuration file in /usr/lib/sysctl.d. At least
one parameter must be specified for the stage.

Add unit test for the new stage.

Fix #790

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-09 11:16:11 +02:00
Tomas Hozza
cf8f851f9c Add a new stage org.osbuild.tmpfilesd for configuring tmpfiles.d
Add a new stage `org.osbuild.tmpfilesd` for creating tmpfiles.d
configuration files in /usr/lib/tmpfiles.d. Provided list of
configuration directives is written as separate lines into
the configuration file. At least one configuration directive
must be specified.

Fix #786

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-08 14:08:40 +02:00
Tomas Hozza
8d85036689 Add a new stage for configuring SELinux state on the system
Add a new `org.osbuild.selinux.config` stage to configure the
SELinux state on the system.

The stage configures the SELinux state on the system in /etc/selinux/config.
The policy enforcement state and active policy type can be configured.

Fix #785

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-08 11:22:55 +02:00
Tomas Hozza
ac02244369 Add a new org.osbuild.dnf.config stage for configuring DNF
Add a new `org.osbuild.dnf.config` stage for changing persistent DNF
configuration. Currently only DNF variables can be defined.

Fix #791

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-07 14:10:09 +02:00
Tomas Hozza
1cab686f7a Add new org.osbuild.tuned stage for setting active TuneD profile
Add a new stage for setting active TuneD profile. The stage checks the
value of chosen TuneD profile(s) with the list of available TuneD
profiles installed in the filesystem root. If any of the chosen profiles
does not exist, the stage raises an exception.

Add unit tests for the new stage.

Fix #792

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-07 10:43:54 +02:00
Christian Kellner
82cc2aeb4e test/data: use cloud-init in ostree image
Although zezere is the official tool, cloud init is easier to use
for local testing so switch to that.
2021-08-28 16:34:37 +02:00
Christian Kellner
c788a74fb5 stages/ostree.deploy: add remote option
Add an optional `remote` to the deployment of the ostree so that
the it is tied to the specified remote. This is needed later for
updating the commit from that remote.
2021-08-28 16:34:37 +02:00
Christian Kellner
7ea9796d9b stages/ostree.pull: add remote parameter
Add an optional `remote` parameter that can be used to specify the
the remote to track for the pulled commits.
2021-08-28 16:34:37 +02:00
Christian Kellner
877f2ba3c3 stages/grub2: greenboot support
Greenboot is the idea of automatically rolling back bad updates,
i.e. updates that do not boot successfully. The implementation
is split between the boot loader and a user space component.
The latter sets two variables `boot_counter`, which indicates
the maximum number of boot attempts and `boot_success` which
tells the boot laoder if a previous boot was successful. The
bootloader on the other hand will decrement the counter variable
and reset the success indicator one.
An implementation of the user space component for rpm-ostree is
called `greenboot`.
2021-08-25 20:53:12 +02:00
Diaa Sami
b93dedf353 stages: add org.osbuild.gzip to compress files
Add a new stage that will take a file from the input and compress
it via gzip.
2021-08-24 20:47:00 +02:00
Christian Kellner
da8f45ef48 stages/grub2: add new uefi.unified option
Fedora 34 and thus RHEL 9 have adopted the unified grub config
scheme[1], where the main config is always placed in the same
location across all platforms, i.e. `boot/grub2`, and a stub
config that redirects to the main config is placed into the ESP.
osbuild has always done that in the case of hybrid boot, but not
for pure EFI systems. The new `uefi.unified` config option can
be used to select that new unified scheme even for the case of
pure EFI systems (aarch64 or non-hybrid boot).

Add a simple test for the grub stage.

[1] https://fedoraproject.org/wiki/Changes/UnifyGrubConfig
2021-08-23 13:55:32 +02:00
Martin Sehnoutka
8b0ea15817 stages: add org.osbuild.ostree.passwd
This stage takes /usr/lib/passwd and /usr/etc/passwd from an OSTree
checkout, merges them into one file, and store it as /etc/passwd in the
buildroot.

It does the same for /etc/group.

The reason for doing this is that there is an issue with unstable UIDs
and GIDs when creating OSTree commits from scratch. When there is a
package that creates a system user or a system group, it can change the
UID and GID of users and groups that are created later.

This is not a problem in traditional deployments because already created
users and groups never change their UIDs and GIDs, but with OSTree we
recreate the files from scratch and then replace the previous one so it
can actually change.

By copying the files to the build root before doing any other
operations, we can make sure that the UIDs and GIDs of already existing
users and groups won't change.

Co-author: Christian Kellner <christian@kellner.me>
2021-08-17 13:53:00 +02:00
Christian Kellner
dbfc04cbb0 test/data: use generic build for fedora-boot
Instead of using the version specific, pre-depsolved f34 build manifest,
use the new version agnostic build manifest (fedora-build.mpp). NB: this
is included directly as mpp so that its variables get defined by the
including manifest. This should make it even easier to update manifests
to new fedora releases.
2021-08-17 10:42:03 +02:00
Christian Kellner
bab3639731 test/data: add version agnostic build manifest
Include a build manifest that is itself not have tied to a specified
version and thus is meant to be included with the following vars
pre-defined as .mpp file:
  - arch          architecture (x86_64)
  - releasever    release version (f34)
  - snapshot      rpmrepo snapshot (20210326)
2021-08-17 10:42:03 +02:00
Christian Kellner
4126a3af7c test/loop: check for data integrity
Add a simple check that data written through the loop device is
actually ending up in the file. NB: this this will _fail_ if the
fd is cleared via `clear_fd` without the use of `flush_buf`. It
seems that the kernel (as of 5.13.8) will indeed not clear the
buffer cache of the loop device if the backing file is detached
via `LOOP_CLR_FD`. On the other hand, if the autoclear flag is,
i.e. the backing file cleared when the last file descriptor of
the loop device is closed, the buffer cached will be cleared as
part of the `release` operation of the block device.
2021-08-13 17:35:32 +02:00
Christian Kellner
8014ab5f1c test/data: use LVM in fedora-ostree-image
Now that we have support for LVM convert the ostree image manifest
to use it for the root partition.
2021-08-13 12:20:54 +02:00
Christian Kellner
c1c9c550d8 test/data: include lvm2 package in v2 build root
Needed to create lvm2 volume groups and logical volumes.
2021-08-13 12:20:54 +02:00
Christian Kellner
2789080d79 test/data: label build root for f34-build-v2
Properly label the build root for the f34 build root manifest v2.
Also label the cp and tar binaries with `install_exec_t` so they
can read and copy labels unknown to the host.
2021-08-13 12:20:54 +02:00
Christian Kellner
363fb88518 device: add org.osbuild.lvm2.lv 2021-08-13 12:20:54 +02:00
Christian Kellner
45d0594b1b device: add support for parent devices
This allows device nesting, i.e. one device being opened inside another
one.
2021-08-13 12:20:54 +02:00
Christian Kellner
6ea5ce1836 test: add rename check for lvm2 module
Check we can create and successfully rename a lvm2 volume group.
2021-08-13 12:20:54 +02:00
Thomas Lavocat
1b145701f4 test: convert to pytest test_noop.py
Split the tests and add verification for access to mount when necessary.
2021-08-12 09:34:33 +02:00
Christian Kellner
78bc042bae test/run: add devices test for loopback devices
Add a new devices run time test and add basic checks for the
org.osbuild.loopback device.
2021-08-11 20:59:51 +02:00
Christian Kellner
2af964a1d5 loop: support for locking via flock
Add support for locking the loopback block device via `flock(2)`.
The main use case for this is to prevent systemd-udevd from
proben the device while any modification is done to it. See the
systemd page, https://www.freedesktop.org/software/systemd, for
more details.
Add the corresponding tests to it.
2021-08-11 20:59:51 +02:00
Christian Kellner
d8e48c0511 loop: add clear_fd_wait method
Add a helper method that clears the fd for a given loop device but
also ensures that the loop device is not bound to the supplied fd
anymore. Check the function documentation for more information.
Add a corresponding test.
2021-08-11 20:59:51 +02:00
Christian Kellner
a367a0df1d loop: add Loop.is_bound_to helper
Add a `Loop.is_bound_to` helper that checks if the looback device is
bound if is so if the backing file refers to the same file as `fd`.
The latter is done by comparing the device and inode information.
2021-08-11 20:59:51 +02:00
Christian Kellner
d6c421faf3 loop: add LoopInfo.is_bound_to helper
Add a helper that will check if the loop devices is backed by
the file identified via the stat(2) result, i.e. the inode on
the correspoding device.
Add a correspoding test for the new helper.
2021-08-11 20:59:51 +02:00
Christian Kellner
61d5f15420 loop: add autoclear propert to LoopInfo
Small convenience property to check if the autoclear flag is set
in the `lo_flags` member of the `LoopInfo`.
Also add a corresponding test for it.
2021-08-11 20:59:51 +02:00
Christian Kellner
73f24c68a2 loop: add get_status method
Implement a `Loop.get_status` method, to get the properties of the
loop device, corresponding to LOOP_GET_STATUS64, and counterpart
to the existing `Loop.set_status` method. Use the new `get_status`
call in the `set_status` call, replacing the existing code that
does the same thing.
Add a basic test for the `get_status` method. Also fix an actual
leak, where the loop device was closed but the fd was not cleared
inside the test.
2021-08-11 20:59:51 +02:00
Christian Kellner
2c64c65608 test/assembler: use new loop_for_fd helper
Use the new `loop_for_fd` helper that got added to `LoopControl`,
which is basically a version of the existing code.
2021-08-11 20:59:51 +02:00
Christian Kellner
d5ded743bd test/mod: add checks for loop module
Add some basic checks for the loop module, including the new
`loop_for_fd` helper method.
2021-08-11 20:59:51 +02:00
Christian Kellner
569697e9a9 stages: add org.osbuild.untar
Add a new stage that can be used to extract a tarball.
2021-08-07 11:50:00 +02:00
Christian Kellner
0c9a54386b test/data: check for nm.conn's path property
Add a check for the new path property of `org.osbuild.nm.conn`.
2021-07-28 22:01:54 +01:00
Christian Kellner
6f7382b885 stages: add org.osbuild.nm.conf
New stage to create NetworkManager configuration files. Currently
only a subset of settings are supported. Include a simple stage
test case for it.
2021-07-28 22:01:54 +01:00
Christian Kellner
7f97f2c12e test/stages: new test for the tar stage
Include a simple test for the `tar` stage that does basic checks,
i.e. that a tarball was successfully created, but also checks
that we do not create entries with a `./` prefix.
2021-07-22 15:23:39 +02:00
Christian Kellner
e1df62fdc8 stages: add org.osbuild.nm.conn
Add a new stage to configure NetworkManger system connections.
Currently only ethernet connections are supported with a limited
set of options.
2021-07-22 15:18:33 +02:00
Christian Kellner
e0db66f624 stages/fstab: support device nodes and partlabel
For the fs spec field, support traditional device nodes as well
as partition labels. Adjust the test accordingly.
2021-07-22 12:05:02 +02:00
Christian Kellner
7fde451ecd stages: extract systemd.unit from systemd
Create a new `org.osbuild.systemd.unit` stage with the code for
create systemd drop-ins extracted from `org.osbuild.systemd`.
In contrast to the old code the new stage can only handle one
drop-in file. This makes the stage and its schema clearer
and simpler. If more than one config file is needed, the stage can
be repeated multiple times. It is also more flexible since we can
in the future specify additional options at the top level which
will then be per-file, like a top level comment.
The file is called `org.osbuild.systemd.unit` since in the future
it could also support creating unit files (not just drop-ins).
2021-07-22 01:05:29 +02:00
Christian Kellner
9a786ce4fc stages/logind: write only one drop-in file
Instead of supporting writing an arbitrary number of configuration
files, just write one. This makes the stage and its schema clearer
and simpler. If more than one config file is needed, the stage can
be repeated multiple times. It is also more flexible since we can
in the future specify additional options at the top level which
will then be per-file, like a top level comment.
2021-07-22 01:05:29 +02:00
Christian Kellner
79c6e65976 stages/logind: configuration_files → config
Rename the `configuration_files` key to `config` key.
2021-07-22 01:05:29 +02:00
Christian Kellner
9f1d97a8b7 stages/modprobe: write only one config file
Instead of supporting writing an arbitrary number of configuration
files, just write one. This makes the stage and its schema clearer
and simpler. If more than one config file is needed, the stage can
be repeated multiple times. It is also more flexible since we can
in the future specify additional options at the top level which
will then be per-file, like a top level comment.
2021-07-22 01:05:29 +02:00
Christian Kellner
0190c991ae stages/dracut: write only one config file
Instead of supporting writing an arbitrary number of configuration
files, just write one. This makes the stage and its schema clearer
and simpler. If more than one config file is needed, the stage can
be repeated multiple times. It is also more flexible since we can
in the future specify additional options at the top level which
will then be per-file, like a top-level comment.
2021-07-22 01:05:29 +02:00
Christian Kellner
a1703dc298 stages/cloud-init: write only one config file
Instead of supporting writing an arbitrary number of configuration
files, just write one. This makes the stage and its schema clearer
and simpler. If more than one config file is needed, the stage can
be repeated multiple times. It is also more flexible since we can
in the future specify the directory, `/etc` vs `/usr` via a new
top level `directory` key.
2021-07-22 01:05:29 +02:00
Christian Kellner
0becf66454 stages/dracut.conf: configuration_files → config
Rename the `configuration_files` key to `config` key.
2021-07-22 01:05:29 +02:00