Currently we always write the kernel command line to the `grubenv`
file, if only to include the root device. Starting with Fedora 33
and thus RHEL 9, the kernel command line included statically in
the BLS snippets and the grubenv `kernelopts` variable not used.
Instead one of the {/usr/lib,/etc}/kernel/cmdline files is read
and the parameters in them used during the creation of the BLS
snippets.
Therefore we add a new `write_cmdline` option that, if set to
FALSE, will prevent us from writing the kernel command line.
The sysconfig stage currently does not produce expected results when
used multiple times within the same pipeline. Specifically, the stage
always truncates respective configuration files for properties `kernel`
and `network`, if if these are not set in the stage options. Due to this
reason, the outcome of the image builds may depend on the order of
multiple occurrences of the sysconfig stage.
The following two pipeline snippets would produce different
configuration files content:
Configuration files are truncated:
```
{
"type": "org.osbuild.sysconfig",
"options": {
"kernel": {
"update_default": true,
"default_kernel": "kernel"
},
"network": {
"networking": true,
"no_zero_conf": true
}
}
},
{
"type": "org.osbuild.sysconfig",
"options": {
"network-scripts": {
"ifcfg": {
"eth0": {
"bootproto": "dhcp",
"device": "eth0",
"ipv6init": false,
"onboot": true,
"peerdns": true,
"type": "Ethernet",
"userctl": true
}
}
}
}
},
```
No configuration files are truncated:
```
{
"type": "org.osbuild.sysconfig",
"options": {
"network-scripts": {
"ifcfg": {
"eth0": {
"bootproto": "dhcp",
"device": "eth0",
"ipv6init": false,
"onboot": true,
"peerdns": true,
"type": "Ethernet",
"userctl": true
}
}
}
}
},
{
"type": "org.osbuild.sysconfig",
"options": {
"kernel": {
"update_default": true,
"default_kernel": "kernel"
},
"network": {
"networking": true,
"no_zero_conf": true
}
}
},
```
Change the stage to not touch respective configuration files if the
`kernel` and `network` properties are not set in the stage options.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The treesum of a filesystem tree is the content hash of all its
files, its directory structure and file metadata.
By storing trees by their treesum we avoid storing duplicates of
identical trees, at the cost of computing the hashes for every
commit to the store.
This has limited benefit as the likelihood of two trees being
identical is slim, in particular when we already have the ability
to cache based on pipeline/stage ID (i.e., we can avoid rebuilding
trees if the pipelines that built them were the same).
Drop the concept of a treesum entirely, even though I very much
liked the idea in theory...
Signed-off-by: Tom Gundersen <teg@jklm.no>
Pacman is the default package manager for Arch Linux and derivates, the
pacman.conf stage generate a valid pacman.conf configuration file.
Co-Authored-By: Jelle van der Waa <jvanderwaa@redhat.com>
This introduces a new dependency resolver to osbuild-mpp for Arch Linux
which uses the pacman package manager. The used solver is determined by
the `solver` field in the `mpp-depsolve` object inside the manifest
file, if it does not exists it falls back to the DepSolver for dnf/rpm.
Co-Authored-By: Jelle van der Waa <jvanderwaa@redhat.com>
Add a new stage `org.osbuild.dnf-automatic.config` for configuring DNF
Automatic.
The stage changes persistent DNF Automatic configuration. Currently, only
a subset of options can be set:
- 'commands' section
- apply_updates
- upgrade_type
Fix#908
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add a new stage `org.osbuild.yum.repos` for creating YUM / DNF `.repo`
files in `/etc/yum.repos.d`. All repo-specific options are supported but
only a subset of options which can be set for a repo as well as in the
[main] section are supported.
Add unit test for the new stage.
Fix#907
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Set the container environment variable to indicate to programs
inside the build root that they are indeed running inside a
container (see also https://systemd.io/CONTAINER_INTERFACE/).
Create a well-defined environment with and use that for the build
root. It is not desirable to have the host's environment leak
into the container. Add a test to ensure that this works.
NB: This was probably an oversight when we switched from systemd-
nspawn to bubblewrap.
Introduce two new command line arguments, which can be used to
specify which monitor class to use (`--monitor`) and what file
descriptor to use for monitoring (`--monitor-fd`). The latter
defaults to standard out. The monitor class, if not specified,
is depended on the `--json` argument.
Wrap the LVM volume group in a LUKS container with the passphrase
`osbuild` (yes, really, super secure). NB: the kernel command line
is changed to include `luks.uuid` which is needed so that dracut
will attempt to open the luks container. This corresponds to an
crypttab entry `luks-uuid UUID`. We cannot use the /etc/crypttab
for ostree based images because the initrd is created at commit
time but they luks volume is created at deployment time, we have
to use the kernel command line instead. See the man page for the
systemd-cryptsetup-generator(8) for more information.
The `cryptsetup` package is included in the build root since it is
needed by the `org.osbuild.luks2.format` stage. All manifests that
are using the `f34-build-v2` build root change as a result.
OSTree tests, especially the fedora-ostree-image one, will soon
need the tight integration with the host for LVM2/LUKS support.
This we cannot run them in github action containers. Move them
to Schutzbot.
Explicitly install the new sub-package until composer gains the
needed requirement.
Add a new callback parameter to `LoopControl` that, if specified,
will be invoked after the loop device is opened but before any
other operation is done, like setting the backing file. Can be
used to perform custom setup tasks.
Add a new signal like callback to the `Loop` class which will be
invoked before the actual loop device is closed, i.e. the loop
device has an open file descriptor to the device node and it is
being closed. Can be used to perform custom cleanup tasks.
As with ostree and lvm2 support, split out the stages and host
device services needed to build images with LUKS2 support into
its own sub-package since not everyone that uses osbuild might
want the additional cryptsetup dependency.
This is the corresponding device for LUKS containers created via the
new `org.osbuild.luks2.format` stage. Needed information are the
parent device and the passphrase used to create the container.
NB: this device always uses the new custom block device udev rule
inhibitor facility.
This gets set in `open` and used in `close` but if the former
fails the latter will explode if we do not properly initialize
it. Also, we should always properly initialize things.
Certain udev rules for block devices are problematic for osbuild.
One prominent example is LVM2 related rules that would trigger
a scan and auto-activation of logical volumes. This rules are
triggered for new block devices or when the backing file of an
loop devices changes. The rules will lead to a `lvm pvscan
--cache --activate ay` via the `lvm2-pvscan@.service` systemd
service. This will auto-activate all LVM2 logical volumes and
thus interfering with our own device handling in `devices/
org.osbuild.lvm2.lv`, where we only want to activate a single
logical volume.
Also, if the lvm2 devices get activated after the manual metadata
change done in `org.osbuild.lvm2.metadata` the volume group names
might conflict which results in all lvm2 based tooling to be very,
ver sad and also said stage to hang since the loopback device can
not be detached since the activate logical volumes keep it open.
To work-around this we therefore implement a udev rule inhibition
mechanism: on the osbuild side a lock file is created via the new
class called `UdevInhibitor` in `utils/udev.py`. A custom set of
udev rules in `10-osbuild-inhibitor.rules` is then acting on the
existence of that lock file and if present will opt-out of certain
further processing. See the udev rules file for more details.
In fact, we want this custom inhibition mechanism, for all block
devices that are under osbuild's control, since these rules are
there to provide automatisms and integrations with the host,
something we never want.
NB: this should not affect the detection of devices, since lvm2
does do a scan of devices when we call `lvdisplay` in `lvm2.lv`.
The call chain as of lvm2 git rev f773040:
_lvdisplay_single [tools/lvdisplay.c
process_each_lv [tools/toollib.c
lvmcache_label_scan [lib/cache/lvmcache.c
label_scan [ibidem, here is the device detection!
lvdisplay_full [lib/display/display.c
New stage to initialize LUKS2 container on a given device, usually a
loopback device bound to a partition. The passphrase and uuid of the
container need to be specified. Optionally the cipher, label, sector
size and sub-label can be specified. Requires the cryptsetup binary
to be install in the build root.
The LVM2 support bits, especially the host service, depend on lvm2
tooling. Since not every user of osbuild may want to build images
with LVM2 split that out into its own sub-package and have that it
depend on LVM2.
Instead of using merify which seems to not do what we want, use
a combination of mergify and automerge. We let mergify review
dependabot PRs. We let mergify dismiss reviews on updates but
exclude those from Schutzbot. We then let Schutzbot update and
merge the PRs via automerge if the `ci:automerge` label is set.
Re-review the PR after rebasing it. Leave a message to make it
clear that it was not the impersonated person but mergify that
did it.
Ideally, if we had premium, we would impersonate Schutzbot so
it is clear who did it and then use mergify to dismiss reviews
on changes but not for Schutzbot.
Add support for `PermitRootLogin` option in the
`org.osbuild.sshd.config` stage.
I kept the "yes" and "no" values for consistency with other stage
options. While it will make the implementation in osbuild-composer
harder, it won't be impossible as we already have a precedence for doing
it this way (e.g. in the `org.osbuild.pam.limits.conf`).
Modify the stage unit tests to check the new option.
Remove the empty `org.osbuild.sshd.config` stage from `a.mpp.json`
since it does not add any value and it actually made the `tree-diff`
tool provide a weird tree diff results.
Fix#910
Signed-off-by: Tomas Hozza <thozza@redhat.com>
