If there are fds to send back to the client, do a check that none
of them are invalid, so that we do not raise an exception in send
later. This allows us to send a proper RemoteError instead of no
reply at all.
When decoding a message, first check that it is not empty and
raise a `ProtocolError` otherwise. This prevent a more obscure
error like "NoneType has no get method".
Since source were converted to host services it now uses a unix
socket instead of stdin to pass the arguments, which includes
the list of items to download. The latter can become quite big,
in fact too big to fit into a single package (NB: SOCK_SEQPACKET
is used for the underlying transport).
Therefore write the actual items to a temporary file and pass
the fd of it along the message.
On the service server side, i.e. the actual host service binary,
when we receive a message that contains file descriptors, clean
then up eagerly, instead relying on the garbage collector.
More importantly, the fds that we get from as a reply, if any,
need to be closed since in the current model the ownership is
transferred to the caller of `dispatch`.
The option got renamed to `compat` (and moved into the `qemu`
object) when the stage was extracted from the `qemu` assembler;
but the code, taken from the assembler, still used the old
`qcow2_compat` name for the option. Fix this.
Port sources to also use the host services infrastructure that is
used by inputs, devices and mounts. Sources are a bit different
from the other services that they don't run for the duration of
the stage but are run before anything is built. By using the same
infrastructure we re-use the process management and inter process
communcation. Additionally, this will forward all messages from
sources to the existing monitoring framework.
Adapt all existing sources and tests.
This moves the check for already downloaded files earlier so
that if all files are already downloaded we don't need to
load the secrets.
This is faster, but also it allows a pre-seeded object store
to run the manifest on a system (like a VM) that isn't subscribed.
Since `.sort()` returns None, we were checking that None == None
which is not what we aimed to do.
Quick reproducer:
```
>>> assert [1,2,3].sort() == [2,3,4].sort()
>>> assert sorted([1,2,3]) == sorted([2,3,4])
Traceback (most recent call last):
File "<input>", line 1, in <module>
assert sorted([1,2,3]) == sorted([2,3,4])
AssertionError
```
Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
Add a new stage `org.osbuild.pam_limits.conf`, which created
configuration files for `pam_limits` module in /etc/security/limits.d.
Add unit test for the new stage.
Fix#788
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Ability to set the bootloader backend that OSTree should use. NB:
normally this should be set to `none` since in modern distros and
bootloaders the BLS is used and the BLS snippets are generated on
`none` but none of the of the specific bootloader tools are run,
like `grub2-mkconfig` for grub.
Update the fedora image manifest to use that config setting.
Add a new stage `org.osbuild.sysctld` for setting kernel parameters at
boot by creating a configuration file in /usr/lib/sysctl.d. At least
one parameter must be specified for the stage.
Add unit test for the new stage.
Fix#790
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add a new stage `org.osbuild.tmpfilesd` for creating tmpfiles.d
configuration files in /usr/lib/tmpfiles.d. Provided list of
configuration directives is written as separate lines into
the configuration file. At least one configuration directive
must be specified.
Fix#786
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add a new `org.osbuild.selinux.config` stage to configure the
SELinux state on the system.
The stage configures the SELinux state on the system in /etc/selinux/config.
The policy enforcement state and active policy type can be configured.
Fix#785
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add a new `org.osbuild.dnf.config` stage for changing persistent DNF
configuration. Currently only DNF variables can be defined.
Fix#791
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add a new stage for setting active TuneD profile. The stage checks the
value of chosen TuneD profile(s) with the list of available TuneD
profiles installed in the filesystem root. If any of the chosen profiles
does not exist, the stage raises an exception.
Add unit tests for the new stage.
Fix#792
Signed-off-by: Tomas Hozza <thozza@redhat.com>
When `get_fallback_rhsm_secrets` was used, `Subscriptions.repositories`
was None, and `get_secrets` never returned the fallback secrets.
So check if `repositories` is None before
iterating over it, otherwise return the fallback secrets.
Use the aarch64 templates if the efi architecture was set to AA64.
NB: since we only support efi booting for aarch64 this should be
good enough for aarch64 selection.
The code of the `org.osbuild.kickstart` stage already supported
adding the `--remote` option for `ostreesetup` via the `remote`
option but it was not included in the schema.
Add an optional `remote` to the deployment of the ostree so that
the it is tied to the specified remote. This is needed later for
updating the commit from that remote.
Greenboot is the idea of automatically rolling back bad updates,
i.e. updates that do not boot successfully. The implementation
is split between the boot loader and a user space component.
The latter sets two variables `boot_counter`, which indicates
the maximum number of boot attempts and `boot_success` which
tells the boot laoder if a previous boot was successful. The
bootloader on the other hand will decrement the counter variable
and reset the success indicator one.
An implementation of the user space component for rpm-ostree is
called `greenboot`.
The `refs` variable is used as parameter to indicate which commits
to checkout, but it also was used as result variable to store the
ids of the commits that actually got checked out; naturally, it
was initialized to the empty array. This of course meant nothing
was ever actually checked out. Doh.
Fedora 34 and thus RHEL 9 have adopted the unified grub config
scheme[1], where the main config is always placed in the same
location across all platforms, i.e. `boot/grub2`, and a stub
config that redirects to the main config is placed into the ESP.
osbuild has always done that in the case of hybrid boot, but not
for pure EFI systems. The new `uefi.unified` config option can
be used to select that new unified scheme even for the case of
pure EFI systems (aarch64 or non-hybrid boot).
Add a simple test for the grub stage.
[1] https://fedoraproject.org/wiki/Changes/UnifyGrubConfig
Previously:
We used images built from pre-mass-rebuild composes but installed packages
from post-mass-rebuild composes. This caused weird stuff like sshd crashing
when installing non-related packages via dnf.
Now:
Both the image and repositories are post-mass-rebuild ones. This should solve
these weird issues.
This stage takes /usr/lib/passwd and /usr/etc/passwd from an OSTree
checkout, merges them into one file, and store it as /etc/passwd in the
buildroot.
It does the same for /etc/group.
The reason for doing this is that there is an issue with unstable UIDs
and GIDs when creating OSTree commits from scratch. When there is a
package that creates a system user or a system group, it can change the
UID and GID of users and groups that are created later.
This is not a problem in traditional deployments because already created
users and groups never change their UIDs and GIDs, but with OSTree we
recreate the files from scratch and then replace the previous one so it
can actually change.
By copying the files to the build root before doing any other
operations, we can make sure that the UIDs and GIDs of already existing
users and groups won't change.
Co-author: Christian Kellner <christian@kellner.me>
New input type that takes a ostree repo and checks out any number
of commits inside that repo to a temporary directory. Each commit
will be checked out to a separate sub-directory. The name of the
dir is the commit id of the corresponding commit.
This input can thus be used to access files and directories of
commits in stages.
Include the packit tool to be able to do updates from the container.
Additionally, include python-boto3 so that `make test-src` now does
not complain about missing `boto3` imports.
Also include ipython3 for quick python code snippet testing and
lsof to debug processes that keep files open.
NB: for packit to work you have to have the right tokens in your
local packit configration file (see the upstream doc). Additionally,
`packit propose-downstream`, to create PRs for a new downstream
release, needs kerberos auth to work correctly.
Instead of using the version specific, pre-depsolved f34 build manifest,
use the new version agnostic build manifest (fedora-build.mpp). NB: this
is included directly as mpp so that its variables get defined by the
including manifest. This should make it even easier to update manifests
to new fedora releases.
Include a build manifest that is itself not have tied to a specified
version and thus is meant to be included with the following vars
pre-defined as .mpp file:
- arch architecture (x86_64)
- releasever release version (f34)
- snapshot rpmrepo snapshot (20210326)
