Add a first outline of the integration points of osbuild and
selinux, from two perspectives: how osbuild interacts with
the host and how osbuild labels the target artefact and the
resulting peculiarties.
Use the "Checks" workflow to trigger gitlab; this workflow should
be much quicker to complete and thus the gitlab ci will trigger
earlier leading to a more parallel ci run.
Sadly `github.event.workflow_run.pull_requests` is empty if the pull
request was opened from another fork. Use the sha to find an open PR,
otherwise assume it's a branch.
This workflow doesn't have access to the original pull request event
that resulted in this workflow being triggered.
Simply use `head_sha` which will contain the PR sha if it was triggered
by a PR's workflow, or the branch sha if it was triggered from a
branch's workflow.
The `workflow_run` event is triggered either when a workflow was
requested or complete (see `types`). We can use this event as a
trigger for the gitlab ci conditioning on a successful workflow
run of the main tests ("Tests" workflow). This will ensure that,
with outside contributor protection turned o, no secrets are
leaked via PRs from non-contributors, but also that gitlab ci is
run for those PRs once they were manually allowed to run.
The only downside is that now the gitlab ci will only run after
the main workflow ("Tests) has completed and thus serializing
both CI runs. OTOH gitlab CI is quite intense so maybe this is
not so bad after all. If in the future we want to parallelize
both CI runs we could have a third "precheck" condition with
maybe the spell checker and the pylint tests that the main tests
as well as the gitlab ci run depend on.
This avoid having the local_vars dictionary getting filled with
a large number of built-in variable that are of no use for this
dictionary.
This commit was created by from Alexander Larsson.
Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
Using this we can now define variables in the variable section using
basic expression that are evaluated as f-string.
For example, you can use the syntax:
```
"mpp-vars": {
"rootfs_uuid": {"mpp-format-string": "{__import__('uuid').uuid1()}"},
"bootfs_uuid": "156f0420-627b-4151-ae6f-fda298097515"
},
```
This will automatically call uuid.uuid1() for rootfs_uuid, thus allowing
to dynamically set the uuid for the rootfs variable.
This variable being able to be overridden via the -D argument of the
osbuild-mpp tool.
In addition, you can also define variable based on variables defined
above, for example:
```
"mpp-vars": {
"rootfs_size": 4294967296,
"homefs_size": {"mpp-format-string": "{rootfs_size}"}
},
```
For context, we're in particular interested in this for the uuid
generation, as we have had issues with xfs failing to mount a filesystem
if the uuid is already mounted. In particular, if we built e.g. a
raspberry pi image using osbuild and then we use that raspberry pi for building
the same manifest (typically a later version of it) the osbuild fails in the
loopback mount case. Being able to easily support reproducible fixed-uuid
images, as well as ones with a real unique uuid makes a lot of sense.
This commit was created with a lot of inputs from Alexander Larsson.
Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
If there are fds to send back to the client, do a check that none
of them are invalid, so that we do not raise an exception in send
later. This allows us to send a proper RemoteError instead of no
reply at all.
When decoding a message, first check that it is not empty and
raise a `ProtocolError` otherwise. This prevent a more obscure
error like "NoneType has no get method".
Since source were converted to host services it now uses a unix
socket instead of stdin to pass the arguments, which includes
the list of items to download. The latter can become quite big,
in fact too big to fit into a single package (NB: SOCK_SEQPACKET
is used for the underlying transport).
Therefore write the actual items to a temporary file and pass
the fd of it along the message.
On the service server side, i.e. the actual host service binary,
when we receive a message that contains file descriptors, clean
then up eagerly, instead relying on the garbage collector.
More importantly, the fds that we get from as a reply, if any,
need to be closed since in the current model the ownership is
transferred to the caller of `dispatch`.
The option got renamed to `compat` (and moved into the `qemu`
object) when the stage was extracted from the `qemu` assembler;
but the code, taken from the assembler, still used the old
`qcow2_compat` name for the option. Fix this.
Port sources to also use the host services infrastructure that is
used by inputs, devices and mounts. Sources are a bit different
from the other services that they don't run for the duration of
the stage but are run before anything is built. By using the same
infrastructure we re-use the process management and inter process
communcation. Additionally, this will forward all messages from
sources to the existing monitoring framework.
Adapt all existing sources and tests.
This moves the check for already downloaded files earlier so
that if all files are already downloaded we don't need to
load the secrets.
This is faster, but also it allows a pre-seeded object store
to run the manifest on a system (like a VM) that isn't subscribed.
Since `.sort()` returns None, we were checking that None == None
which is not what we aimed to do.
Quick reproducer:
```
>>> assert [1,2,3].sort() == [2,3,4].sort()
>>> assert sorted([1,2,3]) == sorted([2,3,4])
Traceback (most recent call last):
File "<input>", line 1, in <module>
assert sorted([1,2,3]) == sorted([2,3,4])
AssertionError
```
Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
Add a new stage `org.osbuild.pam_limits.conf`, which created
configuration files for `pam_limits` module in /etc/security/limits.d.
Add unit test for the new stage.
Fix#788
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Ability to set the bootloader backend that OSTree should use. NB:
normally this should be set to `none` since in modern distros and
bootloaders the BLS is used and the BLS snippets are generated on
`none` but none of the of the specific bootloader tools are run,
like `grub2-mkconfig` for grub.
Update the fedora image manifest to use that config setting.
Add a new stage `org.osbuild.sysctld` for setting kernel parameters at
boot by creating a configuration file in /usr/lib/sysctl.d. At least
one parameter must be specified for the stage.
Add unit test for the new stage.
Fix#790
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add a new stage `org.osbuild.tmpfilesd` for creating tmpfiles.d
configuration files in /usr/lib/tmpfiles.d. Provided list of
configuration directives is written as separate lines into
the configuration file. At least one configuration directive
must be specified.
Fix#786
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add a new `org.osbuild.selinux.config` stage to configure the
SELinux state on the system.
The stage configures the SELinux state on the system in /etc/selinux/config.
The policy enforcement state and active policy type can be configured.
Fix#785
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add a new `org.osbuild.dnf.config` stage for changing persistent DNF
configuration. Currently only DNF variables can be defined.
Fix#791
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add a new stage for setting active TuneD profile. The stage checks the
value of chosen TuneD profile(s) with the list of available TuneD
profiles installed in the filesystem root. If any of the chosen profiles
does not exist, the stage raises an exception.
Add unit tests for the new stage.
Fix#792
Signed-off-by: Tomas Hozza <thozza@redhat.com>
When `get_fallback_rhsm_secrets` was used, `Subscriptions.repositories`
was None, and `get_secrets` never returned the fallback secrets.
So check if `repositories` is None before
iterating over it, otherwise return the fallback secrets.
Use the aarch64 templates if the efi architecture was set to AA64.
NB: since we only support efi booting for aarch64 this should be
good enough for aarch64 selection.
The code of the `org.osbuild.kickstart` stage already supported
adding the `--remote` option for `ostreesetup` via the `remote`
option but it was not included in the schema.
Add an optional `remote` to the deployment of the ostree so that
the it is tied to the specified remote. This is needed later for
updating the commit from that remote.
Greenboot is the idea of automatically rolling back bad updates,
i.e. updates that do not boot successfully. The implementation
is split between the boot loader and a user space component.
The latter sets two variables `boot_counter`, which indicates
the maximum number of boot attempts and `boot_success` which
tells the boot laoder if a previous boot was successful. The
bootloader on the other hand will decrement the counter variable
and reset the success indicator one.
An implementation of the user space component for rpm-ostree is
called `greenboot`.
