On each PR test that osbuild don't break the image generation by testing
against the manifest-db that the produced image-infos are still the same
as the one stored.
When calculating the checksum of the stage, the mount options were
not included. This was maybe deliberate, because if the mounts of
a stage change, it is very likely that previous stages change too.
But the introduction of non-device mounts, like ostree.deployment,
have changed the setting, since the content of the tree will be
different if that mount is applied or not. And even for the device
based mounts it will change the tree if e.g. a device is mounted
at at different path but otherwise is formatted with the very same
options. In the worst case we miss a few cache hits due to changes
in the mount setup that don't lead to tree changes, but that will
rarely happen in practice.
The script will update /etc/greenboot/greenboot.conf if user passes
the parameter in the config. Right now this only tweaks one config but
it will/can be expanded if other use cases come.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
OSTree remotes can be configured with a contenturl that is separate from
the metadata url. When specified, the contenturl is used for content
and the url is used only for metadata.
Updated the description of the url property for clarity.
OSTree remotes can be configured to import keys from a file on disk or a
directory containing multiple keys.
When specified, explicitly enables gpg-verify=true.
Fedora 34 is end of life and we ought to be testing with newer
releases so that we catch issues like the authselect one, that
is detailed in "stages/rpm: support marking install as ostree".
Convert the manifest to use version 2 format. Version 1 is really
not used much anymore because composer was completely ported to
using v2. Welcome to the future, ostree commit.
Include cryptsetup and lvm2 so that commit can be deployed to
images that are encrypted and have lvm2 layout. Additionally,
include greenboot. Remove the explicit dependency on ostree
and systemd-udev, which are automatically pulled in.
An ostree system can be identified as such by the presence of a
marker file: /run/ostree-booted. The `rpm-ostree` tool also
creates this marker during the _installation_ of the system[1,2].
Recently, starting with F36, the authselect package has taken
has become mandatory[3] and is now owning the nsswitch config.
An rpm-ostree system, which has parts of the user database in
/usr, release on the nss-altfiles to read these databases. The
necessary entries are added during the post-processing, which
is called in our `org.osbuild.ostree.preptree` stage[4]. This
installation is skipped though if the nsswitch is the file is
a symlink, indicating that it is owned by some other package,
like authselect. So the F36 authselect change first broke rpm-
ostree[6]. The fix was to check for `/run/ostree-booted` in
the authselect scriptlet and special case this situation[7,8].
Now, our `org.osbuild.rpm` stage does not yet have the ability
to pretend it is a running ostree system and thus we did not
get the special treatment resulting in nss-altfiles not being
enabled in our ostree commits. Therefore the passwd database in
/usr was not read and a lot of daemons and programs without a
valid user, like e.g. `sshd`.
This change introduces a new option, `ostree_booted` that if
set, will create the `/run/ostree-booted` marker and thus our
installation phase will get the same treatments from packages
as rpm-ostree. Hopefully.
[1] 730bec87b1/rust/src/builtins/compose/mod.rs (L24)
[2] 6211d1452e/src/app/rpmostree-compose-builtin-tree.cxx (L501)
[3] https://fedoraproject.org/wiki/Changes/Make_Authselect_Mandatory
[4] 7993c6f565/rust/src/composepost.rs (L635)
[5] d614caeca1/f/0010-spec-fix-detection-of-ostree-system.patch
[6] https://bugzilla.redhat.com/show_bug.cgi?id=2034360
[7] https://src.fedoraproject.org/rpms/authselect/c/d614caeca1a68f55542aefd0d76bda2691c85d24?branch=f36
[8] https://github.com/authselect/authselect/issues/48
Convert the code that sets up the bind mounts to /dev, /sys/ and
/proc from a bash script to python by using the new mount helper
that were moved from the `objectstore`.
This mergest the handling of process_stages() and process_format() into
just one process_format(), which incrementally tracks the call stack
of the formating, which allows it to detect when it is hitting a stage
and can call _process_stage().
This means it is possible to mix things like mpp-if and mpp-join with
stages.
For example, you can do complex combinations like:
pipelines:
- name: rootfs
stages:
mpp-join:
- - type: org.ostree.foo
- mpp-if: use_bar
then:
type: org.osbuild.bar
- mpp-if: use_extra_stages
then:
mpp-eval: extra_stages
This is particularly useful if you included something and you want
to mpp-join something that was set in a variable.
Add to new autopep8 check to the checks suit. Ideally we would
move the `test.src` test over from tests.yml, but running that
in an updated container leads to a whole lot of new warnings,
most of them related to "unspecified encoding" in `open` calls.
Instead of using `subprocess` with `check=True`, which will
echo the command used, including all the files it checked,
check the error code and if non-zero, fail the test with a
nice small error message.
Our format for the global module doc-string is usually:
"""
<short summary>
<longer description>
"""
Introduce the short summary and also reformat the rest of the
doc string a bit.
This stage allows to manifests to define information to be put into a
built image. The information can then later be used by RHSM/Insights.
The facts are stored in `/usr/share/osbuild/rhsm.facts` and then
symlinked into `/etc/rhsm/facts/osbuild.facts`.
The v2 manifest requires the `version` key to be present to be
distinguishable from v1. While technically the manifest can be used
standalong without a `version` key, it does prevent us implementing the
manifest correctly in osbuild (i.e., we are unable to process a v2
manifest without the `version` key, because we are unable to autodetect
it then).
Mark the key as required. It does kind of break backwards compatibility
of the schema, but at the same time we always treated it this way,
anyway. So this should be fine.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
We need to initialize `schema` to `None`, otherwise it will be an access
to an uninitialized variable when looking up invalid schemata:
[...]
File "[...]/osbuild/meta.py", line 583, in get_schema
schema = Schema(schema, name or klass)
UnboundLocalError: local variable 'schema' referenced before assignment
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
