The org.osbuild.container-deploy stage uses podman. Including it
in the build here will allow that stage to be used with this
pipeline as the buildroot.
Include a workaround here for what I consider to be a bug [1] in that
`podman` will create `/etc/containers/networks` on first run if it
doesn't exist. That dir should just be created by an RPM. If we
don't include this workaround then the stage will fail when `podman`
attempts the `mkdir` because `/etc/containers` is mounted in from
the buildroot readonly.
[1] https://github.com/containers/common/pull/2265
Update the fedora manifest template to F41. This is a preparation for
testing a new `org.osbuild.dnf5.sbom.spdx` stage. In addition to that,
F39 is already EOL and unsupported, so moving to a newer version is
desirable.
Regenerate all testing manifests.
Notes about specific changes:
- Remove `pcmciautils` package from ostree manifest, because it has
been deprecated and it is not available on F41 any more.
- Add `python3-dnf` package to the Fedora build pipeline. The reason
is that DNF5 is installed by default since F41 and the module is
needed to test e.g. `org.osbuild.dnf4.sbom.spdx` or
`org.osbuild.dnf4.mark` stages.
- Add `python3-libdnf5` package to the Fedora build pipeline. This is a
preparation for testing a new `org.osbuild.dnf5.sbom.spdx` stage.
- Install `pam` in `authselect` stage test manifest, because otherwise
authselect fails on the nonexistence of /etc/pam.d.
- Adjust the `dnf4.mark` test case to also accept `dnf5` as the marked
package, because on F41, `dnf` is a virtual provide of it.
- Adjust stage tests and their diffs as needed.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
We only support `gpt` here so it would seem this option doesn't
make much sense to add, but it will make it so that the mpp-define-images
from osbuild-mpp can be passed in to `org.osbuild.sgdisk` just as it
can be passed in today to `org.osbuild.sfdisk`.
Partitions by default are indexed starting at 1, but in
some cases, such as CoreOS for IBM Z, it may be usefull
to set the 'partnum' for GPT disks explicitly, without
creating dummy partitions.
Now user can define an image:
```
mpp-define-images:
- id: image
size: 10737418240
table:
uuid: 00000000-0000-4000-a000-000000000001
label: gpt
partitions:
- name: boot
type: 0FC63DAF-8483-4772-8E79-3D69D8477DE4
partnum: 3
size: 786432
- name: root
type: 0FC63DAF-8483-4772-8E79-3D69D8477DE4
partnum: 4
size: 4194304
```
So target disk would look like:
```
Disklabel type: gpt
Disk identifier: 00000000-0000-4000-A000-000000000001
Device Start End Sectors Size Type
/dev/loop0p3 2048 788479 786432 384M Linux filesystem
/dev/loop0p4 788480 4982783 4194304 2G Linux filesystem
```
This patch updates the osbuild-mpp tool and the sgdisk and sfdisk
stages to support this.
Co-authored-by: Dusty Mabe <dusty@dustymabe.com>
Similar to the aleph file created for builds of FCOS based on ostree
commit inputs, this adds an aleph file that contains information about
the initial deployment of data when the disk image was built
A new stage is preferred here as both the org.osbuild.ostree.deploy
and org.osbuild.ostree.deploy.container stages need an aleph file and
use of the aleph file may depend on the project/product. For example,
right now CoreOS is the only project that uses an aleph file, but others
may want it in the future.
The next commit will add a stage test that requires erofs-utils. Let's add it
into the buildroot in a separate commit, so the history is more readable.
This drains some of the logic out of `main()` into a
`ostree_commit_deploy()` function. Doing this will make it easier
to diff this stage with the recently added `ostree.deploy.container`
stage.
This commit also changes the `ref` in the schema to be optional,
which is a fixup for 3cc733d. We need to make the ref optional because
the ref could come from the user in the toplevel schema or it could
come from input commit in the schema.
This will allow a user to specify an input directly to the deploy
stage rather than requiring a ostree.pull stage to be called first.
Adding inputs will also be useful when we soon add support for
deploying from a container rather than just deploying from an existing
ostree commit in a repo.
rpm-ostree 2023.2 dropped the `rpm-ostree container-encapsulate` entrypoint.
Instead, we have to use `rpm-ostree compose container-encapsulate`.
Adjust the code that it selects the correct entrypoint based on the rpm-ostree
version.
Since the new stage now requires python-yaml, add it to the buildroot and
regenerate all manifests.
This commit changes these manifests to use the new fedora-vars.ipp and
fedora-build-v2.ipp infrastructure to remove all hardcoded Fedora versions.
Note that this is currently limited only to stage tests that already use v2
manifests. v1 manifests will come later.
Notes:
The buildroot needs additional packages: zstd and openscap-utils. Thus,
all manifests had to be regenerated.
GPG keys were added where missing.
The oscap.remediation stage now creates a bunch of files with timestamps in
their names under /var/lib/authselect/backups. Thus, the newly introduced
`added_directories` directive is used to handle them.
I reviewed all changes in `diff.json` files, and they all seem sane given
that we jumped 4 releases forward.
What? I can just edit variables in one files and all(*) manifests get updated
content? That's impressive.
(*) We will be able to do all once we migrate all to the new format. For now,
the usual disclaimer applies:
This change is applied only to manifests based on fedora-build-v2 for now.
Include the new journald config stage to configure journald to
persist the journal. This is needed since we don't create the
`/var/log/journal` directory that journald uses to switch the
default to persistent storage. But instead of creating that
directory, we explicitly configure journald via the new stage.
This is also what Fedora CoreOS does.
A recent commit (8a7b6d3) fixed the ostree config stage and thus
we actually properly set the readonly flag for the deployment.
This broke the image since we did not specify the `rw` kernel flag
and as a consequence the ostree mounting code in the initrd broke.
Update the osbuild-ci container and privdocker action to the most recent
builds.
This changes the ostree-image-tests since the Fedora update pulled in
util-linux/fdisk changes that align partition sizes. Hence, the ostree
tests need to be changed to have aligned partition sizes as well. For
more information, see:
commit 921c7da55ec78350e4067b3fd6b7de6f299106ee
Author: Karel Zak <kzak@redhat.com>
Date: Thu Jan 27 10:50:45 2022 +0100
libfdisk: (gpt) align size of partition by default
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
Fedora 34 is end of life and we ought to be testing with newer
releases so that we catch issues like the authselect one, that
is detailed in "stages/rpm: support marking install as ostree".
Wrap the LVM volume group in a LUKS container with the passphrase
`osbuild` (yes, really, super secure). NB: the kernel command line
is changed to include `luks.uuid` which is needed so that dracut
will attempt to open the luks container. This corresponds to an
crypttab entry `luks-uuid UUID`. We cannot use the /etc/crypttab
for ostree based images because the initrd is created at commit
time but they luks volume is created at deployment time, we have
to use the kernel command line instead. See the man page for the
systemd-cryptsetup-generator(8) for more information.
The `cryptsetup` package is included in the build root since it is
needed by the `org.osbuild.luks2.format` stage. All manifests that
are using the `f34-build-v2` build root change as a result.
The `size` option was using `-l`, which in fact was `--extents`.
Fix that to use `--size` and add a new option `extents` that
will in fact call translate to `--extents` and this replace the
current use of the `size` option.
Adapt the `fedora-ostree-image` test manifest to use `extents`.
Ability to set the bootloader backend that OSTree should use. NB:
normally this should be set to `none` since in modern distros and
bootloaders the BLS is used and the BLS snippets are generated on
`none` but none of the of the specific bootloader tools are run,
like `grub2-mkconfig` for grub.
Update the fedora image manifest to use that config setting.
Add an optional `remote` to the deployment of the ostree so that
the it is tied to the specified remote. This is needed later for
updating the commit from that remote.
Greenboot is the idea of automatically rolling back bad updates,
i.e. updates that do not boot successfully. The implementation
is split between the boot loader and a user space component.
The latter sets two variables `boot_counter`, which indicates
the maximum number of boot attempts and `boot_success` which
tells the boot laoder if a previous boot was successful. The
bootloader on the other hand will decrement the counter variable
and reset the success indicator one.
An implementation of the user space component for rpm-ostree is
called `greenboot`.
The order of entries in a dictionary is not specified by the JSON
standard and hard to control when marshalling dictionaries in Go.
Since the order of mounts is important and the wrong order leads
to wrong mount trees change the `mounts` field to an array. This
breaks existing manifests but after careful deliberation it was
concluded that the original schema with mounts as dictionaries
is not something we want to support. Apologies to everyone.
Adjust the schema of the copy and zipl stage accordingly.
The manifests set a "en_US" locale but this causes gnome-terminal to not
run due a non UTF-8 locale being used, which is an unsupported config:
gnome-terminal-server[1899]: Non UTF-8 locale (ISO-8859-1) is not supported!
Reported-by: Stephen Smoogen <ssmoogen@redhat.com>
The logic to sort urls was added globally in `mpp-import-pipeline`
but only the in the v1 code path was the `state.manifest_urls`
variable set and thus for v2 the actual sorting did not happen.
Fix this and set the `manifest_urls` to the `org.osbuild.curl`
items, which makes sense because we only know how to sort those.
In both mpp-depsolve and mpp-import-pipeline, sort the packages to
url dictionary before writing the JSON. This makes it easier to
look for packages but more importantly ensures that the resulting
set of packages has the same ordering in the sources section
independently of how it was assembled.
Instead of operating directly on a file, which was previously specified
by `filename`, operate on a device. This is more flexible since a file
can be accessed via a loop back device; but the inverse is obviously
not true, like other devices can not be accessed via a plain file.
Therefore, re-factor the stage to use a device and adapt the existing
test (`fedora-ostree-image`).
Add a new manifest that creates an ostree commit, deploys that,
creates a raw image and copies the deployment into it. The
resulting artefact is a bootlabel qcow2 image.
