This commit adds example manifests for a bootc.install-to-filesystem
system. It does not do more with them because running a full test
requires a working podman which is difficult to use inside our
GH runners that are already running inside docker.
This adds a `default: true` option for all cases where OSTree
information is specified in schemas and allows for the information
to be picked up from the filesystem.
This is a safe operation because when building disk images there is
no known case where having two deployments makes sense. In the case
there ever were a case then the osname, ref, and serial options still
exist and can be used.
Co-authored-by: Luke Yang <luyang@redhat.com>
Co-authored-by: Michael Vogt <michael.vogt@gmail.com>
Since the stage depends on quite a specific tree state (ostree prepped
tree with boot files), we can't really unit test it any simpler than
generating a tree with and without running the stage and diffing the
tree.
We only support `gpt` here so it would seem this option doesn't
make much sense to add, but it will make it so that the mpp-define-images
from osbuild-mpp can be passed in to `org.osbuild.sgdisk` just as it
can be passed in today to `org.osbuild.sfdisk`.
Partitions by default are indexed starting at 1, but in
some cases, such as CoreOS for IBM Z, it may be usefull
to set the 'partnum' for GPT disks explicitly, without
creating dummy partitions.
Now user can define an image:
```
mpp-define-images:
- id: image
size: 10737418240
table:
uuid: 00000000-0000-4000-a000-000000000001
label: gpt
partitions:
- name: boot
type: 0FC63DAF-8483-4772-8E79-3D69D8477DE4
partnum: 3
size: 786432
- name: root
type: 0FC63DAF-8483-4772-8E79-3D69D8477DE4
partnum: 4
size: 4194304
```
So target disk would look like:
```
Disklabel type: gpt
Disk identifier: 00000000-0000-4000-A000-000000000001
Device Start End Sectors Size Type
/dev/loop0p3 2048 788479 786432 384M Linux filesystem
/dev/loop0p4 788480 4982783 4194304 2G Linux filesystem
```
This patch updates the osbuild-mpp tool and the sgdisk and sfdisk
stages to support this.
Co-authored-by: Dusty Mabe <dusty@dustymabe.com>
Add or remove the immutable bit to the specified mount directory.
The need we have for this right now is for the CoreOS builds where
the immutable bit being set on an OSTree deployment root doesn't
survive the `cp -a --reflink=auto` in the org.osbuild.copy stage when
being copied from the directory tree into the mounted XFS filesystem
we created on the disk image. Thus we have to workaround this loss
of attribute by applying the attribute directly on the mounted
filesystem from the disk.
The stable stream currently doesn't have a new enough bootupd to pass
the tests for the bootupd stage. Let's update to `:testing` for now
and we'll switch back to `:stable` later.
Add the bootupd stage to install GRUB on both BIOS and UEFI systems,
ensuring that your bootloader stays up-to-date.
Signed-off-by: Renata Ravanelli <rravanel@redhat.com>
Similar to the aleph file created for builds of FCOS based on ostree
commit inputs, this adds an aleph file that contains information about
the initial deployment of data when the disk image was built
A new stage is preferred here as both the org.osbuild.ostree.deploy
and org.osbuild.ostree.deploy.container stages need an aleph file and
use of the aleph file may depend on the project/product. For example,
right now CoreOS is the only project that uses an aleph file, but others
may want it in the future.
Diffs for stage tests have changed after the sources were updated.
Update them to match expected behaviour. This was mostly done with
some form of:
```
foo=update-crypto-policies;
sudo tools/gen-stage-test-diff --libdir . --store /var/osbuild/store/ test/data/stages/$foo > test/data/stages/$foo/diff.json
```
For the dracut one I had to figure out what new kernel was used
and the new modules and update the vanilla.json file to get the
test to pass.
For the rpm one I had to also update the metadata.json with something
like:
```
sudo python3 -m osbuild --libdir . --store /var/osbuild/store/ --export tree \
--output-directory /var/osbuild/out/ test/data/stages/rpm/b.json --json \
| jq .metadata >test/data/stages/rpm/metadata.json
```
This adds a new key masked_generators, similar to masked_services,
which masks systemd generators from running at boot, by creating
symlinks to /dev/null in /etc/systemd/systemd-generators, as
described in:
https://www.freedesktop.org/software/systemd/man/latest/systemd.generator.html#Description
This will be useful for the automotive project, as it allows disabling
of unsupported things like sysv or rc.local legacy support, while
improving boot performance.
This commit extends the current support for OpenSCAP
tailoring by accepting an array of key/value overrides.
Users will be able to specify override values for specific
rules that will update the value when remediating the
image.
The next commit will add a stage test that requires erofs-utils. Let's add it
into the buildroot in a separate commit, so the history is more readable.
Add a simple stage test for the erofs stage. It uses dump.erofs
instead of mounting the file because the kernel in the GH runners
do not support mounting erofs just yet.
For our Fedora CoreOS disk images we set the partition labels (name)
for the partitions. This is also supported using the primitives here
in OSBuild, but it wasn't obvious that I needed to set the name in
the mpp-define-images definition. Let's set the name there, but let's
also allow osbuild-mpp to set the `id`, which is what is used later
to access that partition from the `name` too if `id` isn't set.
This means we allow something like:
- name: BIOS-BOOT
type: 21686148-6449-6E6F-744E-656564454649
bootable: true
uuid: FAC7F1FB-3E8D-4137-A512-961DE09A5549
size: 100
rather than requiring something like:
- id: BIOS-BOOT
name: BIOS-BOOT
type: 21686148-6449-6E6F-744E-656564454649
bootable: true
uuid: FAC7F1FB-3E8D-4137-A512-961DE09A5549
size: 100
Quick rename to have our wording be in-line with the new differences
between stage unit tests and stage integration tests; also being applied
to the guides.
This stage calls `update-crypto-policies` to set the
policy applicable for the various cryptographic back-ends,
such as SSL/TLS libraries.
Signed-off-by: Miguel Martín <mmartinv@redhat.com>
Also switch the qcow2 output to be a "qemu" platform image. This
change takes advantage of the recent org.osbuild.kernel-cmdline.bls-append
stage addition to modify kernel arguments in later pipelines.
To get a qemu image output:
- osbuild --output-directory out/ --export qemu test/data/manifests/fedora-coreos-container.json
To get a metal image output:
- osbuild --output-directory out/ --export metal test/data/manifests/fedora-coreos-container.json
This was done via:
```
$ sudo osbuild --export tree --output-directory /tmp/devnull test/data/stages/rpm/b.json --json | jq .metadata >test/data/stages/rpm/metadata.json
```
a outlined in `test/data/stages/rpm`.
Moving to the newer fedora 38 snapshot pulled in some changes
to the `/etc/dnf/automatic.conf` (e.g. [0]) when moving from
dnf 4.16.1 to 4.17.0. This commit updates the diff.
[0] a0acc88efc
The authselect upstream code dropped adding a timestamp to the
auto-geneated content in /etc/authselect [0]. With that the
content diff can be compared again.
[0] 44b9d87f90
Changes in `/etc/shadow` depend on the date, i.e. the last time
the password changed field. So for now exclude them from the
tree diff (until we use `faketime` to fix this :)
With lvm2 the generated fedora fc38 boot image boots in degraded
mode with the following error:
```
[root@localhost ~]# journalctl -u lvm2-monitor.service|more
Nov 13 12:52:04 localhost.localdomain lvm[431]: Failed to create /etc/lvm/devi
ces 2
Nov 13 12:52:04 localhost.localdomain lvm[431]: Failed to set up devices.
Nov 13 12:52:04 localhost.localdomain systemd[1]: lvm2-monitor.service: Main pro
cess exited, code=exited, status=5/NOTINSTALLED
Nov 13 12:52:04 localhost.localdomain systemd[1]: lvm2-monitor.service: Failed w
ith result 'exit-code'.
Nov 13 12:52:04 localhost.localdomain systemd[1]: Failed to start lvm2-monitor.s
ervice - Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress p
olling.
```
This breaks the `test_boot.py` which expects the system after booting
in `running` state (from `systemd is-system-running`).
It looks like this is some sort of race with our generated image,
potentially related to selinux, see
https://github.com/lvmteam/lvm2/blob/v2_03_18/lib/device/dev-cache.c#L1842
and note the lines around dm_prepare_selinux_context(). Note
also that `lvm2-monitor.service` runs with `DefaultDependencies=no`
(c.f.
https://github.com/lvmteam/lvm2/blob/v2_03_18/scripts/lvm2_monitoring_systemd_red_hat.service.in#L7)
Given that the official fc38 cloud image does not use lvm2 and that
it's not needed for the boot test this commit simply removes it
from the fedora-boot manifest. This fixes the test.
The authselect stage will generate timestamps as part of the
update to `/etc/authselect/*`, e.g.:
```
Generate by authselect on Fri Nov 10 16:06:29 2023
...
```
this makes the content diff for those files unusable (until we
start to use faketime which maybe we should ;)
Instead of using the F34-based manifests, let's switch to F38. I tried my
best to import the vars into the new manifest, but I don't think that's
currently supported.
Let's not depend on f34-base.json anymore, but instead of a separate,
standardized manifest for it. The test was modified so it supports
v2 manifests. Also, the new manifest installs just a very minimal system.
There's no need to install the whole @core.
Authconfig was completely retired from Fedora. In order to keep this stage
covered, this commit changes the test to use CentOS Stream 9, which still
ships this package.
This gets rid of the old F34 manifest and migrates the to test to the
standardized V2 one based on F38.
Since the metadata format is much saner in V2, I was able to simplify the
assertion quite a lot.
