Commit graph

287 commits

Author SHA1 Message Date
Michael Vogt
eb91e1780d stages: convert kickstart stage v2 manifest 2023-11-14 10:45:44 -08:00
Ondřej Budai
fd82024153 test/data/manifests: remove F34 manifests
They are no longer used anywhere! We are free!
2023-11-14 10:45:44 -08:00
Ondřej Budai
3bb5a00375 test/run/assemblers: convert to a v2 manifest
Instead of using the F34-based manifests, let's switch to F38. I tried my
best to import the vars into the new manifest, but I don't think that's
currently supported.
2023-11-14 10:45:44 -08:00
Ondřej Budai
5adbecba43 test/run/stages: convert test_selinux to a v2 manifest
Let's not depend on f34-base.json anymore, but instead of a separate,
standardized manifest for it. The test was modified so it supports
v2 manifests. Also, the new manifest installs just a very minimal system.
There's no need to install the whole @core.
2023-11-14 10:45:44 -08:00
Ondřej Budai
c8d252f2fd test/data/stages: convert rest of tests to v2 manifests
These are the boring migrations. The manifests were just replaced by the usual
pipelines and diffs don't have any significant changes.
2023-11-14 10:45:44 -08:00
Ondřej Budai
506d6a7256 test/stages/authconfig: Use centos stream 9 as a base
Authconfig was completely retired from Fedora. In order to keep this stage
covered, this commit changes the test to use CentOS Stream 9, which still
ships this package.
2023-11-14 10:45:44 -08:00
Ondřej Budai
367981d6e4 test/stages/dracut: Update to v2 manifest and F38
I haven't found an automated way to generate vanilla.json. :/ Something
for the future.
2023-11-14 10:45:44 -08:00
Ondřej Budai
60a1570383 test/run/rpm: convert to v2 manifest
This gets rid of the old F34 manifest and migrates the to test to the
standardized V2 one based on F38.

Since the metadata format is much saner in V2, I was able to simplify the
assertion quite a lot.
2023-11-14 10:45:44 -08:00
Ondřej Budai
e4933b4fd6 test/run/boot: migrate to a v2 manifest
This commit migrates the test to a brand new V2, F38-based manifest. It's
actually based on osbuild-composer interpretation of the Fedora Cloud Base
image.
2023-11-14 10:45:44 -08:00
Michael Vogt
1eefb9b55d tests: add kickstart stage test
This commits adds a stage based test for the `kickstart`
configuration.
2023-11-03 13:29:04 +01:00
Ondřej Budai
23de60cd23 stages/mkdir: fix its schema
The items of an array are defined under the `items` key, not under `paths`.
Let's fix this.

Btw, this is possible because JSON Schema itself doesn't use
additionalProperties = false. This allows extending the schemas easily, but
is sadly a bit error-prone.

Sadly, since this issue effectively disabled validation of the stage options,
we also need to relax the schema a bit:

We found out that there are manifests in the wild, that use relative paths,
instead of absolute ones. Thus, this commit changes the validation regex to
allow relative paths. However, this now emits a warning and it's strongly
discouraged. The associated stage test was modified to accommodate for this.

Co-authored-by: Tomáš Hozza <thozza@redhat.com>
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-30 13:31:15 +01:00
Dusty Mabe
41652343a4 manifests: add fedora-coreos-container example/test
This example shows how to build a qcow2 very similar to the one
delivered by Fedora CoreOS. It uses an input container that has
the Fedora CoreOS OSTree already baked into it, which means that
the OSTree isn't built using this manifest, but taken as an input
from a remote registry.
2023-10-24 13:46:10 -04:00
Dusty Mabe
da07300f38 stages/ostree.deploy: create ostree_commit_deploy function
This drains some of the logic out of `main()` into a
`ostree_commit_deploy()` function. Doing this will make it easier
to diff this stage with the recently added `ostree.deploy.container`
stage.

This commit also changes the `ref` in the schema to be optional,
which is a fixup for 3cc733d. We need to make the ref optional because
the ref could come from the user in the toplevel schema or it could
come from input commit in the schema.
2023-10-24 13:46:10 -04:00
Achilleas Koutsou
dd2aa57987 test: update stage diffs
Diffs for stage tests have changed after the sources were updated.
Update them to match expected behaviour.
2023-10-21 07:34:32 +02:00
Dusty Mabe
ca3b831c29 manifests/fedora-vars: bump snapshot date
This should correspond to the snapshots from 6b85149.
2023-10-21 07:34:32 +02:00
Evgeny Kolesnikov
c4de5389e7 stages/oscap.remediation: Properly utilize offline capabilities
The scanner will now properly react to the chroot environment. Also,
there are some optimizations to logs and results.
2023-10-19 12:43:57 +02:00
Michael Ho
258735d8a6 test/stages/users: make test data date agnostic
The test for this stage is failing because etc/shadow changes content
depending on the date that it runs on (due to the "date of last password
change" field). This causes the checksums to not be constant for our
tests and depends on the date.

This commit removes the checksums for etc/shadow from the test so that
they are not checked as part of the test. This worksaround the test
failure issue for now until a solution to the dynamic contents is
determined.

Signed-off-by: Michael Ho <michael.ho@ieee.org>
2023-10-18 13:56:20 +02:00
Achilleas Koutsou
cf14de3558 test/stages/users: extend to add ssh keys
Added three users to the test to test all combinations of ssh key
definitions:
- a user called `onekey` that has a single key defined in the `key`
  option.
- a user called `multikey` that has a multiple keys defined in the
  `keys` array.
- a user called `bothy` that defines both a single key in the `key`
  option and two other keys in the `keys` array.
2023-10-16 18:05:27 +02:00
Dusty Mabe
3cc733d7cd stages/ostree.deploy: support an ostree input
This will allow a user to specify an input directly to the deploy
stage rather than requiring a ostree.pull stage to be called first.

Adding inputs will also be useful when we soon add support for
deploying from a container rather than just deploying from an existing
ostree commit in a repo.
2023-10-06 15:55:26 +02:00
Maxime Ripard
7af2f1a5c1 Create additional option to setup FS geometry
Some platforms like the TI AM62 require a particular FAT geometry for
their CPU to read the file system (and thus the bootloader). Failing
that the CPU will simply not boot and keep looking for a bootloader.

Let's add some options to enforce a particular filesystem geometry
through the -g option of mkfs.fat.

Signed-off-by: Maxime Ripard <mripard@kernel.org>
2023-10-05 20:17:40 +02:00
Colin Walters
2972949d13 tests/ostree-container: Drop hardcoded max layers
This should just defer to rpm-ostree.
xref https://github.com/coreos/rpm-ostree/issues/4530

(If someone cares about reproducibility here, they can specify it;
 alternatively and more reliably, they can create builder container
 images and pin to those)
2023-08-29 10:56:49 +02:00
Ondřej Budai
546c8267d1 test/run/stage: cover the btrfs stages
A simple test to make sure that our btrfs stages work.
Updating the testing container is needed, because the old version
didn't have btrfs-progs in it.
2023-08-25 12:35:56 +02:00
Ondřej Budai
b94048dfd4 mounts/org.osbuild.btrfs: add support for compression
Also, add it to the sample manifest.
2023-08-25 12:35:56 +02:00
Ondřej Budai
3bbded4607 org.osbuild.mkfs.btrfs: add support for -m argument
Osbuild doesn't support creating btrfs over multiple devices, thus it should
be fine to have only single and dup in the schema (other options are for raid)
2023-08-25 12:35:56 +02:00
Ondřej Budai
dae3f884bc samples/fedora-boot-btrfs: convert to use subvolumes
We now have a full support for subvolumes in osbuild, let's use
it in the sample.

Co-authored-by: Christian Kellner <christian@kellner.me>
2023-08-25 12:35:56 +02:00
Ondřej Budai
f145a877f6 test/data: include bootable fedora image with btrfs
So we have a quick reference how to work with btrfs-based images.

Co-authored-by: Christian Kellner <christian@kellner.me>
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-08-25 12:35:56 +02:00
Sayan Paul
61da6124ed stages/systemd:support for user service dropin
systemd unit stage will now be able to add user unit dropin,
this is done by adding an additional field:unit-type
which is set to system by default. It also adds ability
to update the unit config with ConditionPathExists.
Relevant testing updated for the new workflow.

Signed-off-by: Sayan Paul <paul.sayan@gmail.com>
2023-08-16 13:23:16 +02:00
Simon de Vlieger
161fe789af dnf4.mark: mark packages in DNF state database
This adjustment allows the definition of the mark with the RPMs and runs
DNF after installing the RPMs to put the proper markings in the DNF
state database. See #455.

This ensures that packages don't get removed during `autoremove` leading
to broken systems.
2023-08-15 00:37:06 +02:00
Ondřej Budai
c47850b8ae manifests/fedora-ostree-native-container: bump max_layers to 4
rpm-ostree crashes when max_layers is set to <4. Work around that
by bumping the max_layers argument.

See https://github.com/coreos/rpm-ostree/issues/4530
2023-08-08 12:41:17 +02:00
Ondřej Budai
6e254c95af stages/ostree.encapsulate: use new entrypoint on rpm-ostree >=2023.2
rpm-ostree 2023.2 dropped the `rpm-ostree container-encapsulate` entrypoint.
Instead, we have to use `rpm-ostree compose container-encapsulate`.

Adjust the code that it selects the correct entrypoint based on the rpm-ostree
version.

Since the new stage now requires python-yaml, add it to the buildroot and
regenerate all manifests.
2023-08-08 12:41:17 +02:00
Ondřej Budai
09fa9e9def test/data/stages: Document the changes to the tests 2023-08-08 12:41:17 +02:00
Ondřej Budai
fbe6e3ed97 manifests: remove no longer needed f34-build-v2
It's just no longer required by anything, yay!
2023-08-08 12:41:17 +02:00
Ondřej Budai
59edb5ea6c test/data/stages: update to the new structure
This commit changes these manifests to use the new fedora-vars.ipp and
fedora-build-v2.ipp infrastructure to remove all hardcoded Fedora versions.

Note that this is currently limited only to stage tests that already use v2
manifests. v1 manifests will come later.

Notes:
The buildroot needs additional packages: zstd and openscap-utils. Thus,
all manifests had to be regenerated.

GPG keys were added where missing.

The oscap.remediation stage now creates a bunch of files with timestamps in
their names under /var/lib/authselect/backups. Thus, the newly introduced
`added_directories` directive is used to handle them.

I reviewed all changes in `diff.json` files, and they all seem sane given
that we jumped 4 releases forward.
2023-08-08 12:41:17 +02:00
Ondřej Budai
f0c5dfb3a3 manifests/fedora-ostree-tarball: update to the new structure
This commit changes this manifest to use the new fedora-vars.ipp and
fedora-build-v2.ipp infrastructure to remove all hardcoded Fedora versions.
2023-08-08 12:41:17 +02:00
Ondřej Budai
111fc97035 manifests/fedora-ostree-container: update to the new structure
This commit changes this manifest to use the new fedora-vars.ipp and
fedora-build-v2.ipp infrastructure to remove all hardcoded Fedora versions.

We are also able to remove the container manifests from the Makefile,
because they are no longer depending on the old f34-build-v2.json.
2023-08-08 12:41:17 +02:00
Ondřej Budai
9a9834a309 manifests/fedora-ostree-bootiso*: update to the new structure
This commit changes this manifest to use the new fedora-vars.ipp and
fedora-build-v2.ipp infrastructure to remove all hardcoded Fedora versions.

An auto-generated ostree ref and isolabel was added to fedora-vars.

smc-meera-fonts was retired, so this commit changes it to rit-meera-new-fonts.
See
a8fe0d2620

icfg and reiserfs-utils are gone too, and they don't seem to be neccessary for
the image, so they were removed as well. Please correct me, if I am wrong.
2023-08-08 12:41:17 +02:00
Ondřej Budai
aeec48619b manifests/fedora-container: update to the new structure
This commit changes this manifest to use the new fedora-vars.ipp and
fedora-build-v2.ipp infrastructure to remove all hardcoded Fedora versions.
2023-08-08 12:41:17 +02:00
Ondřej Budai
82b7053983 manifests/fedora-vars: update to fedora 38
What? I can just edit variables in one files and all(*) manifests get updated
content? That's impressive.

(*) We will be able to do all once we migrate all to the new format. For now,
    the usual disclaimer applies:

    This change is applied only to manifests based on fedora-build-v2 for now.
2023-08-08 12:41:17 +02:00
Ondřej Budai
baca25f7cb manifests: fix module_platform_id
module_platform_id has a format of `platform:f34`. I have no idea why just
`f34` is used. Let's fix it.

This change is applied only to manifests based on fedora-build-v2 for now.
2023-08-08 12:41:17 +02:00
Ondřej Budai
ed16da70eb manifests: factor repositories and gpg keys out
They are the same everywhere, we can just factor them out and save some bytes.

This change is applied only to manifests based on fedora-build-v2 for now.
2023-08-08 12:41:17 +02:00
Ondřej Budai
9286515821 manifests: move vars to a separate file
This change is applied only to manifests based on fedora-build-v2 for now.
2023-08-08 12:41:17 +02:00
Ondřej Budai
64cc289bf7 manifests: remove vars from fedora-build-v2
Defining variables in the build pipeline is potentially dangerous, because
it can lead to the buildroot using a different package set than the other
pipelines.

Thus, this commit removes the variables from the build manifest. Since now on,
the variables must be defined before the build manifest is included.

Since the build manifest is no longer buildable on its own, let's remove it
from mpp.yaml to ipp.yaml. This is a convention set by the automotive team:
ipp manifests are not meant to be directly buildable, they are only useful
for embedding.
2023-08-08 12:41:17 +02:00
Ondřej Budai
f777eb5ac2 convert all mpp.yaml files to proper YAML
This is the second step of the conversion. All mpp.yaml json files were loaded
and dumped again with pyyaml. It was tweaked to keep the order of keys and to
save multi-line strings in the "flow" style. This was done, so the GPG keys
are kept on one line. Otherwise, they take up too much visual space.

There is no functional change in this commit.
2023-08-08 12:41:17 +02:00
Ondřej Budai
d827be2fb2 rename all .mpp.json files to .mpp.yaml
osbuild-mpp supports yaml files. The advantage of yaml is that it's less
verbose and allows comments. This way, we can start documenting all the
test manifests in this repository.

This is the first step: This commit just renames the files, so we keep the
history intact. Remember: YAML is a super-set(*) of JSON, so this is fine.

There is no functional change in this commit.

(*) There are some caveats, but we don't need to care in this case.
2023-08-08 12:41:17 +02:00
Ondřej Budai
2d23f9c17c samples: remove empty el8-ostree-bootiso.json
No idea why this is here.
2023-08-08 12:41:17 +02:00
Gianluca Zuccarelli
74eed6a33b stages: add openscap autotailor stage
Add a new stage to generate an OpenSCAP tailoring file. The stage
overrides a base OpenSCAP profile by enabling and disabling user
selected rules and creates a new profile name which can be used for
OpenSCAP scanning and remediation.
2023-07-18 15:29:13 -07:00
Ondřej Budai
4439297cc4 test/data: introduce UKI also for CentOS Stream
This is a straightforward port of the UKI from Fedora to CentOS Stream.
Changes:

- Packages that does not exist in CS9 were removed:
  - shim-ia32
  - grub2-efi-ia32-cdboot
  - btrfs-progs
- Squashfs compression was changed from lz4 to gzip, because lz4 for squashfs
  isn't enabled in RHEL 9 kernel
2023-07-18 11:30:09 +02:00
Simon de Vlieger
82e72f1b64 test: update test for sysconfig 2023-07-17 12:15:12 +02:00
Antonio Murdaca
4bfd646b0d stages: add new zstd stage
typo fix test/data/stages/README.md
add updated test-data after adding zstd to fedora-34-build-v2.mpp.json

Signed-off-by: Sarita Mahajan <sarmahaj@redhat.com>
2023-06-14 09:44:00 +02:00
Simon de Vlieger
4bd169c20e stage: add a livesys stage
This stage is necessary for the live image work being done in
`osbuild-composer` at the moment.
2023-05-31 09:30:55 +02:00