This commit adds support for more debug for `qemu-user` options.
When settings:
```
$ sudo IMAGE_BUILDER_EXPERIMENAL=debug-qemu-user bootc-image-builder ...
```
extra debug will be printed. This hopefully helps to track down
the root cause of
https://github.com/podman-desktop/extension-bootc/issues/1475
This commit adds two new helpers:
- util.experimentalflags.get_bool()
- util.experimentalflags.get_string()
similar to what we added in the images library in PR:
https://github.com/osbuild/images/pull/1248
The idea is that we provide experimentalflags for osbuild via
an environment like `OSBUILD_EXPERIMENTAL` and for those we
make no API promises. This will be initially used for better
debug of qemu-user.
We originally required there to be only one kernel modules directory
since we typically only have one kernel installed. Change this check
up a bit since some kernel module RPMs aren't delivering a new kernel
but do deliver kmods that were built against a different kernel.
For example v4l2loopback from https://elrepo.org/
```
bash-5.1# rpm -ql kmod-v4l2loopback
/etc/depmod.d/kmod-v4l2loopback.conf
/lib/modules/5.14.0-427.13.1.el9_4.x86_64
/lib/modules/5.14.0-427.13.1.el9_4.x86_64/extra
/lib/modules/5.14.0-427.13.1.el9_4.x86_64/extra/v4l2loopback
/lib/modules/5.14.0-427.13.1.el9_4.x86_64/extra/v4l2loopback/v4l2loopback.ko
/usr/share/doc/kmod-v4l2loopback-0.12.7
/usr/share/doc/kmod-v4l2loopback-0.12.7/GPL-v2.0.txt
/usr/share/doc/kmod-v4l2loopback-0.12.7/greylist.txt
```
This patch now allows an image built from a Containerfile like the
following to work as input to the live artifacts stage.
```
FROM rhcos
RUN <<EOF
set -eux -o pipefail
dnf install -y https://www.elrepo.org/elrepo-release-9.el9.elrepo.noarch.rpm
dnf install -y v4l2loopback
dnf clean all
ostree container commit
EOF
```
Add a stage build test for the boot_root option of the org.osbuild.rpm
stage. The test defined two manifests that both install the
kernel-uki-virt kernel but differ in the value of the boot_root option.
The diff shows the effect of the value on the install location of the
kernel.
Add support for setting environment variables for kernel-install and its
plugins. These can be used to control the behaviour of kernel-install
during the kernel rpm installation. Currently, we only add support for
$BOOT_ROOT, which we need to control the path where the kernel should be
installed.
Normally, kernel-install scripts will detect the correct path based on
the filesystem layout, specifically the ESP path. However, during our
installation the filesystem is not yet set up, so with this option, we
can control the installation path based on the location we know the ESP
will be mounted in the final image.
This is a very similar override to what we do with the
org.osbuild.fix-bls stage.
The stage option is added under a kernel_install_env object so we can
potentially support more of the known env vars in the future.
Printing 'owner' in an fstring triggers the mypy warning 'str-bytes-safe'
Error: If x = b'abc' then f"{x}" or "{}".format(x) produces "b'abc'",
not "abc".
We can't yet use the nicer run() argument, 'text=True', because we still
target the RHEL 7 build root which only has Python 3.6 and 'text' was
introduced in Python 3.7.
Use a temporary directory for the output of grub2-mkimage.
This makes the stage clean up the grub2-core.img from the build root
after its done. It also has the nice side-effect that unit tests that
call the stage are independent. Previously, a bug in the stage *might*
have been missed if a certain configuration of the stage was not
creating the grub2-core.img. One unit test could create an image at the fixed path
(/var/tmp/grub2-core.img) and then another one could call the stage with
the buggy configuration but the `shutil.copyfile()` call at the end of
the stage would succeed because it would find the image from the
previous stage run.
To accommodate for this change, the unit test with the mocked run call
is adjusted to intercept the random tmp output path and use it to create
a fake file for the stage to succeed.
This commit adds a test that mocks subprocess.run() so that we
can regression test the arguments passed to "grub-mkimage".
This is a regression test for
https://github.com/osbuild/osbuild/issues/2037
When writing the grub2 core image, the format should be set based on the
platform. This was accidentally changed to be `i386-pc` for all
platforms except when building ISOs.
Set the format to the platform and only change it to `i386-pc-eltorito`
when building ISOs.
The main reason is that it comes with license_expression python package
needed for newly added SBOM test cases.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Always return License ref IDs as is, if used as package license,
regardless if license_expression package is available. This will prevent
wrapping them again as extracted license info and generating yet another
license ref ID.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Allow passing a custom license index db file for SBOM generation by
specifying it in the solver configuration.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Instead of completely overriding the default solver configuration with
the one loaded from a file, just extend the default config. This will
allow to specify just desired config options and keeping the defaults
for the rest.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Move the call to `pytest.importorskip()` function into a specific test
case that relies on imported modules. This will make test cases in the
same file to be run, even if importing the modules fail.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Introduce a new class `SpdxLicenseExpressionCreator`, responsible for
converting license texts extracted from packages, into an SPDX-compliant
license expressions. If the `license_expression` Python package is
available on the system, it is used to determine the license text
extracted from a package is a valid SPDX license expression. If it is,
it's returned as is back to the caller. If it is not, or of the package
is not available on the system, the license text is wrapped in a
`ExtractedLicensingInfo` instance.
The `SpdxLicenseExpressionCreator` object keeps track of all generated
`ExtractedLicensingInfo` instances and de-duplicates them based on the
license text. This means that if two packages use the same
SPDX-non-compliant license text, they will be wrapped by an
`ExtractedLicensingInfo` instance with the same `LicenseRef-` ID.
The reason for fallback when `license_expression` package is not
available is that it is not available on RHEL and CentOS Stream. This
implementation allows us to ship the functionality in RHEL and
optionally enabling it by installing `license_expression` from a 3rd
party repository. In any case, the generated SBOM document will always
contain valid SPDX license expressions.
Extend unit tests to cover the newly added functionality.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
FIXUP: sbom/spdx: use compliant license expressions
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the SPDX v2 model to support referencing extracted licensing
information, which is either not in the SPDX license list or can't be
expressed by the SPDX-compliant license expression.
Cover the new functionality by unit tests.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
