Commit graph

3479 commits

Author SHA1 Message Date
Tomáš Hozza
7d7e0a7296 tools/test/depsolve: factor out depsolve sbom test case
Extract testing of SBOM support into a dedicated test case. There's no
added value in running all SBOM test cases for all types of depsolve
transactions.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-18 10:50:19 +01:00
Tomáš Hozza
29f93af0aa tools/test/depsolve: factor out search repo combos test case
Testing all repo config combos for each search test case does not
really increase the test coverage for repo config combos. It just
increases the run time of the test.

Move the repo config combos testing to a dedicated test case, which will
test search for two packages from two different repositories.

For the original `test_search()`, always use repo configs in the
request.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-18 10:50:19 +01:00
Tomáš Hozza
de9f4ba3cb tools/test/depsolve: always pass dnf_config as dict
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-18 10:50:19 +01:00
Tomáš Hozza
b917f6a238 tools/test/depsolve: factor out depsolve repo combos test case
Testing all repo config combos for each depsolve test case does not
really increase the test coverage for repo config combos. It just
increases the run time of the test.

Move the repo config combos testing to a dedicated test case, which will
test depsolving two packages from two different repositories.

For the original `test_depsolve()`, always use repo configs in the
request.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-18 10:50:19 +01:00
Tomáš Hozza
52d027393c tools/test/depsolve: factor out test case repo servers filtering
Extract the code that filters and composes repo servers for a test case
into a separate function. This enables reusing it in all places that did
the same thing. The problem would get more prominent as we would
separate some test scenarios into separate test cases.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-18 10:50:19 +01:00
Tomáš Hozza
3743654178 tools/test/depsolve: clean up test_depsolve_result_api()
The `test_depsolve_result_api()` test case was parametrized based on
`dnf_config`, but in reality, the `depsolve()` call always used an
empty dict as `dnf_config`. Effectively, it was being tested three
times with DNF4.

In addition, don't pass optional arguments to `depsolve()`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-18 10:50:19 +01:00
Tomáš Hozza
e635a8a067 tools/test/depsolve: don't always pass repos, root_dir and opt_metadata
Modify the `config_combos()` to return `repo_configs` and `root_dir`
only if it should be really used. Otherwise, return `None`. Modify all
helper functions for dnf-depsolve API calls to add relevant fields to
the request JSON, only if the relevant values are set. This makes the
test cleaner, since previously, the `root_dir` was always set.

The same applies to `dnf_config`, which could be set to `None` already,
so let's make it optional.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-18 10:50:19 +01:00
Tomáš Hozza
a76d3e406b tools/test/depsolve: factor out func to generate repo config
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-18 10:50:19 +01:00
Michael Vogt
b22cbd3298 monitor: limit the amount of data sent in JSONSeqMontior.result()
This commit limits the output in the json pipeline to a "reasonable"
length. We ran into issues (e.g. [0]) from a combination of a stage
that produce tons of output (dracut, ~256 kb, see issue#1976) and
the consumer ("images" osbuild/monitor.go) that used a golang scanner
with a max default buffer of 64kb before erroring. So limit it
here.

The stage result from via json is mostly for information and any error
will most likely at the end. Plus consumers can collect the individual
log lines on their own if desired via the "log()" messages that are
stream in "real-time" with the added benefit that e.g. timestamps
can be added to the logs etc.

[0] https://issues.redhat.com/browse/RHEL-77988
2025-02-18 10:36:59 +01:00
Dusty Mabe
a08736be49 stages/coreos.live-artifacts: use /var/tmp for mkfs.erofs
This helps reduce the memory requirements of mkfs.erofs since
it won't be writing to tmpfs (memory backed) storage, which is
what is mounted on /tmp/.
2025-02-17 20:39:38 +01:00
Michael Vogt
0c1a8c802f stages: add basic qemu cmdline test
This commit adds a basic fake commandline test for the qemu
stage.
2025-02-17 13:21:40 +01:00
Dusty Mabe
4e033c305e stages: add more options to qemu vmdk disk type
The CoreOS team uses the compat6 and adapter_type options when creating
a VMDK for AWS.

e1943d6adb/src/cosalib/qemuvariants.py (L48)
2025-02-17 03:41:11 -05:00
Sanne Raymaekers
441fbf70d6 org.osbuild.cloud-init: add new datasources and network section
Adds WSL and NoCloud datasources. The network section only allows you to
disable network configuration by cloud-init for now.
2025-02-15 10:38:12 +01:00
schutzbot
0e319018b6 Schutzfile: Update images dependency ref to latest 2025-02-14 20:17:40 +01:00
Tomáš Hozza
8fe28a2e98 GH/update-images: fix missing quote in the script
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-14 17:48:13 +01:00
Nikita Dubrovskii
035ced543b stages/coreos.live-artifacts: update comments/names to reflect erofs related changes 2025-02-13 13:40:20 +01:00
Brian C. Lane
d30cbde6f8 workflows: Switch utility workflows to using ubuntu-24.04
Pin the os version at 24.04 to avoid unexpected changes that can happen
when using ubuntu-latest
2025-02-12 22:23:32 +00:00
Brian C. Lane
26b48e87a9 workflows: Use ubuntu-24.04 for tests
Pin the ubuntu os version at 24.04 to avoid unexpected changes that can
happen when using ubuntu-latest
2025-02-12 22:23:32 +00:00
Michael Vogt
1a2637b1a3 stages: replace "logger" binary in the dracut chroot
This commit replaces the `/usr/bin/logger` binary in the dracut
chroot with a bind mount to `/usr/bin/true` to silence the spam
that we get from dracut during initramfs generation:
```
logger: socket /dev/log: No such file or directory
```

Unfortunately I could not find a nicer way, it seems it is
not possible to simply pass `sysloglvl=0` via the commandline
or an environment.

The extra complication here is that the dracut stage mounts
`devtmpfs` which will likely include:
```
/dev/log -> /run/systemd/journal/dev-log
```
but of course inside this chroot there is no `/run` which
leads to these messages.

Closes: https://github.com/osbuild/osbuild/issues/1976
2025-02-12 15:44:04 +00:00
Paweł Poławski
ed732b36d8 stages: add tests to org.osbuild.ovf
Add some tests for org.osbuild.ovf.

Extracted from https://github.com/osbuild/osbuild/pull/1751
2025-02-12 11:24:47 +01:00
schutzbot
6f69be6897 Post release version bump
[skip ci]
2025-02-12 08:30:17 +00:00
schutzbot
5b15951cb2 schutzfile: Update snapshots to 20250201 2025-02-12 07:05:28 +01:00
Nikita Dubrovskii
ab1f48795b stages/coreos.live-artifacts: add erofs support 2025-02-11 17:41:35 -05:00
Tomáš Hozza
942c74ded1 Tools/osbuild-image-info: make read_selinux_ctx_mismatch more robust
Modify the function able to handle messages about skipped binary
fcontext files and skip them. This started to happen on c10s. Extend the
unit test to cover this new scenario.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-11 20:18:07 +01:00
Tomáš Hozza
3f8fe3381d Tools/osbuild-image-info: add unit test for read_selinux_ctx_mismatch
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-11 20:18:07 +01:00
Simon de Vlieger
b2ec2cd5ee Revert "ostree: introduce optional subpath feature"
This reverts commit 3bb5bedd8e as it
introduces problems in the caching system used by `osbuild` [1]

[1]: https://github.com/osbuild/osbuild/issues/2009
2025-02-11 17:39:45 +01:00
Albert Esteve
a6b64860b3 stages/kernel-cmdline: add max cmdline option
Add an additional option called `kernel_line_size`
to allow setting a maximum cmdline size check
value for custom kernels or other restrictions.

This will override the arch defaults, if not set,
then the size map is checked, and if the current
architecture is not in the map, fallback to
4096, which is the max value allowed for
COMMAND_LINE_SIZE.

Signed-off-by: Albert Esteve <aesteve@redhat.com>
2025-02-11 09:16:17 +00:00
Albert Esteve
7645d717e6 stages/kernel-cmdline: fix link in comment
Signed-off-by: Albert Esteve <aesteve@redhat.com>
2025-02-11 09:16:17 +00:00
Lukas Zapletal
3bb5bedd8e ostree: introduce optional subpath feature 2025-02-10 20:17:36 +01:00
Albert Esteve
bd316ddb8f kernel-cmdline: add size check
Add check to ensure that the size of
the parameters does not exceed the
maximum kernel cmdline size.
Otherwise, the parameters will
be truncated and the command line
will fail.

The size is arch-dependant. In
order to not to over-complicate
the search of the value in the
kernel files (which will probably
not be installed in most cases),
it uses a map with some values
for common architectures.

If architecture is not found in
the map, defaults to 4096, which
is the maximum posible size for
COMMAND_LINE_SIZE.

Signed-off-by: Albert Esteve <aesteve@redhat.com>
2025-02-07 15:18:45 +01:00
Tomáš Hozza
f299c02414 Add initial CODEOWNERS file
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-06 18:33:14 +01:00
Tomáš Hozza
89d8aa623b GH actions: update the osbuild-ci* images to latest-202501201758
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-05 12:00:20 +01:00
Tomáš Hozza
c69b16f333 Tools/osbuild-image-info: rework analyse_iso()
Rework the function to actually fail in case it can't analyze the
provided ISO. Previously, the tool would silently fail to analyze ISO,
generate and generate an empty report. Fix this.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-05 12:00:20 +01:00
Tomáš Hozza
b978559a80 Test/osbuild-image-info: test failures in analyse_iso()
Add unit test for testing failures in analyse_iso(). The function
should fail if it can't analyze the provided ISO.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-05 12:00:20 +01:00
Achilleas Koutsou
45c6006cfc stages: Where is mandatory in systemd.unit.create
`systemd.mount(5)` describes both `What` and `Where` as mandatory
options for the `Mount` section of a unit.
2025-02-04 22:24:02 +01:00
schutzbot
33fd0bd183 Post release version bump
[skip ci]
2025-02-04 10:34:47 +00:00
Tomáš Hozza
ce4bc01b7b osbuild-image-info: exit with non-zero value on empty report
For the purpose of using this tool in tests (specifically for manifest
tests where we diff image-info reports), it is important that the tools
exists with non-zero value if the final report is empty.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-03 12:58:29 +01:00
Tomáš Hozza
542cf4a799 Test/osbuild-image-info: test failure on empty report
Test that the main() function exits with non-zero value if it would
return an empty report.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-03 12:58:29 +01:00
Tomáš Hozza
bf64127c26 osbuild-image-info: use .splitlines() instead of .split("\n")
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-03 12:47:35 +01:00
Tomáš Hozza
35dda557ab GH Action/update-images: escape back-quote when echoing PR body
Make sure that shell does not interpret the text within the back-quote
as a command to execute in a sub-shell.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-03 12:47:35 +01:00
Tomáš Hozza
c9c94380dc update-schutzfile-images: don't catch HTTPError
Catching the exception just to print it and exit with non-zero exit
return code. Let's not catch it at all.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-03 12:47:35 +01:00
schutzbot
d8f353b3b7 Schutzfile: Update images dependency ref to latest 2025-01-31 12:22:25 +01:00
Tomáš Hozza
2622782604 Test/manifest_tests: --osb-store -> --osbuild-store
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-31 10:18:14 +01:00
Tomáš Hozza
b2bf7bc11b GitLab CI/manifest_tests: explicitly specify workdir
Explicitly specify the workdir when running manifest_tests, make
potential debugging of the test case on CI runner easier (because
otherwise the workdir would get removed after failing test).

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-31 10:18:14 +01:00
Tomáš Hozza
0b158c3fd3 Test/manifest_tests: use temporary dir if workdir is not specified
In case the workdir is not provided to the script explicitly as an
argument, the script will use a temporary directory under /var/tmp as
its workdir. In such case, the workdir will be deleted on exit. This
should mitigate potentially confusing behavior when executing the script
multiple times with different arguments, while never specifying the
workdir.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-31 10:18:14 +01:00
Tomáš Hozza
a6c09fd441 osbuild-image-info: refactor opening of LVM LV devices
Extract the opening of LVM LV devices from `discover_lvm()` to
`OSBuildDeviceManager` class as `open_lvm_lv()` method.

`open_lvm_lv()` returns the path to the opened device in the devpath set
in the underlying `DeviceManager`. The `org.osbuild.lvm2.lv`
implementation takes the responsibility for creating and managing
device nodes. This means that we don't need to be creating any device
nodes directly in `osbuild-image-info`, especially in the current
working directory. This was previously causing issues when inspecting
two images with different LVM layout in a sequence.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-31 10:18:14 +01:00
Tomáš Hozza
116bd17244 osbuild-image-info: add wrapper for device.DeviceManager
Add a new class `OSBuildDeviceManager`, which wraps
`devices.DeviceManager`, so that we can consolidate all code that is
opening devices using osbuild, in it. As the fist step, move the
`loop_open()` function to the class.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-31 10:18:14 +01:00
Tomáš Hozza
dbf01e2d1a osbuild-image-info: volume_group_for_device -> lvm_vg_for_device
Rename the function for naming consistency and always include the actual
error from `pvdisplay` when raising RuntimeError.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-31 10:18:14 +01:00
Tomáš Hozza
0fba11369c osbuild-image-info: extract discovery of LV names into a function
Extract the code that discovers LVM LV names for a given VG, from
`discover_lvm()` into a separate function `lvm_lvs_for_vg()`. This
improves the readability of the code. In addition, some values returned
by the `lvdisplay` invocation were never used. Don't request them and
simplify the code. Rename variables that hold LV names to clearly
express that.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-31 10:18:14 +01:00
Tomáš Hozza
9bdde1bc80 Gitlab: don't run manifest-tests on RHEL-8.10
Per discussion in the team, we see little value in rebuilding RHEL-8.10
images on RHEL-8.10 for the purpose of manifest testing in osbuild. So
let's not do that anymore.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-31 10:18:14 +01:00