Manifest tests on Fedora 39 sometimes fail, because checking out the
specific manifest-db commit fails with:
"error: The following untracked working tree files would be
overwritten by checkout"
Use --force when checking out the ref, which will hopefully solve any
error like this.
[1] https://gitlab.com/redhat/services/products/image-builder/ci/osbuild/-/jobs/7317097983
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This is a small adjustment to how the `osbuild.util.mnt.mount` fuction
is imported. While adding unit tests, the tests failed when trying to
patch the function, this commit fixes the issue.
Disable `curl --parallel` by default until the failure in
https://github.com/osbuild/osbuild-composer/pull/4247
is fully understood. It can be enabled via the environment:
```
OSBUILD_SOURCES_CURL_USE_PARALLEL=1
```
in the osbuild-composer test.
When using a modern curl we can download download multiple urls
in parallel which avoids connection setup overhead and is generally
more efficient. Use when it's detected.
TODO: ensure both old and new curl are tested automatically via
the testsuite.
Modern curl (7.68+) has a --parallel option that will download
multiple sources in parallel. This commit adds detection for this
feature as it is only available after RHEL 8.
In addition we need some more feature to properly support --parallel,
i.e. `--write-out` with json and exitcode options. This bumps the
requirements to 7.75+ which is still fine, centos9/RHEL9 have
7.76.
Instead of passing the url and options on the commandline this
commit moves it into a config file. This is not useful just yet
but it will be once we download multiple urls per curl instance.
Setting the user-agent using `--header` is broken in combination with
`--location`, `--proxy`, and an https endpoint which redirects. The
user-agent sent to the proxy changes after the client is redirected,
tripping up proxies.
For more information see https://issues.redhat.com/browse/RHEL-45364
Don't instruct mock to not bootstrap the chroot. This is making builds
to fail on F40 due to DNF5. The error message suggests to bootstrap
chroot.
Example:
https://gitlab.com/redhat/services/products/image-builder/ci/osbuild/-/jobs/7239614060#L1363
Note that e.g. osbuild-composer does not specify the
--no-bootstrap-chroot option either.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Update RHEL runners to the same versions that correspond to the actual
state of which release is currently GA and which is in development
(nightly). Specifically, run jobs only on:
- RHEL-8.10 GA
- RHEL-9.4 GA
- RHEL-9.5 nightly
This is the same set of RHEL workers, as used by 'manifest-db'
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Update manifest-db ref to the latest one. This should solve the problem
with GCP el8 repos being gone in the old manifests.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Similar to what was explained in 2e6d49fbe this commit updates
the l2hash in test_assemblers to the new values from fc40 images.
Sadly it is hard to derive them from first principles (see the
other commit) and given that this is legacy code it is probably
fine this way.
In the release loop upstream changes are merged to Centos every two
weeks. This creates a delay in error detection when new tests
being added upstream.
Running tests in Centos based containers on top of the upstream code
more frequently should speed up error detection.
Newly add building of RPMs for c10s on all supported arches.
Extend the c9s config to build RPMs for all supported arches.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The `tree-diff` tool is used by the `gen-stage-diff` tool to generate
the stage `diff.json` files. A few of these stages have timestamped
files that need to be edited after the tool is run to replace the
`sha256` content hash with `null` so the stage diff tests ignore these
files.
This commit updates the `tree-diff` tool to check through a list of the
files that contain timestamps and to use the `null` value rather than
the `sha256` content hash so the stage tests don't fail.
Since the `/etc/shadow` file contains a timestamp we need to add a
`null` value rather than a `sha256` hash to tell the diff tool to ignore
these fields. The issue is that the timestamp will always be different
meaning the tests will pass for a day, but then fail after that.
I'm not sure what happened, but the test case started failing on the
diff on 'main'. I didn't change anything related to this test case in my
PR. The previous changes adjusted the vars, specifically the Fedora
snapshot date used to generate the manifests, but the test passed on
it.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
With the `oscap-utils-1.3.10` release, the `autotailor` command now
supports importing a JSON tailoring file[1] that is then converted to the
XML tailoring file which is consumed by the `oscap` command in the
remediation stage.
[1] https://github.com/ComplianceAsCode/schemas/blob/main/tailoring/schema.json
Since updating the snapshots the diffs for some stage tests have
changed. This commit updates the diffs accordingly.
I followed the same steps used in 1148a6e.
The optional_metadata_types option isn't available in older versions of
dnf. We could version-guard the option, but let's be more explicit and
check if it exists instead.
If it doesn't we can safely ignore it and rely on dnf's default
behaviour.
The original CentOS Stream GPG key uses SHA-1 in its signature. However,
SHA-1 is by default not allowed by the c10s / el10 crypto policy. As a
result, running the stage tests which use c9s on c10s / el10 are failing
when rpmkeys tries to import the key.
As part of CS-1616 [1], the CS GPG key has been resigned using SHA256,
however only in c10s for now. Let's use the SHA256 signed GPG key from
c10s for c9s manifests, to make tests pass also on c10s / el10.
[1] https://issues.redhat.com/browse/CS-1616
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
When the filelists are enabled in the optional metadata, the local cache
for the depsolve will include a filelist file for each repository.
Count the files matching *filelists* using glob() and compare them with
the number of repositories when the option is enabled.
When the option is not enabled, there should be no filelists.
We originally enabled filelists unconditionally in dnf5. Let's make it
optional as part of the 'optional-metadata' option in the arguments so
that the caller can decide if it's needed, for example when using dnf5
with older repositories that have packages with file dependencies, but
don't download them when they're not needed, since the filelists are
quite large.
Filelist repo metadata is required when using newer versions of libdnf
with old repositories or packages that specify dependencies on files.
For example, depsolving with RHEL 9.3 repos on Fedora 40 fails to
resolve platform-python with the message
nothing provides /usr/libexec/platform-python needed by platform-python-...
Not loading filelists is beneficial because it greatly reduces the size
of the metadata to download. Filelists were downloaded by default for
repositories in the past, but with newer versions of dnf/libdnf (in
Fedora 40), they are downloaded on-demand (e.g. when running a query
that requires file information). Newer package guidelines prohibit
depending on file paths, so filelists are not needed for package
depsolving with newer distro repositories.
Add an 'optional-metadata' property to the 'arguments' object of the
depsolve request, so that callers can enable any optional metadata they
need.
Further reading:
- https://libdnf.readthedocs.io/en/stable/tutorial-py/#case-for-loading-the-filelists
- https://github.com/rpm-software-management/dnf/releases/tag/4.19.0
- https://dnf.readthedocs.io/en/stable/user_faq.html#starting-with-fedora-40-i-noticed-repository-metadata-is-synchronized-much-faster-what-happened
The `test_osbuild_mount_failure_msg` currently fails on fc40 when
run in tmt, see:
https://artifacts.dev.testing-farm.io/c6588a82-a2cb-46df-8ca8-85dd809465f2/
This is because the failure output is slightly different between
a container and a VM/real-machine. The test ensures that we capture
the output of mount and present to the user (for easier debugging).
So this commit updates this test once more for the error string
(that part of the error comes directly from the kernels fsconfig).
If we need another update of the string we should reconsider this
test and e.g. just use `testutil.mock_command()` for this. But
for now it's easier to just add this one more failure string.
