koji and ansible are not in RHEL repositories. Depending on them breaks RHEL
gating (see OSCI-1541): It tries to build a custom image with -tests package
in it but in the build environment there's no EPEL.
This commit makes the RPM independent from EPEL. However, we still need koji
and ansible, so the provision script now enables EPEL and installs the packages
from there. This is not nice but we have to live with that until OSCI-1541 is
solved.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
rhel 8.4 tests are added. The configs are based off of those used for
rhel 8.3. The Schutzbot Mockbuild, Base, Image, Integration, and OSTree
tests are added for 8.4. Repo overrides are added for the rhel 8.4 tests
so that the tests use rpmrepo snapshots.
The mockbuild uses the jenkins rhel84-nightly-repo credential to
override the rhel mock template's repos with rhel 8.4 nightly repos.
These repos are stored in a credential because they are internal links.
The image tests and koji tests need a special distro selector since the
rhel-8 test cases are only for rhel 8 versions less than 8.4. The rhel
8.4 tests are named with the rhel-84 pattern whereas the other rhel 8
versions have the rhel-8 pattern.
Also, instead of having only rhel-8 and rhel-8-beta repo configs for the
tests, we now have a specific repo config for each rhel release we test.
The repo is also now pulled from an rpmrepo snapshot. For whichever
distro is being tested, the approriate repo config will be copied to
/etc/osbuild-composer/repositories as rhel-8 and rhel-8-beta since this
is the naming osbuild-composer looks for. For testing purposes, the
rhel-8 and rhel-8-beta repo should be the same since eventually all rhel
releases will go from beta to not beta. The fedora repo overrides are
already done in tools/provision.sh so the rhel override is set there as
well. Currently, only rhel 8.4 requires an override.
The certificate generation is based on work by Lars Karlitski in our osbuild
CA. The server and client certs now contains Subject Alternative Name making
Python's request module and Go 1.15 happy (they deprecated certificates
without SAN).
Several reasons why we want to switch to the certificate generation:
1) The pre-generated certificates are not documented. If someone wants
to inspect them, he must know the right openssl incantation. This way,
you are able to see what's inside the certificates in a plain text.
2) The pre-generated certificates are going to expire at one point and
someone will be surprised.
3) Shipping private keys in RPMs is iffy. I know, it's just for testing but
still...
4) Auth tests are generating their own certificates. To achieve consistency,
we have two options:
a) Ship also all certificates for auth tests. That's extra 8 ones or
something like that.
b) Generate all certificates on fly. This commit does that.
5) The setup introduced by this commit is very similar to the one in our CA
making the test environment very similar to what's running in production.
tl;dr: I think this is a good step forward.
The test package should be self-contained and contain all the configuration
necessary for a known-good test run (minus secrets). This moves repo
overrides from the test orchestration into the test package.
We want all the external sources (including boot isos and repositories) to
be pinned by their content hash, and never use anything that is not strictly
defined. This moves us in the right direction, but we still have some tests
to update to use these shipped repos rather than official mirrors.
One remaining challenge is that we must make sure all our test runners have
access to the RHEL snapshots, which is not currently the case for the runners
in EC2, but a solution is in the works.
Signed-off-by: Tom Gundersen <teg@jklm.no>
