Extend the integration test with a new case, testing that direct upload
to the cloud works for Koji composes. Test this using a single cloud
provider, specifically AWS.
The test case submits a new osbuild-image build using Koji CLI,
determines the image information once the build finishes and then checks
that such image exists in AWS. The image is then deleted as part of the
test case tear-down.
The AWS credentials are now configured in the worker's configuration, if
the appropriate environment variables are set.
Update the SPEC file with a new test dependency and update the required
osbuild-composer version.
Use latest osbuild and osbuild-composer releases. This is important to
be later able to test direct uploading to the cloud for Koji composes.
Also the mock openid server had to be enhanced to be usable for testing
`koji-osbuild`.
Modify used osbuild-worker configuration to use JWT for authentication
with composer. Update configurations of both, composer and worker to
handle multi-tenancy.
Do not start any services in `schutzbot/deploy.sh`, because at that
point, none of the configuration files are in place.
Ensure that the correct units are started and masked by
`test/integration.sh` to simulate the Service scenario more closely.
This means that the local worker is masked and only remote worker is
started.
Co-authored-by: Jakub Rusz <jrusz@redhat.com>
Switch the authentication method that the koji builder plugin
uses from mTLS to SSO. Since we are using the mock openid
server from the `osbuild-composer-tests` package. Make this
package a dependency of `koji-osbuild-tests`.
This is similar to how other osbuild packages are testing: everything
that's needed for testing is included in the tests package or a
dependency of it. The test runner then runs every executable in
/usr/libexec/tests/<packagename>. This gives a simple test API to
projects depending on this package (notably osbuild-composer).
The local development workflow described in HACKING.md is meant to
continue to work. To ensure this, all relevant scripts gained a
TEST_DATA variable, which defaults to `./test`, but is set from $1 to
the installed path from integration.sh.
Instead of taking podman-plugins from the source directory, use the one
that will be released into RHEL 8.3.1.
This will simplify moving tests into an rpm.
Generate the certificate to be used for koji in make-certs.sh by
the same CA that also generates the composer and client certs.
Create a single certificate that uses the SubjectAltName (SAN)
extension to cover two domains: localhost, org.osbuild.koji.koji,
which previously was done via two separate certificates; this is
the legacy usage which stopped working with go 1.15 (see previous
commit). As a consequence the apache config is modified to use
only one virtual host with a ServerAlias directive.
On RHEL a pre-compiled version is currently installed from the
vendor directory (see the README.md there). On Fedora it is
packaged in podman-plugins and that used to be installed
without explicitly asking for it, but that seems to have changed.
Explicitly install it on Fedora to ensure it is present.
Symptom of the dns plugin missing is that the koji hub container
will not start because it will block waiting for the postgres
container forever.
Move the container directory, containing the container definitions
for all the test containers, to test/, where all the other test-
related files are located (with the exception of `Schutzbot`).
Use `test/build-container.sh` to build the container, instead of
replicating that in `test-integration.sh`.
Upstream composer has introduce a few changes that we need to
adapt for:
- the koji composer API is now exposed on the standard https
port (443). Thus koji hub and web need to move to a different
pair: 8080 (http) and 4343 (https). Change the scripts and
tests for that
- the koji API gained a prefix 'api/composer-koji/v1/'. Change
client and unit tests to use that prefix. Use urljoin to
create new APIs
- composer configuration format (osbuild-composer.toml) has
changed and now also includes configuration for the CA
and allowed domains
- update the composer RPM repositories to the commit for the
21 upstream release.
On Fedora use containers based on Fedora, on RHEL use containers
based on RHEL, so we test the correct integration of the plugins
into the corresponding distribution.
Instead of manually copying credentials around, use the new
copy-creds.sh script. Also don't manually initialize kerberos,
because that is now done automatically by the worker and the
builder.
Instead of creating a compose from integration.sh, use the new
python based integration tests. The integration.sh script will
this just setup the environment and the actual tests are all
found in test/integration/
Try testing a compose via the koji, which involves creating using
the koji command line plugin to make the XMLRPC call to koji hub,
where the osbuild koji hub plugin verifies the parameters and then
creates the task. The osbuild koji plugin for the builder is then
picking up the task, and uses composer's koji API to request a
compose. Once this is successful it will be imported by composer
into koji via the 'CGImport' method.
The `koji osbuild-image` command waits for all this and reports
whether the task was successful or not via its exit code (and
on stdout).
This uses a fleet of containers: a database one, a kerberos kdc
one, another one for the koju hub and finally one for the koji
builder. The pre-build RPMs are used to install the plugins.
NB: On RHEL we need to manually install the `dnsname` podman
plugin, since it is missing, but required so that containers
can address each other by hostnames.
See [schutzbot/vendor/README.md](schutzbot/vendor/README.md)
The CI is in two stages, for each supported distro.
First the RPMs are generated from the spec file in the repo for the
given distro and architecture.
Once all the RPM builds have succeeded successfully, a test machine is
provisioned with osbulid-composer installed, and koji API enabled.
The repository containing the RPMs of the code being tested is also
enabled on the test machine, and the cli client is installed.
Finally, the test/integration.sh script is executed, which currently
does nothing.
Signed-off-by: Tom Gundersen <teg@jklm.no>
