Now that osbuild and osbuild-composer use predictable destinations
for their dnf repositories, those can be used and auto-generated
from the environment and a specific git commit (identified by
the hash).
This will make updating easier, because the only thing we need to
change is the commit hash.
For osbuild-composer the latest release is used, for osbuild the
predictable repos were introduced after the latest release, so
the commit that introduced the feature is used.
Add a new command line option `--skip-tag` that will skip tagging
after a successful build. The help text is copied from the same
option of other sub-commands in the koji client. The hub plugin's
jsonschema was updated accordingly, and the builder plugin will
skip the tag if the option was requested.
Tests were added or augmented for all three plugins to test the
new option.
Instead of just uploading the compose-status.json at the very end
of the build, keep updating it every time it is fetch. This makes
it easier to follow along, especially if there are multiple image
request being built.
Upload the final compose status as JSON, so it can be inspected,
which could be especially handy if multiple image requests were
made and only one of them failed, to quickly pin down the one
that failed, without having to go through all the logs.
Add a '.as_dict()' method to ComposeStatus, which can be used to
serialize the object as JSON or properly log it. Indeed use it
in the debug log to avoid a useless message that contains:
<_koji_plugin__osbuild.ComposeStatus object at 0x7fb80172b820>
For each compose, mock also the "compose/<id>/logs" API endpoint
and just return some string. Add a feature to be able to control
the http status of the route though, so we can simulate failures
during log fetching.
Use the new log API introduced in osbuild-composer >= 24, to fetch
the logs for the compose and attach them to the task. It is a non-
fatal error if fetching the logs fails; in that case a warning is
emitted. NB: logs are attached no matter the compose result.
Logs are per image-request, the content is JSON, but otherwise not
further specified.
Preserve the order of architectures in the image requests inside
the compose requests, i.e. don't transform 'arches' into a set,
which has random order. It is not that anyone should really
depend on it, but there is also no need to mess with the order,
potentially making it harder for humans to match requested arches
with compose request content and logs.
In the builder unit test that checks a successful compose, use
more than one architecture, and check that an image request was
created for all of those. This should expose bugs multi-arch
specific bugs.
Generate the certificate to be used for koji in make-certs.sh by
the same CA that also generates the composer and client certs.
Create a single certificate that uses the SubjectAltName (SAN)
extension to cover two domains: localhost, org.osbuild.koji.koji,
which previously was done via two separate certificates; this is
the legacy usage which stopped working with go 1.15 (see previous
commit). As a consequence the apache config is modified to use
only one virtual host with a ServerAlias directive.
Rework the generation of certificates, in order to make use of
the SubjectAltName (SAN) extension, that is required for modern
usage of TLS (see RFC 2818, or [1]) and now enforced by default
for go version 1.15[2] (Fedora 33). For this a different config
file is used, originally written by Lars, and assign SANs to
the server and client certificates. Additionally, the correct
extensions are used for each of those, so that their usage is
limited to the server or client use case. The client certificate
is renamed from "worker" to "client".
The lifetime of the certificates is increased, as a side effect of
the new config file.
[1] https://github.com/urllib3/urllib3/issues/497
[2] https://golang.org/doc/go1.15#commonname
Co-authored-by: Lars Karlitski <lars@karlitski.net>
The official postgres container image is fetched from docker.io,
which recently introduced rate limiting, which makes introduces
possible CI error if we run into this limit.
Instead use a custom mirror[1] of the official image, hosted on
quay.io[2]. As a side effect this updates the posgres version
from 12 to 13.
[1] osbuild/containers@7db3c68
[2] quay.io/osbuild/postgres:v1
Instead of installing and enabling the koji socket, use the new
api socket. The koji socket is not properly wired up anymore
and any attempt to actually use it with hang.
Also update the Fedora CI to osbuild 23, so that it is in sync
with the RHEL ci. This is important since the koji socket has
been deprecated by the api socket and the koji socket actually
does not work anymore.
In the basic invocation test, specify the expected argument values
not only their type. With the exception of the default image type
they correspond to the various command line arguments given.
Add more supported build architectures to the standard build tag.
This will check the logic for (un-)supported build architectures
in the builder plugin. If we only have one arch, which is the
requested arch, it will not catch logic errors, that invert the
supported vs requested architectures.
The builder plugin checks that all of the requested architectures
are indeed supported, which is determined via the build tag. It
does that by constructing two sets, the requested architectures
and the supported architectures, and then constructing the set
of (asymmetric) differences between the one and the other. This
difference was meant to be "requested" - "supported", which then
will contain architectures that have been requested but are not
supported, or be empty in case all requested architectures are
indeed supported (the good case). However, previously the diff
was done the other way around ("supported" - "requested"), which
will then return a set of architectures that are supported but
were not requested, which is not what we want to check for.
Invert that the argument of the difference to indeed end up with
"requested" - "supported", which results in the correct check.
The hub plugin uses jsonschema, so it must explicitly depend on
it. The dependency is installed "by hand" in the container since
by default, the plugin is not installed via the RPM, but copied
from the source repository and thus all dependencies must be
present already.
On RHEL a pre-compiled version is currently installed from the
vendor directory (see the README.md there). On Fedora it is
packaged in podman-plugins and that used to be installed
without explicitly asking for it, but that seems to have changed.
Explicitly install it on Fedora to ensure it is present.
Symptom of the dns plugin missing is that the koji hub container
will not start because it will block waiting for the postgres
container forever.
Change the use of DATA_DIR across all scripts so that it will
only be set if it does not yet exist. This makes it possible
to change the location of it across all scripts at once.
Change the use of SHARE_DIR across all scripts so that it will
only be set if it does not yet exist. This makes it possible
to change the location of it across all scripts at once.
Move the container directory, containing the container definitions
for all the test containers, to test/, where all the other test-
related files are located (with the exception of `Schutzbot`).
Use `test/build-container.sh` to build the container, instead of
replicating that in `test-integration.sh`.
