Add support for fetching manifests via the compose/<id>/manifests
API endpoint. A failure to fetch them is not critical, since it is
possible the manifests don't exist, e.g. when depsolving fails.
The manifest is attached per image request.
De-serialize the koji init and import logs, required fields in the
ComposeLogs, and if non-empty, attach them to the task.
Update the tests to check for the presence of these logs.
Instead of getting the `koji_build_id` from the direct reply of
the compose request call, use the one returned in the compose
status.
The reason behind this is that composer was changed so that the
CGInitBuild call to koji is now being done by a worker and not
composer itself. This means that once the compose request call
returns, the build id is not yet known. In composer release 24,
the compose request call internally waits for the worker that
does the CGInitBuild API call, but that will be changed, and
the koji_build_id will then not be returned from the compose
request API call anymore. This prepares for that. The tests are
also adapted to simulate the new behavior.
NB: this makes composer 24 a dependency, since the build id is
taken from the ComposeStatus, which was only added there.
Add a new command line option `--skip-tag` that will skip tagging
after a successful build. The help text is copied from the same
option of other sub-commands in the koji client. The hub plugin's
jsonschema was updated accordingly, and the builder plugin will
skip the tag if the option was requested.
Tests were added or augmented for all three plugins to test the
new option.
Instead of just uploading the compose-status.json at the very end
of the build, keep updating it every time it is fetch. This makes
it easier to follow along, especially if there are multiple image
request being built.
Upload the final compose status as JSON, so it can be inspected,
which could be especially handy if multiple image requests were
made and only one of them failed, to quickly pin down the one
that failed, without having to go through all the logs.
Add a '.as_dict()' method to ComposeStatus, which can be used to
serialize the object as JSON or properly log it. Indeed use it
in the debug log to avoid a useless message that contains:
<_koji_plugin__osbuild.ComposeStatus object at 0x7fb80172b820>
Use the new log API introduced in osbuild-composer >= 24, to fetch
the logs for the compose and attach them to the task. It is a non-
fatal error if fetching the logs fails; in that case a warning is
emitted. NB: logs are attached no matter the compose result.
Logs are per image-request, the content is JSON, but otherwise not
further specified.
Preserve the order of architectures in the image requests inside
the compose requests, i.e. don't transform 'arches' into a set,
which has random order. It is not that anyone should really
depend on it, but there is also no need to mess with the order,
potentially making it harder for humans to match requested arches
with compose request content and logs.
The builder plugin checks that all of the requested architectures
are indeed supported, which is determined via the build tag. It
does that by constructing two sets, the requested architectures
and the supported architectures, and then constructing the set
of (asymmetric) differences between the one and the other. This
difference was meant to be "requested" - "supported", which then
will contain architectures that have been requested but are not
supported, or be empty in case all requested architectures are
indeed supported (the good case). However, previously the diff
was done the other way around ("supported" - "requested"), which
will then return a set of architectures that are supported but
were not requested, which is not what we want to check for.
Invert that the argument of the difference to indeed end up with
"requested" - "supported", which results in the correct check.
Upstream composer has introduce a few changes that we need to
adapt for:
- the koji composer API is now exposed on the standard https
port (443). Thus koji hub and web need to move to a different
pair: 8080 (http) and 4343 (https). Change the scripts and
tests for that
- the koji API gained a prefix 'api/composer-koji/v1/'. Change
client and unit tests to use that prefix. Use urljoin to
create new APIs
- composer configuration format (osbuild-composer.toml) has
changed and now also includes configuration for the CA
and allowed domains
- update the composer RPM repositories to the commit for the
21 upstream release.
When a compose / build was successful, tag it with the destination
tag. This will create a sub-task and wait for it. The individual
parameter are the same used for standard "image" koji tasks.
After creating the compose request object, save it as meta-data of
the task. This should come in especially handy when a compose gets
rejected by osbuild-composer.
Instead of having the individual components for a compose request as
argument, make create_compose just take a ComposeRequest object.
This will pave the way to save the compose request as meta-data for
the task, which can be useful if something goes wrong, e.g. creating
the request.
When returning the result from the task handler function, return
a more complete and structured object in all cases. The name of
the sub-object is named after the service the item belongs to;
specifically composer is used for what belongs to (osbuild)-composer.
Currently we were passing the repo information as a comma
separated string, which is fragile, since urls can contain
commas. Just transfer them as arrays of strings.
The requests cert argument accepts a tuple with the key and the
cert, or one string as a path to a combination of both. Create
a new helper and use that to parse the "ssl_cert" configuration
option as well as the command line option. Change the latter
into a string.
Use a global constant `DEFAULT_CONFIG_FILES` for the default
plugin configuration files. Besides giving it more visibility
it also makes it easier to test, because in the unit test,
this can easily be overwritten.
Add support for client side certificates via a new configuration
option: "ssl_cert", which should be a path to the certificate or
a comma separated list of paths to certificates.
Additionally, the "ssl_verify" option controls how the server
certificate is validated. This can either be a boolean ("True",
"False") or a path to a CA file.
NB: The url for the composer host must contain "https" for the
ssl/tls engine to kick in.
When used as a stand-alone koji API client, two new command
line options are provided, --cert and --ca. See their help
text for a description.
Use the requests python package, which is more convenient to use,
and also supports sessions, where global options, like TLS certs
can be set.
Adapt the container and the spec file to install and require
the python package.
The 'koji' parameter for the compose request is called that way to
follow the OpenAPI spec, so silence the "redefined-outer-name"
warning.
The handler function needs have the actual task arguments and thus
on purpose deviates from the argument list of the base class'
handler method. So silence that.
