This is necessary once so we can enable the upstream release bot workflow,
which consists of just pushing a tag and bumping the version number
directly after the release is done.
This adds support for specifing the package sets for repositories;
on the command line this can be done via `--repo-package-set` with
and argument of `;` separated package set names. This will result
in repo information being transported via dict instead of plain
strings. Thus the hub plugin's schema was modified accordingly.
Last but not least, the builder plugin now can decode these dicts
and setup the repos accordingly.
Test were added for plugins as well as the integration test changed
to use this new feature.
The first upstream commit that supports this feature is pinned.
When we are on an RHEL 8.x host we need to supply `rhel-8x` as
target distro. The previous code just used the major version so
we always built `rhel-8`, i.e. RHEL 8.3.
It similar to BuildTask - so it doesn't do much on the builder and
doesn't use a lot of resources. It makes sense to have much lower weight
for this type of task.
OStree compose requests need special options, like the `ref` the
`parent` and the `url`. Add support for those options to all three
plugins:
The command line plugin now takes `--ostree-{parent,ref,url}`
and passes it to koji via the existing options dictionary.
The JSON schemata in the hub plugin was adjusted to allow these
new options.
Finally the builder plugin will look for the new `ostree` dict
inside the options, create an `OSTreeOptions` object from it,
and attach it to each image request.
NB: since the ostree options are per image request and are thus
architecture dependent we support a "$arch" substition in the
`parent` and `ref` options that will be resolved by the plugin;
this allows to builds arch specific commits for with a single
compose request.
Add the respective unit tests.
Map the image types used by the koji API to the image types used
by the cloud api. This should allow for a smooth transition when
the plugin is upgraded, i.e. the pungi configuration can be used
unmodified. After all the plugins are upgraded the pungi config
should be changed to use the native image types and then this
mapping could be removed again.
Take the current list of valid image types currently supported by
the cloud api and validdate it during the compose request. Also
allow a test "image_type" image type which is used all over the
place in the testing code.
Composer now[1] has integrated the koji API into the "cloud API"
and thus we can use this more general purpose and powerful API
instead of using the specialized koji API endpoint.
Adapt the request and response structures as well as the unit
tests to use that.
[1] PR #2214, commit 11e2ae45284bfb0d89ef1c1e0d2aa4ae230ea573
Switch the authentication method that the koji builder plugin
uses from mTLS to SSO. Since we are using the mock openid
server from the `osbuild-composer-tests` package. Make this
package a dependency of `koji-osbuild-tests`.
Implement support for authentication via OAuth2 using the client
credentials "Client Credentials Grant" flow (4.4 of RFC 6749).
For this a new configuration section is added to the config file,
where the client_id, client_secret and token_url have to be
specified.
The impelmention does currently not support "refresh tokens", but
does support refreshing the token if an `expires_in` is present
in the token itself.
Corresponding unit tests have been added.
[1] https://datatracker.ietf.org/doc/html/rfc6749#section-4.4
Change the integration test so that it adapts to the host, i.e. it
will take the name, version and architecture for the tags and urls
for the repos from the host it is running on. This should make it
more future proof since we now have a central place where this is
configured: the ci configuration, i.e. `.gitlab-ci.yml`.
When the builder is used as an command line binary: have `repo`
be a proper command line argument. It was currently specified
before arch which can be passed multiple times, but so does
repo; hence one of them needs to be optional.
The `run-test.sh` helper is meant to execute the tests as if it
was CI but using a container. If it is detected that we are
already running inside a container assume it is prepare so that
we can exectue the tests directly and do so.
when building the containers the initial `dnf upgrade` will download
content from the latest nightly trees which turns the container into a
Beta system and subsequent `dnf isntall` gets confused!
s3cmd sync actually downloads metadata for all objects in a s3 bucket.
We have built a lot of RPMs, thus this takes 5 minutes on AWS and 25 minutes
on my laptop (!!!).
Let's use recursive put instead. This doesn't delete any files on the remote
side. As we upload RPMs only once, this also shouldn't fail on "the
object already exists". Using this method, we should be able to upload the
RPMs in seconds.
The same patch was applied in osbuild-composer cf73edd2
