Commit graph

20 commits

Author SHA1 Message Date
Tomáš Hozza
3d1c24fdc0 Hub: use anyOf for upload_options schema
It turned out, that the upload options for AWS EC2 or GCP with just the
required properties specified, would match both schemas. This is causing
the validation fail with `oneOf` used for `upload_options`.

This will be fixed in osbuild-composer PR#3018. However, we can't use
the same approach for koji-osbuild, while keeping the schema backward
compatible and sane.

Another discussed option would be to define `upload_options` as an empty
object with `additionalProperties` set to `True`. This would
effectively mean no validation of `upload_options`. None of the plugins
actually modify the `upload_options` in any way. It is passed as
provided to `osbuild-composer`.

I think that the change in this commit is a compromise. The validation
of the `upload_options` schema is kept, but it is relaxed to `anyOf`.

[1] https://github.com/osbuild/osbuild-composer/pull/3018

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-11-21 12:06:35 +01:00
Tomáš Hozza
7ccd349999 hub: make location optional for AzureUploadOptions
The Azure Resource Group Location is no longer required and
osbuild-composer can determine the correct Location from the
provided Resource Group.

Related to https://github.com/osbuild/osbuild-composer/pull/3093.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-11-21 12:06:35 +01:00
Tomáš Hozza
8939c3515f hub: make bucket optional for GCPUploadOptions
The GCP Bucket may be (and usually will be) set on the composer-worker.

Related to https://github.com/osbuild/osbuild-composer/pull/3023.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-11-21 12:06:35 +01:00
Simon Steinbeiss
9975f38990 hub: Log adding tasks to Koji's db
Log both the entrypoint and the return value from adding a task to
Koji's database. We can measure both to ensure a task has been
successfully added to the database as a service level indicator.
2022-11-16 18:39:41 +01:00
Tomas Hozza
f21a2de39b Hub: support image_type being an array for backwards compatibility
The support for specifying multiple `image_types` for a single compose
has been removed by [1]. This turned out to be problematic, because e.g.
Pungi uses the array type when triggering image builds via osbuild.

Bring back the support for specifying the `image_type` as an array, but
restrict it to a single item. This will cover the Pungi use-case, since
it is always passing a single `image_type` in the array. The array is
then converted to a string in the Hub plugin and passed as such to the
Builder plugin.

Extend unit tests covering the introduced compatibility layer.

[1] c725265081
2022-09-02 15:15:21 +02:00
Tomas Hozza
c76e97ddc9 Support specifying upload options for image builds
The upload options are expected to be provided as a JSON file. The same
options will be used for all image type and architecture combinations,
similarly as it is done for ostree options.

Extend unit tests to cover the newly added functionality.
2022-08-31 13:13:24 +02:00
Tomas Hozza
c725265081 Drop support for specifying more than one image type
While it is technically possible to build more than one image type
as part of a Koji compose in osbuild-composer, this option is not used
in reality and it also makes very little sense. If the user wants to
build more than one image type, they should submit multiple Koji builds.

Adjust affected unit tests.
2022-08-22 09:58:06 +02:00
Christian Kellner
591a55aad5 plugins: add support for customizations
The Cloud API supports passing in a variety of image customizations,
like e.g. extra packages or pre-defining users.

Add a new command line option to the client `--customizations` which
takes a path to a JSON file which contains the customziations; they
will be passed via the existing `opts` argument to the hub.

Add support for `customizations` to the `opts`/`options` arguments
to the hub plugin. No validation to the object is done. Instead we
rely in Composer for the validation of the content.

Add support for `customizations` the image `ComposeRequest` in the
builder plugin. All specified values are just passed through to
composer as-is.

Add tests for the respective plugins.
2022-05-03 12:28:05 +02:00
Christian Kellner
f559c18079 plugins: support for repo package sets
This adds support for specifing the package sets for repositories;
on the command line this can be done via `--repo-package-set` with
and argument of `;` separated package set names. This will result
in repo information being transported via dict instead of plain
strings. Thus the hub plugin's schema was modified accordingly.
Last but not least, the builder plugin now can decode these dicts
and setup the repos accordingly.
Test were added for plugins as well as the integration test changed
to use this new feature.
The first upstream commit that supports this feature is pinned.
2022-02-15 17:35:29 +01:00
Christian Kellner
78ed04dbd6 plugins: support for ostree specific options
OStree compose requests need special options, like the `ref` the
`parent` and the `url`. Add support for those options to all three
plugins:
  The command line plugin now takes `--ostree-{parent,ref,url}`
  and passes it to koji via the existing options dictionary.

  The JSON schemata in the hub plugin was adjusted to allow these
  new options.

  Finally the builder plugin will look for the new `ostree` dict
  inside the options, create an `OSTreeOptions` object from it,
  and attach it to each image request.

NB: since the ostree options are per image request and are thus
architecture dependent we support a "$arch" substition in the
`parent` and `ref` options that will be resolved by the plugin;
this allows to builds arch specific commits for with a single
compose request.

Add the respective unit tests.
2022-02-14 13:14:19 +01:00
Christian Kellner
d7bfaee189 plugins: ability to skip the tagging
Add a new command line option `--skip-tag` that will skip tagging
after a successful build. The help text is copied from the same
option of other sub-commands in the koji client. The hub plugin's
jsonschema was updated accordingly, and the builder plugin will
skip the tag if the option was requested.
Tests were added or augmented for all three plugins to test the
new option.
2020-11-16 17:00:40 +01:00
Christian Kellner
24850d3303 plugin/hub: specify type of architectures array
In the json schema used to validate the input, specify the type of
the architectures array as a string.
2020-09-17 16:33:14 +01:00
Christian Kellner
1d9612ca05 plugin: pass repos as arrays
Currently we were passing the repo information as a comma
separated string, which is fragile, since urls can contain
commas. Just transfer them as arrays of strings.
2020-09-17 16:33:14 +01:00
Christian Kellner
d0d167d2f8 plugin/hub: fix typo
It is schema, not schmea. That is not even a word.
2020-09-17 10:48:37 +02:00
Christian Kellner
5ba7028340 plugin/hub: raise ParameterError for invalid input
Convert the jsonschema.exceptions.ValidationError into the more
appropriate koji.ParameterError.
2020-09-13 13:20:19 +02:00
Christian Kellner
0bf3c47113 plugin/hub: clean up imports
Remove an unused import and sort the imports properly.
Also silence the for the kojihub import.
2020-09-11 23:18:04 +01:00
Christian Kellner
af4e66d2b2 plugin/hub: use jsonschema to validate input
Use jsonschema to validate the input to the XMLRPC call to catch
mistakes early, i.e. before creating the task.
2020-09-10 18:41:13 +01:00
Christian Kellner
8fb1342631 Make image_type and distro required arguments
Distro, in composer terms "distribution", and "image_type" are
required for composer, so make that explicit everywhere in the
code.
2020-09-08 16:13:48 +02:00
Christian Kellner
a9e187d38a plugin/hub: rename method to osbuildImage
Remove the 'Test' suffix.
2020-09-07 19:03:15 +02:00
Christian Kellner
aefe1218b5 initial import 2020-09-02 19:15:25 +02:00