Commit graph

97 commits

Author SHA1 Message Date
Sanne Raymaekers
135dd94de5 cloudapi/v2: include details in case internal error is set
In case details aren't explicitly given, and the internal error is set,
include the internal error message in the details.
2024-06-25 17:27:28 +02:00
Brian C. Lane
66c5c5ecf9 cloudapi: Add UploadTypesLocal for local_save status reports
This allows the compose status to reflect that it was saved locally, not
uploaded to a remote service. Without this it returns an error of
'Compose has unknown upload target'
2024-05-13 13:27:12 -07:00
Brian C. Lane
57ebfb4011 cloudapi: Use distro repos if none included in imageRequest
In order to support cloudapi blueprint requests from the cmdline using
composer-cli it needs to select the repositories based on the selected
distribution instead of requiring the user to include them with the
request.

If the image request includes repositories they are used, which matches
the current behavior. If the repository list is empty it will use the
distribution name to select from the repositories shipped with
osbuild-composer.
2024-03-11 03:04:54 -07:00
Ondřej Budai
504a51089c cloudapi: split GetImageRequests off PostCompose
This allows us to do some testing of the ComposeRequest => []imageRequest
conversion without actually enqueueing any jobs (which requires us to
do quite a lot of setup in the testsuite).

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2024-02-21 12:09:01 +01:00
Ondřej Budai
28ef0bc855 cloudapi: move manifest seed into an image request
The goal of this commit is primarily to simplify the API of the enqueue
methods. This way, basically everything needed to generate manifests
is in the imageRequest structure, which simplifies the amount of structures
that we need to think about.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2024-02-21 12:09:01 +01:00
Ondřej Budai
7385cab165 cloudapi: move blueprint into an image request
The goal of this commit is primarily to simplify the API of the enqueue
methods. This way, basically everything needed to generate manifests
is in the imageRequest structure, which simplifies the amount of structures
that we need to think about.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2024-02-21 12:09:01 +01:00
Ondřej Budai
ab8c1ae4f7 clouadpi: remove distribution from enqueue methods
They can just find it from the image type.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2024-02-21 12:09:01 +01:00
Ondřej Budai
c48bf0a143 cloudapi: remove arch from imageRequest
You can absolutely just get it from imageType, let's not duplicate
information, that's error-prone.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2024-02-21 12:09:01 +01:00
Ondřej Budai
08aabe1bef cloudapi: add minimal-raw 2024-02-21 12:09:01 +01:00
Achilleas Koutsou
52d4b196a3 cloudapi: enable iot-bootable-container image type 2024-02-02 10:34:41 +01:00
djach7
4d241b684b edge: add iot-simplified-installer image type
Adds iot-simplified-installer image type to enable koji composes.

Signed-off-by: djach7 <djachimo@redhat.com>
2024-01-18 21:45:23 +01:00
Brian C. Lane
7438e29375 cloudapi: Create a compose using the blueprint data
If the request includes a blueprint (and not customizations) it uses
that blueprint for the compose.
2024-01-15 11:48:35 +01:00
Ondrej Ezr
4b3b942dde cloudapi: Add module_hotfixes flag
Allow passing module_hotfixes flag through the cloudapi.
This will enable depsolving on repositories that might be affected by modularity filtering.

Refs HMS-3202
2023-12-20 09:02:06 +01:00
Gianluca Zuccarelli
b711e302ba cloudapi: add pulp upload target
Add the pulp.ostree upload target to the cloud API and enable it for
edge/iot commits.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2023-11-17 16:48:16 +01:00
Achilleas Koutsou
5a93166f6b cloudapi: support multiple upload statuses in ComposeStatus
Add the new upload_statuses under the image_status in the result of the
ComposeStatus object.  The first status is also included in the old
top-level 'upload_status' property for backwards compatibility.

Tests are updated to match the new results.
2023-11-17 16:48:16 +01:00
Achilleas Koutsou
9fc4551fdf cloudapi: check upload targets in request validation
It is now valid for UploadOptions to be nil but only if there is at
least one UploadTarget defined.
2023-11-17 16:48:16 +01:00
Achilleas Koutsou
d7ab1f2112 cloudapi: multiple upload targets in request
Add an array of targets in the imageRequest and return an array from
ImageRequest.GetTargets() (renamed from GetTarget()).  Currently, the
function still only returns one target, the default for the image type
with the top level upload options.
2023-11-17 16:48:16 +01:00
Brian C. Lane
d0877e68dc cloudapi: Add partitioning_mode support to the API
This adds a 'partitioning_mode' field that can be set to 'auto-lvm',
'lvm' or 'raw'. It defaults to 'auto-lvm'.
2023-10-23 14:52:30 +02:00
Sanne Raymaekers
a7794dc634 cloudapi/v2: add oci image type to cloudapi 2023-09-19 22:57:06 +02:00
Sanne Raymaekers
e5c4640923 cloudapi/v2: expose wsl image type 2023-08-28 14:51:01 +02:00
Brian C. Lane
139bf4dec2 cloudapi: Add ability to skip uploading and save image locally
During development it can be very useful to store the results locally
instead of uploading to a remote system. This implements a development
only option to help with that.

To use it you need to add OSBUILD_LOCALSAVE to the server's environment.
This can be done by editing /usr/lib/systemd/system/osbuild-composer.service
and adding:

Environment="OSBUILD_LOCALSAVE=1"

You can then use an 'upload_options' object to skip trying to upload to
the default service for the type of image, eg:

    "image_requests": [
    {
      "architecture": "x86_64",
      "image_type": "guest-image",
      "upload_options": {
          "local_save": true
      },
      ...
    }]

The results will be saved to /var/lib/osbuild-composer/artifacts/UUID/
using the default filename for the image type.

If local_save is used without OSBUILD_LOCALSAVE being set it will return
an error with id=36 saying 'local_save is not enabled'.
2023-08-09 16:48:46 +02:00
Brian C. Lane
c1e52aebc3 cloudapi: Refactor handler.go code to make testing easier
This moves some of the code from the PostCompose function in handler.go
into methods on the OpenAPI ComposeRequest and ImageRequest structs.

In compose.go I have added several methods.
GetBlueprintWithCustomizations takes the ComposeRequest customizations
and builds a Blueprint struct.

GetPayloadRepositories returns the custom payload repos.

GetSubscription returns the ImageOptions setup with optional
subscription information from the request.

In imagerequest.go I have added GetTarget which takes the upload
options and returns a Target. This moves the giant switch statement,
which may also benefit from further simplification at some point.

GetOSTreeOptions returns the OSTree ImageOptions if there are ostree
settings in the ImageRequest.

GetImageOptions returns the distro.ImageOptions with the size set.

This commit only moves the code, making PostCompose easier to read. All
tests still pass.

cloudapi: Move the size handling to a method on ImageRequest
2023-08-05 11:28:29 +02:00
Brian C. Lane
818d434303 cloudapi: Add optional image size support
This adds a 'size' parameter to the image_request object. It can be used
to specify the minimum image size in bytes

This behaves in the same way as the size parameter of the weldr API

For raw images the root partition is grown to fill the available space.
For LVM images the PV uses the available space, but the LV does not,
leaving space available for other LVs to be created after boot.

See COMPOSER-1883
2023-08-05 11:28:29 +02:00
Simon de Vlieger
326f0cfa2f cloudapi: add live-installer
This was missing to enable `live-installer` in the CloudAPI.
2023-07-20 18:53:28 +02:00
Achilleas Koutsou
0e4a9e586f split: replace internal packages with images library
Remove all the internal package that are now in the
github.com/osbuild/images package and vendor it.

A new function in internal/blueprint/ converts from an osbuild-composer
blueprint to an images blueprint.  This is necessary for keeping the
blueprint implementation in both packages.  In the future, the images
package will change the blueprint (and most likely rename it) and it
will only be part of the osbuild-composer internals and interface.  The
Convert() function will be responsible for converting the blueprint into
the new configuration object.
2023-07-10 21:11:19 +02:00
Gianluca Zuccarelli
fa6f3ba197 cloudapi: openscap integration
Enable the openscap customizations in the cloudapi.
2023-06-29 13:42:49 +01:00
Achilleas Koutsou
0eb999d510 cloudapi: remove ostree from imageRequest
Use ostree.ImageOptions for the request parameters instead of a
SourceSpec on the imageRequest.

When preparing the image request, add the ostree values from the API's
compose request to the ostree options on the image options of the image
request.

It's not necessary to create a source spec and it's also not necessary
to add the default ref when it's not specified in the request for an
ostree-based image type.  Both of these will be handled by the Manifest
generation based on the ostree options (imageOptions.OSTree).  The image
functions will take care of setting any missing parameters or returning
errors if any required parameters are missing.
2023-06-14 11:19:29 +02:00
Sanne Raymaekers
fb66b4bbf3 cloudapi: add vsphere-ova type 2023-06-05 11:22:07 +02:00
Achilleas Koutsou
db431a565d ostree: move OSTreeImageOptions to the ostree package
Move the ostree image options to the ostree package and rename the type
to ImageOptions (ostree.ImageOptions).
2023-05-31 16:40:07 +02:00
Achilleas Koutsou
ffa1e1df17 rhsm: move FactsImageOptions to the rhsm/facts package
Move the FactsImageOptions from distro to the new rhsm/facts package.
At the same time define the values we use as an enum, including the
"test-manifest" value.
Though the values don't really matter, the test value is defined first
so it takes the 0 value, which feels nicer conceptually.

The field in the distro.ImageOptions is changed to be a pointer to allow
for nil values.
2023-05-31 16:40:07 +02:00
Achilleas Koutsou
fc5461e9e3 ostree: rename RequestParams to SourceSpec
Same as with the container SourceSpec, the struct specifies the required
information to resolve an ostree commit from a source (URL, ref, and
optional parent).
Renaming for consistency.
2023-05-31 16:40:07 +02:00
Achilleas Koutsou
c7aecdf31c distro: delete distro.Manifest type
Delete the distro.Manifest type and its tests.
Change all occurrences of distro.Manifest to manifest.OSBuildManifest.
2023-05-31 16:40:07 +02:00
Achilleas Koutsou
0f925c445c subscription: new package for subscription options
Move the subscription options from distro to its own package.
Now we can import the manifest package into the distro package (instead
of the other way around) so we can work with the manifest.Manifest type
in distro.
2023-05-31 16:40:07 +02:00
Gianluca Zuccarelli
ce299dfa0e internal/rpmmd: change ignoressl to pointer
Change the `IgnoreSSL` field in `rpmmd.RepoConfig`
to a pointer. This will be later used to configure
the `SSLVerify` field in the yum repo stage.
2023-05-31 16:24:36 +02:00
Tomáš Hozza
7c9d671eae Cloud API: explicitly specify boot mode in AWS upload target
Explicitly specify the AMI boot mode in AWS upload target in Cloud API
compose handler. The value is determined based on image type's boot
mode.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-19 13:24:39 +02:00
Gianluca Zuccarelli
79284f3447 internal/cloudapi: fix custom repo fields
Minor fixes to cloudapi custom repos:
- add missing `priority` field
- rename `repo_check_gpg` to `check_repo_gpg`
  to match `payload_repositories`
2023-05-03 18:27:06 +02:00
Gianluca Zuccarelli
fb63d7ad9a internal/cloudapi: expose custom repos
Expose the custom repository customizations for
the cloudapi.
2023-04-26 14:16:16 +01:00
Gianluca Zuccarelli
d44703cdc8 rpmmd/repository: repoconfig pointers
Convert some of the fields in the `RepoConfig` struct
to pointers. Since `RepoConfig` will be used to convert
custom repositories to an array of `osbuild.YumRepository`,
we need to ensure that fields that are not set explicitly
are not saved to the `/etc/yum.repos.d` repository files.
2023-04-21 17:40:00 +02:00
Gianluca Zuccarelli
4d42808b6a internal/rpmmd: RepoConfig baseurl change
Update the internal RepoConfig object to
accept a slice of baseurls rather than a
single field. This change was needed to
align RepoConfig with the dnf spec [1].

Additionally, this change adds custom json
marshal and unmarshal functions to ensure
backwards compatibility with older workers.
Add json tags to the internal rpmmd config
since this is serialized in dnfjson.
Add unit tests to check the serialization
is okay.

[1] See dnf.config
2023-04-21 17:40:00 +02:00
Sanne Raymaekers
14a9b2395d cloudapi/v2: expose repo metadata verification 2023-04-13 22:08:17 +02:00
Achilleas Koutsou
2a87a8b7f3 cloudapi: enable azure-eap7-rhui image type 2023-03-30 18:40:12 +02:00
Tomáš Hozza
e9af3bda64 Cloud API: expose directory and file customizations
Expose the Directory and File customizations in the Cloud API. Also
validate the provided customizations while processing the compose
request by trying to convert them to internal representations
`fsnode.File` and `fsnode.Directory`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Brian C. Lane
b0e388d115 cloudapi: Add subscription option for rhc
When rhc is selected it will install the required packages, register
using rhc and always enable insights.

When rhc is not selected it will use subscription manager for
registration, and optionally enable insights. Also installing required
packages.
2023-02-09 09:41:15 +01:00
Brian C. Lane
d12447408b Change RepoConfig.GPGKey to an array of key strings
DNF supports more than one GPG key. It is possible that one may be used for
signing packages, and another to sign the repository metadata. This
renamed GPGKey to GPGKeys internally. It does not change the on-disk
repository json format.
2023-02-01 10:27:58 +01:00
Sanne Raymaekers
4b90cb6fa4 cloudapi/v2: set ostree rhsm option on image options
The ostree options are used during the ostree resolve job, but when
generating the manifest the rhsm value comes from the image options, so
it's necessary to set it on both.
2022-12-07 16:31:46 +01:00
Brian C. Lane
a1a3e5ba2d cloudapi: Add azure-sap-rhui support to cloudapi
and Update openapi.v2.gen.go
by running `./tools/prepare-source.sh` which runs `go generate ./...`
among other things.
2022-11-18 16:53:22 +01:00
Sanne Raymaekers
acc1575cf6 cloudapi/v2: expose ostree contenturl and rhsm options 2022-11-04 11:41:31 +01:00
Tomáš Hozza
1af01ad135 cloudapi: make location optional in Azure Upload Options
Providing the `location` is no longer required for Azure Upload Options.
If it is not provided, the implementation determines the location from
the provided Resource Group. This will make the API nicer for any
client, since they won't need to provide redundant information.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-10-27 19:33:43 +02:00
Sanne Raymaekers
620536fd61 internal/cloudapi: add ostree options for all otree image types
b01792d9dd broke this behaviour. All
ostree image types should have an ostree resolve job.
2022-10-21 22:32:24 +02:00
Sanne Raymaekers
8fdd158799 cloudapi/v2: use the ostree resolve job to resolve ostree refs 2022-10-19 18:14:10 +02:00