Commit graph

3321 commits

Author SHA1 Message Date
Lukas Zapletal
f2bf47814f deps: upgrade go-systemd from v1 to v22 2025-06-16 17:46:45 +02:00
Achilleas Koutsou
5a5e6d732c clouapi: test Disk customization validation
Two new tests are added, one with valid customizations and another with
invalid customizations.
Invalid customizations will return one of two error types, depending on
where exactly the failures was generated.
When the openapi schema fails to validate a request, it will return
IMAGE-BUILDLER-30.
When the error occurs while converting the openapi customizations to the
blueprint types, it will have the IMAGE-BUILDER-35 type instead.
2025-06-16 13:33:20 +02:00
Achilleas Koutsou
45929efbcf cloudapi: validate Disk customization before returining
After converting a Disk customization from the API type to the Blueprint
equivalent, run the validator and return an error if it fails to
validate.
The schema doesn't catch all invalid cases, such as the mountpoint being
required when the fs_type is not "swap", so calling the validator
ensures that any such cases are caught without needing to duplicate the
checks.
2025-06-16 13:33:20 +02:00
Achilleas Koutsou
ea0b5e8661 cloudapi: make fs_type required for filesystems and logical volumes
While technically this is a backwards incompatible API change, the
core blueprint schema didn't allow for empty or undefined fs_type [1,2],
so the general behaviour hasn't changed, we're just catching it early
now.

[1] de18902e3f/pkg/blueprint/disk_customizations.go (L624-L627)
[2] de18902e3f/pkg/blueprint/disk_customizations.go (L677-L680)
2025-06-16 13:33:20 +02:00
Achilleas Koutsou
a03d01c92c cloudapi: make type mandatory for non-plain partitions
The type is required for partitions with a type other than plain.

This commit also adds the empty string to the valid values in the
conversion switch case and is treated like "plain".
2025-06-16 13:33:20 +02:00
Achilleas Koutsou
6cd4bbc6d2 cloudapi: support swap partitions and volume groups
Supporting "swap" as the fs_type requires making the mountpoint an
optional field.  When setting fs_type="swap", the mountpoint must be
empty.
2025-06-16 13:33:20 +02:00
Achilleas Koutsou
edb9b284d5 cloudapi: return http error when failing to convert disk customizations
Return an HTTP error with the invalid customization message when the
convertDiskCustomizations() function returns an error.

Secondary: The conversion was being run twice by mistake.  This has also
been fixed.
2025-06-16 13:33:20 +02:00
Brian C. Lane
12dd0b0be4 cloudapi: Fix listing composes with empty result
It was returning a null body instead of an empty list.

Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
b33f86783f cloudapi: Add artifact cleanup to delete handler
Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
56fc58cca3 cloudapi: Add DeleteCompose to delete a job by UUID
This adds the handler for DELETE /composes/{id} which will delete a job and
all of its dependencies, and any artifacts.

Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
a569ac0f7b cloudapi: Add DELETE /composes/{id} route to delete a job
This will be used to delete jobs and their artifacts.

Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
f1a2c24563 worker: Add CleanupArtifacts function
This removes all artifact directories, and their contents, if there
isn't an associated Job. This is used to clean up local artifacts after
the compose job has been deleted.

Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
c3394ae589 jobqueuetest: Add job delete tests
This adds tests for retrieving all root jobs, and deleting jobs
and unused dependencies. These tests are run against the fsjobqueue for
unit testing, and against dbjobqueue for integration testing.

Resolves: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
5cddc4223d dbjobqueue: Add AllRootJobIDs implementation
Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
d8285a0b74 jobqueue: Add DeleteJob function
This allows jobs to be deleted from the database.
Currently only implemented by fsjobqueue. The function for
dbjobqueue currently returns nil.

This will remove all the job files used by the root job UUID as long as
no other job depends on them. ie. It starts at the top, and moves down
the dependency tree until it finds a job that is also used by another
job, removes the job to be deleted from its dependants list, and moves
back up the tree only deleting jobs with empty dependants lists.

Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
5961b69caa jsondb: Add Delete function
This allows database entries to be deleted.

Related: RHEL-60120
2025-06-05 10:32:56 +02:00
Brian C. Lane
4cb47a424d cloudapi: Fix missing specs for koji ContainerResolve
It looks like this got lost in a refactor back in commit 6e4efabf24
when it used to populate the Specs list in a loop.
2025-05-13 20:38:38 +02:00
rverdile
1559b4d760 cloudapi/v2: add templates to subscription image options
Adds optional templateName and templateUUID paramaters to image options,
each used if registering a system to a template a boot.
2025-05-12 10:07:19 +02:00
Michael Vogt
1a6c70b649 internal: drop internal/fsnode package
This package has an identical copy in `images` and is also (AFAICT)
unused. Hence this commit removes it.
2025-05-06 10:45:21 +03:00
Achilleas Koutsou
e621386caf cloudapi: drop ImageRequest.GetImageOptions() method
This method is not particularly useful anymore.  Its purpose was to
initialise the ImageOptions from an ImageRequest with the appropriate
size and partitioning mode.  However, the partitioning mode was also
being set later using request.GetPartitioningMode().  More importantly,
setting the size on the ImageOptions caused issues with the interaction
between filesystem and partitioning customizations as well as the image
request size (see #4705).  The correct thing to do here is to map the
ImageRequest.Size directly onto ImageOptions.Size, without taking into
account ImageType or the Blueprint Customizations.  The rest are
considered when generating the manifest in images, either when preparing
the Manifest() call or when generating the partition table.  This makes
it easier to trace and reason about the effect of each option.  This
kind of decision making in the API layer makes it difficult to maintain
the logic, since it requires duplicating the decision making or, as we
had now, making certain specific combinations impossible.
2025-05-05 20:37:19 +02:00
Achilleas Koutsou
045364cbf2 cloudapi: reduce minsize type to just string
The type of the minsize parameter in the new disk customization was
meant to support both integers (size in bytes) and strings (size with
unit).  However, the schema wasn't done properly, which made any input
result in an error:

    GenericError: Failed to create the compose request:
    {"code":"IMAGE-BUILDER-COMPOSER-30","details":"request body has an
    error: doesn't match schema #/components/schemas/ComposeRequest:
    Error at \"/customizations/disk/partitions/0\": doesn't match schema
    due to: Error at \"/minsize\": input matches more than one oneOf
    schemas

Reducing it to just support strings simplifies the schema.  It's also
not an important feature reduction since sizes as integers (for
filesystems that are typically in GiB) aren't very convenient.
2025-04-29 23:40:59 +02:00
Tomáš Hozza
7036e2bcb9 Many: expose rhsm.subscription_manager.rhsm.auto_enable_yum_plugins
Expose the ${SUBJ} customization in Cloud API. Weldr API inherited it
automatically with the update of `osbuild/blueprint` to v1.6.0.

Adjust relevant test cases and unit tests to use this new customization.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-04-29 18:22:40 +03:00
Ondřej Budai
401f832cc8 client: adjust the tests to new image type names in Fedora
See the comment inline.
2025-04-25 09:26:59 +02:00
Simon de Vlieger
242564cc5c test: update type names to new names
The image type names for Fedora were changed in images 0.138.0, aliases
were left behind so the old names still exist.

Split out the testcases between rhel and fedora so we test for the new
names being returned.

Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
2025-04-25 09:26:59 +02:00
rverdile
983ac27484 cloudapi/v2: add insights proxy to subscription image options 2025-04-22 18:00:58 +02:00
Achilleas Koutsou
cf956ff5a6 Delete internal/blueprint/ and import from osbuild/blueprint
Import osbuild/blueprint v1.6.0
2025-04-17 11:17:58 +02:00
Tomáš Hozza
169f9a4308 client/blueprints_test.go: delete condition for composer version < 83
The version check is failing on the "dot" version, while trying to
convert "118.1" to an integer. Delete the condition for skipping the
test, because it will never be running on anything older than v83
anyway.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-04-03 17:09:15 +02:00
Achilleas Koutsou
ad0cb80bed cloudapi/v2: minsize can be a string or integer
Define a reusable minsize property in the cloud API schema.
The minsize type is either a uint64 or a string with data units.

Internally, this creates a union (json.RawMessage) type called Minsize
and optional uses of the property are of type *Minsize.  We define an
internal decodeMinsize() function to convert *Minsize to uint64.  This
is similar to the decodeSize() function in the blueprint, but made
specifically to handle *Minsize.
2025-03-27 11:33:56 +01:00
Achilleas Koutsou
82ac185e7c cloudapi: Disk customization conversion
Convert from cloudapi Disk customizations to the blueprint types.
2025-03-27 11:33:56 +01:00
Achilleas Koutsou
fe19c87dd9 common: new utility function: DerefOrDefault()
This is sort of the opposite of ToPtr().  It dereferences a pointer to
its base value or returns the default value for the type if the pointer
is nil.
2025-03-27 11:33:56 +01:00
Achilleas Koutsou
1a65e573eb cloudapi: add new Disk customizations
Following up from #4535 which added support for advanced partitioning
customizations to the blueprint.
These are now added to the cloud api spec.
2025-03-27 11:33:56 +01:00
Achilleas Koutsou
330ce26715 blueprint: update Disk customization with new features
- Partition table type: https://github.com/osbuild/images/pull/1085
- Partition type IDs:   https://github.com/osbuild/images/pull/1115
2025-03-27 11:33:56 +01:00
Sanne Raymaekers
7652af83d7 cloudapi/v2: adapt to new oapi-codegen
Fixed distribution list type, function signatures (later versions parse
uuids in the generated code), defaults actually being set and added
x-go-type where needed.
2025-03-26 11:13:14 +01:00
Sanne Raymaekers
17416bf60b worker: adapt to new oapi-codegen 2025-03-26 11:13:14 +01:00
Sanne Raymaekers
b2700903ae go.mod: bump github.com/getkin/kin-openapi to v0.131.0
As deepmap/oapi-codegen didn't work with this newer version, upgrade to
oapi-codegen/oapi-codegen v2.

Mitigating CVE-2025-30153
2025-03-26 11:13:14 +01:00
Sanne Raymaekers
0b11dc6b59 cloudapi/v2: mark md5 signature as required in package metadata
Technically osbuild/osbuild-composer#4564 broke the api spec by marking
a required field as non-required. Fix this by using allOf.
2025-03-21 10:06:42 +01:00
Sanne Raymaekers
ec93197b7a cloudapi/v2: add some unit tests for testing modules 2025-03-20 14:07:52 +01:00
Sanne Raymaekers
241aae6fd4 cloudapi: support dnf modules
Exposes enabled modules in the api.
2025-03-20 14:07:52 +01:00
Sanne Raymaekers
d6fe9d85cf weldr: adapt tests to modularity 2025-03-20 14:07:52 +01:00
Sanne Raymaekers
52e5fe9627 weldr: add enabled modules to depsolve handler 2025-03-20 14:07:52 +01:00
Sanne Raymaekers
9c62e68aa7 blueprint: add enabled modules to blueprint 2025-03-20 14:07:52 +01:00
Michael Vogt
4584d05eff cloudapi: silence logrus in tests
This commit discards the logrus output when the tests are run. That
might be a controversial change but the amount of redundant output
that drowns the actual failures makes it very hard to work with
these tests.
2025-03-10 20:20:00 +01:00
Sanne Raymaekers
536b7d95c5 go.mod: update osbuild/images to v0.123.0
Includes modularity support.
2025-03-10 17:56:19 +01:00
Brian C. Lane
5e3d6aff54 cloudapi: Add a test for metadata returning the compose request
Related: RHEL-60120
2025-03-05 12:36:36 +01:00
Brian C. Lane
d8e9a86921 cloudapi: save and return compose request details
The original compose request contains useful details that are not
preserved when it is converted to a manifest. Things like the
distribution, arch, image type, blueprint or customizations are useful
when examining builds later.

This saves the original request json using the job id and a new
directory (ComposeRequest) under the artifacts directory. The original
request, if present, is then added to the compose/<id>/metadata response
alongside the package list.

Related: RHEL-60120
2025-03-05 12:36:36 +01:00
Brian C. Lane
199a3d31f8 worker: Expose the ArtifactsDir path
This will help make it easier to write the original compose request json
to the same directory tree.

Related: RHEL-60120
2025-03-05 12:36:36 +01:00
Brian C. Lane
a394d1a498 cloudapi: Fix href for metadata responses
Related: RHEL-60120
2025-03-05 12:36:36 +01:00
Brian C. Lane
c0bf89b51f openapi: Add ComposeRequest as part of the ComposeMetadata response
This will allow clients to display more information about a compose,
including the image type created, arch and distro, and blueprint to
customizations used to create it.

Related: RHEL-60120
2025-03-05 12:36:36 +01:00
Beñat Gartzia Arruabarrena
d5a77ffcb5 internal/cloud/gcp/compute: Add TDX_CAPABLE guest OS feature
Latest RHEL images (from 9.6 on) should be able to run as TDX guests.
CentOS guests also fully support it at the moment.

See: https://issues.redhat.com/browse/COS-3111
See: https://github.com/coreos/coreos-assembler/pull/4006
See: https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9/-/merge_requests/5979
2025-02-27 13:33:22 +01:00
Brian C. Lane
96715c3106 cloudapi: Use constants for distro in compose_test.go
This will make it easier to update next time the distribution version
needs to be bumped.
2025-02-24 13:26:40 +01:00