Commit graph

3292 commits

Author SHA1 Message Date
Achilleas Koutsou
fe19c87dd9 common: new utility function: DerefOrDefault()
This is sort of the opposite of ToPtr().  It dereferences a pointer to
its base value or returns the default value for the type if the pointer
is nil.
2025-03-27 11:33:56 +01:00
Achilleas Koutsou
1a65e573eb cloudapi: add new Disk customizations
Following up from #4535 which added support for advanced partitioning
customizations to the blueprint.
These are now added to the cloud api spec.
2025-03-27 11:33:56 +01:00
Achilleas Koutsou
330ce26715 blueprint: update Disk customization with new features
- Partition table type: https://github.com/osbuild/images/pull/1085
- Partition type IDs:   https://github.com/osbuild/images/pull/1115
2025-03-27 11:33:56 +01:00
Sanne Raymaekers
7652af83d7 cloudapi/v2: adapt to new oapi-codegen
Fixed distribution list type, function signatures (later versions parse
uuids in the generated code), defaults actually being set and added
x-go-type where needed.
2025-03-26 11:13:14 +01:00
Sanne Raymaekers
17416bf60b worker: adapt to new oapi-codegen 2025-03-26 11:13:14 +01:00
Sanne Raymaekers
b2700903ae go.mod: bump github.com/getkin/kin-openapi to v0.131.0
As deepmap/oapi-codegen didn't work with this newer version, upgrade to
oapi-codegen/oapi-codegen v2.

Mitigating CVE-2025-30153
2025-03-26 11:13:14 +01:00
Sanne Raymaekers
0b11dc6b59 cloudapi/v2: mark md5 signature as required in package metadata
Technically osbuild/osbuild-composer#4564 broke the api spec by marking
a required field as non-required. Fix this by using allOf.
2025-03-21 10:06:42 +01:00
Sanne Raymaekers
ec93197b7a cloudapi/v2: add some unit tests for testing modules 2025-03-20 14:07:52 +01:00
Sanne Raymaekers
241aae6fd4 cloudapi: support dnf modules
Exposes enabled modules in the api.
2025-03-20 14:07:52 +01:00
Sanne Raymaekers
d6fe9d85cf weldr: adapt tests to modularity 2025-03-20 14:07:52 +01:00
Sanne Raymaekers
52e5fe9627 weldr: add enabled modules to depsolve handler 2025-03-20 14:07:52 +01:00
Sanne Raymaekers
9c62e68aa7 blueprint: add enabled modules to blueprint 2025-03-20 14:07:52 +01:00
Michael Vogt
4584d05eff cloudapi: silence logrus in tests
This commit discards the logrus output when the tests are run. That
might be a controversial change but the amount of redundant output
that drowns the actual failures makes it very hard to work with
these tests.
2025-03-10 20:20:00 +01:00
Sanne Raymaekers
536b7d95c5 go.mod: update osbuild/images to v0.123.0
Includes modularity support.
2025-03-10 17:56:19 +01:00
Brian C. Lane
5e3d6aff54 cloudapi: Add a test for metadata returning the compose request
Related: RHEL-60120
2025-03-05 12:36:36 +01:00
Brian C. Lane
d8e9a86921 cloudapi: save and return compose request details
The original compose request contains useful details that are not
preserved when it is converted to a manifest. Things like the
distribution, arch, image type, blueprint or customizations are useful
when examining builds later.

This saves the original request json using the job id and a new
directory (ComposeRequest) under the artifacts directory. The original
request, if present, is then added to the compose/<id>/metadata response
alongside the package list.

Related: RHEL-60120
2025-03-05 12:36:36 +01:00
Brian C. Lane
199a3d31f8 worker: Expose the ArtifactsDir path
This will help make it easier to write the original compose request json
to the same directory tree.

Related: RHEL-60120
2025-03-05 12:36:36 +01:00
Brian C. Lane
a394d1a498 cloudapi: Fix href for metadata responses
Related: RHEL-60120
2025-03-05 12:36:36 +01:00
Brian C. Lane
c0bf89b51f openapi: Add ComposeRequest as part of the ComposeMetadata response
This will allow clients to display more information about a compose,
including the image type created, arch and distro, and blueprint to
customizations used to create it.

Related: RHEL-60120
2025-03-05 12:36:36 +01:00
Beñat Gartzia Arruabarrena
d5a77ffcb5 internal/cloud/gcp/compute: Add TDX_CAPABLE guest OS feature
Latest RHEL images (from 9.6 on) should be able to run as TDX guests.
CentOS guests also fully support it at the moment.

See: https://issues.redhat.com/browse/COS-3111
See: https://github.com/coreos/coreos-assembler/pull/4006
See: https://gitlab.com/redhat/centos-stream/src/kernel/centos-stream-9/-/merge_requests/5979
2025-02-27 13:33:22 +01:00
Brian C. Lane
96715c3106 cloudapi: Use constants for distro in compose_test.go
This will make it easier to update next time the distribution version
needs to be bumped.
2025-02-24 13:26:40 +01:00
Brian C. Lane
0256e09031 lint: Clean up golangci-lint v1.60 complaints
This cleans up the linting results by adding checks for
integer underflow/overflow in several places, suppressing the error in
places where it has been checked, or fixing the types when possible.
2025-02-24 04:20:42 -08:00
Michael Vogt
f237af9bda cloudapi: move fedora-39 test forward to fedora-42
This commit fixes the failing TestGetImageRequests_BlueprintDistro
test. It fails because fedora-39 is no longer part of the supported
distros and can no longer be build. Move to fedora-42 so that we
have a bit time before we need to update it again.

It also updates TestGetImageRequests_NoRepositories to point to
fedora-42.
2025-02-19 19:19:42 +01:00
Michael Vogt
35167ce83f many: update for new reporegistry.New() api (c.f. pr#1179)
This commit updates osbuild-composer for the new API in images
for the `reporegistry.New()`. The main incompatible change is
that the `/repositories` part is not longer automatically added
inside the library so we need to add it on the call-site.

This needs https://github.com/osbuild/images/pull/1179
2025-02-19 19:19:42 +01:00
Michael Vogt
5906c554f6 cloudapi: update test to use repos from images
This commit updates the cloudapi test that uses the real repositories
to use the version of those from the "images" library. Composer
no longer carries the default repos.

Note that this can most likely be simplified once
https://github.com/osbuild/images/pull/1179
is merged.
2025-02-13 14:00:45 +01:00
Brian C. Lane
25e8bb1202 cloudapi: Fix artefact typo 2025-02-11 16:09:27 +01:00
Brian C. Lane
d47102716c v2_test: Add tests for Download
This enables the artifact directory during the tests, it mocks up a
download artifact, and tests that it can be downloaded. The file
contains JSON because the TestRoute helper expects that as the response.

Related: RHEL-60142
2025-02-11 16:09:27 +01:00
Brian C. Lane
a6f48bcd2f cloudapi: Add error information to ErrorUnknownUploadTarget
It can include useful extra details, whether or not the artifacts
directory is setup, whether the file is missing, etc.

Related: RHEL-60142
2025-02-11 16:09:27 +01:00
Brian C. Lane
3781820b45 server: Return the path alongside the error from JobArtifactLocation
Even though the file isn't there it can be useful to have the full path
that it was checking for.

Related: RHEL-60142
2025-02-11 16:09:27 +01:00
Brian C. Lane
d0be06278c cloudapi: Hook up the /composes/{id}/download handler
Resolves: RHEL-60142
2025-02-11 16:09:27 +01:00
Brian C. Lane
1d0d198e67 openapi: Add composes/{id}/download route
This will download the build artifact from a locally saved osbuild
compose. It will set the filename to the the UUID of the build with the
artifact filename appended. eg. 1dbcc86e-745b-4061-812f-e50f06fa7cbe-disk.qcow2

Related: RHEL-60142
2025-02-11 16:09:27 +01:00
Achilleas Koutsou
49271e614b blueprint: test the conversion 2025-02-05 11:32:52 +01:00
Achilleas Koutsou
68dd8699ed blueprint: add local DiskCustomization
Add DiskCustomization and all its children to the internal blueprint.
Add the conversion to the images counterpart to the Convert() function.
2025-02-05 11:32:52 +01:00
Brian C. Lane
7622a69d60 v2_test: Add a test for GetDistributionsList
Related: RHEL-60133
2025-02-04 15:15:14 -08:00
Brian C. Lane
44ac65b70c cloudapi: Add /distributions to return distro:arch:image-type
This adds support for listing all of the supported distributions,
their arches, the image types, and their repository details.

This returns 3 nested json objects. The keys for the first layer are the
distribution names. The 2nd layer's keys are the architectures supported
by that distribution, and the 3rd layer's keys are the image types
supported by that distribution:architecture pair. The value of the 3rd
layer is the repository information.

Resolves: RHEL-60133
2025-02-04 15:15:14 -08:00
Brian C. Lane
325a018c75 cloudapi: Return the compose status of all root compose jobs
This returns the status using the same structure as it does for
requesting individual statuses for the jobs.

Related: RHEL-60120
2025-02-03 17:27:31 -08:00
Brian C. Lane
24b939d4c7 cloudapi: Move most of getComposeStatusImpl into a function
This will make it easier to use the status in responses.

Related: RHEL-60120
2025-02-03 17:27:31 -08:00
Brian C. Lane
b2d803306c cloudapi: Add /composes route to list root job UUIDs
This will be used to list the top level job UUIDs.

Related: RHEL-60120
2025-02-03 17:27:31 -08:00
Brian C. Lane
87c0462a33 jobqueue: Add AllRootJobIDs function to jobqueue
This lists the root job UUIDs (the jobs with no dependants).
Currently only implemented by fsjobqueue. The function for
dbjobqueue currently returns nil.

Related: RHEL-60120
2025-02-03 17:27:31 -08:00
Achilleas Koutsou
bec893e37c go.mod: update github.com/vmware/govmomi to v0.48.0
Needs manual change of import paths.
2025-02-01 18:14:18 +01:00
Brian C. Lane
a613e8cb37 DepsolveJobResult: Remove unused Error and ErrorType
These fields are not set by the depsolve job, they are only set and used
in tests so remove them. Errors are reported in the result.JobError

Related: Related: RHEL-60125
2025-01-30 08:00:12 +01:00
Brian C. Lane
29f81a029a v2_test: Add a test for /search/packages
Add a new mockSearch job handler to return static results, add tests for
searching, for distro error, and for arch error.

Related: RHEL-60136
2025-01-30 08:00:12 +01:00
Brian C. Lane
17bebb547c v2 tests: Make it easier to add mock handlers to newV2Server
This refactors the server setup, splitting the depsolve and ostree
resolve goroutine creation into helper functions. It also removes the
use of channels, which was always set to "" (and in the case of the
multi-tenancy test an empty list, which acts the same).

Related: RHEL-60136
2025-01-30 08:00:12 +01:00
Brian C. Lane
532f1b0396 cloudapi: Hook up the /search/packages handler
This connects all the pieces needed to implement the search.

If you POST a request to /search/packages like this:

    {
      "packages": [
        "tmux"
      ],
      "distribution": "fedora-41",
      "architecture": "x86_64"
    }

It will return details about the tmux packages that looks like this:

{
  "packages": [
    {
      "arch": "x86_64",
      "buildtime": "2024-10-10T00:19:06Z",
      "description": "tmux is ...",
      "license": "ISC AND BSD-2-Clause AND BSD-3-Clause AND SSH-short AND LicenseRef-Fedora-Public-Domain",
      "name": "tmux",
      "release": "2.fc41",
      "summary": "A terminal multiplexer",
      "url": "https://tmux.github.io/",
      "version": "3.5a"
    }
  ]
}

Resolves: RHEL-60136
2025-01-30 08:00:12 +01:00
Brian C. Lane
234e8a09eb openapi: Add /search/packages route to the openapi schema
The request is similar to a depsolve request, it must include the
distribution and architecture. It can optionally include a list of
repositories to search, but if they are not included it searches the
default repos for the distro:arch

Related: RHEL-60136
2025-01-30 08:00:12 +01:00
Brian C. Lane
bd55670dd9 worker: Add worker server support for Search job
This adds support for sending a search job to the worker client,
gathering results, and handling errors.

The errors returned are the same as for the Depsolve job, since they
both use the osbuild-depsolve-dnf script via images/pkg/dnfjson.

Related: RHEL-60136
2025-01-30 08:00:12 +01:00
Brian C. Lane
d8df7e7cd4 worker: Add search job implementation to worker client
This is similar to the depsolve job, and it shares the solver (which
supports locking, as does DNF itself). This will allow searching for
specific package names, names with globs, or names as substrings of
other names using * as the wildcard.

Related: RHEL-60136
2025-01-30 08:00:12 +01:00
Michael Vogt
e8a0e8ff49 weldr: update depsolve calls in weldr API
Update the weldr API to work with the new depsolve API.
Update tests to match (adding repo_id).

Co-authored-by: Achilleas Koutsou <achilleas@koutsou.net>
2025-01-29 18:03:11 +01:00
Achilleas Koutsou
dab836de19 weldr/test: fix test run name 2025-01-29 18:03:11 +01:00
Michael Vogt
a6ba0785b0 cloudapi: fix manifestSource.Serialize() 2025-01-29 18:03:11 +01:00