Include the osbuild version used to build the image, in the image extra
metadata attached to the Koji build and image output.
Extend `koji.sh` to verify that the version is set and of the expected
value.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This will make it easier to reproduce image builds using the same
tooling which was used by the worker.
This change will enable to include the osbuild version in Koji build
metadata.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Expose the extra metadata attached to each manifest output, also to the
build extra metadata under `osbuild_manifest` property. The value is a
map of all manifest metadata with keys being the filename of each
manifest.
This will expose the information about osbuild-composer and
osbuild/images versions used to produce the manifest in the Koji
buildinfo, including the Web UI.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Include the osbuild-composer and its dependencies versions in the extra
metadata associated with the Manifest output when importing it to Koji.
This will make it possible to pin-point the exact version combination
which was used to generate the osbuild manifest used to built the image
imported to Koji.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Copy the Manifest info data from the Manifest job result to the Koji
target result, so that this information can be then imported to Koji
build metadata by the koji-finalize job.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Include the osbuild/images module version in the Manifest job result.
The module has direct impact on image definitions and the content of
produced manifest, therefore including this information in the Manifest
job result is very helpful for various purposes (debugging,
traceability).
This will enable to embed this information in the Koji build metadata.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the Manifest job result structure to hold information about
osbuild-composer version, which produced the manifest. This will be
useful for other job types which depend on it and can then push this
information further as needed.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Expose the osbuild information used to produce the image, in the image
extra metadata under the `osbuild_artifact` property.
This information will get included in the image / build extra metadata
and make it explicit how to reproduce the image build using osbuild and
the attached manifest.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add the information about osbuid artifact to the target result.
Specifically the name of the osbuild pipeline which was exported for the
specific target, and the filename of the exported file.
This will later enable embedding this information in Koji build metadata
to make it easy to reproduce the image build using the attached
manifest.
Add any non-Koji upload target results attached to an OSBuild result, to
the image extra metadata. This will make it easy to locate any image
from Koji uploaded to cloud, in the target cloud environment.
The rationale behind including only non-Koji target results is that one
can find it only in Koji, so there is no added value in including the
Koji target results at all.
Extend the `koji.sh` to check the target results in image metadata when
testing Koji scenario with cloud upload.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Import osbuild manifest and build log to the Koji build as outputs. Also
note the respective filenames in the image output extra metadata.
Note that the osbuild manifest is imported as a log file for now. Koji
has very limited set of output types defined and I still need to
determine the best way to use a custom output type in Koji instances (as
other content generators do).
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The `KojiTargetResultOptions` previously contained information only
about the uploaded image file. And even then, some information, such as
the filename, were scattered in other structures such as
`KojiFinalizeJob` struct.
Since the plan is to start uploading also osbuild manifest and osbuild
build log to Koji, we need to extend the result options structure to
hold more information and also make it specific to which file is the
information related.
Rework the `KojiTargetResultOptions` to contain information about:
- the built image
- build log
- osbuild manifest
Information about each file contains:
- filename
- checksum type
- file checksum
- file size
For now, only the built image information is set and consumed by the
worker.
Add custom JSON (un)marshaler for `KojiTargetResultOptions` to handle
backward compatibility when old version of worker or composer server
interact with each other. Cover them with unit tests.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
These use 'pkg1' when depsolving, so they need an entry in the manifest
with the mocked checksum:
sha256:e50ddb78a37f5851d1a5c37a4c77d59123153c156e628e064b9daa378f45a2fe
Commit 312d87c6d5fab7ffd085a303e27b8db41111c86e adds validation of the
checksums to the ImageType.Manifest call, so it requires a valid looking
checksum, otherwise it will fail with a 'ManifestCreationFailed' error
when running the unit tests.
Uploads an OCI image to OCI object storage, and generates a
pre-authenticated request for the object, which can be used to import it
into custom images.
Add support to the cloudapi for generating the tailoring file used
to customize the OpenSCAP remediation. This allows users to select and
unselect rules for the remediation and the `autotailor` stage generates
the tailoring file.
resp.IsResumable crashes if resp.MultipartUploadResponse == nil. This happens
for instance when authentication fails. Fix this by also checking the
MultipartUploadResponse field.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Previously, the worker would happily report success if osbuild succeeded,
there was no JobError, but the job actually panicked in the meantime.
Let's fix this by adding a recovery mechanism.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
The SEV-SNP support was added since RHEL-9.1, so we need to keep the
original Guest OS Feature set when importing RHEL-9.0 images to GCP.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Previously, nil values in the conversion source were in some cases converted
to empty arrays or empty objects. This is undesirable, because it can be in
certain cases changing the semantics of the blueprint. See e.g.
f317064da5/pkg/distro/rhel7/imagetype.go (L239C7-L239C7)
This commit modifies the conversion process so nil values are converted
without any changes. Also, the `Convert` function was covered with a unit
test.
During development it can be very useful to store the results locally
instead of uploading to a remote system. This implements a development
only option to help with that.
To use it you need to add OSBUILD_LOCALSAVE to the server's environment.
This can be done by editing /usr/lib/systemd/system/osbuild-composer.service
and adding:
Environment="OSBUILD_LOCALSAVE=1"
You can then use an 'upload_options' object to skip trying to upload to
the default service for the type of image, eg:
"image_requests": [
{
"architecture": "x86_64",
"image_type": "guest-image",
"upload_options": {
"local_save": true
},
...
}]
The results will be saved to /var/lib/osbuild-composer/artifacts/UUID/
using the default filename for the image type.
If local_save is used without OSBUILD_LOCALSAVE being set it will return
an error with id=36 saying 'local_save is not enabled'.
Also extend the Koji test case to verify that the boot mode information
is in the build extra metadata and that it contains valid value.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add the information about the image boot mode to the OSBuildJob options
as well as to the OSBuildJobResult options. The intention is that the
worker will simply copy the value from the job options to job result,
so that the `KojiFinalize` job can then access this information and
upload it to Koji as extra metadata.
This information is required in Koji in order for Red Hat's SP tooling
to know how to import image to the cloud environment in case the boot
mode affects the import parameters.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Expose the extra metadata information for each image output stored in
`ImageExtraInfo` also in the build extra metadata. The extra metadata
for each image is nested under key corresponding to the image filename.
Extend the Koji test case to check information in the buildinfo output
and specifically check for all expected metadata in the build Extra
metadata field.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The original Koji implementation expected that the output of a content
generator is only an image. While in reality, we will eventually upload
other types of files as outputs to Koji, such as logs and osbuild
manifest.
Rename Koji structures and their members to better map to the upstream
Koji documentation and their JSON representation. Add comments to
structures. Define type aliases and constants for string values which
are more like enums, than a free-form values.
These changes have no effect on the actual JSON representation of any of
the structures
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This moves some of the code from the PostCompose function in handler.go
into methods on the OpenAPI ComposeRequest and ImageRequest structs.
In compose.go I have added several methods.
GetBlueprintWithCustomizations takes the ComposeRequest customizations
and builds a Blueprint struct.
GetPayloadRepositories returns the custom payload repos.
GetSubscription returns the ImageOptions setup with optional
subscription information from the request.
In imagerequest.go I have added GetTarget which takes the upload
options and returns a Target. This moves the giant switch statement,
which may also benefit from further simplification at some point.
GetOSTreeOptions returns the OSTree ImageOptions if there are ostree
settings in the ImageRequest.
GetImageOptions returns the distro.ImageOptions with the size set.
This commit only moves the code, making PostCompose easier to read. All
tests still pass.
cloudapi: Move the size handling to a method on ImageRequest
This adds a 'size' parameter to the image_request object. It can be used
to specify the minimum image size in bytes
This behaves in the same way as the size parameter of the weldr API
For raw images the root partition is grown to fill the available space.
For LVM images the PV uses the available space, but the LV does not,
leaving space available for other LVs to be created after boot.
See COMPOSER-1883
UBI and the oldest support Fedora (37) now all have go 1.19, so we are
cleared to switch.
gofmt now reformats comments in certain cases, so that explains the formatting
changes in this commit.
See https://go.dev/doc/go1.19#go-doc
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
