Also extend the Koji test case to verify that the boot mode information
is in the build extra metadata and that it contains valid value.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Expose the extra metadata information for each image output stored in
`ImageExtraInfo` also in the build extra metadata. The extra metadata
for each image is nested under key corresponding to the image filename.
Extend the Koji test case to check information in the buildinfo output
and specifically check for all expected metadata in the build Extra
metadata field.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The original Koji implementation expected that the output of a content
generator is only an image. While in reality, we will eventually upload
other types of files as outputs to Koji, such as logs and osbuild
manifest.
Rename Koji structures and their members to better map to the upstream
Koji documentation and their JSON representation. Add comments to
structures. Define type aliases and constants for string values which
are more like enums, than a free-form values.
These changes have no effect on the actual JSON representation of any of
the structures
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Remove all the internal package that are now in the
github.com/osbuild/images package and vendor it.
A new function in internal/blueprint/ converts from an osbuild-composer
blueprint to an images blueprint. This is necessary for keeping the
blueprint implementation in both packages. In the future, the images
package will change the blueprint (and most likely rename it) and it
will only be part of the osbuild-composer internals and interface. The
Convert() function will be responsible for converting the blueprint into
the new configuration object.
The ellipsis operator was used as a hack to not need to pass any details
as an argument, but it makes what the end object will actually look like
less obvious. It also makes it impossible to pass an array to details
without getting a nested array.
Fixes#2874
Enhance the `koji-finalize` job implementation to be able to cope with
multiple upload targets being specified for an `OSBuildJob`.
Implement a convenience method `OSBuildJobResult.TargetResultsByName()`
for filtering the target results attached to the job result by their
name. Cover the method with an unit test. And lastly use this method in
the `koji-finalize` job to find the appropriate Koji upload target
results.
This is a preparation for enabling cloud uploads for Koji composes.
Enhance the `koji-finalize` job implementation to use deferred function
to ensure that the job status is always reported back to the composer.
In addition, if the `JobError` is set, also fail the Koji job.
Previously, composer and Koji were not updated in some corner cases when
the job would fail.
Koji API removed by the previous commit was the last user of osbuild-koji job.
Let's remove it since nothing uses it. This also removes all of the
compatibility code in Cloud API, see concerns below:
Compatibility concerns:
- the internal deployment was moved to a completely different composer
instance, thus there are no old jobs
- Fedora deployment is still unused in prod, thus we don't care about keeping
backward compatibility of the old jobs
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
This commit moves the field to the koji struct where it actually belongs.
Also, it renames it to relax_timeout_factor for the sake of consistency.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Adjust the `koji-finalize` job implementation to be able to handle
results from both the `osbuild` and `osbuild-koji` jobs.
In case of `osbuild` job, the result is of type
`worker.OSBuildJobResult` and the important values are stored in the
Koji upload target options. For now assume that there may be only a
single upload target results.
In case of `osbuild-koji` job, the result is of type
`worker.OSBuildKojiJobResult` and the important values are already part
of the structure. Add "Old" suffix to all functions handling this case.
Move `OSBuildMetadataToRPMs()` and `PackageMetadataToSignature()`
functions from the `rpmmd` package to `osbuild2` package to prevent
import cycles while de-duplicating `rpmStageInputs()` function from
`stage_inputs.go` of distro definitions.
Rename `PackageMetadataToSignature()` to
`RPMPackageMetadataToSignature()`, since it takes specifically
`RPMPackageMetadata` type as an argument.
Adjust affected parts of code (unit tests, cloudapi, worker).
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Before accessing a field of the `OSBuildOutput`, which itself is a
field of the `osbuildKojiResults` struct, check if it is actually
is set (non-nill), otherwise dereferencing it will crash the
worker.
The field will be null if osbuild has not been invoked at all or
if osbuild crashed or refused to accept the input.
Implement the structured errors as defined by the worker client.
Every error for each of the job types now returns a structured
error with a reason and a specific error code. This will make
it possible to differentiate between 4xx errors and 5xx errors.
This commit refactors the way errors are implemented in the workers,
but maintains backwards compatability in composer by checking for
both kinds of errors.
Pipeline names are added to each job before adding to the queue. When a
job is finished, the names are copied to the Result object as well. This
is done for both OSBuild and Koji jobs.
The pipeline names in the result are primarily used to separate package
lists into build and payload/image packages in two cases:
1. Koji builds: for reporting the build root and image package lists to
Koji (in Koji finalize).
2. Cloud API (v1 and v2): for reporting the payload packages in the
metadata request.
The pipeline names are also used to print the system log output in the
order in which pipelines are executed. This still isn't used when
printing the OSBuild Result (osbuild2.Result.Write()) and we still rely
on sorting by pipeline name
(see https://github.com/osbuild/osbuild-composer/pull/1330).
- koji-finalize:
Use v2 result type to collect RPM metadata.
The separation between the "build" pipeline and the rest is based on the
pipeline name, which isn't completely reliable since pipeline names can
be arbitrary.
Koji will fail a build if it specifies duplicate packages, so the RPM
lists are deduplicated. The "build" pipeline package list is also
deduplicated in case there are multiple build stages in the same
pipeline.
- osbuild:
Use v2 result type for printing build result to log.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Move the OSBuildStagesToRPMs function, associated test, and RPM type
from the worker into the rpmmd subpackge. We will use this function in
the cloud API to compile the NEVRAs for the new metadata endpoint.
Previously, the checks that dependencies were successful were all over the
Run() method. This led to a issue #1101 (lovely binary number btw).
This commit rewrites the Run() method to:
1) Extract dynamic args. Return an error if they cannot be unmarshalled.
2) Check if dependencies were successful. If not, call kojiFail, update the
job and return.
3) Create the CGImport metadata and call kojiImport.
Fixes#1101
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Serializing an interface does not work, let us simply use the string
representation and treat the empty string as no error. This is
compatible with the current API in the success case, and fixes the
error case, which is currently broken.
Also extend the test matrix for the kojiapi to ensure that all the
different kinds of errors can be serialized correctly and leads to
the correct status being returned.
Fixes#1079 and #1080.
The three new job types osbuild-koji, koji-init, and koji-finalize
allows the different tasks to be split appart and in particular for
there to be several builds on different architectures as part of a
given compose.
