Copy the Manifest info data from the Manifest job result to the Koji
target result, so that this information can be then imported to Koji
build metadata by the koji-finalize job.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Include the osbuild/images module version in the Manifest job result.
The module has direct impact on image definitions and the content of
produced manifest, therefore including this information in the Manifest
job result is very helpful for various purposes (debugging,
traceability).
This will enable to embed this information in the Koji build metadata.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the Manifest job result structure to hold information about
osbuild-composer version, which produced the manifest. This will be
useful for other job types which depend on it and can then push this
information further as needed.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Expose the osbuild information used to produce the image, in the image
extra metadata under the `osbuild_artifact` property.
This information will get included in the image / build extra metadata
and make it explicit how to reproduce the image build using osbuild and
the attached manifest.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add the information about osbuid artifact to the target result.
Specifically the name of the osbuild pipeline which was exported for the
specific target, and the filename of the exported file.
This will later enable embedding this information in Koji build metadata
to make it easy to reproduce the image build using the attached
manifest.
Add any non-Koji upload target results attached to an OSBuild result, to
the image extra metadata. This will make it easy to locate any image
from Koji uploaded to cloud, in the target cloud environment.
The rationale behind including only non-Koji target results is that one
can find it only in Koji, so there is no added value in including the
Koji target results at all.
Extend the `koji.sh` to check the target results in image metadata when
testing Koji scenario with cloud upload.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Import osbuild manifest and build log to the Koji build as outputs. Also
note the respective filenames in the image output extra metadata.
Note that the osbuild manifest is imported as a log file for now. Koji
has very limited set of output types defined and I still need to
determine the best way to use a custom output type in Koji instances (as
other content generators do).
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Rename the variable to make it easily tell what it does at the first
glance. Previous name made it look as if it held build (job?) arguments,
which was not the case.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the Koji target handling in the OSBuild job implementation and
upload also the osbuild manifest and osbuild output log to Koji. Add all
the necessary metadata to the Koji target result options, so that
KojiFinalize job can then import them to the build.
Note that none of these files is yet imported by the KojiFinalize job.
Also note that the osbuild output log is still a JSON, which is not
great to read by humans. Adjustments to make it nice will be done in
following commits.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The `KojiTargetResultOptions` previously contained information only
about the uploaded image file. And even then, some information, such as
the filename, were scattered in other structures such as
`KojiFinalizeJob` struct.
Since the plan is to start uploading also osbuild manifest and osbuild
build log to Koji, we need to extend the result options structure to
hold more information and also make it specific to which file is the
information related.
Rework the `KojiTargetResultOptions` to contain information about:
- the built image
- build log
- osbuild manifest
Information about each file contains:
- filename
- checksum type
- file checksum
- file size
For now, only the built image information is set and consumed by the
worker.
Add custom JSON (un)marshaler for `KojiTargetResultOptions` to handle
backward compatibility when old version of worker or composer server
interact with each other. Cover them with unit tests.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Refactor koji.sh:
- Store Koji hub URL in a variable and use it instead of a literal.
- Do not inspect builinfo output and don't parse it using grep and sed,
but instead directly call Koji XMLRPC functions (using `koji call`)
with JSON output.
In addition, test that the image archive extra metadata are the same as
those stored in the build extra metadata, stored under the image
filename.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
These use 'pkg1' when depsolving, so they need an entry in the manifest
with the mocked checksum:
sha256:e50ddb78a37f5851d1a5c37a4c77d59123153c156e628e064b9daa378f45a2fe
Commit 312d87c6d5fab7ffd085a303e27b8db41111c86e adds validation of the
checksums to the ImageType.Manifest call, so it requires a valid looking
checksum, otherwise it will fail with a 'ManifestCreationFailed' error
when running the unit tests.
The fluentd sidecar had the same request/limit as the service container,
and the migrate init-container had the fluentd request/limit. It should
be the other way round.
The job won't run if it doesn't get scheduled within 30 minutes. This
prevents the job running multiple times in a row if it didn't get
scheduled, for instance due to resource limits.
Uploads an OCI image to OCI object storage, and generates a
pre-authenticated request for the object, which can be used to import it
into custom images.
Run a basic check to see that the tailoring file has been created when
adding tailoring rules to the OpenSCAP config. Since the OpenSCAP scan
requires more memory and breaks some of the API tests, a full scan is
not run in the cloudapi tests.
Add support to the cloudapi for generating the tailoring file used
to customize the OpenSCAP remediation. This allows users to select and
unselect rules for the remediation and the `autotailor` stage generates
the tailoring file.
resp.IsResumable crashes if resp.MultipartUploadResponse == nil. This happens
for instance when authentication fails. Fix this by also checking the
MultipartUploadResponse field.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Previously, the worker would happily report success if osbuild succeeded,
there was no JobError, but the job actually panicked in the meantime.
Let's fix this by adding a recovery mechanism.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
test logs print:
[2023-08-22T10:18:14-04:00] 💿 Install image via installer(ISO) on VM
WARNING --os-type is deprecated and does nothing. Please stop using it.
