Instead of going directly fatal inside the handlers this commit provides
a wrapper function allowing handlers to instead return an `error`; this
will allow for checking deferred errors.
Add any errors to job error details when an osbuild build fails.
Otherwise these won't show up in the worker log, which makes
debugging issues harder.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add new configuration option `distro_aliases`, which is a map of
strings, allowing to specify distro name alias for supported
distributions.
Define aliases for RHEL major versions without the minor version
specified.
For now, the distro aliases map is not used by any API
implementation and it is ignored.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Drop `common.CurrentArch()` implementation and use
`arch.Current().String()` from the osbuild/images instead.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Drop `common.GetHostDistroName()` implementation and use
`distro.GetHostDistroName()` from the osbuild/images instead.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Previously, the old dnfjson cache dirs for unsupported distributions
were deleted in the osbuild-composer binary on startup. This is no
longer possible, since the supported distros are determined by loading
available repositories. Loading repositories happens in the Weldr API
constructor. Move the cleanup code there.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Update the osbuild/images to the version which introduces "dot notation"
for distro release versions.
- Replace all uses of distroregistry by distrofactory.
- Delete local version of reporegistry and use the one from the
osbuild/images.
- Weldr: unify `createWeldrAPI()` and `createWeldrAPI2()` into a single
`createTestWeldrAPI()` function`.
- store/fixture: rework fixtures to allow overriding the host distro
name and host architecture name. A cleanup function to restore the
host distro and arch names is always part of the fixture struct.
- Delete `distro_mock` package, since it is no longer used.
- Bump the required version of osbuild to 98, because the OSCAP
customization is using the 'compress_results' stage option, which is
not available in older versions of osbuild.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This command is part of the osbuild/images repo, where all image
definitions live. Having it in the osbuild-composer repository does not
add any value.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This command was used by the original Python scripts used to generate
test manifests. These scripts were deleted, thus there is no need to
keep this command around any more.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This command is part of the osbuild/images repo, where all image
definitions live. Having it in the osbuild-composer repository does not
add any value.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This command is part of the osbuild/images repo, where all image
definitions live. Having it in the osbuild-composer repository does not
add any value.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add a local socket for communicating with the cloudapi. It is started by
osbuild-composer.socket and is located at /run/cloudapi/api.socket
cloudapi requests can be passed to it using curl like this:
curl -k --unix-socket /run/cloudapi/api.socket --header 'Content-Type: application/json' \
--data request.json http://localhost/api/image-builder-composer/v2/compose
A simple request.json looks like this:
{
"distribution": "fedora-38",
"image_request":
{
"architecture": "x86_64",
"image_type": "guest-image",
"upload_options": {},
"repositories": [
{
"name": "fedora",
"metalink": "https://mirrors.fedoraproject.org/metalink?repo=fedora-38&arch=x86_64",
"check_gpg": false
},
{
"name": "updates",
"metalink": "https://mirrors.fedoraproject.org/metalink?repo=updates-released-f38&arch=x86_64",
"check_gpg": false
}
]
}
}
Allow passing module_hotfixes flag through the cloudapi.
This will enable depsolving on repositories that might be affected by modularity filtering.
Refs HMS-3202
Since the UploadAndDistributeCommit() function now waits for all tasks
to finish, update the wording of the final message to indicate that the
commit is available.
The latest images introduced a new blueprint option: minimal. Since we use
direct type-conversion, we need to add it also in gen-manifests. A warning
is also added, since this feature is considered experimental.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Indent the osbuild manifest before uploading it to Koji. This will make
it much nicer for reading by humans.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Include the osbuild version used to build the image, in the image extra
metadata attached to the Koji build and image output.
Extend `koji.sh` to verify that the version is set and of the expected
value.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This will make it easier to reproduce image builds using the same
tooling which was used by the worker.
This change will enable to include the osbuild version in Koji build
metadata.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Expose the extra metadata attached to each manifest output, also to the
build extra metadata under `osbuild_manifest` property. The value is a
map of all manifest metadata with keys being the filename of each
manifest.
This will expose the information about osbuild-composer and
osbuild/images versions used to produce the manifest in the Koji
buildinfo, including the Web UI.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Include the osbuild-composer and its dependencies versions in the extra
metadata associated with the Manifest output when importing it to Koji.
This will make it possible to pin-point the exact version combination
which was used to generate the osbuild manifest used to built the image
imported to Koji.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Copy the Manifest info data from the Manifest job result to the Koji
target result, so that this information can be then imported to Koji
build metadata by the koji-finalize job.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Expose the osbuild information used to produce the image, in the image
extra metadata under the `osbuild_artifact` property.
This information will get included in the image / build extra metadata
and make it explicit how to reproduce the image build using osbuild and
the attached manifest.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add the information about osbuid artifact to the target result.
Specifically the name of the osbuild pipeline which was exported for the
specific target, and the filename of the exported file.
This will later enable embedding this information in Koji build metadata
to make it easy to reproduce the image build using the attached
manifest.
Add any non-Koji upload target results attached to an OSBuild result, to
the image extra metadata. This will make it easy to locate any image
from Koji uploaded to cloud, in the target cloud environment.
The rationale behind including only non-Koji target results is that one
can find it only in Koji, so there is no added value in including the
Koji target results at all.
Extend the `koji.sh` to check the target results in image metadata when
testing Koji scenario with cloud upload.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Import osbuild manifest and build log to the Koji build as outputs. Also
note the respective filenames in the image output extra metadata.
Note that the osbuild manifest is imported as a log file for now. Koji
has very limited set of output types defined and I still need to
determine the best way to use a custom output type in Koji instances (as
other content generators do).
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Rename the variable to make it easily tell what it does at the first
glance. Previous name made it look as if it held build (job?) arguments,
which was not the case.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the Koji target handling in the OSBuild job implementation and
upload also the osbuild manifest and osbuild output log to Koji. Add all
the necessary metadata to the Koji target result options, so that
KojiFinalize job can then import them to the build.
Note that none of these files is yet imported by the KojiFinalize job.
Also note that the osbuild output log is still a JSON, which is not
great to read by humans. Adjustments to make it nice will be done in
following commits.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The `KojiTargetResultOptions` previously contained information only
about the uploaded image file. And even then, some information, such as
the filename, were scattered in other structures such as
`KojiFinalizeJob` struct.
Since the plan is to start uploading also osbuild manifest and osbuild
build log to Koji, we need to extend the result options structure to
hold more information and also make it specific to which file is the
information related.
Rework the `KojiTargetResultOptions` to contain information about:
- the built image
- build log
- osbuild manifest
Information about each file contains:
- filename
- checksum type
- file checksum
- file size
For now, only the built image information is set and consumed by the
worker.
Add custom JSON (un)marshaler for `KojiTargetResultOptions` to handle
backward compatibility when old version of worker or composer server
interact with each other. Cover them with unit tests.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
