osbuild/images#751 wrapped the errors in the images/dnfjson package to
provide more details, the depsolve job should take this into account to
map the dnfjson error to the correct worker client error.
This caused user input errors errors to be misclassified as internal
errors, triggering depsolve job failure alerts.
Previously, the worker was determining the GCE image guest OS Features
on its own, based on the OS name. This caused problems, in case the
osbuild-composer was of a newer version than the worker.
Example:
osbuild-composer contained support for c10s GCE image type and its
implementation also contained the proper guest OS Features list for it.
However, when the worker got the osbuild job, it built it and tried to
fetch the guest OS Features for the distro. Since its implementation was
too old, it didn't contain the code that added the actual support for
c10s GCE images and got no guest OS features list (which is the default
for unsupported distros). The image was successfully uploaded and
shared, but it does not boot in GCP, because it does not know that it
should use UEFI to boot it.
This behavior could be considered a bug. The worker should be dumb. It
should not be making decisions about the image features, but instead it
should take them from the upload target options. And composer should be
the authoritative source of truth for this. Because otherwise, we
basically have two components that need to be updated in sync to add
support for GCE images on a new distro.
Move the GCE image guest OS features to the GCP upload target options.
The worker will just take what is specified there and use it when
importing the image to GCP. As a compatibility layer for the case when
the composer would be older than the worker (unlikely, but still),
worker will try to determine the image guest OS features in case the
list in the upload target options is empty.
Extend the GCP functional tests to check that the imported image has at
least some guest OS features set.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The workerClientErrorFrom() was returning an `*clienterrors.Error` and
an `error` (if something with the conversation goes wrong.
But the calling code was expecting that even if an `error` is returned
the `*clienterrors.Error` is still valid. The caller would then just
log the error. As returning a valid `value` even when there is an
`error` is an unexpected pattern this commit changes the code to
always return a `*clienterrors.Error` and log any issue via the
logger.
The usual convention to create new object is to prefix `New*` so
this commit renames the `WorkerClientError`. Initially I thought
it would be `NewWorkerClientError()` but looking at the package
prefix it seems unneeded, i.e. `clienterrors.New()` already
provides enough context it seems and it's the only error we
construct.
We could consider renaming it to `clienterror` (singular) too
but that could be a followup.
I would also like to make `clienterror.Error` implement the
`error` interface but that should be a followup to make this
(mechanical) rename trivial to review.
This is an alternative/complementary fix for PR#4137. It is very
simple so should be uncontroverisal.
It fixes an issue that @schuellerf discovered, i.e. that when an error
interface is passed into clienterrors.Error.Details the details get
lost because the json.Marshaler will not know how to handler an
error interface.
To find the problematic uses of `error` a custom vet checker was
build in https://github.com/mvo5/osbuild-cvet. With that the
result is:
```
$ go run github.com/mvo5/osbuild-cvet@latest ./...
/home/mvogt/devel/osbuild/osbuild-composer/cmd/osbuild-worker/jobimpl-depsolve.go:93:26: do not pass 'error' to WorkerClientError() details, use error.Error() instead
/home/mvogt/devel/osbuild/osbuild-composer/cmd/osbuild-worker/jobimpl-osbuild.go:404:31: do not pass 'error' to WorkerClientError() details, use error.Error() instead
/home/mvogt/devel/osbuild/osbuild-composer/cmd/osbuild-worker/jobimpl-osbuild.go:519:31: do not pass 'error' to WorkerClientError() details, use error.Error() instead
/home/mvogt/devel/osbuild/osbuild-composer/cmd/osbuild-worker/jobimpl-osbuild.go:556:31: do not pass '[]error' to WorkerClientError() details, use []string instead
```
and once this commit is in no more errors.
Just like PR#4137 this is not perfect because it will not do a
recursive check for the passed argument.
We need the ability to use different CloudWatch group for the
osbuild-executor on Fedora workers in staging and production
environment.
Extend the worker confguration to allow configuring the CloudWatch group
name used by the osbuild-executor. Extend the secure instance code to
instruct cloud-init via user data to create /tmp/cloud_init_vars file
with the CloudWatch group name in the osbuild-executor instance, to make
it possible for the executor to configure its logging differently based
on the value.
Cover new changes by unit tests.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Multiple blueprint fixes:
- Extend the blueprint service customizations to accept services to be
masked.
- The `storage-path` and `container-transport` fields were removed in
imagees 41.0 in order to simplify the way local storage containers are
handled.
Add any errors to job error details when an osbuild build fails.
Otherwise these won't show up in the worker log, which makes
debugging issues harder.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Drop `common.CurrentArch()` implementation and use
`arch.Current().String()` from the osbuild/images instead.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Drop `common.GetHostDistroName()` implementation and use
`distro.GetHostDistroName()` from the osbuild/images instead.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Update the osbuild/images to the version which introduces "dot notation"
for distro release versions.
- Replace all uses of distroregistry by distrofactory.
- Delete local version of reporegistry and use the one from the
osbuild/images.
- Weldr: unify `createWeldrAPI()` and `createWeldrAPI2()` into a single
`createTestWeldrAPI()` function`.
- store/fixture: rework fixtures to allow overriding the host distro
name and host architecture name. A cleanup function to restore the
host distro and arch names is always part of the fixture struct.
- Delete `distro_mock` package, since it is no longer used.
- Bump the required version of osbuild to 98, because the OSCAP
customization is using the 'compress_results' stage option, which is
not available in older versions of osbuild.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Indent the osbuild manifest before uploading it to Koji. This will make
it much nicer for reading by humans.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Include the osbuild version used to build the image, in the image extra
metadata attached to the Koji build and image output.
Extend `koji.sh` to verify that the version is set and of the expected
value.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This will make it easier to reproduce image builds using the same
tooling which was used by the worker.
This change will enable to include the osbuild version in Koji build
metadata.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Expose the extra metadata attached to each manifest output, also to the
build extra metadata under `osbuild_manifest` property. The value is a
map of all manifest metadata with keys being the filename of each
manifest.
This will expose the information about osbuild-composer and
osbuild/images versions used to produce the manifest in the Koji
buildinfo, including the Web UI.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Include the osbuild-composer and its dependencies versions in the extra
metadata associated with the Manifest output when importing it to Koji.
This will make it possible to pin-point the exact version combination
which was used to generate the osbuild manifest used to built the image
imported to Koji.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Copy the Manifest info data from the Manifest job result to the Koji
target result, so that this information can be then imported to Koji
build metadata by the koji-finalize job.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Expose the osbuild information used to produce the image, in the image
extra metadata under the `osbuild_artifact` property.
This information will get included in the image / build extra metadata
and make it explicit how to reproduce the image build using osbuild and
the attached manifest.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add the information about osbuid artifact to the target result.
Specifically the name of the osbuild pipeline which was exported for the
specific target, and the filename of the exported file.
This will later enable embedding this information in Koji build metadata
to make it easy to reproduce the image build using the attached
manifest.
Add any non-Koji upload target results attached to an OSBuild result, to
the image extra metadata. This will make it easy to locate any image
from Koji uploaded to cloud, in the target cloud environment.
The rationale behind including only non-Koji target results is that one
can find it only in Koji, so there is no added value in including the
Koji target results at all.
Extend the `koji.sh` to check the target results in image metadata when
testing Koji scenario with cloud upload.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
