It turned out that the upload target was never adopted by the service,
thus we are removing it as part of upload code consolidation.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
With the upload code consolidation to osbuild/images, we need to make
sure to configure the logger used by the library to keep logging the
same (or similar) messages when running osbuild-composer.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
v1.60 seems to have some issues [1] with something in our dependency
chain. Update to v1.61 and fix all new issues.
New issues are all instances of potential integer overflow from int ->
uint conversions. Added guards where appropriate and disabled the check
when when it's not needed.
[1] https://github.com/osbuild/osbuild-composer/actions/runs/16624417387/job/47037518471
Delete the `internal/upload/koji` package and replace it with
`pkg/upload/koji` package provided by `osbuild/images`.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Modify the Koji image extra metadata to not depend on the upload target
data structure for the OSBuild Artifact. This is the last dependency on
the internal osbuild-composer package, allowing the move of the Koji
upload code to the osbuild/images repository.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Modify the Koji image extra metadata to not depend on the upload target
data structures for upload results. The target results are supposed to
be specific to the uploader implementation, which will eventually
change. Moreover, the definition is internal to osbuild-composer, so
this would create a problem once the Koji upload implementation is moved
to osbuild/images.
The Koji upload implementation does not really care about the exact
structure of the upload results. It is just a list of JSON objects.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The use of rpmmd.RPM is exclusive to the Koji upload implementation.
Consolidate the metadata structure for RPMs into the koji package
codebase, together with any required functions for converting osbuild
stage metadata. The respective code in osbuild/images will be eventually
removed.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The upload/koji package functions were creating a logger and then were
using it. This is not ideal for a library implementation.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This cleans up the linting results by adding checks for
integer underflow/overflow in several places, suppressing the error in
places where it has been checked, or fixing the types when possible.
Previously it would not return a result if there was an error. This adds
a deferred function that always returns the current contents of result,
and if there is an error it logs it.
Related: RHEL-60125
This is similar to the depsolve job, and it shares the solver (which
supports locking, as does DNF itself). This will allow searching for
specific package names, names with globs, or names as substrings of
other names using * as the wildcard.
Related: RHEL-60136
This is a bit of an RFC commit, I noticed that when we discussed
a crash from the worker we looked at individual message from
syslog/journald for the stacktrace deatils. I was wondering if
having a more direct crash report would be more useful? We can
of course also add more logrus features to flag those with tags
like "crash" or something (I did not do that in this PR, I don't
know much about the operational side, sorry).
Extend the Koji target handling in the osbuild job, to also upload SBOM
documents attached to the related depsolve job result.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
If the Koji target result contains information about any uploaded SBOM
documents, import them to Koji as part of the finalize task.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Adjust all paces that call `Solver.Depsolve()`, to cope with the changes
that enabled SBOM support.
Fix loading of testing repositories in the CloudAPI unit tests.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
osbuild/images#751 wrapped the errors in the images/dnfjson package to
provide more details, the depsolve job should take this into account to
map the dnfjson error to the correct worker client error.
This caused user input errors errors to be misclassified as internal
errors, triggering depsolve job failure alerts.
Previously, the worker was determining the GCE image guest OS Features
on its own, based on the OS name. This caused problems, in case the
osbuild-composer was of a newer version than the worker.
Example:
osbuild-composer contained support for c10s GCE image type and its
implementation also contained the proper guest OS Features list for it.
However, when the worker got the osbuild job, it built it and tried to
fetch the guest OS Features for the distro. Since its implementation was
too old, it didn't contain the code that added the actual support for
c10s GCE images and got no guest OS features list (which is the default
for unsupported distros). The image was successfully uploaded and
shared, but it does not boot in GCP, because it does not know that it
should use UEFI to boot it.
This behavior could be considered a bug. The worker should be dumb. It
should not be making decisions about the image features, but instead it
should take them from the upload target options. And composer should be
the authoritative source of truth for this. Because otherwise, we
basically have two components that need to be updated in sync to add
support for GCE images on a new distro.
Move the GCE image guest OS features to the GCP upload target options.
The worker will just take what is specified there and use it when
importing the image to GCP. As a compatibility layer for the case when
the composer would be older than the worker (unlikely, but still),
worker will try to determine the image guest OS features in case the
list in the upload target options is empty.
Extend the GCP functional tests to check that the imported image has at
least some guest OS features set.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The workerClientErrorFrom() was returning an `*clienterrors.Error` and
an `error` (if something with the conversation goes wrong.
But the calling code was expecting that even if an `error` is returned
the `*clienterrors.Error` is still valid. The caller would then just
log the error. As returning a valid `value` even when there is an
`error` is an unexpected pattern this commit changes the code to
always return a `*clienterrors.Error` and log any issue via the
logger.
The usual convention to create new object is to prefix `New*` so
this commit renames the `WorkerClientError`. Initially I thought
it would be `NewWorkerClientError()` but looking at the package
prefix it seems unneeded, i.e. `clienterrors.New()` already
provides enough context it seems and it's the only error we
construct.
We could consider renaming it to `clienterror` (singular) too
but that could be a followup.
I would also like to make `clienterror.Error` implement the
`error` interface but that should be a followup to make this
(mechanical) rename trivial to review.
This is an alternative/complementary fix for PR#4137. It is very
simple so should be uncontroverisal.
It fixes an issue that @schuellerf discovered, i.e. that when an error
interface is passed into clienterrors.Error.Details the details get
lost because the json.Marshaler will not know how to handler an
error interface.
To find the problematic uses of `error` a custom vet checker was
build in https://github.com/mvo5/osbuild-cvet. With that the
result is:
```
$ go run github.com/mvo5/osbuild-cvet@latest ./...
/home/mvogt/devel/osbuild/osbuild-composer/cmd/osbuild-worker/jobimpl-depsolve.go:93:26: do not pass 'error' to WorkerClientError() details, use error.Error() instead
/home/mvogt/devel/osbuild/osbuild-composer/cmd/osbuild-worker/jobimpl-osbuild.go:404:31: do not pass 'error' to WorkerClientError() details, use error.Error() instead
/home/mvogt/devel/osbuild/osbuild-composer/cmd/osbuild-worker/jobimpl-osbuild.go:519:31: do not pass 'error' to WorkerClientError() details, use error.Error() instead
/home/mvogt/devel/osbuild/osbuild-composer/cmd/osbuild-worker/jobimpl-osbuild.go:556:31: do not pass '[]error' to WorkerClientError() details, use []string instead
```
and once this commit is in no more errors.
Just like PR#4137 this is not perfect because it will not do a
recursive check for the passed argument.
