Since the workers will use structured error messages
going forward, it is necessary to maintain backwards
compatability for there errors in composer. Tests have
been added to the various apis to ensure that each api
checks for both kinds of errors, old and new.
Pipeline names are added to each job before adding to the queue. When a
job is finished, the names are copied to the Result object as well. This
is done for both OSBuild and Koji jobs.
The pipeline names in the result are primarily used to separate package
lists into build and payload/image packages in two cases:
1. Koji builds: for reporting the build root and image package lists to
Koji (in Koji finalize).
2. Cloud API (v1 and v2): for reporting the payload packages in the
metadata request.
The pipeline names are also used to print the system log output in the
order in which pipelines are executed. This still isn't used when
printing the OSBuild Result (osbuild2.Result.Write()) and we still rely
on sorting by pipeline name
(see https://github.com/osbuild/osbuild-composer/pull/1330).
Reading stage metadata using osbuild's v2 result format.
For RPM stages we only want the core (OS) RPMs (not the build root
RPMs). Skip the build pipeline by name, but this should be handled
better since names are arbitrary.
Using type switch to convert metadata types instead of relying on the
type string of the stage result.
The rpmmd helper function isn't used anymore since that requires two
conversion passes (osbuild.StageMetadata -> rpmmd.RPM ->
cloudapi.PackageMetadata).
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
The variables are set to the git revision from which the build is
triggered and rpm version from the spec file, if it is build using RPM.
This can be later used to query exact source version while
running osbuild-composer.
It is necessary to use both, because none of them is available in all
possible scenarios.
Use either git-rev (preferably) or RPM version (NEVRA) instead of the
"devel" build type. It was just a placeholder.
weldr-client returns different json structures than the original
composer-cli does. It is being replaced in RHEL-9 and these changes make
it possible to run the tests with both old composer-cli and
weldr-client.
The problem: osbuild-composer used to have a rather uncomplete logic for
selecting client certificates and keys while fetching data from
repositories that use the "subscription model". In this scenario, every
repo requires the user to use a client-side TLS certificate. The problem
is that every repo can use its own CA and require a different pair of
a certificate and a key. This case wasn't handled at all in composer.
Furthermore, osbuild-composer can use remote workers which complicates
things even more.
Assumptions: The problem outlined above is hard to solve in the general
case, but Red Hat Subscription Manager places certain limitations on how
subscriptions might be used. For example, a subscription must be tight to
a host system, so there is no way to use such a repository in osbuild-composer
without it being available on the host system as well.
Also, if a user wishes to use a certain repository in osbuild-composer it
must be available on both hosts: the composer and the worker. It will come
with different pair of a client certificate and a key but otherwise, its
configuration remains the same.
The solution: Expect all the subscriptions to be registered in the
/etc/yum.repos.d/redhat.repo file. Read the mapping of URLs to certificates
and keys from there and use it. Don't change the manifest format and let
osbuild guess the appropriate subscription to use.
Allow globing patterns in distro-specific image type deny list of Weldr
API configuration. Extend unit tests to verify simple globing patterns.
Update NEWS entry.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Refactor the `composeHandler()` method to send the actual error
returned by `getImageType()` as an API response.
Modify tests to handle the changed error message in API calls.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Rename the `checkImageTypeDenylist()` method to `isImageTypeAllowed()`
and return boolean value instead of error.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Change the Image Type denylist in Weldr API from being applied to all
distributions to being distribution-specific. A special name `*`
can be used in the configuration to match any distribution
or any image type.
Modify NEWS entry and unit tests to reflect this change.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Extend Weldr API to accept a list of denied image types, which should
not be exposed via API for any supported distribution. This
functionality will be needed to not expose image types which can't be
successfully built outside of Red Hat VPN. Example of such images are
the official RHEL EC2 images, which include RHUI client packages not
available publicly.
Image Types are filters when listing available compose types and
creating a new compose using Weldr API.
Extend osbuild-composer configuration to allow specifying the list of
denied Image Types for Weldr API.
Add unit tests for implemented changes.
Add NEWS entry describing the newly introduced functionality.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The system sources allow specification of the rhsm parameter, but it
isn't available in the sources configured over the Weldr API. This patch
implements support for it.
Some distributions do not have repositories and therefore cannot be
built. This filters the list of supported distributions by checking for
repos when starting up. All other requests use the api.distros list or
api.getDistro() function.
The name of the distro you get from distros.FromHost() may not match any of
the names in the registry's list. Use the actual name of the distro
instead of the mangled name.
Also removes api.distro which is unused.
This uses the image type based on the distribution selected by the
blueprint, or the host distro if none is present. This enables compose
to build images for the selected distribution.
It adds a helper, getImageType(), to return the ImageType based on the
distro name and compose type.
The host distro needs to be passed to New in the first position, AND
second so that it ends up in the distro map. Without this
distros.GetDistro() will fail because it cannot lookup the host distro
name.
An optional distribution name can be included with the blueprint. If is
is not then the blueprint will be depsolved/built using the current host
distribution.
depsolveBlueprint and depsolveBlueprintForImageType check for the empty
Distro name and set it to the host distro before using it. The function
signatures have also been changed to use the value instead of a pointer
so that changes don't effect anything outside the depsolve function.
Move OSTree option handling outside of the weldr API to make it usable
by other packages. New subpackage at internal/ostree.
Add support for ostree options ("Ref" and "URL") in the Cloud API.
Validate OSTree options and resolve the parent reference the same way as
in the Weldr API.
Unlike the Weldr API, the Cloud API doesn't support specifying the
Parent reference directly.
The exports list is included in the job information on the queue.
Modify composer to use RepoRegistry, instead of loading the host
repositories, when initializing WeldrAPI.
Modify WeldrAPI to use RepoRegistry, instead of a map of repository
definitions. Make sure that the RepoRegistry method specific to image
type is used in Welder where appropriate. Specifically when depsolving a
Blueprint, which is used to build a specific image type. Update Weldr
API unit tests to reflect the change.
Add a new method to RepoRegistry, allowing to get list of repositories,
which should be used for building an image for a given architecture,
without specifying the exact image type. Add relevant unit tests.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
fedoratest was yet another dummy distribution used by unit tests. After
the rework of test_distro, there is no reason to not use it as the only
distro implementation for testing purposes.
Remove fedoratest distro and replace it with test_distro in all affected
tests.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
