Blueprint package set is now depsolved together with the OS package set
in a chain. The result is stored in the package specs sets under the OS
package set name.
In reality, the code was able to handle a `nil` package specs to be
passed to pipelines, however some parts were looking for the kernel
version in the blueprint package specs, which would be a bug.
Regenerated affected image test cases.
Blueprint package set is now depsolved together with the OS package set
in a chain. The result is stored in the package specs sets under the OS
package set name.
In reality, the code was able to handle a `nil` package specs to be
passed to pipelines, however some parts were looking for the kernel
version in the blueprint package specs, which would be a bug.
Regenerated affected image test cases.
Blueprint package set is now depsolved together with the OS package set
in a chain. The result is stored in the package specs sets under the OS
package set name.
In reality, the code was able to handle a `nil` package specs to be
passed to pipelines, however some parts were looking for the kernel
version in the blueprint package specs, which would be a bug.
Regenerated affected image test cases.
Blueprint package set is now depsolved together with the OS package set
in a chain. The result is stored in the package specs sets under the OS
package set name.
In reality, the code was able to handle a `nil` package specs to be
passed to pipelines, however some parts were looking for the kernel
version in the blueprint package specs, which would be a bug.
Regenerated affected image test cases.
Use `rpmmd.DepsolvePackageSets()` in Weldr API compose request handler,
instead of `rpmmd.Depsolve()`.
Extract common code from `API.allRepositories()` and
`API.allRepositoriesByImageType()` to a new method
`API.payloadRepositories()`.
Modify `API.allRepositoriesByImageType()` to return payload repositories
(repositories defined by user) as a separate slice to enable the use of
`rpmmd.DepsolvePackageSets()`, which requires the package-set-specific
repositories to be passed separately.
Keep using `rpmmd.Depsolve()` in Weldr where appropriate. The
implementation depsolves various simple package sets for multiple API
request handlers and it does not make sense to complicate the code by
moving to `rpmmd.DepsolvePackageSets()`.
Return dummy values from the following methods:
- PackageSets
- PayloadPackageSets
- PackageSetsChains
Use package set names commonly used by recent distro definitions.
Package sets are based on values used by rpmmd mock implementation.
Adjust two Weldr API unit test check for the dummy values. Without
this fix, these unit tests would start failing after the move to
`rpmmd.DepsolvePackageSets()` in Weldr API compose handler.
Add a convenience method `DepsolvePackageSets()` to the `RPMMD`
interface. The method is expected to depsolve all provided package sets
in a chain or separately, based on the provided arguments, and return
depsolved PackageSpecs sets.
The intention is to have a single implementation of how are package sets
depsolved and then use it from all places in composer (API and tools
implementations).
Adjust necessary mock implementations and add a unit test testing the
new interface method implementation.
Introduce a new method `PackageSetsChains()` to the `ImageType`
interface, which returns a named lists of package sets, which should be
depolved together in a chain.
Extend all distro implementations with the new method.
Add a unit test ensuring that if an image type defines some package set
name chains, that all of the listed package set names are present in the
package set map returned by the image type.
The method is currently not used anywhere. This is a preparation for
switching from current way of depsolving to the chain depsolving.
Add a new `rpmmdImpl` method `chainDepsolve`, which is able to
depsolve multiple chained package sets as separate DNF transactions
layered on top of each other.
This new method allows to depsolve the `blueprint` package set on top of
the base image package set (usually called `packages`).
Introduce a helper function `chainPackageSets` for constructing
arguments to the `chainDepsolve` method based on the provided arguments:
- slice of package set names to chain as transactions
- map of package sets
- slice of system repositories used by all package sets
- map of package-set-specific repositories
Extend `dnf-json` with a new command `chain-depsolve` allowing to
depsolve multiple transaction in a row, layered on top of each other.
Add unit tests where appropriate.
New function that ensures that a partition can hold the total sum of all
the required sizes of specific directories on the partition. The
function sums the required directory sizes grouped by their mountpoint
and then resizes the entity path of that Mountable.
The generated gcp name had an invalid `.tar.gz` extension. This
extension still needs to be supplied for the object name however.
The integration tests supply the image name rather than relying in the
generated one, which is why this slipped through.
Add support for importing the GCE image into GCP using Weldr API. The
credentials to be used can be specified in the upload settings and will
be then used by the worker to authenticate with GCP.
The GCP target credentials are passed to Weldr API as base64 encoded
content of the GCP credentials JSON file. The reason is that the JSON
file contains many values and its format could change in the future.
This way, the Weldr API does not rely on the credentials file content
format in any way.
Add a new test case for the GCP upload via Weldr and run it in CI.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Refactor the handling of GCP credentials in the worker to be equivalent
to what is done for AWS. The main idea is that the code decides which
credentials to use when processing each job. This change will allow
preferring credentials passed via upload `TargetOptions` with the job,
over the credentials configured in worker's configuration or the default
way of authenticating implemented by the Google library.
Move loading of GCP credentials to the internal `gcp` library into
`NewFromFile()` function accepting path to the file with credentials.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Delete all internal `cloud/gcp` API related to importing virtual images
to GCP using Cloud Build API. This API is no longer needed.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Introduce a new `ComputeImageInsert()` method for importing images into
GCP. It uses the `compute.Images.Insert()` API [1], which has many
advantages over the currently used way of importing images using the
CloudBuild API. The advantages are mainly that the image is imported as
is and no additional cache files or VMs are created as part of the
import process. Therefore there is no need to do additional cleanup of
cache files after importing the image.
In addition, the import itself is approximately 30% faster for RHEL
images when using the `Insert()` call.
Nevertheless the `Insert()` call accepts only gzip-ed tarball with a RAW
disk, unlike the `Import()` call, which accepts basically any virtual
disk format.
[1] https://cloud.google.com/compute/docs/reference/rest/v1/images/insert
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `gce-rhui` image type intended for Google Compute Engine. The image
uses Google's RHUI infrastructure to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Extend RHEL-84 `imageTypeS2` structure to contain pipelines generator
function. Previously, the `imageTypeS2` implementation defaulted to only
a single pipelines generator method for EDGE image types. The ability to
pass a different generator function implementation is important to
enable addition of new image types relying on osbuild Manifest v2.
Rename the original pipeline generator method to `edgePipelines`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `gce-rhui` image type intended for Google Compute Engine. The image
uses Google's RHUI infrastructure to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `gce-rhui` image type intended for Google Compute Engine. The image
uses Google's RHUI infrastructure to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Introduce `tarArchivePipeline()` function returning a pipeline,
which creates a Tar archive from another pipeline tree referenced by the
pipeline name.
Replace `tarStage()` with `osbuild.NewTarStage()`
Use the `tarArchivePipeline()` function in respective image type
pipelines.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Introduce `tarArchivePipeline()` function returning a pipeline, which
creates a Tar archive from another pipeline tree referenced by the
pipeline name.
Replace `tarStage()` with `osbuild.NewTarStage()`
Use the `tarArchivePipeline()` function in respective image type
pipelines.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Introduce `tarArchivePipeline()` function returning a pipeline, which
creates a Tar archive from another pipeline tree referenced by the
pipeline name.
Replace `tarStage()` with `osbuild.NewTarStage()`
Use the `tarArchivePipeline()` function in respective image type
pipelines.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The `tar` stage options contain three boolean values. All of them
default to `true` in the osbuild stage implementation [1]. However
if these values were explicitly set to `false`, they would be omitted
from the resulting JSON structure. As a result, it was impossible to use
any non-default values.
Use `*bool` instead of `bool`, to ensure that explicitly set `false`
values will end up in the JSON structure passed to osbuild.
[1] 8102f20d23/stages/org.osbuild.tar (L39-L53)
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add depsolve job error dependency test cases for
regular composes and koji composes. The error furthest
up the chain should be returned in the details field
of the job error.
