The VMDK image is already produced as stream-optimized. Therefore stop
setting the `StreamOptimized` option in `OSBuildJob` structure by both,
Weldr and Cloud APIs.
Keep the handling of the option in worker for backward compatibility,
in case an older instance of Composer server is used, which does not
produce VMDK manifests as stream-optimized. In such case, the worker
needs to convert the image.
Modify pipelines in all distro definitions to produce stream-optimized VMDK
image.
Regenerate all VMDK test cases.
Bump worker dependency on osbuild to the version supporting VMDK
subformat in both QEMU assembler and stage
We would benefit from having support for 9.1 downstream so let's add it in
the form of an alias. This is a bare minimum for having a proper 9.1 support.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
In the internal deployment, we want to talk with composer over a http/https
proxy. This proxy adds new composer.proxy field to the worker config that
causes the worker to connect to composer and the oauth server using
a specified proxy.
NB: The proxy is not supported when connection to composer via unix sockets.
For testing this, I added a small HTTP proxy implementation, pls don't
use this in production, it's just good enough for tests.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Just so we don't need to care about all the server-side setup in individual
test cases and we can just reuse the setup.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Firstly, let's use t.TempDir(), it's less code.
Secondly, let's remove all the code that touches distributions, we can just
use random values, both worker server and client actually do't inspect
any values so they can be completely random.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
This test actually verifies that the client code for OAuth works. As this was
the only code that tests client in the file, I think it deserves its own one.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Whenever we create a new mountpoint due to a user customization,
ensure the layout uses LVM, i.e. convert plain layouts to it, if
needed. This does not apply to rpm-ostree based systems.
Add "lvm2" to the build pipeline and thus generate new manifests
and image infos.
Adjust the existing tests that assumed we can not create more
than 4 partitions on mbr layouts, since that is now not true
anymore.
This is a port from rhel86, commit 63aa155
The change in osPipeline() is required now to fix the Prefix for the
bootloader specification when LVM is used. The unspecified Prefix, which
was previously used for all cases, defaults to "/boot". When the layout
is converted to LVM, a boot partition is created and the BLS Prefix
should be set to "".
In the case where we don't have a partition table, the BLS stage is not
needed, but it was done unconditionally before, so keep the default
image definitions unchanged.
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
/ and /usr have minimum sizes defined (1 GiB and 2 GiB respectively).
When /usr is not defined, the minimum size of /usr gets added to the
minimum size for /.
This new test runs through a few scenarios and checks whether the sizes
fit.
This tests that the clampFSSize() function ensures all user-defined
mountpoints are at least 1 GiB.
Added a blueprint with < 1 GiB minsizes to test this.
Testing all blueprints in TestCreatePartitionTable() now.
Currently, we only specify a minimum size for
- `/` (1 GiB), and
- `/usr` (2 GiB).
This ensures that
- a separate `/usr` partition is at least 2 GiB,
- `/` is always at least 1 GiB,
- if `/usr` it not a separate partition, `/` is at least 3 GiB.
We could (or should), in the future, make it possible for image types to
override this mapping as part of their default config, for example, if
an image type by default requires a larger `/usr`.
Makes the test values more readable (without needing comments).
Some values in the default partition table were fixed, e.g., cases where
we had `Size: 1024000, // 500 MB`.
When the partition table did not have a boot partition, we created it
but then _unconditionally_ returned, which meant that we did not create
the LVM skeleton and wrap the root partition. Properly handle this case
and also re-initialize the `rootPath` in this case since we change the
underlying `Partition[]` array in `PartitionTable` object. Add an extra
blueprint with only one customization which exposes this bug.
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
Blueprint package set is now depsolved together with the OS package set
in a chain. The result is stored in the package specs sets under the OS
package set name.
In reality, the code was able to handle a `nil` package specs to be
passed to pipelines, however some parts were looking for the kernel
version in the blueprint package specs, which would be a bug.
Regenerated affected image test cases.
Blueprint package set is now depsolved together with the OS package set
in a chain. The result is stored in the package specs sets under the OS
package set name.
In reality, the code was able to handle a `nil` package specs to be
passed to pipelines, however some parts were looking for the kernel
version in the blueprint package specs, which would be a bug.
Regenerated affected image test cases.
Blueprint package set is now depsolved together with the OS package set
in a chain. The result is stored in the package specs sets under the OS
package set name.
In reality, the code was able to handle a `nil` package specs to be
passed to pipelines, however some parts were looking for the kernel
version in the blueprint package specs, which would be a bug.
Regenerated affected image test cases.
Blueprint package set is now depsolved together with the OS package set
in a chain. The result is stored in the package specs sets under the OS
package set name.
In reality, the code was able to handle a `nil` package specs to be
passed to pipelines, however some parts were looking for the kernel
version in the blueprint package specs, which would be a bug.
Regenerated affected image test cases.
Use `rpmmd.DepsolvePackageSets()` in Weldr API compose request handler,
instead of `rpmmd.Depsolve()`.
Extract common code from `API.allRepositories()` and
`API.allRepositoriesByImageType()` to a new method
`API.payloadRepositories()`.
Modify `API.allRepositoriesByImageType()` to return payload repositories
(repositories defined by user) as a separate slice to enable the use of
`rpmmd.DepsolvePackageSets()`, which requires the package-set-specific
repositories to be passed separately.
Keep using `rpmmd.Depsolve()` in Weldr where appropriate. The
implementation depsolves various simple package sets for multiple API
request handlers and it does not make sense to complicate the code by
moving to `rpmmd.DepsolvePackageSets()`.
Return dummy values from the following methods:
- PackageSets
- PayloadPackageSets
- PackageSetsChains
Use package set names commonly used by recent distro definitions.
Package sets are based on values used by rpmmd mock implementation.
Adjust two Weldr API unit test check for the dummy values. Without
this fix, these unit tests would start failing after the move to
`rpmmd.DepsolvePackageSets()` in Weldr API compose handler.
Add a convenience method `DepsolvePackageSets()` to the `RPMMD`
interface. The method is expected to depsolve all provided package sets
in a chain or separately, based on the provided arguments, and return
depsolved PackageSpecs sets.
The intention is to have a single implementation of how are package sets
depsolved and then use it from all places in composer (API and tools
implementations).
Adjust necessary mock implementations and add a unit test testing the
new interface method implementation.
Introduce a new method `PackageSetsChains()` to the `ImageType`
interface, which returns a named lists of package sets, which should be
depolved together in a chain.
Extend all distro implementations with the new method.
Add a unit test ensuring that if an image type defines some package set
name chains, that all of the listed package set names are present in the
package set map returned by the image type.
The method is currently not used anywhere. This is a preparation for
switching from current way of depsolving to the chain depsolving.
Add a new `rpmmdImpl` method `chainDepsolve`, which is able to
depsolve multiple chained package sets as separate DNF transactions
layered on top of each other.
This new method allows to depsolve the `blueprint` package set on top of
the base image package set (usually called `packages`).
Introduce a helper function `chainPackageSets` for constructing
arguments to the `chainDepsolve` method based on the provided arguments:
- slice of package set names to chain as transactions
- map of package sets
- slice of system repositories used by all package sets
- map of package-set-specific repositories
Extend `dnf-json` with a new command `chain-depsolve` allowing to
depsolve multiple transaction in a row, layered on top of each other.
Add unit tests where appropriate.
New function that ensures that a partition can hold the total sum of all
the required sizes of specific directories on the partition. The
function sums the required directory sizes grouped by their mountpoint
and then resizes the entity path of that Mountable.
The generated gcp name had an invalid `.tar.gz` extension. This
extension still needs to be supplied for the object name however.
The integration tests supply the image name rather than relying in the
generated one, which is why this slipped through.
