Modify pipelines in all distro definitions to produce stream-optimized VMDK
image.
Regenerate all VMDK test cases.
Bump worker dependency on osbuild to the version supporting VMDK
subformat in both QEMU assembler and stage
Whenever we create a new mountpoint due to a user customization,
ensure the layout uses LVM, i.e. convert plain layouts to it, if
needed. This does not apply to rpm-ostree based systems.
Add "lvm2" to the build pipeline and thus generate new manifests
and image infos.
Adjust the existing tests that assumed we can not create more
than 4 partitions on mbr layouts, since that is now not true
anymore.
This is a port from rhel86, commit 63aa155
The change in osPipeline() is required now to fix the Prefix for the
bootloader specification when LVM is used. The unspecified Prefix, which
was previously used for all cases, defaults to "/boot". When the layout
is converted to LVM, a boot partition is created and the BLS Prefix
should be set to "".
In the case where we don't have a partition table, the BLS stage is not
needed, but it was done unconditionally before, so keep the default
image definitions unchanged.
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
Blueprint package set is now depsolved together with the OS package set
in a chain. The result is stored in the package specs sets under the OS
package set name.
In reality, the code was able to handle a `nil` package specs to be
passed to pipelines, however some parts were looking for the kernel
version in the blueprint package specs, which would be a bug.
Regenerated affected image test cases.
Introduce a new method `PackageSetsChains()` to the `ImageType`
interface, which returns a named lists of package sets, which should be
depolved together in a chain.
Extend all distro implementations with the new method.
Add a unit test ensuring that if an image type defines some package set
name chains, that all of the listed package set names are present in the
package set map returned by the image type.
The method is currently not used anywhere. This is a preparation for
switching from current way of depsolving to the chain depsolving.
Add the `gce-rhui` image type intended for Google Compute Engine. The image
uses Google's RHUI infrastructure to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Introduce `tarArchivePipeline()` function returning a pipeline,
which creates a Tar archive from another pipeline tree referenced by the
pipeline name.
Replace `tarStage()` with `osbuild.NewTarStage()`
Use the `tarArchivePipeline()` function in respective image type
pipelines.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Specify a size for the root filesystem in the partition table,
which basically equates to a minimum size. In reality all image
types specify a larger image size and thus we enlarge the root
file system to more than the specified size for plain layouts.
Does not change any existing manifests.
This also prepares the enablement of auto-LVM conversion, since
in that case we need to have a size for the root file system
specified.
De-duplicate the `qemuStageOptions()` implementation from many distro
implementations and move it to the `osbuild2` package as
`NewQEMUStageOptions()`.
Adjust all affected pipeline implementations.
The `NewQEMUStageOptions()` function takes an optional format-specific
options argument, instead of a dedicated format-specific options (the
previously used qcow2 compat version). This is a preparation to produce
VMDK images with a different than default subformat.
Add unit test for `NewQEMUStageOptions()`.
Define a new aliased type `QEMUFormat` for the format type used by the
osbuild2 QEMU stage and define constants for all allowed values.
Use QEMU format type constants in all relevant places, instead of string
literals. Not using string literals minimizes the room for making a
typo.
The image type is only ever known (externally) as image-installer.
Renaming the internal variables and functions to reflect the name makes
the code easier to navigate.
The users anaconda module enables users to create user accounts at
install time if one is not already created in the payload. This is
required for the cloud API (Image Builder service) for the image
installer where user customizations are not supported. Without it, user
creation isn't possible on the installed system.
The module also enables user creation at install time through the
kickstart file for both the image-installer and the edge-installer
(Anaconda only).
Therefore, for the image-installer, the users and groups are no longer
created as part of the payload.
This commit adapts the changes from the following commits (originally
made in the RHEL 8.6 and RHEL 9.0 distros) to the rest of the RHEL
distro definitions:
ebc3330cbd5825294dad
Use single NewAnacondaStageOptions() from osbuild2 instead of
implementing in each distro.
Followup from 974b0a434b, copied to the
rest of the RHEL distro definitions.
Use single NewKickstartStageOptions() and replace image-type-specific
implementation from each distro.
- Followup from cb186df208, copied to the
rest of the RHEL distro definitions.
NB: The change was not made in the Fedora distro definitions as they are
currently being rewritten.
Use single NewGroupsStageOptions() from osbuild1 and osbuild2 instead of
implementing in each distro.
- Followup from 2eef6e6e2d, copied to the
rest of the RHEL distro definitions.
- Added NewGroupsStageOptions() to osbuild1 for rhel8 and rhel84.
NB: The change was not made in the Fedora distro definitions as they are
currently being rewritten.
Use single NewUsersStageOptions() from osbuild1 and osbuild2 instead of
implementing in each distro.
- Followup from ca8b371142, copied to the
rest of the RHEL distro definitions.
- Added NewUsersStageOptions() to osbuild1 for rhel8 and rhel84.
NB: The change was not made in the Fedora distro definitions as they are
currently being rewritten.
Followup from, f34380d5b5 and
3a1765a5a8, copied to the rest of the RHEL
distro definitions.
For now, these customizations have no effect on the manifest.
The new `with-users` variants of the edge-installer test cases include
the user customizations in the blueprint, but the manifests are
(currently) the same as the corresponding base cases.
Deduplicate the `rpmStageOptions()` function implemented in every distro
`stage_options.go` to `osbuild2` package.
RHEL-8.4 and code using osbuild v1 manifests are not touched.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Install `cloud-init` by default on the VMDK image on RHEL-85/86/90 and
as a result also CentOS Stream 8/9.
Regenerate image test cases.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
test cases fixup
A future change to `NewGrub2StageOptions` will result in a change of
`Config.Default` to `saved` if a kernel is set. Preserve the current
behavior by manually resetting it to the empty string.
Add a new parameter `lvmify` to `NewPartitionTable` that, if set to
`true`, will cause the root partition to be wrapped in LVM in case
it is not in a LVM volume group. Set this to `false` for now so no
actual change should happen anywhere. Layouts where the root is
directly on a LUKS container are not yet supported.
Add tests for this.
This commit fixes#2347 by ensuring that a minimum
size of 1GB is set for all file systems. The only
exception to this is the `/usr` which is set to 2GB,
since this was the only mountpoint that was previously
being checked.
The new `GenImageKernelOptions` will retrun all needed kernel options
that are derived from the partition table. The only case - for now -
is if the partition table contains a LUKS decice, where the uuid of it
needs to be passed to the on the kernel command line so that it will
get unlocked.
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
Use the new `GenImage{Prepare,Finish}Stages` in all liveImagePipelines
methods. This should allow us to handle partition tables with LUKS2 and
LVM2 transparently. Since none of the existing layouts contain any of
those, nothing should currently change.
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
The only thing we needed from the passed in device was the filename,
so just pass in that directly.
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
The `PartitionTable.FindMountable` method is the more generic version of
`PartitionTable.RootFilesystem` which returns a `Mountable` interface,
instead of a specific `Filesystem` object. This commit thus prepares for
the more generic case, like when root is a Btrfs sub-volume.
Co-Authored-By: Christian Kellner <christian@kellner.me>
Replace the old CreateParittionTable() function with the new one called
NewPartitionTable() which works with the new interface types and
supports container-type setups (LUKS, LVM ,and Btrfs).
Changed usage in distro packages to take and carry around a pointer to
the new PartitionTable rather than a concrete type. The
NewPartitionTable() function returns a deep clone of the base
PartitionTable so the new pointer type can be moved and (if necessary)
modified freely without affecting the distro base PT.
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
Use single NewGrub2StageOptions() and NewGrub2InstStageOptions()
functions from osbuild2 instead of implementing them in distros.
A small option flip is required outside the function for RHEL 8.5 and
RHEL 8.6 to maintain the old behaviour.
Co-Authored-By: Christian Kellner <christian@kellner.me>
Use a single GenCopyFSTreeOptions() function from osbuild2 instead of
implementing it multiple times in distros.
Co-Authored-By: Christian Kellner <christian@kellner.me>
Use a single GenMkfsStages() function from osbuild2 instead of
implementing it multiple times in distros.
Co-Authored-By: Christian Kellner <christian@kellner.me>
Use bytes internally everywhere and convert to sectors only when writing
the options for the stages.
Changed the AlignUp() method to not do the alignment if the input is
already aligned. This changes the behaviour when the size is 0, but
that's not a realistic use case. Updated unit tests to match.
Manifests are unaffected.
Co-Authored-By: Christian Kellner <christian@kellner.me>
Return an error if the maximum numbers of partitions has been
reached and thus creating further partitions would result in
errors.
Currently we limit MBR partition types to 4 as we dont support
logical partitions and GPT layouts to 128. According to the
UEFI specificatio (2.8) a minimum of 16384 bytes are reserved
for the partition entries array. Each entry is 128 bytes wide
thus resulting in at least 128 entries; we choose this to be
the maximum as well for now.
This is needed so that we can do different things depending on the
given layout; this will be used in tests for now only. Only GPT
allows for arbitrary number of partitions and once we assert this
in code we will need to adjust the tests accordingly.
NB: This method might be removed again in the future, once generic
LVM support is added everywhere and the ability to differentiate
between MBR and GPT layouts is not needed anymore.
Do not rely on `distro.imageOptions` having any size information,
i.e. `Size` being `0`. Instead use `imageType.Size()` and the
information in the blueprint customization to calculate the size.
This makes the individual distro definitions idenpendent of the
API entry points that currently calculate the size, e.g.:
internal/cloudapi/v1/v1.go:PostCompose line 184
internal/cloudapi/v2/v2.go:PostCompose line 197
internal/kojiapi/server.go:PostCompose line 135
internal/weldr/api.go:composeHandler line 2289
Instead of hard coding a padding of 100 sectors for all layouts, i.e.
MBR and GTP, adjust the needed space depending on the layout: for MBR
we don't need to reserve any space at all since it does not have a
secondary header. For GTP we reserve 33 sectors, as indicated in the
UEFI specific, which allows for the header itself and up to 128 entries.
To not modify the layout of already released distributions, like RHEL
8.4 and 8.5, a new member called `ExtraPadding` is added to `Partition
Table` and then used in the corresponding layouts to preserve the
existing padding of 100.
Modify the signature of `CreatePartitionTable` so that it is
possible to return errors from the function. This is not yet
used, but will be in the near future. Change all call sites
accordingly: in most cases we can just bubble up the error.
Pass the `basePartitionTable` argument of `CreatePartitionTable`.
Now that we clone the partition table at the beginning of the
method there is no need to pass a copy of the partition table.
In various places we are interested in the root and boot file-
systems. Currently those were accessed via by retrieving the
partition that contain them and the accessing the filesystem
member. Add accessors to `PartitionTable` that directly return
the needed filesystem. This will help if the file system is
stored inside a container like LVM or LUKS instead of directly
on a partition.
Move the `ostreePullStageInputs()` function duplicated in all
distro definitions to the `osbuild2` package as
`NewOstreePullStageInputs()`.
Delete `stage_inputs.go` from all distro definitions.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Move the `bootISOMonoStageInputs()` function duplicated in all
distro definitions to the `osbuild2` package as
`NewBootISOMonoStagePipelineTreeInputs()`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
