Commit graph

1740 commits

Author SHA1 Message Date
Ondřej Budai
5e291bd98f cloudapi/test: remove dependency on rpmmd_mock
rpmmd_mock fixture are complex and unneeded in the context of cloudapi, let's
just copy 3 lines from them and drop the dependency.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-02 16:20:11 +01:00
Ondřej Budai
c29a1fa9da cloudapi: drop unused rpmmdMetadata parameter & struct member
cloudapi no longer calls dnf-json directly so we can clean some code, yay!

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-02 16:20:11 +01:00
Antonio Murdaca
c1a896dd1d rhel{86,90}: define simplified installer build packages set
The edge installer and edge simplified installer build roots have
diverged, i.e. the latter need clevis/luks, so define a new pkg
set for the simplified installer extending the edge installer one.

Co-Authored-By: Christian Kellner <christian@kellner.me>
2022-02-28 23:05:45 +00:00
Tom Gundersen
973b5141b3 ditro/rhel86: set volid like in rhel9.0
This is only required in RHEL9.0, but best practice is to always pin these things
down. Also increases uniformity between distros.

Simplify a bit the volid generator by making it require `rand.Rand` rather than
`io.Reader`, and hence eliminating the need for error handling.
2022-02-28 23:05:45 +00:00
Antonio Murdaca
3be67ad965 Simplified installer: add support for encrypted raw images
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-02-28 23:05:45 +00:00
Achilleas Koutsou
437dd397cd disk: test for duplicate Entities when cloning PartitionTables
Collect all entities and compare pointers.
2022-02-28 23:05:45 +00:00
Achilleas Koutsou
10095e382a disk: fix Payload cloning bugs
In LUKSContainer and LVMLogicalVolume we neglected to clone the Payload
which means we would modify the base PartitionTable when manipulating
the clone.
2022-02-28 23:05:45 +00:00
Christian Kellner
e921b7aadd rhel86: specify a minimum root file system size
Specify a size for the root filesystem in the partition table,
which basically equates to a minimum size. In reality all image
types specify a larger image size and thus we enlarge the root
file system to more than the specified size for plain layouts.
But if we auto-convert an partition layout to LVM we need a size
for the root partition.
Does not change any existing manifests.
This does not apply for ostree based systems like the simplified
installer.
2022-02-28 21:36:25 +01:00
Christian Kellner
ddc0126a36 test: check we always have a boot on lvmification
Check that the `ensureLVM` code creates a boot partition, if it does
not yet exist.
2022-02-28 21:36:25 +01:00
Christian Kellner
63aa1556fc rhel86: automatically convert to LVM on fs customizations
Whenever we create a new mountpoint due to a user customization,
ensure the layout uses LVM, i.e. convert plain layouts to it, if
needed. It uses the existing lvm-ification code but enhances it
so that we also create a `/boot` partition in case it does not
yet exist.
Adjust the existing tests that assumed we can not create more
than 4 partitions on mbr layouts, since that is now not true
anymore.
2022-02-28 21:36:25 +01:00
Christian Kellner
9e5b265a58 osbuild2: lock loopback devices during sfdisk, mkfs
Since udev will probe block devices it is advisable to hold a lock
on the device when modifying its partition table or the superblock
of the filesystem (see [1]). osbuild loopback devices do support
this via the `lock` option. Set this option for all operation that
involve changing block device "metadata" that could potentionally
race with udev, such as sfdisk, mkfs, creating a luks2 container
and creating LVM2 volume groups and logical volumes.
NB: osbuild also has its own device inhibition logic to prevent
udev/lvm2 from auto activating devices and in general to limit the
interaction between the host and devices used by osbuild. See [2]
for more information.
NB: this also locks the loopback device in situation where we the
it is strickly not the right thing to do, e.g. when creating a fs
on a logical voume that is located on a loopback device, since in
this case the device we would need to lock is the logical volume.
Sadly, LVM/DM devices are exempt from block device locking. But,
due to a bug in osbuild < 50, the udev inhibitor does *not* work
for loopback devices and therefore we have to use the actual lock
to preven LVM device auto-activation via `69-dm-lvm-metad.rules`.
The change was implemented by adding a new boolean to `getDevices`
indicating if the loopback device should be locked or not. Once
we depend on osbuild 50 we can change the logic in `getDevices`
to only lock the loopback device if the number of devices is one,
i.e. we are working directly on the loopback device.

[1] https://systemd.io/BLOCK_DEVICE_LOCKING/
[2] /usr/lib/udev/rules.d/10-osbuild-inhibitor.rules
2022-02-28 17:09:30 +01:00
Christian Kellner
19e7890f63 rhel90: automatically convert to LVM on fs customizations
Whenever we create a new mountpoint due to a user customization,
ensure the layout uses LVM, i.e. convert plain layouts to it, if
needed. This does not apply to rpm-ostree based systems, e.g. the
simplified installer since they will be using LUKS in 9.0.
Add "lvm2" to the build pipeline and thus generate new manifests
and image infos.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-28 17:09:30 +01:00
Christian Kellner
375c3c5cff rhel90: specify a minimum root size in the partition table
Specify a size for the root filesystem in the partition table,
which basically equates to a minimum size. In reality all image
types specify a lager image size and thus we enlarge the root
file system to more than the specified size for plain layouts.
But if we auto-convert an partiton layout to LVM we need a size
for the root partition.
Does not change any existing manifests.
This does not apply for ostree based systems like the simplified
installer.
2022-02-28 17:09:30 +01:00
Christian Kellner
ed4e0a94a4 disk: honour the fact that some containers have metadata
Re-introduce the VolumeContainer interface but with a different
meaning: it is supposed to be implemented by all container that
contain volumes and as a result have themselves a size, like eg
LVM2, LUKS2 and PartitionTable (the latter is not yet included).
The sole method on the interface for now is MetadataSize, which
should return the metadata for the container itself.
Use that new `VolumeContainer.MetadataSize` method when we up-
date the sizes of elements in `resizeEntitybranch`.
2022-02-28 17:09:30 +01:00
Christian Kellner
17fa96b84a disk: rename VolumeContainer → MountpointCreator
Rename the interface after the one method it has and the attribute
it describes for the implementing entities: being able to create
mountpoints.
2022-02-28 17:09:30 +01:00
Christian Kellner
7bfd0bb49f disk: rename Create{Volume → Mountpoint}
The function is indeed creating a mounpoint not a Volume; the
latter is not even well defined in our "ontology".
2022-02-28 17:09:30 +01:00
Christian Kellner
45f898c05c disk: remove unused CreateFilesystem method
The `CreateFilesystem` method on `PartitionTable` is not used
anymore since it got replaced by `CreateVolume`.
2022-02-28 17:09:30 +01:00
Christian Kellner
e57cccc3fe disk: NewPartitionTable can wrap plain partitions in LVM
Add a new parameter `lvmify` to `NewPartitionTable` that, if set to
`true`, will cause the root partition to be wrapped in LVM in case
it is not in a LVM volume group. Set this to `false` for now so no
actual change should happen anywhere. Layouts where the root is
directly on a LUKS container are not yet supported.
Add tests for this.
2022-02-28 17:09:30 +01:00
Christian Kellner
f91d8c6292 distro/rhel86: add support for azure marketplace
Add support for building images for the Azure marketplace: add a
new image type "azure-rhui" that can be used to build images
tailored to the Azure marketplace.
Add two sample manifests for 8.5 and 8.6, but note that even the
8.5 is using the 8.6 distro definitions. Also no image-info is
included since `image-info` cannot (yet) handle LVM setups and
the azure marketplace images use the LVM setup.
2022-02-27 22:03:36 +01:00
Christian Kellner
a26103312a osbuild2: properly calculate grub2 core location
The previous code assumed that the first partition is the location to
write the grub2 core image, implying it needs to always be a PReP or
BIOS-Boot partition. This is not an assumption we can easily make;
instead we now try to detect the correct partition based on its type.
2022-02-27 22:03:36 +01:00
Christian Kellner
ca61baf03b disk: add partition type helpers
Add helper methods on `Partition` so that we can easily "detect" if
they are of the type BIOS-BOOT or PReP. Add the PReP GUID for GPT
as well.
2022-02-27 22:03:36 +01:00
Christian Kellner
1060885386 disk: extract GPT header calculation code
Extract the code that calculates the GPT header size into its own
method so that we can use it in other places as well.
2022-02-27 22:03:36 +01:00
Christian Kellner
69ac301af8 distro: add Grub2Config to ImageConfig
Add it to all distros (8.6, 9.0) that support using ImageConfig
2022-02-27 22:03:36 +01:00
Christian Kellner
a8b0d0e255 osbuild2: add GRUB2Config to GRUB2StageOptions
A new `config` section was introduced in osbuild with several new
fields to configure grub2.
2022-02-27 22:03:36 +01:00
Christian Kellner
86f3ae55e4 osbuild2: add X11Keymap option to KeymapStageOptions
This was introduced in osbuild but not yet exposed.
2022-02-27 22:03:36 +01:00
Christian Kellner
f7fb727847 distro: add WAAgentConf to ImageConfig
Add it to all distros (8.6, 9.0) that support using ImageConfig
2022-02-27 22:03:36 +01:00
Christian Kellner
fcbaf80d67 osbuild2: add org.osbuild.waagent.config stage
Add the necessary structs for the `org.osbuild.waagent.config` stage.
2022-02-27 22:03:36 +01:00
Christian Kellner
8f746698f0 distro: add PwQuality to ImageConfig
Add it to all distros (8.6, 9.0) that support using ImageConfig
2022-02-27 22:03:36 +01:00
Christian Kellner
adfdb2cd39 distro: add Authconfig to ImageConfig
Add it to all distros (8.6, 9.0) that support using ImageConfig.
2022-02-27 22:03:36 +01:00
Christian Kellner
9bdb67de8d osbuild2: need specify unit for lvm logical volumes
When creating the logical volumes for the `LVM2CreateStage` we
need to explicitly specify the unit byte (`B`) since lvcreate
defaults to megabytes.
2022-02-27 22:03:36 +01:00
Christian Kellner
a9ef16a95e disk/lvm: generate logical volume name
When creating a new logical volume via the `CreateVolume` method,
the logical volume name was left blank. Generate an name based
on the mountpoint.
We will detect collisions for names and will try to correct them
by attaching a suffix. We do give up after 100 attempts though.
Add a simple test for it.
2022-02-27 22:03:36 +01:00
Achilleas Koutsou
41e4b17f89 distro/rhel86: add lvm2 to base build package set
We want to support LVM on all image types (optionally) so let's make
lvm2 available in all build roots.

Manifests and image info updated for RHEL 8.6 and CentOS Stream 8.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-27 13:04:55 +01:00
Achilleas Koutsou
685b53919f distro/rhel86: rearrange main build package set
One package per line.
Better for diffs.
2022-02-27 13:04:55 +01:00
Achilleas Koutsou
a7fbe916b7 distro/rhel86+90: set selinux in permissive mode in installers
SELinux in the installer environment is in permissive mode in regular
installation ISOs
2022-02-26 15:55:56 +00:00
Achilleas Koutsou
0a535ab09e distro: load ostree parameters from test manifest if specified
If the compose request in the test manifest contains ostree parameters,
add them to the test.
2022-02-26 15:55:56 +00:00
Tomas Hozza
ccd0be8c34 RHEL-90: use C.UTF-8 for images that only have glibc-minimal-langpack
Change the default locale to `C.UTF-8` for RHEL-9.0.
For all the images which install `langpack-en`, keep using the
`en_US.UTF-8` locale. `C.UTF-8` is used as the default for:
 - edge-commit
 - edge-container
 - image-installer
 - qcow2
 - tar

Also change the default locale for the edge-simplified-installer
installer-tree pipeline, since its package set contains only
`glibc-minimal-langpack`.

Regenerate RHEL-90 and c9s image tests.

Fix #2206

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-26 11:29:35 +01:00
Ondřej Budai
d8d161480e rhel86/90: change isolevel of image-installer to 3
So it can contain payload bigger than 4GiB.

Related: rhbz#2056451
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-24 17:56:22 +00:00
Antonio Murdaca
522ef73f4c internal/{8,9}: add clevis deps to simplified installer
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-02-24 17:07:34 +01:00
Antonio Murdaca
74540bcee7 distro/rhel90: add fido dracut module and enable customizing onboarding
FDO secure device onboarding support for RHEL90.

Co-Authored-By: Christian Kellner <christian@kellner.me>
2022-02-24 17:07:34 +01:00
Antonio Murdaca
e2bd3506e2 distro/rhel86: add fido dracut module and enable customizing onboarding
This patch also introduces customizations to fully enable onboarding
options for FDO.

Co-Authored-By: Christian Kellner <christian@kellner.me>
2022-02-24 17:07:34 +01:00
Christian Kellner
892cfb67cc osbuild2: add new FDOStage for org.osbuild.fdo
The stage takes no options but a single file input that contains the
root certs.
Add the stage with a helper constructor that will generate the needed
reference based on the actual certificate data.
Add corresponding tests.
2022-02-24 17:07:34 +01:00
Christian Kellner
6d383ec89a osbuild2: add org.osbuild.inline source
Add support for the `org.osbuild.inline` source via a new `InlineSource`
and the corresponding `InlineSourceItem`. A utility function `AddItem`
can be used to conveniently add items based on data (as strings).
2022-02-24 17:07:34 +01:00
Achilleas Koutsou
f71b4f69a8 distro/rhel86: remove greenboot condition for CS8
greenboot-default-health-checks is available in CS8 as well now.
2022-02-24 17:07:34 +01:00
Achilleas Koutsou
be0a45fe83 distro/rhel86: sort -u anaconda package set 2022-02-24 16:30:04 +01:00
Achilleas Koutsou
c4d6fcbe5b distro/rhel90: add lldpad and fcoe-utils to anaconda packages
Required by dracut modules.
2022-02-24 16:30:04 +01:00
Achilleas Koutsou
cae6a8d2b1 distro/rhel86: add lldpad to anaconda packages
Required by dracut modules.
2022-02-24 16:30:04 +01:00
Achilleas Koutsou
1762336891 distro/rhel86+90: add imageInstaller to aarch64 2022-02-24 16:30:04 +01:00
Achilleas Koutsou
07705aea7a distro/rhel90: split out arch-specific packages for anaconda 2022-02-24 16:30:04 +01:00
Achilleas Koutsou
82bec696b2 distro/rhel86+90: pipeline function rename 2022-02-24 16:30:04 +01:00
Achilleas Koutsou
9a8271a524 distro/rhel86+90: variable rename
Reflect the external name of the image type for clarity.
2022-02-24 16:30:04 +01:00