Commit graph

237 commits

Author SHA1 Message Date
Achilleas Koutsou
5825294dad distro/rhel86+90: enable the anaconda user modeule for image-installer
Enable the user module unconditionally for the image-installer:
- If users are specified for the kickstart file, the module is required
  to set up the users.
- If no users are specified, the module can be used at install time to
  create users.

Updated relevant test cases (manifests).
2022-03-28 14:08:57 +02:00
Achilleas Koutsou
fe514b9c2e test: create new test cases (manifests)
New test cases for edge- and image-installer with users.
2022-03-28 14:08:57 +02:00
Achilleas Koutsou
2eef6e6e2d osbuild2: deduplicate groupStageOptions()
Use single NewGroupsStageOptions() from osbuild2 instead of implementing
in each distro.

The new function does not set the Group.Name field anymore.  The field
does not exist in the osbuild schema and was silently ignored.
The field in the stage has been marked 'omitempty' and the relevant
manifests have been updated.
2022-03-28 14:08:57 +02:00
Jakub Rusz
f1006c5adb ci/tests: re-enable Installers.sh test
Related bug has been fixed. Updating snapshots and enabling the test.
2022-03-25 16:00:39 +01:00
Antonio Murdaca
bf46113251 test/data/repositories: bump cs8
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-03-24 18:23:39 +01:00
Jakub Rusz
7bf019efcb test/repositories: update rhel-90 repository gpg keys
SHA-1 keys are deprecated, updating to the newly used ones.
2022-03-24 18:23:39 +01:00
Tomas Hozza
2ec55b4c53 RHEL-90: use XBOOTLDR partition GUID for /boot
Use the Extended Bootloader Partition GUID for `/boot`, instead of the
Linux filesystem data GUID. This is useful for autodetection of a
partition purpose based on its GUID without reading the `/etc/fstab`
first.

Ensure that when creating mountpoints, e.g. when converting the
partitions layout to LVM, the `/boot` partition get the proper GUID
assigned.

Regenerate RHEL-90 and centos-9 image test cases.

Related to https://bugzilla.redhat.com/show_bug.cgi?id=2057231
2022-03-24 15:40:10 +01:00
Tomas Hozza
60db6ad06f image tests: update GPG keys used for RHEL-9.0 repos
SHA-1 key is no longer supported, update it with a new one from
redhat-release.

Related rhbz#2058497

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-03-23 13:22:13 +01:00
Jakub Rusz
15c2044b3c tests/upgrade: update gpg key
We need to use a new gpg key after the SHA-1 deprecation. Also don't
fail immediately on compose failure to be able to retrieve logs from the
test VM.
2022-03-22 10:54:30 +01:00
Antonio Murdaca
b2d18166de test/data/manifests: regenerate
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-03-14 17:31:40 +01:00
Jakub Rusz
485fc68a98 tests/upgrade: add user choice for vdo devices check
This is a user choice that has to be picked before the upgrade or it
acts as an inhibitor.
2022-03-08 09:23:05 +01:00
Christian Kellner
923a627de4 distro/rhel86: set GPGKeyFiles for azure-rhui
Import the appropriate GPG keys for Red Hat contained in the package
`redhat-release` and Microsoft contained in `rhui-azure-rhel8`.
2022-03-05 21:04:24 +01:00
Christian Kellner
5b802f699a distro/rhel86: sync packages for azure-rhui
Sync the list of packages with the existing market place images.
2022-03-05 21:04:24 +01:00
Christian Kellner
320dcb0f9d distro/rhel86: update sysconfig options
Set `kernel-core` to be the defaul kernel instead of `kernel`.
2022-03-05 21:04:24 +01:00
Christian Kellner
e60517f0af distro/rhel86: remove authconfig call from azure-rhui
authconfig is actually a RHEL 7 thing and we should not call it for
RHEL 8.
2022-03-05 21:04:24 +01:00
Christian Kellner
4b53312b7c test/data/repositories: bump snapshots to 20220301
Use the latest snapshot for the not yet released versions of RHEL and
their CentOS counterparts.
2022-03-05 11:37:12 +01:00
yih
f0522e758b test: fix ansible mount point check issue when encrypted
The device file is not `/dev/vda` anymore since we are no using a LUKS2
container, so we now have to look for `/dev/mapper/luks-`.
2022-02-28 23:05:45 +00:00
Antonio Murdaca
c1a896dd1d rhel{86,90}: define simplified installer build packages set
The edge installer and edge simplified installer build roots have
diverged, i.e. the latter need clevis/luks, so define a new pkg
set for the simplified installer extending the edge installer one.

Co-Authored-By: Christian Kellner <christian@kellner.me>
2022-02-28 23:05:45 +00:00
Antonio Murdaca
39db7653b5 test/data/respositories: bump snapshots to 0227
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-02-28 23:05:45 +00:00
Tom Gundersen
973b5141b3 ditro/rhel86: set volid like in rhel9.0
This is only required in RHEL9.0, but best practice is to always pin these things
down. Also increases uniformity between distros.

Simplify a bit the volid generator by making it require `rand.Rand` rather than
`io.Reader`, and hence eliminating the need for error handling.
2022-02-28 23:05:45 +00:00
Achilleas Koutsou
154e966cda test: update manifests and image info
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-28 23:05:45 +00:00
Christian Kellner
9e5b265a58 osbuild2: lock loopback devices during sfdisk, mkfs
Since udev will probe block devices it is advisable to hold a lock
on the device when modifying its partition table or the superblock
of the filesystem (see [1]). osbuild loopback devices do support
this via the `lock` option. Set this option for all operation that
involve changing block device "metadata" that could potentionally
race with udev, such as sfdisk, mkfs, creating a luks2 container
and creating LVM2 volume groups and logical volumes.
NB: osbuild also has its own device inhibition logic to prevent
udev/lvm2 from auto activating devices and in general to limit the
interaction between the host and devices used by osbuild. See [2]
for more information.
NB: this also locks the loopback device in situation where we the
it is strickly not the right thing to do, e.g. when creating a fs
on a logical voume that is located on a loopback device, since in
this case the device we would need to lock is the logical volume.
Sadly, LVM/DM devices are exempt from block device locking. But,
due to a bug in osbuild < 50, the udev inhibitor does *not* work
for loopback devices and therefore we have to use the actual lock
to preven LVM device auto-activation via `69-dm-lvm-metad.rules`.
The change was implemented by adding a new boolean to `getDevices`
indicating if the loopback device should be locked or not. Once
we depend on osbuild 50 we can change the logic in `getDevices`
to only lock the loopback device if the number of devices is one,
i.e. we are working directly on the loopback device.

[1] https://systemd.io/BLOCK_DEVICE_LOCKING/
[2] /usr/lib/udev/rules.d/10-osbuild-inhibitor.rules
2022-02-28 17:09:30 +01:00
Christian Kellner
19e7890f63 rhel90: automatically convert to LVM on fs customizations
Whenever we create a new mountpoint due to a user customization,
ensure the layout uses LVM, i.e. convert plain layouts to it, if
needed. This does not apply to rpm-ostree based systems, e.g. the
simplified installer since they will be using LUKS in 9.0.
Add "lvm2" to the build pipeline and thus generate new manifests
and image infos.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-28 17:09:30 +01:00
Christian Kellner
f91d8c6292 distro/rhel86: add support for azure marketplace
Add support for building images for the Azure marketplace: add a
new image type "azure-rhui" that can be used to build images
tailored to the Azure marketplace.
Add two sample manifests for 8.5 and 8.6, but note that even the
8.5 is using the 8.6 distro definitions. Also no image-info is
included since `image-info` cannot (yet) handle LVM setups and
the azure marketplace images use the LVM setup.
2022-02-27 22:03:36 +01:00
Achilleas Koutsou
41e4b17f89 distro/rhel86: add lvm2 to base build package set
We want to support LVM on all image types (optionally) so let's make
lvm2 available in all build roots.

Manifests and image info updated for RHEL 8.6 and CentOS Stream 8.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-27 13:04:55 +01:00
Achilleas Koutsou
a7fbe916b7 distro/rhel86+90: set selinux in permissive mode in installers
SELinux in the installer environment is in permissive mode in regular
installation ISOs
2022-02-26 15:55:56 +00:00
Achilleas Koutsou
365d754827 test: add new manifests for edge installers
Edge installer manifests (both simplified and anaconda) added for RHEL
8.6, RHEL 9.0, CS8, and CS9 for both supported architectures.
2022-02-26 15:55:56 +00:00
Tomas Hozza
ccd0be8c34 RHEL-90: use C.UTF-8 for images that only have glibc-minimal-langpack
Change the default locale to `C.UTF-8` for RHEL-9.0.
For all the images which install `langpack-en`, keep using the
`en_US.UTF-8` locale. `C.UTF-8` is used as the default for:
 - edge-commit
 - edge-container
 - image-installer
 - qcow2
 - tar

Also change the default locale for the edge-simplified-installer
installer-tree pipeline, since its package set contains only
`glibc-minimal-langpack`.

Regenerate RHEL-90 and c9s image tests.

Fix #2206

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-26 11:29:35 +01:00
Tom Gundersen
5f894281d5 test/manifest/image-installer: fix conflicting merge
The new image-installer tests were merged at the same time as the isolevel support. Adjust the test cases.
2022-02-24 18:53:21 +00:00
Ondřej Budai
d8d161480e rhel86/90: change isolevel of image-installer to 3
So it can contain payload bigger than 4GiB.

Related: rhbz#2056451
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-24 17:56:22 +00:00
Achilleas Koutsou
45a2c308f7 repos: bump RHEL 8.6 test repositories 2022-02-24 17:07:34 +01:00
Achilleas Koutsou
7f17757122 test: update manifests and image info
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-02-24 17:07:34 +01:00
Achilleas Koutsou
efbef95bf1 test: change condition for greenboot checks in ansible playbooks
Both CentOS versions are now part of the same condition.
2022-02-24 17:07:34 +01:00
Achilleas Koutsou
4ff0916caa test: update CS8 manifests and image info 2022-02-24 17:07:34 +01:00
Achilleas Koutsou
eda78a9a99 Update CentOS Stream 8 repositories 2022-02-24 17:07:34 +01:00
Achilleas Koutsou
fa209fe747 test: generate aarch64 image installer manifests
Added to imagetype map for test case generators and generate new
manifests.
2022-02-24 16:30:04 +01:00
Achilleas Koutsou
91d9417f5a distro/rhel90: use new unified grub2 stage options
With the new grub2 stage options we have the following changes to the
grub2 stage:
- Set the WriteCmdLine flag to false to not write kernel command line
  options to grubenv.
- Don't set the kernel command line options in the stage options.

The kernel command line options are now only specified in the Kernel
command line stage (org.osbuild.kernel-cmdline) so we add all options
there:
- Image type options
- Blueprint options
- Partition-table-specific options

Updated manifests and image info.

Co-Authored-By: Christian Kellner <christian@kellner.me>
2022-02-23 22:42:59 +01:00
Xiaofeng Wang
05cbbf0cc5 test: Skip rollback test only on UEFI VM of ostree-rebase.sh 2022-02-23 11:08:24 +01:00
Xiaofeng Wang
bc9d1431bc test: Add OSTree rebase test for OSTree parent ref support
To run ostree-rebase.sh, like RHEL 8.6 and RHEL 9, the CS8 VM
needs "large" flavor for more CPU and memory resource.
2022-02-23 11:08:24 +01:00
Achilleas Koutsou
602111dedd osbuild2: generalise copy stage option generation
Use entity based method `ForEachMountable` and `getDevices` to generate
all mounts and devices in a generic way, which then allows for mounts on
arbitrarily nested devices.

Update manifests and image info:
- New device names generated by `pathdot()` to avoid basename
  collisions.
- Some partitions are generated in a different order now which changes
  the order they appear in the manifest and their UUIDs.

Co-Authored-By: Christian Kellner <christian@kellner.me>
2022-02-22 19:23:41 +00:00
Christian Kellner
b7abef54e8 disk: calculate padding for secondary GPT header
Instead of hard coding a padding of 100 sectors for all layouts, i.e.
MBR and GTP, adjust the needed space depending on the layout: for MBR 
we don't need to reserve any space at all since it does not have a 
secondary header. For GTP we reserve 33 sectors, as indicated in the
UEFI specific, which allows for the header itself and up to 128 entries.
To not modify the layout of already released distributions, like RHEL
8.4 and 8.5, a new member called `ExtraPadding` is added to `Partition
Table` and then used in the corresponding layouts to preserve the
existing padding of 100.
2022-02-22 19:23:41 +00:00
Christian Kellner
d9ef268809 disk: use GenerateUUIDs in CreatePartitionTable
Instead of generating the UUIDs directly when new partitions are
created and separately for the boot and root partition, use the
new `PartitionTable.GenerateUUIDs` method to generate all UUIDs
that are missing in one go. Since this changes the order in
which the uuids are generated the test manifests UUIDs changed
and needed to be updated:
I used to following patch to get the updated manifests:

--- a/internal/distro/distro_test_common/distro_test_common.go
+++ b/internal/distro/distro_test_common/distro_test_common.go
@@ -105,6 +105,12 @@ func TestDistro_Manifest(t *testing.T, pipelinePath string, prefix string, regis
                                require.NoError(t, err)

                                diff := cmp.Diff(expected, actual)
+                               if diff != "" {
+                                       tt.Manifest = got
+                                       data, _ := json.MarshalIndent(tt, "", "  ")
+                                       path := filepath.Join("/tmp", filepath.Base(fileName))
+                                       _ = ioutil.WriteFile(path, data, 0644)
+                               }
                                require.Emptyf(t, diff, "Distro: %s\nArch: %s\nImage type: %s\nTest case file: %s\n", d.Name(), arch.Name(), imageType.Name(), fileName)
                        }
                })

And the following fish snippet to update the existing ones, using the
jq and sponge utilities:

for file in /tmp/rhel_85-*.json
  set filename (basename $file)
  jq -s '.[0].manifest = .[1].manifest | .[0]' test/data/manifests/$filename /tmp/$filename | sponge test/data/manifests/$filename
end
2022-02-22 19:23:41 +00:00
Achilleas Koutsou
12cbae3891 test: wait longer for reboot during ostree rollback test
Co-Authored-By: Xiaofeng Wang <henrywangxf@me.com>
2022-02-18 10:39:08 +00:00
Achilleas Koutsou
22f23ec38e test: update manifests and image info 2022-02-18 10:39:08 +00:00
Achilleas Koutsou
96e09e0322 Update cs9 test repos to latest snapshot 2022-02-18 10:39:08 +00:00
Achilleas Koutsou
6bc4afbcbd test,repositories: add cs9 repositories
Add rpmrepo repositories for testing.
Symlink centos-9 to centos-stream-9.
Add rpmrepo snapshot repositories to test and
test-case-generator repos.
Remove unused repositories from cs9 test repositories.
2022-02-18 10:39:08 +00:00
Jakub Rusz
43bfcc5389 tests/upgrade: gather more logs
Also disable firwalld AllowZoneDrifting because it's not supported in
RHEL-9.
2022-02-18 09:25:01 +01:00
Xiaofeng Wang
b378fee2e5 test: conditional greenboot package checks in ansible playbooks
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-16 10:49:55 +01:00
Achilleas Koutsou
1b0b54f853 test: regenerate RHEL 8.6 and 9.0 manifests
Repositories were update for the two distros so we've regenerated all
the manifests.
2022-02-16 10:49:55 +01:00
Achilleas Koutsou
e0822bff44 repos: update repos for test case generators
test/data/repositories:
    new snapshots for RHEL 8.6 (20220201) and 9.0 (20220208)

Switching RHEL 8.6 and RHEL 9.0 to the latest rpmrepo snapshots:
    We need updated packages for greenboot, used by edge image types, so
    we technically only need to change the architectures that are
    supported by edge, but let's update all of them for consistency.

test/data/repositories:
    Remove rhel-90 symlink to beta

Keep the rhel-90-beta.json repository definitions, but make rhel-90.json
a file that contains the previously named rhel-90-ga.json definitions.
2022-02-16 10:49:55 +01:00