Commit graph

265 commits

Author SHA1 Message Date
Achilleas Koutsou
2b2181867f osbuild2: use source-array-ref for rpm stage inputs
The (new) source-array-ref object is an array of objects, which allows
for input references to have options (metadata) but still maintain their
order when (de)serialised.

Fixes https://github.com/osbuild/osbuild-composer/issues/2577
Related to https://github.com/osbuild/osbuild/pull/1003
2022-04-27 11:51:25 +02:00
Tomas Hozza
d474db38d5 RHEL-86/EC2 SAP: install compat-sap-c++-10
Related to https://bugzilla.redhat.com/show_bug.cgi?id=2074114
2022-04-21 16:20:50 +02:00
Tomas Hozza
082ce34a93 RHEL-90/EC2 SAP: install @Server package group
Related to:
- 9.0: https://bugzilla.redhat.com/show_bug.cgi?id=2076763
- 9.2: https://bugzilla.redhat.com/show_bug.cgi?id=2076761
2022-04-21 16:20:50 +02:00
Tomas Hozza
54ba98f8c1 RHEL-86/EC2 SAP: install @Server package group
Related to https://bugzilla.redhat.com/show_bug.cgi?id=2074107
2022-04-21 16:20:50 +02:00
Tomas Hozza
4bba54b1b3 RHEL-90/EC2 SAP: adjust PAM limits configuration
Related to:
- 9.0: https://bugzilla.redhat.com/show_bug.cgi?id=2076759
- 9.2: https://bugzilla.redhat.com/show_bug.cgi?id=2076760
2022-04-21 16:20:50 +02:00
Tomas Hozza
88fc9f9869 RHEL-86/EC2 SAP: adjust PAM limits configuration
Related to https://bugzilla.redhat.com/show_bug.cgi?id=2074101
2022-04-21 16:20:50 +02:00
Tomas Hozza
f45bdf3fb3 Don't pass GPG keys for GCP repos and don't verify signatures on el9/c9s
Google repositories use RSA/SHA1 for signing packages. However the SHA1
has been disabled by default on el9/c9s. Since osbuild-composer imports
GPG keys specified in the repository definition unconditionally, this
creates issues when installing rpms signed with the key by osbuild [1].

Remove GPG keys in all el9/c9s GCP repo definitions and disable GPG
signature verification until [2] is resolved.

[1] https://github.com/osbuild/osbuild/issues/991
[2] https://issuetracker.google.com/issues/223626963

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Tomas Hozza
e7f6b95032 RHEL-84: add gce-rhui image type
Add the `gce-rhui` image type intended for Google Compute Engine. The image
uses Google's RHUI infrastructure to access Red Hat content.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Tomas Hozza
795294d98f RHEL-84: add gce image type
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Tomas Hozza
77b5ef459d RHEL-85: add gce-rhui image type
Add the `gce-rhui` image type intended for Google Compute Engine. The image
uses Google's RHUI infrastructure to access Red Hat content.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Tomas Hozza
6ffe63442e RHEL-85: add gce image type
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Tomas Hozza
0219ad1f43 RHEL-86: add gce-rhui image type
Add the `gce-rhui` image type intended for Google Compute Engine. The image
uses Google's RHUI infrastructure to access Red Hat content.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Tomas Hozza
db75c39c4d RHEL-86: add gce image type
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Tomas Hozza
5d27b7c784 RHEL-90: add gce image type
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Tomas Hozza
bd81506831 image-info: read the firewall default zone
Modify affected image manifests.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Christian Kellner
aaee939f9b distro/rhel86: enable auto-registration for azure
We want to ensure that cloud images connect to Red Hat[1] independently
of how the content was acquired (PAYG, BYOS, or marketplace).
This auto-registration feature is already enabled for AWS and this
patch enables it for Azure with the same recommended settings:

Services:
  rhsmcertd: Enabled (already done, so not changed in the patch)

/etc/rhsm/rhsm.conf:
  auto_registration: enabled
  auto_registration_interval: 60 (the default, so not explicitly set)
  manage_repos: false

The latter value `manage_repos` is left enabled (the default) in case
the user explicitly requested to have the system subscribed, i.e. the
`RHSMConfigWithSubscription` code path.

Regenerate the relevant test manifests and image information.

[1] https://cloud.redhat.com
[2] https://docs.google.com/document/d/1VeZFJxNUlyZMQJh6s3NA3RLvadqATsGxVet6uuP87_4
2022-04-06 22:40:01 +02:00
Jakub Rusz
7666049210 ci: re-enabled Installer test on centos-stream-9
Anaconda with the fix released in the centos-stream-9 repos. Also fixing
ostree runner matrix.
2022-04-06 12:53:35 +02:00
Achilleas Koutsou
8316cf7ef0 rhel85: enable anaconda users module in installers
The users anaconda module enables users to create user accounts at
install time if one is not already created in the payload.  This is
required for the cloud API (Image Builder service) for the image
installer where user customizations are not supported.  Without it, user
creation isn't possible on the installed system.

The module also enables user creation at install time through the
kickstart file for both the image-installer and the edge-installer
(Anaconda only).
Therefore, for the image-installer, the users and groups are no longer
created as part of the payload.

This commit adapts the changes from the following commits (originally
made in the RHEL 8.6 and RHEL 9.0 distros) to the rest of the RHEL
distro definitions:
ebc3330cbd
5825294dad
2022-04-06 12:09:09 +02:00
Achilleas Koutsou
42364f2cc6 distro: deduplicate groupStageOptions() in RHEL pipelines
Use single NewGroupsStageOptions() from osbuild1 and osbuild2 instead of
implementing in each distro.
- Followup from 2eef6e6e2d, copied to the
  rest of the RHEL distro definitions.
- Added NewGroupsStageOptions() to osbuild1 for rhel8 and rhel84.

NB: The change was not made in the Fedora distro definitions as they are
    currently being rewritten.
2022-04-06 12:09:09 +02:00
Achilleas Koutsou
15243f1dc5 distro: allow user and group customizations for edge-installer
Followup from, f34380d5b5 and
3a1765a5a8, copied to the rest of the RHEL
distro definitions.

For now, these customizations have no effect on the manifest.

The new `with-users` variants of the edge-installer test cases include
the user customizations in the blueprint, but the manifests are
(currently) the same as the corresponding base cases.
2022-04-06 12:09:09 +02:00
Achilleas Koutsou
d3f217d40f test: (re)generate all supported installer test manifests
New test cases for RHEL 8.5.
New RPM GPG keys for RHEL 9.0.
2022-04-06 12:09:09 +02:00
Achilleas Koutsou
11be84c225 test: add RHEL 8.4 rhel-edge-installer test case
Include RHEL 8.4 rhel-edge-installer in the test cases.
2022-04-06 12:09:09 +02:00
Tomas Hozza
c6507c21b9 distro: add cloud-init to VMDK image on RHEL-85/86/90
Install `cloud-init` by default on the VMDK image on RHEL-85/86/90 and
as a result also CentOS Stream 8/9.

Regenerate image test cases.

Signed-off-by: Tomas Hozza <thozza@redhat.com>

test cases fixup
2022-04-02 08:47:02 +01:00
Ondřej Budai
fd49ed9a07 repositories: update key for RHEL 9.0
RHEL 9.0 will ship a brand new auxiliary key. Let's use it everywhere in our
RHEL 9 stuff. Taken from current RHEL 9.0's redhat-release package.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-04-01 19:22:23 +02:00
Xiaofeng Wang
87679fcab2 test: Add retries on ubi8 image and greenboot package downloading
To fix issue: https://github.com/osbuild/osbuild-composer/issues/2492,
retry 30 times when downloading ubi8 image and greenboot package.
2022-03-30 16:35:11 +02:00
Christian Kellner
45b1fc3cd1 osbuild2: honor GPG key setting for rpm inputs
We should honour `pkg.CheckGPG` when creating the file inputs for the
rpm stage. This was lost in the transition from v1 to v2 manifests.

Regenerate image test manifests.

Co-authored-by: Tomas Hozza <thozza@redhat.com>
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-03-30 14:32:28 +02:00
Tomas Hozza
947acf74ab Regenerate affected image test cases 2022-03-29 11:43:54 +02:00
Xiaofeng Wang
21b6e95136 test: Clean up and improve ostree-simplified-installer.sh
Remove comment out code
Use three different IP address for different test scenarios
Move /boot/device-credentials file checking into playbook
Some shell script improvements
2022-03-29 08:09:37 +02:00
Achilleas Koutsou
5825294dad distro/rhel86+90: enable the anaconda user modeule for image-installer
Enable the user module unconditionally for the image-installer:
- If users are specified for the kickstart file, the module is required
  to set up the users.
- If no users are specified, the module can be used at install time to
  create users.

Updated relevant test cases (manifests).
2022-03-28 14:08:57 +02:00
Achilleas Koutsou
fe514b9c2e test: create new test cases (manifests)
New test cases for edge- and image-installer with users.
2022-03-28 14:08:57 +02:00
Achilleas Koutsou
2eef6e6e2d osbuild2: deduplicate groupStageOptions()
Use single NewGroupsStageOptions() from osbuild2 instead of implementing
in each distro.

The new function does not set the Group.Name field anymore.  The field
does not exist in the osbuild schema and was silently ignored.
The field in the stage has been marked 'omitempty' and the relevant
manifests have been updated.
2022-03-28 14:08:57 +02:00
Jakub Rusz
f1006c5adb ci/tests: re-enable Installers.sh test
Related bug has been fixed. Updating snapshots and enabling the test.
2022-03-25 16:00:39 +01:00
Antonio Murdaca
bf46113251 test/data/repositories: bump cs8
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-03-24 18:23:39 +01:00
Jakub Rusz
7bf019efcb test/repositories: update rhel-90 repository gpg keys
SHA-1 keys are deprecated, updating to the newly used ones.
2022-03-24 18:23:39 +01:00
Tomas Hozza
2ec55b4c53 RHEL-90: use XBOOTLDR partition GUID for /boot
Use the Extended Bootloader Partition GUID for `/boot`, instead of the
Linux filesystem data GUID. This is useful for autodetection of a
partition purpose based on its GUID without reading the `/etc/fstab`
first.

Ensure that when creating mountpoints, e.g. when converting the
partitions layout to LVM, the `/boot` partition get the proper GUID
assigned.

Regenerate RHEL-90 and centos-9 image test cases.

Related to https://bugzilla.redhat.com/show_bug.cgi?id=2057231
2022-03-24 15:40:10 +01:00
Tomas Hozza
60db6ad06f image tests: update GPG keys used for RHEL-9.0 repos
SHA-1 key is no longer supported, update it with a new one from
redhat-release.

Related rhbz#2058497

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-03-23 13:22:13 +01:00
Jakub Rusz
15c2044b3c tests/upgrade: update gpg key
We need to use a new gpg key after the SHA-1 deprecation. Also don't
fail immediately on compose failure to be able to retrieve logs from the
test VM.
2022-03-22 10:54:30 +01:00
Antonio Murdaca
b2d18166de test/data/manifests: regenerate
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-03-14 17:31:40 +01:00
Jakub Rusz
485fc68a98 tests/upgrade: add user choice for vdo devices check
This is a user choice that has to be picked before the upgrade or it
acts as an inhibitor.
2022-03-08 09:23:05 +01:00
Christian Kellner
923a627de4 distro/rhel86: set GPGKeyFiles for azure-rhui
Import the appropriate GPG keys for Red Hat contained in the package
`redhat-release` and Microsoft contained in `rhui-azure-rhel8`.
2022-03-05 21:04:24 +01:00
Christian Kellner
5b802f699a distro/rhel86: sync packages for azure-rhui
Sync the list of packages with the existing market place images.
2022-03-05 21:04:24 +01:00
Christian Kellner
320dcb0f9d distro/rhel86: update sysconfig options
Set `kernel-core` to be the defaul kernel instead of `kernel`.
2022-03-05 21:04:24 +01:00
Christian Kellner
e60517f0af distro/rhel86: remove authconfig call from azure-rhui
authconfig is actually a RHEL 7 thing and we should not call it for
RHEL 8.
2022-03-05 21:04:24 +01:00
Christian Kellner
4b53312b7c test/data/repositories: bump snapshots to 20220301
Use the latest snapshot for the not yet released versions of RHEL and
their CentOS counterparts.
2022-03-05 11:37:12 +01:00
yih
f0522e758b test: fix ansible mount point check issue when encrypted
The device file is not `/dev/vda` anymore since we are no using a LUKS2
container, so we now have to look for `/dev/mapper/luks-`.
2022-02-28 23:05:45 +00:00
Antonio Murdaca
c1a896dd1d rhel{86,90}: define simplified installer build packages set
The edge installer and edge simplified installer build roots have
diverged, i.e. the latter need clevis/luks, so define a new pkg
set for the simplified installer extending the edge installer one.

Co-Authored-By: Christian Kellner <christian@kellner.me>
2022-02-28 23:05:45 +00:00
Antonio Murdaca
39db7653b5 test/data/respositories: bump snapshots to 0227
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-02-28 23:05:45 +00:00
Tom Gundersen
973b5141b3 ditro/rhel86: set volid like in rhel9.0
This is only required in RHEL9.0, but best practice is to always pin these things
down. Also increases uniformity between distros.

Simplify a bit the volid generator by making it require `rand.Rand` rather than
`io.Reader`, and hence eliminating the need for error handling.
2022-02-28 23:05:45 +00:00
Achilleas Koutsou
154e966cda test: update manifests and image info
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-28 23:05:45 +00:00
Christian Kellner
9e5b265a58 osbuild2: lock loopback devices during sfdisk, mkfs
Since udev will probe block devices it is advisable to hold a lock
on the device when modifying its partition table or the superblock
of the filesystem (see [1]). osbuild loopback devices do support
this via the `lock` option. Set this option for all operation that
involve changing block device "metadata" that could potentionally
race with udev, such as sfdisk, mkfs, creating a luks2 container
and creating LVM2 volume groups and logical volumes.
NB: osbuild also has its own device inhibition logic to prevent
udev/lvm2 from auto activating devices and in general to limit the
interaction between the host and devices used by osbuild. See [2]
for more information.
NB: this also locks the loopback device in situation where we the
it is strickly not the right thing to do, e.g. when creating a fs
on a logical voume that is located on a loopback device, since in
this case the device we would need to lock is the logical volume.
Sadly, LVM/DM devices are exempt from block device locking. But,
due to a bug in osbuild < 50, the udev inhibitor does *not* work
for loopback devices and therefore we have to use the actual lock
to preven LVM device auto-activation via `69-dm-lvm-metad.rules`.
The change was implemented by adding a new boolean to `getDevices`
indicating if the loopback device should be locked or not. Once
we depend on osbuild 50 we can change the logic in `getDevices`
to only lock the loopback device if the number of devices is one,
i.e. we are working directly on the loopback device.

[1] https://systemd.io/BLOCK_DEVICE_LOCKING/
[2] /usr/lib/udev/rules.d/10-osbuild-inhibitor.rules
2022-02-28 17:09:30 +01:00