On RHEL-8, the x86_64 AMI / EC2 images used a BIOS-only partition table
layout, because the base partition table unification happened in the
past only on RHEL-9 and Fedora (inherited from RHEL-9).
To make things consistent and uniform across RHEL-8 and RHEL-9, I copied
the base partition table used by RHEL-9 AMI / EC2 images to RHEL-8. This
has a side-effect for aarch64 AMI / EC2, where the `/boot` partition
size changed from 512 MiB to 500 MiB, together with the partition GUID
to "Extended Boot Loader Partition GUID".
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Move the handling of the list of enabled and disabled systemd services
more to the end of the `os` pipeline, just before the SELinux stage.
This has no functional effect on produced images, but it will make it
nicer once the handling of the custom files and directories will be
added to the pipeline. Specifically it should be added right before the
services stage to allow enabling custom service files, but after all
other configurations that are applied to the image.
Regenerate all manifests.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Changes:
- Removed unneeded RPMs from the build root.
- /usr/bin/tar removed from selinux stage.
- Changed order of the rhsm stage. This will not affect functionality.
`amdgpu` module is causing error to be printed in the system log on AWS
instances. After investigation, it turns out that it is not needed.
Disable it by default on all AWS images.
Related to COMPOSER-1807
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Blueprint package set is now depsolved together with the OS package set
in a chain. The result is stored in the package specs sets under the OS
package set name.
In reality, the code was able to handle a `nil` package specs to be
passed to pipelines, however some parts were looking for the kernel
version in the blueprint package specs, which would be a bug.
Regenerated affected image test cases.
We should honour `pkg.CheckGPG` when creating the file inputs for the
rpm stage. This was lost in the transition from v1 to v2 manifests.
Regenerate image test manifests.
Co-authored-by: Tomas Hozza <thozza@redhat.com>
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Since udev will probe block devices it is advisable to hold a lock
on the device when modifying its partition table or the superblock
of the filesystem (see [1]). osbuild loopback devices do support
this via the `lock` option. Set this option for all operation that
involve changing block device "metadata" that could potentionally
race with udev, such as sfdisk, mkfs, creating a luks2 container
and creating LVM2 volume groups and logical volumes.
NB: osbuild also has its own device inhibition logic to prevent
udev/lvm2 from auto activating devices and in general to limit the
interaction between the host and devices used by osbuild. See [2]
for more information.
NB: this also locks the loopback device in situation where we the
it is strickly not the right thing to do, e.g. when creating a fs
on a logical voume that is located on a loopback device, since in
this case the device we would need to lock is the logical volume.
Sadly, LVM/DM devices are exempt from block device locking. But,
due to a bug in osbuild < 50, the udev inhibitor does *not* work
for loopback devices and therefore we have to use the actual lock
to preven LVM device auto-activation via `69-dm-lvm-metad.rules`.
The change was implemented by adding a new boolean to `getDevices`
indicating if the loopback device should be locked or not. Once
we depend on osbuild 50 we can change the logic in `getDevices`
to only lock the loopback device if the number of devices is one,
i.e. we are working directly on the loopback device.
[1] https://systemd.io/BLOCK_DEVICE_LOCKING/
[2] /usr/lib/udev/rules.d/10-osbuild-inhibitor.rules
We want to support LVM on all image types (optionally) so let's make
lvm2 available in all build roots.
Manifests and image info updated for RHEL 8.6 and CentOS Stream 8.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Use entity based method `ForEachMountable` and `getDevices` to generate
all mounts and devices in a generic way, which then allows for mounts on
arbitrarily nested devices.
Update manifests and image info:
- New device names generated by `pathdot()` to avoid basename
collisions.
- Some partitions are generated in a different order now which changes
the order they appear in the manifest and their UUIDs.
Co-Authored-By: Christian Kellner <christian@kellner.me>
Instead of hard coding a padding of 100 sectors for all layouts, i.e.
MBR and GTP, adjust the needed space depending on the layout: for MBR
we don't need to reserve any space at all since it does not have a
secondary header. For GTP we reserve 33 sectors, as indicated in the
UEFI specific, which allows for the header itself and up to 128 entries.
To not modify the layout of already released distributions, like RHEL
8.4 and 8.5, a new member called `ExtraPadding` is added to `Partition
Table` and then used in the corresponding layouts to preserve the
existing padding of 100.
Disable loging in via password authentication since this is an
official Amazon marketplace requirement
Linux-based AMIs must not allow SSH password authentication.
Disable password authentication via your sshd_config file by
setting PasswordAuthentication to NO.
Section "Security policies" from
https://docs.aws.amazon.com/marketplace/latest/userguide/product-and-ami-policies.html
Port all of the pipeline refactoring done to RHEL-90 to RHEL-86. Both
distros now use the same approach.
Regenerate all RHEL-8.6 and CentOS 8 image test cases.
[1] https://git.centos.org/centos/kickstarts/tree/master
Signed-off-by: Tomas Hozza <thozza@redhat.com>
This is a preparation for defining EC2 images for RHEL-8.5.0. These
extensions to image-info tool represent modifications done to the
official EC2 images currently produced as RHEL release. It is important
to be able to analyse these aspects of images, before we define them in
osbuild-composer, to ensure that the resulting images will be consistent
with the current state.
- Read non-empty lines from /etc/hosts file and add them to the report.
- Read content of /etc/machine-id and add it to the report.
- Read uncommented key/values from /etc/systemd/logind.conf and add them
to the report.
- Read all ifcfg-* files from /etc/sysconfig/network-scripts/ and add
their values to the report.
- Read content of /etc/locale.conf and add it to the report.
- Read SELinux configuration from /etc/selinux/config and add it to the
report.
- Inspect the filesystem tree for SELinux context mismatches and add
them to the report.
- Read configuration files from /etc/modprobe.d/ and for now report only
all blacklisted kernel modules.
- Read RHSM configuration from /etc/rhsm/rhsm.conf and add it to the
report.
- Read cloud-init configuration from /etc/cloud/cloud.conf and add it to
the report.
- Read all *.conf files from /etc/dracut.conf.d/ and add their content to
the report.
- Read VC and X11 keyboard configuration and add it to the report.
- Read specific configuration directives from Chrony configuration and
add them to the report. Specifically 'server', 'pool', 'peer' and
'leapsectz'.
- Read drop-in configurations for *.service unit files from
/etc/systemd/system/ and add them to the report.
- Read all configuration files from /etc/tmpfiles.d/ and add them to the
report.
- Read all configuration files from /etc/sysctl.d/ and add them to the
report.
- Read the Tuned active profile and profile mode and add them to the
report.
- Read all configuration files from /etc/security/limits.d and add them
to the report.
- Read sudoers configuration from /etc/sudoers and files in
/etc/sudoers.d/ and add uncommented lines to the report. No
sophisticated parsing is done, because the configuration format
grammar is too complicated for the purpose of image-info.
- Read udev rules configuration files from /etc/udev/rules.d/ and add
them to the report.
- Read DNF configuration and defined vars and add them to the report.
- Read profile ID and enabled features used by authselect.
- Enable SELinux, extended attributes and POSIX ACLs support when
unpacking 'tar' image type to prevent potential mismatches
Regenerate all image test cases to reflect changes in the image-info
output. Modify the distro-arch-imagetype-map.json to cover all
combinations currently covered by existing image test cases.
Add doc strings to all read_* functions.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Change the "image-format" from a string to a dict, with a "type":
$value entry, where $value contains the previous plain string
data.
Additionally, include the qcow2 format version, if the given
image is indeed a qcow2.
Adapt all manifest test accordingly (partly done by Ondřej)
Python 3 script used for conversion of manifest tests:
import os
import json
for name in os.listdir(os.getcwd()):
if not name.endswith(".json"):
continue
print(name)
with open(name, "r") as old:
data = json.load(old)
info = data.get("image-info", {})
format = info.get("image-format")
if not format:
continue
info["image-format"] = {
"type": format
}
if format != "qcow2":
continue
info["image-format"]["compat"] = "1.1"
with open(name + ".new", "w") as new:
json.dump(data, new, indent=2)
new.write("\n")
new.flush()
os.rename(name+".new", name)
test: use the new image-info format in all test manifests
The previous commit converted only qcow2 and openstack manifests but this change
is actually needed for all manifests produced by the qemu assembler.
Co-Developed-by: Ondřej Budai <ondrej@budai.cz>
Use en_US.UTF-8 as default for LANG, which is what previously was
used and is also needed to properly work on non-us/latin setups[1].
In the customization tests, use a different value than the default
one to check that the customization does in fact work.
[1] http://git.app.eng.bos.redhat.com/git/spin-kickstarts.git/tree/rhel8/rhel-8.2-kvm-x86_64.ks#n4
Co-authored-by: Achilleas Koutsou <achilleas@koutsou.net>
We have integration tests that boot most of the image types, let
the image tests focus on the actual content of the images, shaving
off considerable CI time.
The only missing one is OpenStack (which should be ported over) and
tar (which we need to define the integration test for).
Signed-off-by: Tom Gundersen <teg@jklm.no>
The image definition is shared with the latest RHEL 8.y one (8.4 currently).
I expect that we the introduction of 8.5 support, we point the centos 8
distro at it.
The test repositories and manifests use the official CentOS composes. From
what I can tell, they are persistent. This is not guaranteed though, so we
might need to switch to RPMRepo at some point.
The "classic" CentOS 8 should also be buildable but due to the chicken and egg
issue (this commit will get into Centos "8.4" but Centos "8.4" isn't a thing
yet), we cannot test it and therefore it might be broken.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
