Since udev will probe block devices it is advisable to hold a lock
on the device when modifying its partition table or the superblock
of the filesystem (see [1]). osbuild loopback devices do support
this via the `lock` option. Set this option for all operation that
involve changing block device "metadata" that could potentionally
race with udev, such as sfdisk, mkfs, creating a luks2 container
and creating LVM2 volume groups and logical volumes.
NB: osbuild also has its own device inhibition logic to prevent
udev/lvm2 from auto activating devices and in general to limit the
interaction between the host and devices used by osbuild. See [2]
for more information.
NB: this also locks the loopback device in situation where we the
it is strickly not the right thing to do, e.g. when creating a fs
on a logical voume that is located on a loopback device, since in
this case the device we would need to lock is the logical volume.
Sadly, LVM/DM devices are exempt from block device locking. But,
due to a bug in osbuild < 50, the udev inhibitor does *not* work
for loopback devices and therefore we have to use the actual lock
to preven LVM device auto-activation via `69-dm-lvm-metad.rules`.
The change was implemented by adding a new boolean to `getDevices`
indicating if the loopback device should be locked or not. Once
we depend on osbuild 50 we can change the logic in `getDevices`
to only lock the loopback device if the number of devices is one,
i.e. we are working directly on the loopback device.
[1] https://systemd.io/BLOCK_DEVICE_LOCKING/
[2] /usr/lib/udev/rules.d/10-osbuild-inhibitor.rules
Whenever we create a new mountpoint due to a user customization,
ensure the layout uses LVM, i.e. convert plain layouts to it, if
needed. This does not apply to rpm-ostree based systems, e.g. the
simplified installer since they will be using LUKS in 9.0.
Add "lvm2" to the build pipeline and thus generate new manifests
and image infos.
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
Add support for building images for the Azure marketplace: add a
new image type "azure-rhui" that can be used to build images
tailored to the Azure marketplace.
Add two sample manifests for 8.5 and 8.6, but note that even the
8.5 is using the 8.6 distro definitions. Also no image-info is
included since `image-info` cannot (yet) handle LVM setups and
the azure marketplace images use the LVM setup.
We want to support LVM on all image types (optionally) so let's make
lvm2 available in all build roots.
Manifests and image info updated for RHEL 8.6 and CentOS Stream 8.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Don't remove the pre-existing mount-points from the blueprint b/c we
still need to assert on those.
Add `/tmp` and `/var/tmp` to exercise the case where mountpoints have
matching suffixes.
Change the default locale to `C.UTF-8` for RHEL-9.0.
For all the images which install `langpack-en`, keep using the
`en_US.UTF-8` locale. `C.UTF-8` is used as the default for:
- edge-commit
- edge-container
- image-installer
- qcow2
- tar
Also change the default locale for the edge-simplified-installer
installer-tree pipeline, since its package set contains only
`glibc-minimal-langpack`.
Regenerate RHEL-90 and c9s image tests.
Fix#2206
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Filesystem mountpoints with matching suffixes, i.e.
`/tmp` and `/var/tmp` was causing image builds to
fail. Since this was fixed with #2141 the integration
tests have been updated to check for this.
With the new grub2 stage options we have the following changes to the
grub2 stage:
- Set the WriteCmdLine flag to false to not write kernel command line
options to grubenv.
- Don't set the kernel command line options in the stage options.
The kernel command line options are now only specified in the Kernel
command line stage (org.osbuild.kernel-cmdline) so we add all options
there:
- Image type options
- Blueprint options
- Partition-table-specific options
Updated manifests and image info.
Co-Authored-By: Christian Kellner <christian@kellner.me>
Specifying a parent commit ID without a URL isn't allowed now. The
upgrade commit is built without a parent. The OS upgrade should still
work without it.
Getting the ID of the first commit is not necessary now.
Use entity based method `ForEachMountable` and `getDevices` to generate
all mounts and devices in a generic way, which then allows for mounts on
arbitrarily nested devices.
Update manifests and image info:
- New device names generated by `pathdot()` to avoid basename
collisions.
- Some partitions are generated in a different order now which changes
the order they appear in the manifest and their UUIDs.
Co-Authored-By: Christian Kellner <christian@kellner.me>
Instead of hard coding a padding of 100 sectors for all layouts, i.e.
MBR and GTP, adjust the needed space depending on the layout: for MBR
we don't need to reserve any space at all since it does not have a
secondary header. For GTP we reserve 33 sectors, as indicated in the
UEFI specific, which allows for the header itself and up to 128 entries.
To not modify the layout of already released distributions, like RHEL
8.4 and 8.5, a new member called `ExtraPadding` is added to `Partition
Table` and then used in the corresponding layouts to preserve the
existing padding of 100.
Instead of generating the UUIDs directly when new partitions are
created and separately for the boot and root partition, use the
new `PartitionTable.GenerateUUIDs` method to generate all UUIDs
that are missing in one go. Since this changes the order in
which the uuids are generated the test manifests UUIDs changed
and needed to be updated:
I used to following patch to get the updated manifests:
--- a/internal/distro/distro_test_common/distro_test_common.go
+++ b/internal/distro/distro_test_common/distro_test_common.go
@@ -105,6 +105,12 @@ func TestDistro_Manifest(t *testing.T, pipelinePath string, prefix string, regis
require.NoError(t, err)
diff := cmp.Diff(expected, actual)
+ if diff != "" {
+ tt.Manifest = got
+ data, _ := json.MarshalIndent(tt, "", " ")
+ path := filepath.Join("/tmp", filepath.Base(fileName))
+ _ = ioutil.WriteFile(path, data, 0644)
+ }
require.Emptyf(t, diff, "Distro: %s\nArch: %s\nImage type: %s\nTest case file: %s\n", d.Name(), arch.Name(), imageType.Name(), fileName)
}
})
And the following fish snippet to update the existing ones, using the
jq and sponge utilities:
for file in /tmp/rhel_85-*.json
set filename (basename $file)
jq -s '.[0].manifest = .[1].manifest | .[0]' test/data/manifests/$filename /tmp/$filename | sponge test/data/manifests/$filename
end
We (mistakenly) don't enforce a minimum size for /var,
so setting it to 1024 (1kB) causes the image build to fail.
CI does not expose this in a helpful way at the moment,
so this is a bit tricky to debug.
Also skip customizations for the AWS.S3 upload type. Not all the
image types with this upload type support filesystem customizations
and that's as expected. We could make a more fine-grained test in
the future, but testing with a coulpe of targets should be
sufficient.
Add rpmrepo repositories for testing.
Symlink centos-9 to centos-stream-9.
Add rpmrepo snapshot repositories to test and
test-case-generator repos.
Remove unused repositories from cs9 test repositories.
test/data/repositories:
new snapshots for RHEL 8.6 (20220201) and 9.0 (20220208)
Switching RHEL 8.6 and RHEL 9.0 to the latest rpmrepo snapshots:
We need updated packages for greenboot, used by edge image types, so
we technically only need to change the architectures that are
supported by edge, but let's update all of them for consistency.
test/data/repositories:
Remove rhel-90 symlink to beta
Keep the rhel-90-beta.json repository definitions, but make rhel-90.json
a file that contains the previously named rhel-90-ga.json definitions.
This is needed to satisfy RPM dependencies of GCP guest tools, which
will be installed on the `gce` image type.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
When deploying an ostree commit, specify a remote, currently hard-
coded to `rhel-edge`, so that updates work automatically, if they
are served from the same location as the initial commit is pulled
from.
NB: now that the remote is specified in the raw image, remove the
corresponding bits form the tests.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
