Commit graph

1241 commits

Author SHA1 Message Date
Gianluca Zuccarelli
b782ecaf1f distro: rhel85 internal tests 2021-08-24 00:22:29 +02:00
Gianluca Zuccarelli
d4403a03c0 distro: rhel85 add sub mounts to allow list 2021-08-24 00:22:29 +02:00
Tomas Hozza
a8d503e5b9 RHEL-85/90: modify the x86_64 EC2 images dracut configuration
Change the x86_64-specific dracut configuration of RHEL-8.5 and RHEL-9.0
EC2 and AMI images to not include `xen-netfront` driver and add `nvme`
driver, which was previously not included. Since the configuration is no
longer Xen-specific, rename the configuration file to `ec2.conf`.

Justification:
There is no reason to put `xen-netfront` to initramfs as EC2 images don't
boot from network root. In addition, add `nvme` driver to handle the case
when initramfs is getting forcefully rebuild on a Xen instance (and not able
to boot on Nitro after that).

Related to https://issues.redhat.com/browse/COMPOSER-1096.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-23 16:01:11 +02:00
Christian Kellner
20258bd45e distro/rhel90: use unified grub config
Feodra 34 and thus RHEL 9 switched to a unified grub configuration,
which means that the main grub config is always located in the same
location, /boot/grub2/grub.cfg.[1] osbuild has used this scheme for
hybrid boot on x64 but not on pure efi systems like aarch64. The
new osbuild option `uefi.unified` was introduced to select that new
unified grug cfg scheme also for those, pure efi, systems. Use that.

[1] https://fedoraproject.org/wiki/Changes/UnifyGrubConfig
2021-08-23 13:48:26 +02:00
Christian Kellner
22bff1df71 osbuild2/grub2: add unified option for uefi
The `uefi.unified` option indicates whether the `org.osbuild.grub2`
will use the unified grub configration scheme[1] used by Fedora 34
and thus RHEL 9.
NB: This requires osbuild version >= 32.

[1] https://fedoraproject.org/wiki/Changes/UnifyGrubConfig
2021-08-23 13:48:26 +02:00
Christian Kellner
fcace40710 osbuild2/grub2: add install option for uefi
The `uefi.install` option indicates whether the `org.osbuild.grub2`
stage will copy the efi binaries from the build root to the `/boot`
directory in the tree.

Co-Developed-by: Achilleas Koutsou <achilleas@koutsou.net>
Co-Developed-by: Antonio Murdaca <runcom@linux.com>
2021-08-23 13:48:26 +02:00
Achilleas Koutsou
321dcc56bf distro/rhel90: copy devices and mounts changes
Rebase brings RHEL 9.0 definition with old Mounts and Devices. Changes
applied to new distro pipelines and stage options.
2021-08-21 13:39:10 +02:00
Achilleas Koutsou
745443181e osbuild2: remove stage-specific device types
Devices unlike stage options, shouldn't be stage specific.
There is only one type of device so far, the loopback device, which
is already defined as a separate type.

The top level Devices type is simply an alias to a Device map.

The mkfs stages require a single device with a specific key ("device").
These stages accept only one device in their NewStage() function for
convenience and create the Stage struct with the required key.

The zipl.inst stage requires a device labeled 'disk' as well as the rest
of the devices that correspond to each partition. The disk device is
passed to the New stage function separately and added to the Stage
devices with the required key.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-08-21 13:39:10 +02:00
Achilleas Koutsou
c74d13daf8 osbuild2: remove stage-specific mount types
Mounts unlike stage options, shouldn't be stage specific. We have
filesystem specific mount types, differentiated by their type string.
Mounts can define their own additional options if necessary.

The top level Mounts type is simply an alias to a Mount array.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-08-21 13:39:10 +02:00
Gianluca Zuccarelli
613ad0b862 disk: refactor partition table size & start points 2021-08-21 02:54:38 +02:00
Gianluca Zuccarelli
280e1b6e3d distro: rhel85 declarative partition tables 2021-08-21 02:54:38 +02:00
Gianluca Zuccarelli
e21eee273c disk: move partition creation to disk package 2021-08-21 02:54:38 +02:00
Gianluca Zuccarelli
87c176503a distro: rhel85 streamline to single function 2021-08-21 02:54:38 +02:00
Gianluca Zuccarelli
6c4da2b821 distro: rhel85 create partition helper functions 2021-08-21 02:54:38 +02:00
Gianluca Zuccarelli
346eef7477 distro: rhel85 validate /usr mountpoint size 2021-08-21 02:54:38 +02:00
Gianluca Zuccarelli
ad39aab854 distro: allow valid rhel85 mountpoint subdirs 2021-08-21 02:54:38 +02:00
Gianluca Zuccarelli
6abe88f36d distro: update rhel8.5 partition table creation 2021-08-21 02:54:38 +02:00
Gianluca Zuccarelli
dad535d295 distro: add mountpoint allow list for rhel8.5 2021-08-21 02:54:38 +02:00
sanne
9fab5def90 dbjobqueue: Reduce error noise in rollback check
If the transaction is already closed don't log the rollback failure as
an error, it means it was successfully committed.
2021-08-20 15:42:57 +02:00
Ondřej Budai
1e2ba4da64 upload/azure: use cheaper storage accounts
Previously, we used RAGRS which means that all our data was always replicated
to at least two regions for increased safety. This is cool but expensive, this PR
switches the API to use LRS that just uses one region.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-17 17:51:23 +02:00
Tomas Hozza
a8ac012ebe rhel90: fix a typo in RHBZ number in a comment
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-13 08:24:37 +02:00
Tomas Hozza
6edf4aad49 osbuild2: add prefix parameter to the fix-bls stage
The `fix-bls` stage supports a `prefix` argument, which was not
supported in composer. Specifying this argument is necessary in case the
`/boot` mountpoint is on a separate partition.

Add the `prefix` argument to the `fix-bls` stage. Amend unit tests.

The RHEL-8.5 and RHEL-9.0 `aarch64` `ec2` and `ami` images use partitioning
with `/boot` on a separate partition. Due to this, the pipeline must specify
a non-default prefix to the `fix-bls` stage.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-13 08:24:37 +02:00
Tomas Hozza
7ac1d68fb6 rhel85/90: allow specifying the boot partition for the grub2 stage
Specifying the boot partition filesystem UUID in grub2 stage is required
in case the `/boot` mountpoint is on a separate partition. This is the
case of RHEL-8.5 and RHEL-9.0 `ami` and `ec2` images.

Extend `disk.PartitionTable` with a new `BootPartition` method, which
returns a pointer to partition with FS mountpoint `/boot` if there is
such partition, or `nil` otherwise.

Extend the RHEL-8.5 and RHEL-9.0 code creating options structure for
grub2 osbuild stage to include the boot partition in case it has been
provided.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-13 08:24:37 +02:00
Tomas Hozza
a9c280a2fc rhel85/90: use random FS UUID for /boot partition on ec2 and ami
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-13 08:24:37 +02:00
Tomas Hozza
4a83c0596f rhel85/90: don't use the same part UUID for /boot and / on ec2 images
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-13 08:24:37 +02:00
Tomas Hozza
aa4886670a rhel85/90: fix FS type for /boot on ami and ec2 images
The `/boot` partition had incorrect FS type `EFI System partition`,
instead of `Linux filesystem data`. Fix this.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-13 08:24:37 +02:00
Ondřej Budai
3d7345d445 distro/rhel90: temporarily remove some packages from tar-installer
They don't exist in RHEL 9.0, we need to investigate.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00
Ondřej Budai
9e4e734cd3 distro/rhel90: temporarily remove nss-altfiles from edge-commit
It isn't shipped in RHEL 9 yet, see:
https://bugzilla.redhat.com/show_bug.cgi?id=1985020

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00
Ondřej Budai
20ef62c3d1 distro/rhel90: add kernel-cmdline stage to all image types
Otherwise, kernel-install will just pick the cmdline from /proc/cmdline
that is actually the host's one. This way, I managed to leak the cmdline
from my Fedora running on btrfs to RHEL 9 image which led to a very weird
results.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00
Ondřej Budai
374ac58f51 distro/rhel90: drop dhcp-client from the package set
It wasn't in the previous definition.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00
Ondřej Budai
77306faeda distro/rhel90: drop net-tools from the package set
It wasn't in the previous definition.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00
Ondřej Budai
7f039e7a7c distro/rhel90: do not exclude fwupd from qcow2
Otherwise, the image type cannot be depsolved because it's hard dependency
of kernel.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00
Ondřej Budai
9bbacb885c distro/rhel90: drop dracut-norescue from package sets
It's no longer shipped in rhel-90, its functionality is now just a part
of the dracut package.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00
Ondřej Budai
c1acdcc82b distro/rhel90: drop python36 from the basic packageset
python36 is not shipped in RHEL 9, RIP.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00
Ondřej Budai
afa267cebc distro: fix distro mangling for rhel-90
We want to also mangle RHEL 9 in the same style as we do 8.4+.

RHEL 8.0 => rhel-80
RHEL 8.1 => rhel-81
etc

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00
Ondřej Budai
7b03db05c3 distro/rhel90: drop rhel86 alias
It doesn't ever make sense to alias 90 as 86.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00
Ondřej Budai
6693ebc59b distro/rhel90: rename everything from 85 to 90
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00
Ondřej Budai
d6ade6386d distro/rhel90: copy everything from rhel85
rm internal/distro/rhel90/*
cp internal/distro/rhel85/* internal/distro/rhel90/

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00
Achilleas Koutsou
280b06381f distro/rhel85: bare metal package set
Used by the image-installer. The package set is intended to replicate a
default RHEL installation, but may need some changes.
2021-08-11 01:31:47 +02:00
Achilleas Koutsou
ab264abd76 distro/rhel85: tar-installer -> image-installer
Renamed tar-installer to image-installer.
This is a more appropriate name:
- It disassociates the image type from the "tar" image type. The two
  should not be perceived to be connected.
- It's more descriptive. The format of the payload (tar) isn't relevant
  to the purpose of the image type.
2021-08-11 01:31:47 +02:00
Achilleas Koutsou
48466ae440 distro/rhel85: offset first partition for ppc
First partition for ppc64le starts at 2048.
2021-08-10 22:38:36 +02:00
Martin Sehnoutka
130e357268 internal/rhsm: implement fallback to the previous mechanism
It turns out there are edge cases where the previous mechanism worked
and the new one doesn't. Employee subscription is one example, where the
key can be used to access basically any content, yet nothing it written
in the redhat.repo file. This should have no effect on hosts running
RHSM the usual way.
2021-08-09 12:40:23 +02:00
Martin Sehnoutka
1227633e7c internal/rpmmd: return helpful error message when subscriptions == nil
It can happen that the system is not subscribed and the user requests a
source with rhsm set to "true". Return useful error message in such case
informing the user what to do about it.
2021-08-09 12:40:23 +02:00
Martin Sehnoutka
ac51c5087f internal/distro: make it possible to use rhsm with fedora
It should be possible to use rhsm with fedora repos as well.
2021-08-09 12:40:23 +02:00
Martin Sehnoutka
1ada606ed8 internal/rhsm: introduce package that handles subscriptions
The problem: osbuild-composer used to have a rather uncomplete logic for
selecting client certificates and keys while fetching data from
repositories that use the "subscription model". In this scenario, every
repo requires the user to use a client-side TLS certificate. The problem
is that every repo can use its own CA and require a different pair of
a certificate and a key. This case wasn't handled at all in composer.

Furthermore, osbuild-composer can use remote workers which complicates
things even more.

Assumptions: The problem outlined above is hard to solve in the general
case, but Red Hat Subscription Manager places certain limitations on how
subscriptions might be used. For example, a subscription must be tight to
a host system, so there is no way to use such a repository in osbuild-composer
without it being available on the host system as well.

Also, if a user wishes to use a certain repository in osbuild-composer it
must be available on both hosts: the composer and the worker. It will come
with different pair of a client certificate and a key but otherwise, its
configuration remains the same.

The solution: Expect all the subscriptions to be registered in the
/etc/yum.repos.d/redhat.repo file. Read the mapping of URLs to certificates
and keys from there and use it. Don't change the manifest format and let
osbuild guess the appropriate subscription to use.
2021-08-09 12:40:23 +02:00
Achilleas Koutsou
fe19d5a1d4 distro/rhel85: prepend kernel cmdline stage
For s390x, prepend a kernel cmdline stage to the start of the OS
pipeline. This is a noop for other architectures for now.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-08-06 07:30:05 +02:00
Achilleas Koutsou
844f77f7ca distro/rhel85: add zipl.inst stage for s390x
Installs the zipl bootloader on the disk image.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-08-06 07:30:05 +02:00
Achilleas Koutsou
0ed4f057c2 osbuild2: zipl install stage
New stage: org.osbuild.zipl.inst

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-08-06 07:30:05 +02:00
Achilleas Koutsou
22faa79880 distro/rhel85: bootloaderInstStage() function
Returns a single stage that installs the bootloader.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-08-06 07:30:05 +02:00
Achilleas Koutsou
38e88daea6 osbuild2: Don't add nil stages
Do nothing if a nil value is passed to pipeline.AddStage().

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-08-06 07:30:05 +02:00