Commit graph

1463 commits

Author SHA1 Message Date
Achilleas Koutsou
4126f4e17a distro/rhel85: name param for xorrisofs input
It might not always be `bootiso-tree`.
2021-08-28 09:20:19 +02:00
Christian Kellner
3781cc0f47 distro/rhel85: make anaconda optional for dracut
Add an argument to `dracutStageOptions` so that additional dracut
modules can be supplied. Remove `anaconda` from the list of mods
that are included by default, and manually include it for the
anaconda based installer.

Co-Developed-by: Achilleas Koutsou <achilleas@koutsou.net>
Co-Developed-by: Antonio Murdaca <runcom@linux.com>
2021-08-28 09:20:19 +02:00
Antonio Murdaca
10e4a853bd osbuild2: ostree options for the fstab stage
osbuild, since version 30, supports the ostree deployments for the
fstab stage.

Co-Developed-by: Achilleas Koutsou <achilleas@koutsou.net>
Co-Developed-by: Christian Kellner <christian@kellner.me>
2021-08-28 09:20:19 +02:00
Achilleas Koutsou
a98be74135 osbuild2/ostree.preptree: add missing options
Add the missing options for the `org.osbuild.ostree.preptree` stage.
2021-08-28 09:20:19 +02:00
Christian Kellner
cda1884de3 osbuild2: files input is a valid copy stage input
Mark the `FilesInput` as valid input for the copy stage input so it
can be used in the constructor helper function(s).

Co-Developed-by: Achilleas Koutsou <achilleas@koutsou.net>
Co-Developed-by: Antonio Murdaca <runcom@linux.com>
2021-08-28 09:20:19 +02:00
Christian Kellner
d27596b24a osbuild2: make inputs generic for copy stage ctor
Add a level of indirection for the copy stage constructor function
in order to be able to use it with existing input types, like the
files input.

Co-Developed-by: Achilleas Koutsou <achilleas@koutsou.net>
Co-Developed-by: Antonio Murdaca <runcom@linux.com>
2021-08-28 09:20:19 +02:00
Achilleas Koutsou
2a98885953 osbuild2: OSTreeRemote → OSTreeSourceRemote
Properly name the ostree remote source struct.
2021-08-28 09:20:19 +02:00
Christian Kellner
f08fa8c5ca distro/rhel85: add install option for grub2 stage
Add a new `install` option to the `grub2StageOptions` which directly
translates to the `uefi.install` option for the grub2 stage. This
should be false for all existing pipelines, where RPM directly is
used to fill `/boot/efi/EFI/<vendor>`.

Co-Developed-by: Achilleas Koutsou <achilleas@koutsou.net>
Co-Developed-by: Antonio Murdaca <runcom@linux.com>
2021-08-28 09:20:19 +02:00
Antonio Murdaca
4fdf4dbd01 osbuild2: add stages for raw images and grub iso
Add the new stages included in osbuild 30 that can be used to
deploy an OSTree commit as well as the grub iso stage that
is used to configure grub2 (efi) for isos.
Bump the spec file requirement accordingly.

Co-Developed-by: Achilleas Koutsou <achilleas@koutsou.net>
Co-Developed-by: Christian Kellner <christian@kellner.me>
2021-08-28 09:20:19 +02:00
Achilleas Koutsou
14b5e98be8 osbuild2/xorrisofs: make Boot section optional
Should be a pointer otherwise it creates an empty object in the manifest
which makes the stage fail.

Co-Developed-by: Christian Kellner <christian@kellner.me>
2021-08-28 09:20:19 +02:00
Achilleas Koutsou
51bb4ea3a1 distro/rhel85: grub2 prefix path based on partition
If the partition table has a /boot partition, the grub2 prefix path
should be /grub2.
2021-08-28 09:20:19 +02:00
Achilleas Koutsou
39b0cf39d6 osbuild2: add support for LVM stages
Stage options implement a validate() method for checking values against
schema constraints.
2021-08-28 09:20:19 +02:00
Christian Kellner
df30073d65 distro/rhel85: pre-load uid/gid database for edge
When building RHEL for Edge commits and a parent together with an
URL was specified, add a `org.osbuild.ostree.passwd` stage which
then will pre-load the uid/gid database with the data from the
parent commit. This ensures that uids and gids do not change for
the "child" commit.
2021-08-27 12:56:54 +02:00
Christian Kellner
f8c8f28ac8 osbuild2: new org.osbuild.ostree.passwd stage
Add support for the `org.osbuild.ostree.passwd` and with it also
support for the `org.osbuild.ostree.checkout` input.
This stage can be used to pre-load the user and groups database
from an existing commit to ensure that uids/gids are stable.
2021-08-27 12:56:54 +02:00
Gianluca Zuccarelli
3b5783d8a0 distro: clean rhel90 image-installer package sets 2021-08-27 11:02:26 +02:00
Achilleas Koutsou
ca56714aa4 rhel85: use nginx with custom config for container
Running the container on Openshift requires that the process inside the
container run without special permissions.
Switching to nginx and setting the following options that don't require
root privileges:
- Port 8080 (> 1024)
- pid file in '/tmp' instead of the default '/run' path

Also, the log file is chmod-ed to be world writable. Nginx always writes
to the default log file on startup, even if a different log file path is
specified in the configuration.

See rhbz#1945238

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-08-27 09:29:28 +02:00
Achilleas Koutsou
a717a7245b osbuild2: new stages
org.osbuild.chmod: runs chmod on one or more files
org.osbuild.nginx.conf: write nginx config file

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-08-27 09:29:28 +02:00
Gianluca Zuccarelli
78e042b47a distro: rhel90 tar-installer image type 2021-08-26 01:52:03 +02:00
Jakub Rusz
37b14facd3 tests: update base tests for weldr-client
weldr-client returns different json structures than the original
composer-cli does. It is being replaced in RHEL-9 and these changes make
it possible to run the tests with both old composer-cli and
weldr-client.
2021-08-25 19:16:04 +02:00
Chloe Kaubisch
8c26614049 cloudapi: rework return values
Change instances of panic on error to return StatusInternalServerError
instead. Additionally change return code 200s to http.StatusOK.

Fixes #1606
2021-08-25 17:48:29 +02:00
Gianluca Zuccarelli
8374af3e6d distro: rhel90 custom filesystem support 2021-08-25 13:18:12 +02:00
Gianluca Zuccarelli
382b5370c0 distro: export common types
squash

unused type
2021-08-25 13:18:12 +02:00
Gianluca Zuccarelli
b782ecaf1f distro: rhel85 internal tests 2021-08-24 00:22:29 +02:00
Gianluca Zuccarelli
d4403a03c0 distro: rhel85 add sub mounts to allow list 2021-08-24 00:22:29 +02:00
Tomas Hozza
a8d503e5b9 RHEL-85/90: modify the x86_64 EC2 images dracut configuration
Change the x86_64-specific dracut configuration of RHEL-8.5 and RHEL-9.0
EC2 and AMI images to not include `xen-netfront` driver and add `nvme`
driver, which was previously not included. Since the configuration is no
longer Xen-specific, rename the configuration file to `ec2.conf`.

Justification:
There is no reason to put `xen-netfront` to initramfs as EC2 images don't
boot from network root. In addition, add `nvme` driver to handle the case
when initramfs is getting forcefully rebuild on a Xen instance (and not able
to boot on Nitro after that).

Related to https://issues.redhat.com/browse/COMPOSER-1096.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-23 16:01:11 +02:00
Christian Kellner
20258bd45e distro/rhel90: use unified grub config
Feodra 34 and thus RHEL 9 switched to a unified grub configuration,
which means that the main grub config is always located in the same
location, /boot/grub2/grub.cfg.[1] osbuild has used this scheme for
hybrid boot on x64 but not on pure efi systems like aarch64. The
new osbuild option `uefi.unified` was introduced to select that new
unified grug cfg scheme also for those, pure efi, systems. Use that.

[1] https://fedoraproject.org/wiki/Changes/UnifyGrubConfig
2021-08-23 13:48:26 +02:00
Christian Kellner
22bff1df71 osbuild2/grub2: add unified option for uefi
The `uefi.unified` option indicates whether the `org.osbuild.grub2`
will use the unified grub configration scheme[1] used by Fedora 34
and thus RHEL 9.
NB: This requires osbuild version >= 32.

[1] https://fedoraproject.org/wiki/Changes/UnifyGrubConfig
2021-08-23 13:48:26 +02:00
Christian Kellner
fcace40710 osbuild2/grub2: add install option for uefi
The `uefi.install` option indicates whether the `org.osbuild.grub2`
stage will copy the efi binaries from the build root to the `/boot`
directory in the tree.

Co-Developed-by: Achilleas Koutsou <achilleas@koutsou.net>
Co-Developed-by: Antonio Murdaca <runcom@linux.com>
2021-08-23 13:48:26 +02:00
Achilleas Koutsou
321dcc56bf distro/rhel90: copy devices and mounts changes
Rebase brings RHEL 9.0 definition with old Mounts and Devices. Changes
applied to new distro pipelines and stage options.
2021-08-21 13:39:10 +02:00
Achilleas Koutsou
745443181e osbuild2: remove stage-specific device types
Devices unlike stage options, shouldn't be stage specific.
There is only one type of device so far, the loopback device, which
is already defined as a separate type.

The top level Devices type is simply an alias to a Device map.

The mkfs stages require a single device with a specific key ("device").
These stages accept only one device in their NewStage() function for
convenience and create the Stage struct with the required key.

The zipl.inst stage requires a device labeled 'disk' as well as the rest
of the devices that correspond to each partition. The disk device is
passed to the New stage function separately and added to the Stage
devices with the required key.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-08-21 13:39:10 +02:00
Achilleas Koutsou
c74d13daf8 osbuild2: remove stage-specific mount types
Mounts unlike stage options, shouldn't be stage specific. We have
filesystem specific mount types, differentiated by their type string.
Mounts can define their own additional options if necessary.

The top level Mounts type is simply an alias to a Mount array.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-08-21 13:39:10 +02:00
Gianluca Zuccarelli
613ad0b862 disk: refactor partition table size & start points 2021-08-21 02:54:38 +02:00
Gianluca Zuccarelli
280e1b6e3d distro: rhel85 declarative partition tables 2021-08-21 02:54:38 +02:00
Gianluca Zuccarelli
e21eee273c disk: move partition creation to disk package 2021-08-21 02:54:38 +02:00
Gianluca Zuccarelli
87c176503a distro: rhel85 streamline to single function 2021-08-21 02:54:38 +02:00
Gianluca Zuccarelli
6c4da2b821 distro: rhel85 create partition helper functions 2021-08-21 02:54:38 +02:00
Gianluca Zuccarelli
346eef7477 distro: rhel85 validate /usr mountpoint size 2021-08-21 02:54:38 +02:00
Gianluca Zuccarelli
ad39aab854 distro: allow valid rhel85 mountpoint subdirs 2021-08-21 02:54:38 +02:00
Gianluca Zuccarelli
6abe88f36d distro: update rhel8.5 partition table creation 2021-08-21 02:54:38 +02:00
Gianluca Zuccarelli
dad535d295 distro: add mountpoint allow list for rhel8.5 2021-08-21 02:54:38 +02:00
sanne
9fab5def90 dbjobqueue: Reduce error noise in rollback check
If the transaction is already closed don't log the rollback failure as
an error, it means it was successfully committed.
2021-08-20 15:42:57 +02:00
Ondřej Budai
1e2ba4da64 upload/azure: use cheaper storage accounts
Previously, we used RAGRS which means that all our data was always replicated
to at least two regions for increased safety. This is cool but expensive, this PR
switches the API to use LRS that just uses one region.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-17 17:51:23 +02:00
Tomas Hozza
a8ac012ebe rhel90: fix a typo in RHBZ number in a comment
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-13 08:24:37 +02:00
Tomas Hozza
6edf4aad49 osbuild2: add prefix parameter to the fix-bls stage
The `fix-bls` stage supports a `prefix` argument, which was not
supported in composer. Specifying this argument is necessary in case the
`/boot` mountpoint is on a separate partition.

Add the `prefix` argument to the `fix-bls` stage. Amend unit tests.

The RHEL-8.5 and RHEL-9.0 `aarch64` `ec2` and `ami` images use partitioning
with `/boot` on a separate partition. Due to this, the pipeline must specify
a non-default prefix to the `fix-bls` stage.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-13 08:24:37 +02:00
Tomas Hozza
7ac1d68fb6 rhel85/90: allow specifying the boot partition for the grub2 stage
Specifying the boot partition filesystem UUID in grub2 stage is required
in case the `/boot` mountpoint is on a separate partition. This is the
case of RHEL-8.5 and RHEL-9.0 `ami` and `ec2` images.

Extend `disk.PartitionTable` with a new `BootPartition` method, which
returns a pointer to partition with FS mountpoint `/boot` if there is
such partition, or `nil` otherwise.

Extend the RHEL-8.5 and RHEL-9.0 code creating options structure for
grub2 osbuild stage to include the boot partition in case it has been
provided.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-13 08:24:37 +02:00
Tomas Hozza
a9c280a2fc rhel85/90: use random FS UUID for /boot partition on ec2 and ami
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-13 08:24:37 +02:00
Tomas Hozza
4a83c0596f rhel85/90: don't use the same part UUID for /boot and / on ec2 images
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-13 08:24:37 +02:00
Tomas Hozza
aa4886670a rhel85/90: fix FS type for /boot on ami and ec2 images
The `/boot` partition had incorrect FS type `EFI System partition`,
instead of `Linux filesystem data`. Fix this.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-13 08:24:37 +02:00
Ondřej Budai
3d7345d445 distro/rhel90: temporarily remove some packages from tar-installer
They don't exist in RHEL 9.0, we need to investigate.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00
Ondřej Budai
9e4e734cd3 distro/rhel90: temporarily remove nss-altfiles from edge-commit
It isn't shipped in RHEL 9 yet, see:
https://bugzilla.redhat.com/show_bug.cgi?id=1985020

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00