Commit graph

4973 commits

Author SHA1 Message Date
Xiaofeng Wang
422e70c8ce test: Update nightly compose URL to updates URL
Since RHEL 9.1 and 8.7 released, the nightly compose URL moved to
update URL, like /rhel-8/nightly/updates/RHEL-8/latest-RHEL-8.7.0/
2022-11-25 03:47:30 +01:00
Xiaofeng Wang
f732af9e8b test: Do not enable UEFI secure boot on CS9 2022-11-25 03:47:30 +01:00
schutzbot
1c130ef6ed schutzfile: Update snapshots to 20221115 2022-11-25 03:47:30 +01:00
Simon de Vlieger
3443484502 image: image_installer, use gzip compression 2022-11-24 17:59:56 +01:00
Brian C. Lane
0b864c4306 store: Add blueprints to the stored changes
Previously the blueprint change was only present until a reboot. The
change and its version was recorded, but the actual blueprint was not
saved.

This adds saving the blueprint when it is changed, and loading the old
blueprint versions when they are present in the store.

No version change is needed, if the blueprint is not present it is
loaded as an empty blueprint which was already being detected by
checking the length of the Name.

This will allow specific blueprint changes to be recalled after a
restart of the server.

This also includes tests for the new behavior.
2022-11-24 11:19:14 +01:00
Brian C. Lane
088ca6ec72 client: Add GetBlueprintChangeV1
Add a function to recall a specific blueprint change. Also includes
tests.
2022-11-24 11:19:14 +01:00
Brian C. Lane
e2011652e2 weldr: Add test for new /blueprints/change/... route 2022-11-24 11:19:14 +01:00
Brian C. Lane
a4e361ccd2 weldr: Add /blueprints/change/NAME/COMMIT route
This allows clients to request a specific blueprint change. This can
then be used to locally implement diff support.
2022-11-24 11:19:14 +01:00
Antonio Murdaca
a295ed7c90 check FDO re-encryption happens
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2022-11-24 09:33:00 +00:00
Ondřej Budai
ccb3a67d94 ci: use the latest terraform config to fix missing images
cloud-cleaner deleted all our nightly and CS9 testing images, let's replace
them.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-11-24 09:02:29 +01:00
Achilleas Koutsou
5130be4ccc tools: silence version comparison in get_build_info()
The get_build_info() function uses nvrGreaterOrEqual() to determine how
to parse the output from composer-cli.  The function prints the result
of the comparison but is often used in a loop when waiting for a compose
to finish, which makes CI logs very annoying.

Silence the version comparison output when called form get_build_info().
2022-11-23 13:35:57 +01:00
Tomáš Hozza
f24c6ad952 SPEC: run the %preun commands in worker package only on removal
PR#553 [1] introduced a change to the worker subpackage and replaced
the systemd %preun RPM macro with a literal calls to `systemctl` due
to missing functionality in systemd. However, the change omitted the
fact, that the RPM macro executes commands only on package removal,
not upgrade.

As a result a local or remote worker which is running on the system
while the osbuild-composer-worker package is updated, gets stopped.

Fix the scriptlet and run the commands only on package removal.

[1] https://github.com/osbuild/osbuild-composer/pull/553

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-11-23 10:42:01 +01:00
Sarita Mahajan
dc3c0d9725 simplified-installer: do not mandate FDO section in simplified provisioning
- build simplified installer iso without mentioning FDO section.
  - change done for rhel8 and rhel9
  - add test case for this use case in test/case/ostree-simplified-installer.shovisioning
  - fixed review comments

Signed-off-by: Sarita Mahajan <sarmahaj@redhat.com>
2022-11-22 11:36:54 +01:00
Sanne Raymaekers
44f4225c02 rhsm: remove CA from consumer secrets
The `/etc/rhsm/ca/redhat-uep.pem` CA is not valid for consumer
certificates.

As a result resolving the ostree ref should use the system's CA cert
pool.
2022-11-21 10:27:15 +01:00
Achilleas Koutsou
6f37df7777 tools/define-compose-url: fix RHEL 9.1 compose URL
RHEL 9.1 currently doesn't have a 'latest' compose.  Use most recent
date-stamped build.
2022-11-21 09:45:26 +01:00
Simon de Vlieger
0e2f8bb115 image-installer: move conditionals into distro
Try and keep `image/image_installer.go` as clean as possible and instead
pass any customizations one layer up.
2022-11-21 09:45:26 +01:00
Simon de Vlieger
d4d100383e image: create installer image types for fedora
This adds the `fedora-image-installer` and
`fedora-image-installer-preview` images.

The image installer type installs anaconda-webui on Fedora >= 38 to use
the new UI. It also writes its setting to
`/usr/share/anaconda/interactive-defaults.ks` as the current
anaconda-webui has not yet been tested in kickstart mode.

To do so manifest.Anaconda was expanded to take a (subset) of options
for a KickstartStage which is will write into interactive-defaults.ks.
And to take a list of additional modules to enable, so we can set up
Anaconda with all default modules.
2022-11-21 09:45:26 +01:00
Christian Kellner
9ea58d1486 disk: align LVM2 volumes to the extent size
When the size of a logical volume is not aligned to the extent size of
the volume group, LVM2 will automatically align it by rounding up[1]:
	Rounding up size to full physical extent 29.80 GiB
	Rounding up size to full physical extent <3.82 GiB

Since we don't take that into account when we create a new volume or
set the size of an existing one, the size for the whole volume group
will be short by that amount and thus the creation of the last volume
will fail:
  	Volume group <uuid> has insufficient free space (975 extents): 977 required.

To fix this a new `AlignUp` method is added to the `MountpointCreator`
creator interface. It will align a given size to the requirements of
the implementing container, like e.g. `LVMVolumeGroup`. It is then
used by a new `alignEntityBranch` which takes a size and walks the
entity path, calling `AlignUp` for all entities that implement said
`MountpointCreator` interface; thus the resulting size should fullfil
the alignment requirement for all elements in the path.
NB: `PartitionTable` already had an `AlignUp` method.

Add a corresponding test.

[1]: 8686657664/lib/metadata/metadata.c (L1072)

Co-authored-by: Achilleas Koutsou <achilleas@koutsou.net>
2022-11-18 18:44:12 +01:00
Christian Kellner
93875576e9 disk/lvm: split up CreateMountpoint
Extract a `CreateLogicalVolume` method from `CreateMountpoint`
and implement the latter via the former. This makes it possible
to create a Logical Volume for an existing payload.
2022-11-18 18:44:12 +01:00
Brian C. Lane
a1a3e5ba2d cloudapi: Add azure-sap-rhui support to cloudapi
and Update openapi.v2.gen.go
by running `./tools/prepare-source.sh` which runs `go generate ./...`
among other things.
2022-11-18 16:53:22 +01:00
Brian C. Lane
a00cde7424 test: Add manifest for RHEL 8 azure-sap-rhui image
Generated with:
go run ./cmd/gen-manifests -output ./test/data/manifests -cache /var/tmp/manifest-cache/

Includes some changes to ec2-sap manifests due to using the same repos.
2022-11-18 16:53:22 +01:00
Brian C. Lane
f9b3f95615 distro: Install ansible for sap on 8.5 as well 2022-11-18 16:53:22 +01:00
Brian C. Lane
e4f7e94dd0 distro: Use GibiByte constant for Azure 64GiB Images
instead of using 68719476736, use 64 * common.GibiByte
2022-11-18 16:53:22 +01:00
Brian C. Lane
12c7f12d16 tools: Add new azure-sap-rhui image type
It needs to be listed in format-request-map.json, and the repo snapshots
required to depsolve the packages need to be added to repos.json

NOTE: The snapshots are from https://github.com/osbuild/rpmrepo/
2022-11-18 16:53:22 +01:00
Brian C. Lane
8c07d65fe0 distro: Add azure-sap-rhui image type to RHEL8 2022-11-18 16:53:22 +01:00
Brian C. Lane
a47c239d11 distro: Move alsa-lib out of azureCommonPackageSet Exclude 2022-11-18 16:53:22 +01:00
Brian C. Lane
010d92eed1 distro: Separate out RHEL8 SAP package set into its own function
This also moves cloud specific client packages into that cloud's
PackageSet so that the SAP packages are common to all users.
2022-11-18 16:53:22 +01:00
Brian C. Lane
ed7aaec12d distro: Move RHEL 8 SAP config data into sap.go
This can be shared between cloud providers so move it out of the EC2 SAP
config into its own file and drop the X86_64 from the name (there is
nothing arch specific in it, even if it is only ever used on X86).
2022-11-18 16:53:22 +01:00
Sanne Raymaekers
bff6403b37 cloudapi/v2: pass rhsm requirement to ostree resolve job 2022-11-18 13:07:36 +01:00
Sanne Raymaekers
28319f50d8 worker: log failures in ostree resolve job 2022-11-17 12:55:02 +01:00
Ondřej Budai
422281698a packer: add a workaround for broken podman from 9.1
Podman doesn't work when installed from 9.1 repositories on 9.0 image.

I found that upgrading the whole system helps. Sadly, that requires removing
python-unversioned-command for some reason.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-11-16 22:31:43 +01:00
schutzbot
5669e05264 Post release version bump
[skip ci]
2022-11-16 08:29:16 +00:00
Tomáš Hozza
587e043668 distro/RHEL-9: exclude unwanted packages from the EC2-SAP image
Exclude unwanted packages from the EC2-SAP image. These packages have
been pulled into RHEL-9 image due to the fact that we moved away
from using `@core` package group by default and as a result we dropped
explicit package excludes. However the SAP image includes the
`@Server` package group, which pulls in these unwanted packages, thus
we need to explicitly exclude them in the SAP package set.

Related to COMPOSER-1829

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-11-15 12:04:42 +01:00
Tomáš Hozza
61478254cd distro/RHEL-9: disable amdgpu module on all AWS images
`amdgpu` module is causing error to be printed in the system log on AWS
instances. After investigation, it turns out that it is not needed.
Disable it by default on all AWS images.

Related to COMPOSER-1807

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-11-15 12:04:42 +01:00
Tomáš Hozza
a930073e4a distro/RHEL-8: disable amdgpu module on all AWS images
`amdgpu` module is causing error to be printed in the system log on AWS
instances. After investigation, it turns out that it is not needed.
Disable it by default on all AWS images.

Related to COMPOSER-1807

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-11-15 12:04:42 +01:00
Brian C. Lane
050403fbe4 diff-manifests.sh: Use shared_lib for greenprint and redprint 2022-11-15 12:00:00 +01:00
Paul Whalen
560905108c fedora/package_sets: Update sil-scheherazade-new-fonts
F37 no longer ships sil-scheherazade-fonts, but
sil-scheherazade-new-fonts instead. Let's change this. The repos for
test manifests must have been updated in order to get the new package.

Co-authored-by: Ondřej Budai <ondrej@budai.cz>
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-11-14 12:01:47 +01:00
Ondřej Budai
7e97c6475e distro/fedora: print an error if PackageSets cannot initialize a manifest
Previously, this just happened silently and let to extremely odd errors. Let's
just print the error to simplify debugging the next time.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-11-11 18:39:18 +01:00
Ondřej Budai
194351e681 distro/fedora: fix iot-installer package set generation
8fdd158799 modified the Cloud API to resolve
ostree commits using a separate job. This change caused the API handler
to call PackageSets without any ostree options (because they are not resolved
yet).

Unfortunately, the new implementation of PackageSets initializes the manifest.
The initialization checks the options and if the type is iot-installer and
it doesn't have the fetch checksum for IoT, it just returns an error.

To work around this (we need an initialized manifest to create the chains),
this commit just gives the initialization method a dummy checksum. The ostree
options currently don't have any effect on the package sets, so this should
be fine.

In order to make this workaround at least slightly sane, a warning is printed,
there's a new test just for this behaviour and a long comment to remember to
delete these lines.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-11-11 18:39:18 +01:00
Thomas Lavocat
4e7b021514 ci: add tags to AWS instances
The new terraform runners support an extended lists of tags to get a
greater precision on the stats we can extract from AWS cost center.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-11-10 20:56:21 +01:00
Tomáš Hozza
754a9b59b3 tools/provision.sh: copy RHEL repo overrides using wildcard
Don't list specific RHEL repos which should be copied over to the repo
overrides directory, but instead copy all of them, similar to how it's
done for Fedora and CentOS Stream. This is less error prone when support
for a new RHEL release is added to composer.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-11-09 20:25:01 +01:00
Brian C. Lane
ff823a93e1 distro: Separate out RHEL9 SAP package set into its own function
This also moves cloud specific client packages into that cloud's
PackageSet so that the SAP packages are common to all users.
2022-11-09 20:17:10 +01:00
Brian C. Lane
b785af25ed distro: Move RHEL 9 SAP config data into sap.go
This can be shared between cloud providers so move it out of the EC2 SAP
config into its own file and drop the X86_64 from the name (there is
nothing arch specific in it, even if it is only ever used on X86).
2022-11-09 20:17:10 +01:00
Simon de Vlieger
f6bd62e838 distro: SELinux should be the last stage
SELinux relabels the entire filesystem tree and should always be the
last stage in the os pipelines.
2022-11-09 15:41:40 +01:00
Brian C. Lane
d73171fdde Use go install instead of deprecated go get
In go 1.18 'go install' needs to be used to install an executable. See
https://go.dev/doc/go-get-install-deprecation and
https://go.dev/doc/manage-install
2022-11-09 12:08:43 +01:00
Simon de Vlieger
1612d9cb80 ci: add my SSH keys to the CI ssh keys 2022-11-09 11:31:57 +01:00
Simon de Vlieger
a7f1dd4065 docker-compose: remove unavailable --dnf-json 2022-11-09 11:28:56 +01:00
Sanne Raymaekers
0a9cf9b6a7 dbjobqueue: check context errors after trying to dequeue
This fixes a race condition where the context might have been canceled
or timed out in between the preliminary check and trying to dequeue, and
consequently returning the wrong error.

Instead of doing the preliminary check, just check for the context
errors when trying to dequeue.
2022-11-08 07:37:32 -05:00
Sanne Raymaekers
26b8e2ff6e dbjobqueue: acquire a new connection for each listen query
This fixes a bug where the listen function would keep trying to use a
closed, unrecoverable connection to listen for a notification. This
continued failing, which essentially made the queue instance useless.
2022-11-08 07:37:32 -05:00
Tomáš Hozza
63f1b8d9fb internal/disk: fix potential nil pointer dereference
The `LVMVolumeGroup.Clone()` method could end up dereferencing a `nil`
pointer in the `lv` variable, if there would be a `nil` logical volume
in the LVM volume group. Such situation would be an error of its own.
There is no point in checking if the cloned logical volume is not `nil`
and casting it to another variable. The logic should check if the cloned
logical volume is `nil` and panic in such situation. The following code
can then cast the clone to a different variable without issues and there
is no risk of dereferencing a `nil` pointer.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-11-07 16:10:41 +01:00