Commit graph

1037 commits

Author SHA1 Message Date
Achilleas Koutsou
a9efe3ebb9 test: regenerate RHEL 9.x manifests
All RHEL 9.x manifests have been affected except edge-installer,
edge-raw-image, and edge-simplified-installer.  These are image types
that carry a pre-built payload so don't have an OS pipeline and don't
include RHSM facts.
2022-11-30 10:04:16 +01:00
Achilleas Koutsou
4bd2c0cb1d test: regenerate manifests
Regenerate manifests with the added RHSM fact.
RHEL 9 manifests haven't changed because the stage is missing in the OS
pipeline (bug).
2022-11-30 10:04:16 +01:00
Achilleas Koutsou
7da0cde855 test: regenerate image-installer and edge-installer manifests 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
d4a8775751 test: update installer manifests
- Added users module.
- Set squashfs compression to xz for RHEL 9.x and CentOS Stream 9.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
ff494d66f6 test: update RHEL 9.x and CS9 tar manifests 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
c4af0a1886 test: update RHEL 9.x and CS9 image-installer manifests 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
3400c27153 test: update manifests with containers
Update test manifests that have embedded containers with the new info.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
5c8b5995bf test: update manifests with embedded containers
Updated manifests (and generated new ones) that embed a container:
- edge-commit on RHEL 9.x and CS9
- qcow2 on RHEL 9.1
- iot-commit on all Fedora versions (new)
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
5086f1e413 test: update RHEL 9.x and CS9 gce manifests
Regenerate manifests for the gce and gce-rhui image types with the
rewritten definitions:
- remove unnecessary RPMs in the build root
  artifact
- some stages changed order, but the pipelines be functionally
  equivalent
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
1a0a16bc46 test: update RHEL 9.x and CS9 azure and vhd manifests
Regenerate manifests for the Azure and VHD image types with the
rewritten definitions:
- remove unnecessary RPMs in the build root
- changed order of rhsm stage: this should have no effect functionally
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
f3e43a01bc test: update RHEL 9.x and CS9 manifests
Adding python3-pyyaml to the build root for manifests that contain
cloud-init in the OS pipeline.
Adding python3-iniparse to the build root for manifests that contain
rhsm or dnf.config in the OS pipeline.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
b0574daa90 test: update RHEL 9.x ec2 and ec2-ha manifests
Regenerate manifests for the EC2 and EC2-HA image types with the
rewritten definitions:
- remove unnecessary RPMs in the build root
- added dofstools to the image
- internal raw image filename changed: this has no effect on the final
  artifact, since it's compressed and renamed before exporting.
- rhsm stage order changed.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
c5bd001071 test: update RHEL 9.x and CS9 ami manifests
Regenerated manifests for the AMI image type with the rewritten
definitions:
- removed RPMs in the build root: packages are added to the build-root
  on-demand by the payload pipelines.
- added dosfstools to the image: our new pipelines add fs tools for
  every filesystem type that appears in the partition table.
- rhsm stage order changed.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
a2a3ac8ffe test: regenerate manifests with chrony stages
Regenerate all manifests that used the old chrony stage options with the
'timeservers' field.  This is now replaced by the 'servers' field, which
is a slice of objects with a 'hostname' field instead of a slice of
strings.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
0ba1a5ff73 test: update RHEL 9.x and CS9 vmdk manifests
Regenerated manifests for the vmdk image type with the rewritten
definitions:
- removed RPMs in the build root: packages are added to the build-root
  on-demand by the payload pipelines.  All removed package were
  unnecessary for building the image.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
88ff103621 test: update RHEL 9.x and CS9 edge manifests
Regenerated manifests for the edge image types with the rewritten
definitions:
- removed RPMs in the build root: packages are added to the build-root
  on-demand by the payload pipelines.  All removed package were
  unnecessary for building the image.
- pipeline names changed.
- rpm stage for os pipeline now sets dbpath and ostree-booted option
  (see [1] for original change in Fedora).
- installer pipelines changed to no longer use bootiso.mono stage, but
  instead break it down to component stages (see [2] for original change
  in Fedora).

Fedora IoT manifests updated to include selinux labelling changes.

The skopeo and containers storage stage have been temporarily dropped
since the new pipeline generator functions don't support it yet.

[1] https://github.com/osbuild/osbuild-composer/pull/2914
[2] https://github.com/osbuild/osbuild-composer/pull/3020
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
9d34362bbf test: update RHEL 9.x and CS9 qcow2, oci, and openstack manifests
Regenerated manifests for the QCOW2, OCI, and openstack image types with
the rewritten definitions.

The skopeo stage is not yet added by the live image pipeline.

The following changes in the manifests should affect the final image:
- Removed RPMs in the build root: packages are added to the build-root
  on-demand by the payload pipelines.  All removed packages were
  unnecessary for building the image.
- /usr/bin/tar removed form selinux stage: tar should be added on-demand
  if it's needed by any of the payload pipelines.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
e9746bed32 test: regenerate all edge-commit and qcow2 manifests
Create new manifests for the new edge-commit-with-container
configuration and the qcow2 for RHEL 9.1
2022-11-28 17:20:49 +01:00
fkolwa
401a59b6e4 distro/rhel7: Add insights-client to Azure package set
Add insights-client package to azureRhuiCommonPackageSet conditionally
Regenerate test cases for RHEL 7 azure-rhui
2022-11-25 13:15:49 +01:00
Xiaofeng Wang
f732af9e8b test: Do not enable UEFI secure boot on CS9 2022-11-25 03:47:30 +01:00
Simon de Vlieger
3443484502 image: image_installer, use gzip compression 2022-11-24 17:59:56 +01:00
Antonio Murdaca
a295ed7c90 check FDO re-encryption happens
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2022-11-24 09:33:00 +00:00
Achilleas Koutsou
5130be4ccc tools: silence version comparison in get_build_info()
The get_build_info() function uses nvrGreaterOrEqual() to determine how
to parse the output from composer-cli.  The function prints the result
of the comparison but is often used in a loop when waiting for a compose
to finish, which makes CI logs very annoying.

Silence the version comparison output when called form get_build_info().
2022-11-23 13:35:57 +01:00
Sarita Mahajan
dc3c0d9725 simplified-installer: do not mandate FDO section in simplified provisioning
- build simplified installer iso without mentioning FDO section.
  - change done for rhel8 and rhel9
  - add test case for this use case in test/case/ostree-simplified-installer.shovisioning
  - fixed review comments

Signed-off-by: Sarita Mahajan <sarmahaj@redhat.com>
2022-11-22 11:36:54 +01:00
Simon de Vlieger
d4d100383e image: create installer image types for fedora
This adds the `fedora-image-installer` and
`fedora-image-installer-preview` images.

The image installer type installs anaconda-webui on Fedora >= 38 to use
the new UI. It also writes its setting to
`/usr/share/anaconda/interactive-defaults.ks` as the current
anaconda-webui has not yet been tested in kickstart mode.

To do so manifest.Anaconda was expanded to take a (subset) of options
for a KickstartStage which is will write into interactive-defaults.ks.
And to take a list of additional modules to enable, so we can set up
Anaconda with all default modules.
2022-11-21 09:45:26 +01:00
Brian C. Lane
a00cde7424 test: Add manifest for RHEL 8 azure-sap-rhui image
Generated with:
go run ./cmd/gen-manifests -output ./test/data/manifests -cache /var/tmp/manifest-cache/

Includes some changes to ec2-sap manifests due to using the same repos.
2022-11-18 16:53:22 +01:00
Tomáš Hozza
587e043668 distro/RHEL-9: exclude unwanted packages from the EC2-SAP image
Exclude unwanted packages from the EC2-SAP image. These packages have
been pulled into RHEL-9 image due to the fact that we moved away
from using `@core` package group by default and as a result we dropped
explicit package excludes. However the SAP image includes the
`@Server` package group, which pulls in these unwanted packages, thus
we need to explicitly exclude them in the SAP package set.

Related to COMPOSER-1829

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-11-15 12:04:42 +01:00
Tomáš Hozza
61478254cd distro/RHEL-9: disable amdgpu module on all AWS images
`amdgpu` module is causing error to be printed in the system log on AWS
instances. After investigation, it turns out that it is not needed.
Disable it by default on all AWS images.

Related to COMPOSER-1807

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-11-15 12:04:42 +01:00
Tomáš Hozza
a930073e4a distro/RHEL-8: disable amdgpu module on all AWS images
`amdgpu` module is causing error to be printed in the system log on AWS
instances. After investigation, it turns out that it is not needed.
Disable it by default on all AWS images.

Related to COMPOSER-1807

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-11-15 12:04:42 +01:00
Brian C. Lane
050403fbe4 diff-manifests.sh: Use shared_lib for greenprint and redprint 2022-11-15 12:00:00 +01:00
Paul Whalen
560905108c fedora/package_sets: Update sil-scheherazade-new-fonts
F37 no longer ships sil-scheherazade-fonts, but
sil-scheherazade-new-fonts instead. Let's change this. The repos for
test manifests must have been updated in order to get the new package.

Co-authored-by: Ondřej Budai <ondrej@budai.cz>
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-11-14 12:01:47 +01:00
Simon de Vlieger
f6bd62e838 distro: SELinux should be the last stage
SELinux relabels the entire filesystem tree and should always be the
last stage in the os pipelines.
2022-11-09 15:41:40 +01:00
Brian C. Lane
d73171fdde Use go install instead of deprecated go get
In go 1.18 'go install' needs to be used to install an executable. See
https://go.dev/doc/go-get-install-deprecation and
https://go.dev/doc/manage-install
2022-11-09 12:08:43 +01:00
Tom Gundersen
626530818d worker/server: requeue unresponsive jobs
If a job is unresponsive the worker has most likely crashed or been shut
down and the in-progress job been lost.

Instead of failing these jobs, requeue them up to two times. Once a job is lost
a third time it fails. This avoids infinite loops.

This is implemented by extending FinishJob to RequeuOrFinish job. It takes a
max number of requeues as an argument, and if that is 0, it has the same
behavior as FinishJob used to have.

If the maximum number of requeues has not yet been reached, then the running
job is returned to pending state to be picked up again.
2022-11-02 15:26:00 +01:00
Tomáš Hozza
a001da966a distro: add support for RHEL-9.2
Add relevant repo definitions and generate image test manifests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-10-31 08:07:09 +01:00
Tomáš Hozza
fc95dac86a distro: add support for RHEL-8.8
Add relevant repo definitions and generate image test manifests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-10-31 08:07:09 +01:00
Tomáš Hozza
5a0d286d6b test/api: don't provide Resource Group Location when testing Azure
Let's default to not providing the Resource Group Location when testing
Cloud API with Azure and leave it up to the implementation to determine
the correct location to use.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-10-27 19:33:43 +02:00
Alexander Todorov
1778d84d68 Remove azure/main.tf file b/c not actually used 2022-10-24 10:16:01 +02:00
Alexander Todorov
699812d70b tests: Update the version of azurerm terraform provider 2022-10-24 10:16:01 +02:00
Sanne Raymaekers
8fdd158799 cloudapi/v2: use the ostree resolve job to resolve ostree refs 2022-10-19 18:14:10 +02:00
Tomáš Hozza
ae04c56c32 distro/rhel9: adjust RHSM-related config on EC2 9.1+ images
The EC2 images starting with 9.1 should:
 - not configure RHSM using osbuild
 - install `redhat-cloud-client-configuration` package which ships the
   RHSM configuration.

Regenerate affected image manifests.

Related to COMPOSER-1805

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-10-19 17:07:36 +02:00
Tomáš Hozza
b65bb9491f distro/rhel8: adjust RHSM-related config on EC2 8.7+ images
The EC2 images starting with 8.7 should:
 - not configure RHSM using osbuild
 - install `redhat-cloud-client-configuration` package which ships the
   RHSM configuration.

Regenerate affected image manifests

Related to COMPOSER-1804.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-10-19 17:07:36 +02:00
Tomáš Hozza
10432d6103 tools: update repos used to generate 8.7 and 9.1 image manifests
Also drop repos for RHEL-9.0 Beta, which are not used any more.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2022-10-19 17:07:36 +02:00
Antonio Murdaca
50fb688d43 Use FDO AIO server for testing
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-10-19 12:08:34 +02:00
Tomáš Hozza
dc476671e4 weldr/upload/gcp: make Object optional
Previously, it was expected from the user to provide the Object name
when uploading image to GCP. The object name does not matter much,
because the object is deleted once image import finishes. Make
the specification of the object name optional and generate it if not
provided.

Adjust the GCP Weldr test case to not provide the Object name when
uploading the image.

The user can still provide the Object name if needed.
2022-10-11 13:23:18 +02:00
Tomáš Hozza
809107cd45 cloudapi: make Bucket optional in GCPUploadOptions
GCP Bucket to use can be now configured in the worker configuration.
Make the `Bucket` optional in the Cloud API when uploading image to GCP.

Adjust the Cloud API test case to configure GCP Bucket on the worker and
not provide it in the API request.
2022-10-11 13:23:18 +02:00
Achilleas Koutsou
7ebe3f550c distro/rhel: set ostree sysroot.readonly to false
We used to always set the sysroot.readonly setting to true, but this
never worked because of a bug in osbuild [1].
The bug is now fixed and the RHEL and CentOS edge-raw images are crated
with sysroot.readonly = true, and the images aren't booting.

Fixing the option to false.  This changes the manifests, but not the
generated images because of the change in osbuild.

If sysroot is meant to be readonly, we will change it in a future
update.

[1] https://github.com/osbuild/osbuild/pull/1129
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
55b1e2b6ee test: only set boot args in virt-install version condition
Make it clearer that the only difference between the two cases are the
arguments for --boot.
2022-10-11 10:00:22 +02:00
Achilleas Koutsou
58eeb74c98 test: update raw image script to skip BIOS boot on Fedora 2022-10-11 10:00:22 +02:00
Achilleas Koutsou
97dddd24a5 fedora: remove BIOS boot for IoT raw images 2022-10-11 10:00:22 +02:00