Commit graph

2179 commits

Author SHA1 Message Date
Achilleas Koutsou
afa07bc87b CI: disable tests on RHEL 8.5
Main image types aren't get supported for RHEL 8.5
Installer test needs more adjustments so disabling for now to keep CI
from blocking other work.
2021-07-02 17:57:22 +02:00
Gianluca Zuccarelli
ea5b7e113a fix: small typo in code comments 2021-07-02 10:38:04 +02:00
Gianluca Zuccarelli
9aa8ebdfff fix: unused function arg 2021-07-02 10:38:04 +02:00
Xiaofeng Wang
a0ec753e05 test: Update check_ostree.yaml to work with RHEL 8.5
There's no any error message in dmesg output on RHEL 8.5 Edge image.
Update playbook to support that.
2021-07-01 13:34:12 +02:00
Xiaofeng Wang
bc76f437a0 Pin osbuild to latest commit to fix bz#1975554 2021-07-01 13:34:12 +02:00
Xiaofeng Wang
f87ce3fa3a Update schutzbot/terraform to point to latest gitlab-ci-terraform
repo commit
2021-07-01 13:34:12 +02:00
Xiaofeng Wang
35263e5ab4 test: Remove fedora 32 test
GitLab CI dropped Fedora 32 support, fedora 32 test should be
removed.
2021-07-01 13:34:12 +02:00
Achilleas Koutsou
f557ab57bf test/data: update edge test manifests
Core stages were updated to include the fix-bls stage.
Updating existing RHEL 8.5 test manifests.
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
a304806d7c GitLab CI: Enable RHEL 8.5 and installers.sh
Add RHEL 8.5 runners and run new installers.sh integration test.
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
a877326671 test: add installer integration test
Builds a tar-installer, installs the OS, boots it, and checks if the
packages selected in the blueprint are installed.
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
c4aeb256a5 test/data: test case for tar x86_64 2021-07-01 12:48:32 +02:00
Achilleas Koutsou
5f53b3865f rhel85: append "boot" package sets to "packages"
If an image type is marked "bootable", append the package set with  name
"boot".
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
e8dd5b3248 rhel85: move package sets to separate file
Package sets shared between multiple image types are statically defined
in functions.
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
49df4a70af rhel85: add base build package set to distribution 2021-07-01 12:48:32 +02:00
Achilleas Koutsou
f1e805658c rhel85: distro and arch package sets
Add package sets to the top level distribution and the
architectures. Merge package sets when requested through
`imageType.PackageSets()`.
This allows us to have "cascading" package sets:
    distro > arch > image type

Previously this was only supported for the build packages. The idea is
generalised based on the idea of arbitrary package sets.

Certain package sets have special meaning (for the distro):
- packages: blueprint packages and special option packages are merged
  into this one.
- build: is not expected to be defined on the image type so it is merged
  from the distro and arch explicitly.
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
38f5b23e87 rhel85: tar image type
Traditional tar image type.
Pipelines generation function for tar image type uses the same pipelines
and stages as the tar installer, but exports the OS image directly.
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
ec381fefb9 rhel85: generalise and simplify tar stage creation 2021-07-01 12:48:32 +02:00
Achilleas Koutsou
7e1ab365c8 rhel85: change kickstart file location
Global const for defining kickstart path for all installer types.
Kernel options added to point to path.

Kickstart file renamed to osbuild.ks for all types and placed in the
bootiso root.

Fixes #1363
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
32c69b3cf9 rhel85: add subscription stage to core pipeline 2021-07-01 12:48:32 +02:00
Achilleas Koutsou
242f6d3c44 rhel85: tar installer main pipeline
Define a core set of packages and stages shared by images.
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
19d15af788 rhel85: new image type tar-installer 2021-07-01 12:48:32 +02:00
Ondřej Budai
e1e4a1f845 distro/rhel90: add s390utils-core to the build pipeline for s390x
A recent update to crypto-policies changed its requirements from:

Recommends: grubby

to:

Recommends: (grubby if kernel)

We don't install kernel in the build pipeline, thus grubby was now no longer
installed. This caused also s390utils-core to not be installed on s390x.
s390utils-core has to be in the build pipeline though because we use
/usr/sbin/zipl from it to install the bootloader to s390x images.

Long story short, images for s390x currently cannot be build because
/usr/sbin/zipl is no longer in the build pipeline. This commit fixes that
by explicitly adding s390utils-core to the package list for s390x.

I verified this fix on an actual s390x machine. The test manifest was
regenerated but it's actually still the same because we use an older
snapshot that doesn't contain the newest crypto-policies change.

Kudos to Tomáš Hozza for helping me out!

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-06-30 14:48:57 +02:00
Yuxin Sun
6cd6e81e85 Add subscription test case and Azure boot up case
Signed-off-by: Yuxin Sun <yuxisun@redhat.com>
2021-06-30 12:29:33 +02:00
Achilleas Koutsou
1e54cd4493 CI: run ostree tests in RHEL 8.5 2021-06-29 17:26:43 +01:00
Achilleas Koutsou
f557bf9a8b test/cases/ostree: test new image types
ostree.sh: add case for RHEL 8.5 edge-commit
ostree-ng.sh: conditional container and installer type and file names
2021-06-29 17:26:43 +01:00
Achilleas Koutsou
034bdaafab distro-arch-imagetype-map: add new distro and types 2021-06-29 17:26:43 +01:00
Achilleas Koutsou
0bac6806f5 tools/provision: set up nightly repos for RHEL 8.5 2021-06-29 17:26:43 +01:00
Achilleas Koutsou
54a811c3f1 schutzbot/mockbuild.sh: RHEL 8.5 support
Add RHEL 8.5 support to the mockbuild script.
2021-06-29 17:26:43 +01:00
Achilleas Koutsou
c8bf77bad8 news: add entry about RHEL 8.5 Edge images 2021-06-29 17:26:43 +01:00
Achilleas Koutsou
0b6452bb34 test: new manifests for RHEL 8.5 edge types 2021-06-29 17:26:43 +01:00
Achilleas Koutsou
779103cb96 rhel85: add tests 2021-06-29 17:26:43 +01:00
Achilleas Koutsou
4651b8ba22 tools: add 8.5 edge types to format request map
For test case generation. These are the new names used by the 8.5 edge
image types.
2021-06-29 17:26:43 +01:00
Achilleas Koutsou
e388706712 store: add new image type mappings and fix typos 2021-06-29 17:26:43 +01:00
Achilleas Koutsou
139c8eb3c4 Revert "rhel85: nolint package for unused private functions"
Pipeline generation functions are now used by the new image definitions.

This reverts commit cca084d5ce.
2021-06-29 17:26:43 +01:00
Achilleas Koutsou
f789f785a4 rhel85: rhel-edge image definitions
Image definitions added for all three edge types:
edge-commit
edge-container
edge-installer

Manifest pipelines generation methods attached to image types.
2021-06-29 17:26:43 +01:00
Xiaofeng Wang
9a7016f451 Remove the subscription check on mock config for RHEL 8.5 2021-06-29 16:14:45 +02:00
Xiaofeng Wang
e00ba2f075 gitlab-ci: Add two new RHEL 8.5 runners on rpmbuild stage
Now CI is able to build RHEL 8.5 RPM package on both x86_64 and
aarch64
2021-06-29 16:14:45 +02:00
Xiaofeng Wang
bc43eab535 Update schutzbot/terraform to point to latest gitlab-ci-terraform
repo commit

The latest gitlab-ci-terraform repo commit includes new runners:
1. openstack/rhel-8.5-x86_64
2. aws/rhel-8.5-x86_64
3. aws/rhel-8.5-aarch64
2021-06-29 16:14:45 +02:00
Ondřej Budai
419ac4c769 ci: remove EXTRA_REPO_PATH_SEGMENT
EXTRA_REPO_PATH_SEGMENT was needed when both Jenkins and Gitlab CI
were running in parallel (so they don't override their artifacts).

Jenkins is now decommissioned so we can drop the variable.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-06-29 13:58:58 +02:00
Ondřej Budai
cc2c354371 tests: regenerate manifests for rhel 9.0
Previous commits bumped the repo snapshots and changed the manifest generation
for RHEL 9.0.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-06-29 13:56:26 +02:00
Ondřej Budai
39f1d4d485 tests: update snapshot for rhel 9.0
We have a newer one so let's use it.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-06-29 13:56:26 +02:00
Ondřej Budai
9bcd219528 distro/rhel90: make selinux stage the last one
There are currently some context mismatches in the image because of the new
files from the sysconfig stage. Let's move the selinux stage to the end so
we fix them.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-06-29 13:56:26 +02:00
Ondřej Budai
4c16b2f77e distro/rhel90: include qemu-guest-agent in qcow2
Fixes COMPOSER-990

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-06-29 13:56:26 +02:00
Ondřej Budai
c37705e7d5 distro/rhel90: move all package lists to a separate file
There's currently the Package Deletion Process going on in CentOS Stream 9.
When a maintainer wants to get rid of a package, several steps must be taken
before the package is completely removed from all places. We want to include
osbuild-composer in the process so packages that are used here are not deleted
without us noticing it. Thus, maintainers going through the process should be
able to easily see all packages that osbuild-composer includes in images.

This commit moves all package lists from rhel90/distro to rhel90/packages
to allow CentOS Stream maintainers to easily packages required by composer.

Not a functional change.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-06-29 13:56:26 +02:00
Ondřej Budai
42e4a5f6ac ci: remove fedora 34 hack from mockbuild
This is apparently not needed anymore, I checked with 20 jobs here:
https://gitlab.com/osbuild/ci/osbuild-composer/-/pipelines/328779200

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-06-29 11:53:56 +02:00
Achilleas Koutsou
0ddfc32957 test: verify selected package appears in metadata
Add postgresql to all builds and verify that it's listed in the metadata
2021-06-29 09:33:05 +01:00
Achilleas Koutsou
cfb244ab34 cloudapi: check if metadata is nil before converting 2021-06-29 09:33:05 +01:00
Achilleas Koutsou
51503cc000 cloudapi: collect RPMs from v2 result stages
Fix for v1 results: collect RPMs for main stages, not build.
Support for v2 results: filter out build stages from stage results,
which include stages from all pipelines (flattened).
2021-06-29 09:33:05 +01:00
Achilleas Koutsou
b46addea5c osbuild1: convert stage metadata from v2
While unmarshalling osbuild output and converting from v1 to v2, convert
the metadata as well.

Added checks to successful parsing and conversion test.
2021-06-29 09:33:05 +01:00
Achilleas Koutsou
1dcaa26966 osbuild2: add support for reading stage metadata
Metadata from stages in the output from osbuild in the v2 schema are
stored at the top level of the result structure.

Adding the same Metadata types from v1:
- org.osbuild.rpm stage
- org.osbuild.ostree.commit stage
- Raw metadata (for every other stage type)
2021-06-29 09:33:05 +01:00