Commit graph

487 commits

Author SHA1 Message Date
Tomáš Hozza
58e3716b89 distro: add support for RHEL-8.9 and RHEL-9.3
Add a basic support for building RHEL-8.9 and RHEL-9.3 images with
composer.

Add 8.9 and 9.3 repositories to the multitude of places where we have
them.

Generate image test manifests for 8.9 and 9.3. No functional testing is
added for 8.9 or 9.3 at this moment.

This change unblocks the RHEL Gating (since unit tests are currently
failing on 8.9 and 9.3 as on unknown distribution).

Related to https://issues.redhat.com/browse/COMPOSER-1924

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-03-27 11:39:24 +02:00
Simon de Vlieger
23627e9ce0 distro/fedora: add support for Fedora 39 2023-03-15 09:31:23 +01:00
Sanne Raymaekers
ca8a05bd3a templates/packer: subscribe packer machines
To avoid a mismatch between the RPMs (which are build using CDN content)
and the packer instances (RHUI, which might be older).
2023-03-03 13:00:05 +01:00
Sanne Raymaekers
41d98c85a7 tools/appsre-ansible: revert sub removal on rpmbuild machines
The rhel-9 templates used in mock don't work with RHUI, they require the
CDN. This reintroduces the RHUI-CDN repo sync issue again.

This reverts commit d9f2ac14bc.
2023-03-03 13:00:05 +01:00
yih
266cfdcc44 fix ansible error on centos9 2023-03-02 16:03:11 +01:00
Sanne Raymaekers
75e6333b9e Revert "tools/appsre-ansible: workaround missing authselect-compat"
This reverts commit 4833f99070.
2023-03-01 20:05:38 +01:00
Sanne Raymaekers
0096ff3689 Revert "Packer: workaround missing authselect-compat-1.2.5-2.el9_1 in RHUI repos"
This reverts commit 0a4a75e19e.
2023-03-01 20:05:38 +01:00
Sanne Raymaekers
d9f2ac14bc tools/appsre-ansible: don't subscribe machines used for rpmbuild
We actually use rhui/cloud access images for rpmbuild as well. And the
sync issue between cdn and rhui repos can cause issues when rpms are
build against older packages.
2023-03-01 20:05:38 +01:00
Alexander Todorov
72b223c064 Remove duplicate function and use the shared implementation 2023-02-28 11:15:49 +01:00
Alexander Todorov
e064c12449 Update test suite after rebase to weldr-client-35.9
in order to avoid failures in nightly pipelines b/c JSON format has
changed
2023-02-28 11:15:49 +01:00
Tomáš Hozza
1c64d15a61 Test/fedora: add iot-container test manifests with customizations
Add `iot-container` test manifests with customizations. Specifically with
files, directories and services.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
df26be35cc Test/fedora: add iot-commit test manifests with customizations
Add `iot-commit` test manifests with customizations. Specifically with
files, directories and services.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
e16fdeb7ed Test/el/cs: add edge-container test manifests with customizations
Add `edge-container` test manifests with customizations. Specifically with
files, directories and services.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
5a7092bfd5 Test/el/cs: add edge-commit test manifests with customizations
Add `edge-commit` test manifests with customizations. Specifically with
files, directories and services.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
1ea58566ff tools/tc-generators/format-req-map: drop all rhel-edge-* requests
Drop unused `rhel-edge-*` requests, which are not used any more. They
have been replaced by `edge-*` requests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
aa9373cb18 Test: use dir and file customizations in qcow2 image manifests
Add the new directories and files customization to the customized qcow2
image manifests used for testing.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Thomas Lavocat
aba981c508 tools/image-info: bad syntax on rhel8 python
We can't use:

with (a() as b, c() as d):
    pass

On older versions of python.
2023-02-14 14:37:10 +01:00
Thomas Lavocat
15d80a0cd1 tools/image-info: cosmetics, autopep8 2023-02-14 14:37:10 +01:00
Thomas Lavocat
36d890d452 toos/image-info: use SECTOR-SIZE
Instead of numerical value
2023-02-14 14:37:10 +01:00
Thomas Lavocat
0324e02659 tools/image-info: support iso
Add the support for ISOs. Mounting then as a loopack device and then
accessing the content as a tarball
2023-02-14 14:37:10 +01:00
Thomas Lavocat
d0be1ec5a4 tools/image-info: validate with osbuild's schemas
Before we were invoking osbuild's stages/devices/mounter directly
without taking the time to validate that the options sent were actually
valid and supported.
This commit adds the support of the validation schema into image-info so
that we're sure we don't mess with the internals when we call them.
2023-02-14 14:37:10 +01:00
Thomas Lavocat
ffd45d635a tools/image-info: mount with the mount options
Before the new version of image-info was ignoring the mount options for
the devices leading to errors in the resulting json.
2023-02-14 14:37:10 +01:00
Thomas Lavocat
62e6bed23d tools/image-info: clean placeholder try block
Was only there to ease diffing previous commit.
2023-02-14 14:37:10 +01:00
Thomas Lavocat
31e3729236 tools/image-info: use osbuild's internal to mount
On the newest versions of rhel 92, 88, a change in behavior makes the
previous version of image-info failing to mount loopback devices. We've
tracked down this error to be a race condition on udev, yet without
understanding what changed for now.

Osbuild had for some time already a cleaner way to mount partitions.
osbuild has some machinery to opt out of block device handling in udev
48a4419705/devices/org.osbuild.loopback (L69)
Using this fixes the issue at hand.

This changes the way we need to mount all the partitions, including the
LVM ones. This new mechanism might also pave the way to include lusks fs.
2023-02-14 14:37:10 +01:00
Thomas Lavocat
aaddca445d tools/image-info: fix mypy errors 2023-02-14 14:37:10 +01:00
Achilleas Koutsou
5e27e01e5e test-case-generators: enable rhel8 azure aarch64 manifest generation
Add the rhui-azure rpmrepo snapshots to aarch64 repositories.
The repos are labelled as x86_64 but they're actually 'noarch' and we
name them x86_64 when creating the snapshots.
2023-01-30 17:40:18 +01:00
Achilleas Koutsou
db97690b84 tools: add RHEL 9.1 repos for test case generators
In commit 7c8b9f4da8 I intended to make a
copy of the RHEL 9.1 repos under the key "rhel-9" so that the
RHEL 9 distribution, which is currently an alias to RHEL 9.1 (the
current GA) would have valid test repositories available for the test
case generator.  Instead, I accidentally replaced the "rhel-91"
repositories with the ones for "rhel-9".
This commit fixes that mistake.
2023-01-26 20:32:38 +01:00
Sarita Mahajan
0d16e2310e image: add new image type "minimal-raw" for fedora
Signed-off-by: Sarita Mahajan <sarmahaj@redhat.com>
2023-01-25 19:19:45 +01:00
Sanne Raymaekers
4833f99070 tools/appsre-ansible: workaround missing authselect-compat
See PR#3237.
2023-01-24 16:04:40 +01:00
Tomáš Hozza
0a4a75e19e Packer: workaround missing authselect-compat-1.2.5-2.el9_1 in RHUI repos
`authselect-compat-1.2.5-2.el9_1` package is currently missing in AWS
RHUI el9 AppStream repositories, which makes `dnf upgrade` fail on
RHEL-9.1. This is a RHUI-specific issue, since the package is available
in CDN repos.

In order to workaround the issue for now, `authselect-compat` needs to
be removed as part of the upgrade in order for it to succeed. Use
`--allowerasing` instead of just removing the issue, because this will
ensure that `authselect-compat` will be upgraded just fine, once the
issue is resolved.

Fix the issue in the CI script that builds the image using Packer, as
well as the Ansible playbook used by Packer to build the image.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-24 15:40:02 +01:00
Achilleas Koutsou
73882b7289 test: update customized test manifests to include bluez
Our 'customize' test manifests include an option to disable the
bluetooth.service.  Originally this option was added for image types
that included bluez in their default package set (Fedora IoT commit) but
it was later copied to the qcow2 image type as a way of testing
customizations.

Until recently, building these caused no issues.  On distros with more
recent versions of systemd, disabling a non-existent service causes an
error and these manifests fail to build.

Added the 'bluez' package to all manifests that include the 'disable
bluetooth.service' customization and updated the manifests.  These
should all be buildable now.
2023-01-23 19:54:39 +01:00
Achilleas Koutsou
7f1a0a76b1 test: update manifests for "rhel-8"
The format-request-map is updated to remove the override for the
customized qcow for rhel-8.
The rhel-8 manifests are now identical to the rhel-87 counterparts.
2023-01-18 11:50:35 +01:00
Achilleas Koutsou
24ef584be9 Update repository configs for RHEL 8 (no minor)
- repositories/: add google-compute-engine and google-cloud-sdk repos to
  package repositories.
- test/data/repositories/: add rt, rhui, and rhui-azure to test
  repositories.
- test-case-generators/: update unversioned rhel-8 repos to point to
  RHEL 8.7 snapshots.
2023-01-18 11:50:35 +01:00
Achilleas Koutsou
7c8b9f4da8 Add repository configs for RHEL 9 (no minor)
Release repositories (in repositories/) for RHEL 9 are the CDN repos
without a minor release, which should always track GA.

Test repositories (in test/data/ and test-case-generators/) point to
RHEL 9.1, the current GA.
2023-01-18 11:50:35 +01:00
Achilleas Koutsou
fad1d528ee tools/format-request-map: override kernel-rt for CS8
Override the kernel-rt test case for CS8 with kernel-rt-core.
The kernel-rt package resolves to kernel-rt-core (no kernel-rt
metapackage exists).

More details can be found at https://github.com/osbuild/osbuild-composer/issues/3211
2023-01-18 11:50:35 +01:00
Antonio Murdaca
560756a164 regenerate manifests
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-01-17 14:02:02 +01:00
Alexander Todorov
00dd497173 libvirt.sh: Adjust virt-install parameters for 9.2 2023-01-10 00:28:23 +01:00
Alexander Todorov
b41353f01c Revert "test: Update nightly compose URL to updates URL"
This reverts commit 422e70c8ce because it
breaks nightly CI testing with 8.8 and 9.2.
2023-01-10 00:28:23 +01:00
Ondřej Budai
0359647a82 go.mod: update to Go 1.18
Fedora 35 support was dropped, so we can update to a newer Go.

Stable RHEL 8 and 9 and Fedora 36 ships Go 1.18, so let's switch to it.

"//go:build" directives are now apparently enforced by go fmt, so that's why
there were added.

Also, all the github actions were adjusted to use Go 1.18.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-01-09 14:03:18 +01:00
Ondřej Budai
4bafe1ead9 remove Fedora 35 support
Fedora 35 is going EOL on Tue 2022-12-13. At the time of writing this commit
message, that's the next day. As we do releases on Wednesdays, the next
release will never find its way to F35 and thus, there's no point in keeping
support for it.

Let's delete everything that relates to Fedora 35. If there's something that
cannot be deleted (e.g. CI containers based on F35), let's upgrade it to F37.

TestCrossArchDepsolve now uses CentOS Stream 8 because RHEL 8.4 cannot read
F37 repository metadata. This is a similar issue to
https://bugzilla.redhat.com/show_bug.cgi?id=2004853 . Basically, newer
repositories can be only read by libmodulemd >= 2.11.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-01-06 11:34:50 +01:00
Ondřej Budai
1f782900f0 tools/koji-composer: add encoding to open()
Newer versions of pylint complain about a missing encoding in the open()
function.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-01-06 11:34:50 +01:00
Achilleas Koutsou
a9842d5e9c test-case-generators: enable azure aarch64 manifest generation
Add the rhui-azure rpmrepo snapshots to aarch64 repositories.
The repos are labelled as x86_64 but they're actually 'noarch' and we
name them x86_64 when creating the snapshots.

Co-Authored-By: Christian Kellner <christian@kellner.me>
2022-12-16 15:40:09 +01:00
Sanne Raymaekers
07a8f3d5ea test: adapt to shellcheck v0.9.0 2022-12-13 13:55:53 +01:00
Sanne Raymaekers
369f237d89 gen-manifests: add ostree rhsm option 2022-12-12 13:42:52 +01:00
Achilleas Koutsou
c2fe05950c tools: update test case repos for CS9 2022-12-11 12:37:37 +00:00
Achilleas Koutsou
b9098c64c4 test: add valid parent commit ID to test manifests
The previous value for parent was not a valid commit ID.  In a regular
compose request, the parent ref is resolved before being added as a
source in the manifest.  However, when building test manifests, since
the values aren't real, we don't resolve the ostree refs, meaning that
the test manifests weren't valid — they failed validity checks if run in
osbuild.

Replaced the fake parent ref with a fake parent commit ID
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff".
The manifests still aren't buildable, since they don't point to a valid
ostree repository or commit, but now they're at least valid (in the
technical sense).
2022-12-11 12:37:37 +00:00
Ondřej Budai
fd72a6d16e run-koji-container.sh: drop the duplicate vhd.xz format
When the test upload a vhd.xz image to koji, it returns `koji.GenericError:
multiple matches for file extension: vhd.xz`. It seems like the default
schema is not valid for vhd.xz images because it contains two archive types
for them which koji cannot handle. I reported this issue as

https://pagure.io/koji/issue/3605

This commit works around that by removing one of the archive types, so koji
isn't confused by two same records.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-12-01 20:59:30 +01:00
Achilleas Koutsou
9ae3d03f88 tools: add package to image-installer manifest request
Add podman to the image installer's packages when generating test
manifests.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
c5f6aca53b format-request-map: iot-commit with container
Add manifests for container embedding in iot-commits for Fedora.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
04a7cd36c7 format-request-map: new manifests for container embedding
Add manifests for container embedding in edge-commits and qcow2s.
Limit qcow2 container embedding to RHEL 9.1 for now since it's not
supported on Fedora or RHEL 8 yet.
2022-11-28 17:20:49 +01:00