Commit graph

605 commits

Author SHA1 Message Date
Alexander Todorov
4aff9076ea Update rpmrepos & manifest files via command
go run ./cmd/gen-manifests/ -workers 50
2023-06-15 08:54:57 +02:00
Alexander Todorov
22437f7784 Add RPM repository definitions for 8.9 and 9.3 2023-06-15 08:54:57 +02:00
Achilleas Koutsou
403fe74ce4 format-request-map: clean up ostree params
Don't specify ref ID for parent, instead only specify a ref and URL and
let the manifest generator resolve the ID using the sha256 hash.
2023-06-14 11:19:29 +02:00
Irene Diez
a8f7ff487d distro/rhel9: add the edge-ami image type based on edgeRawImage
Creates the 'edge-ami' image type based on edgeRawImage, which generates
a raw image (x86_64, aarch64) ready to upload to AWS EC2.

This 'edge-ami' image type has Ignition support.

Signed-off-by: Irene Diez <idiez@redhat.com>
2023-06-07 13:18:01 +02:00
Ondřej Budai
73c8a482f7 repositories: update fedora 37-39 snapshots
Fedora 37 and 38 still pointed at branched/rawhide, oops. Let's just update
them all.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-06 13:04:10 +02:00
Ondřej Budai
eaa4e9b5fa repositories: fix GPG keys
Oh my, we had the Fedora 37 key even in Fedora 38 and 39 repositories. This
commit fixes that.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-06 13:04:10 +02:00
Ondřej Budai
894e6042c6 tools/test-case-generators/repos: remove fedora-36
It's EOL and already removed from the codebase. This was apparently forgotten
during the purge.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-06 13:04:10 +02:00
Sanne Raymaekers
f5b01125b6 tools/test-case-generators: add ovf image type 2023-05-25 10:14:32 +02:00
Tomáš Hozza
2a50092920 Test: add Fedora iot-raw-image test manifests with customizations
Add Fedora iot-raw-image test manifests with supported customizations,
specifically:
 - custom users and groups
 - custom enabled services
 - custom files and directories

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-22 16:30:14 +02:00
Ondřej Budai
ce5e41f980 tools/run-koji-container: allow passing a custom certificate dir
This helps with running the script locally.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-05-18 09:25:22 +02:00
Ondřej Budai
25161a8316 libvirt_test: stop setting nvram in virt-install on RHEL 9
We use CDN repositories when running tests on GA RHEL. Thus, our RHEL 9.1
machines are actually pulling packages from RHEL 9.2. Therefore, we are
now getting virt-install >= 4. This version brings firmware auto-detection
and doesn't support nvram_template anymore. Let's remove it then.

See 966049ec3c

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-05-12 11:02:49 +02:00
Sanne Raymaekers
c7d2eaa5f7 test: use non-deprecated gitlab CI variables
CI_BUILD_ID was replaced with CI_JOB_ID in gitlab 9+.
2023-05-12 11:02:49 +02:00
Gianluca Zuccarelli
1284642ab2 tools/test-case-generator: add custom repos 2023-04-21 17:40:00 +02:00
Achilleas Koutsou
140571bf48 test: add "name" to test manifests with container
Add name field to the manifest-list-test container in the test
request.  The value is the same as the source but with a `v1` tag
added.

In the manifests, the name field for the manifest-list-test is added to
the skopeo stage.  The `name` option of the fedora-minimal container in
the skopeo stage is also changed to reflect the full source reference
including the `latest` tag.
2023-04-19 20:07:40 +02:00
Tomáš Hozza
7a439ba59d Test/manifests: update Azure RHUI client repo snapshots
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-04-19 15:04:42 +02:00
Ondřej Budai
d8342862d5 customizations: create all groups unconditionally
We had this weird condition in code that prevented composer to create groups
with the same name as a user has. This unfortunately means that you are not
able to create a user with a primary group with a certain GID that has the
same name as the user. There's the gid field in the user customization,
but it requires that the group already exists.

In order to allow that, we need to remove the condition. From now on, it's
possible to create groups with the same name as a user has, which can be used
to create primary groups with a custom gid.

Note that the lorax compatibility behaviour was actually wrong. When lorax was
given a custom gid for a user, it didn't require the gid to exist. When it
didn't, the group was just created. Thus, we still don't have full backward
compatibility, but at least we now have support for this.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-04-19 11:50:49 +02:00
Achilleas Koutsou
5a455b4406 test: add container source with manifest-list to manifests
Add a second container to the ostree-commit test manifests that refers
to a manifest list on the osbuild-composer registry on gitlab.
2023-04-17 18:30:41 +02:00
Achilleas Koutsou
4e3d94f9a7 test-case-generators: add request for Azure EAP 7 image type 2023-03-30 18:40:12 +02:00
Tomáš Hozza
58e3716b89 distro: add support for RHEL-8.9 and RHEL-9.3
Add a basic support for building RHEL-8.9 and RHEL-9.3 images with
composer.

Add 8.9 and 9.3 repositories to the multitude of places where we have
them.

Generate image test manifests for 8.9 and 9.3. No functional testing is
added for 8.9 or 9.3 at this moment.

This change unblocks the RHEL Gating (since unit tests are currently
failing on 8.9 and 9.3 as on unknown distribution).

Related to https://issues.redhat.com/browse/COMPOSER-1924

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-03-27 11:39:24 +02:00
Simon de Vlieger
23627e9ce0 distro/fedora: add support for Fedora 39 2023-03-15 09:31:23 +01:00
Sanne Raymaekers
ca8a05bd3a templates/packer: subscribe packer machines
To avoid a mismatch between the RPMs (which are build using CDN content)
and the packer instances (RHUI, which might be older).
2023-03-03 13:00:05 +01:00
Sanne Raymaekers
41d98c85a7 tools/appsre-ansible: revert sub removal on rpmbuild machines
The rhel-9 templates used in mock don't work with RHUI, they require the
CDN. This reintroduces the RHUI-CDN repo sync issue again.

This reverts commit d9f2ac14bc.
2023-03-03 13:00:05 +01:00
yih
266cfdcc44 fix ansible error on centos9 2023-03-02 16:03:11 +01:00
Sanne Raymaekers
75e6333b9e Revert "tools/appsre-ansible: workaround missing authselect-compat"
This reverts commit 4833f99070.
2023-03-01 20:05:38 +01:00
Sanne Raymaekers
0096ff3689 Revert "Packer: workaround missing authselect-compat-1.2.5-2.el9_1 in RHUI repos"
This reverts commit 0a4a75e19e.
2023-03-01 20:05:38 +01:00
Sanne Raymaekers
d9f2ac14bc tools/appsre-ansible: don't subscribe machines used for rpmbuild
We actually use rhui/cloud access images for rpmbuild as well. And the
sync issue between cdn and rhui repos can cause issues when rpms are
build against older packages.
2023-03-01 20:05:38 +01:00
Alexander Todorov
72b223c064 Remove duplicate function and use the shared implementation 2023-02-28 11:15:49 +01:00
Alexander Todorov
e064c12449 Update test suite after rebase to weldr-client-35.9
in order to avoid failures in nightly pipelines b/c JSON format has
changed
2023-02-28 11:15:49 +01:00
Tomáš Hozza
1c64d15a61 Test/fedora: add iot-container test manifests with customizations
Add `iot-container` test manifests with customizations. Specifically with
files, directories and services.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
df26be35cc Test/fedora: add iot-commit test manifests with customizations
Add `iot-commit` test manifests with customizations. Specifically with
files, directories and services.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
e16fdeb7ed Test/el/cs: add edge-container test manifests with customizations
Add `edge-container` test manifests with customizations. Specifically with
files, directories and services.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
5a7092bfd5 Test/el/cs: add edge-commit test manifests with customizations
Add `edge-commit` test manifests with customizations. Specifically with
files, directories and services.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
1ea58566ff tools/tc-generators/format-req-map: drop all rhel-edge-* requests
Drop unused `rhel-edge-*` requests, which are not used any more. They
have been replaced by `edge-*` requests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
aa9373cb18 Test: use dir and file customizations in qcow2 image manifests
Add the new directories and files customization to the customized qcow2
image manifests used for testing.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Thomas Lavocat
aba981c508 tools/image-info: bad syntax on rhel8 python
We can't use:

with (a() as b, c() as d):
    pass

On older versions of python.
2023-02-14 14:37:10 +01:00
Thomas Lavocat
15d80a0cd1 tools/image-info: cosmetics, autopep8 2023-02-14 14:37:10 +01:00
Thomas Lavocat
36d890d452 toos/image-info: use SECTOR-SIZE
Instead of numerical value
2023-02-14 14:37:10 +01:00
Thomas Lavocat
0324e02659 tools/image-info: support iso
Add the support for ISOs. Mounting then as a loopack device and then
accessing the content as a tarball
2023-02-14 14:37:10 +01:00
Thomas Lavocat
d0be1ec5a4 tools/image-info: validate with osbuild's schemas
Before we were invoking osbuild's stages/devices/mounter directly
without taking the time to validate that the options sent were actually
valid and supported.
This commit adds the support of the validation schema into image-info so
that we're sure we don't mess with the internals when we call them.
2023-02-14 14:37:10 +01:00
Thomas Lavocat
ffd45d635a tools/image-info: mount with the mount options
Before the new version of image-info was ignoring the mount options for
the devices leading to errors in the resulting json.
2023-02-14 14:37:10 +01:00
Thomas Lavocat
62e6bed23d tools/image-info: clean placeholder try block
Was only there to ease diffing previous commit.
2023-02-14 14:37:10 +01:00
Thomas Lavocat
31e3729236 tools/image-info: use osbuild's internal to mount
On the newest versions of rhel 92, 88, a change in behavior makes the
previous version of image-info failing to mount loopback devices. We've
tracked down this error to be a race condition on udev, yet without
understanding what changed for now.

Osbuild had for some time already a cleaner way to mount partitions.
osbuild has some machinery to opt out of block device handling in udev
48a4419705/devices/org.osbuild.loopback (L69)
Using this fixes the issue at hand.

This changes the way we need to mount all the partitions, including the
LVM ones. This new mechanism might also pave the way to include lusks fs.
2023-02-14 14:37:10 +01:00
Thomas Lavocat
aaddca445d tools/image-info: fix mypy errors 2023-02-14 14:37:10 +01:00
Achilleas Koutsou
5e27e01e5e test-case-generators: enable rhel8 azure aarch64 manifest generation
Add the rhui-azure rpmrepo snapshots to aarch64 repositories.
The repos are labelled as x86_64 but they're actually 'noarch' and we
name them x86_64 when creating the snapshots.
2023-01-30 17:40:18 +01:00
Achilleas Koutsou
db97690b84 tools: add RHEL 9.1 repos for test case generators
In commit 7c8b9f4da8 I intended to make a
copy of the RHEL 9.1 repos under the key "rhel-9" so that the
RHEL 9 distribution, which is currently an alias to RHEL 9.1 (the
current GA) would have valid test repositories available for the test
case generator.  Instead, I accidentally replaced the "rhel-91"
repositories with the ones for "rhel-9".
This commit fixes that mistake.
2023-01-26 20:32:38 +01:00
Sarita Mahajan
0d16e2310e image: add new image type "minimal-raw" for fedora
Signed-off-by: Sarita Mahajan <sarmahaj@redhat.com>
2023-01-25 19:19:45 +01:00
Sanne Raymaekers
4833f99070 tools/appsre-ansible: workaround missing authselect-compat
See PR#3237.
2023-01-24 16:04:40 +01:00
Tomáš Hozza
0a4a75e19e Packer: workaround missing authselect-compat-1.2.5-2.el9_1 in RHUI repos
`authselect-compat-1.2.5-2.el9_1` package is currently missing in AWS
RHUI el9 AppStream repositories, which makes `dnf upgrade` fail on
RHEL-9.1. This is a RHUI-specific issue, since the package is available
in CDN repos.

In order to workaround the issue for now, `authselect-compat` needs to
be removed as part of the upgrade in order for it to succeed. Use
`--allowerasing` instead of just removing the issue, because this will
ensure that `authselect-compat` will be upgraded just fine, once the
issue is resolved.

Fix the issue in the CI script that builds the image using Packer, as
well as the Ansible playbook used by Packer to build the image.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-24 15:40:02 +01:00
Achilleas Koutsou
73882b7289 test: update customized test manifests to include bluez
Our 'customize' test manifests include an option to disable the
bluetooth.service.  Originally this option was added for image types
that included bluez in their default package set (Fedora IoT commit) but
it was later copied to the qcow2 image type as a way of testing
customizations.

Until recently, building these caused no issues.  On distros with more
recent versions of systemd, disabling a non-existent service causes an
error and these manifests fail to build.

Added the 'bluez' package to all manifests that include the 'disable
bluetooth.service' customization and updated the manifests.  These
should all be buildable now.
2023-01-23 19:54:39 +01:00
Achilleas Koutsou
7f1a0a76b1 test: update manifests for "rhel-8"
The format-request-map is updated to remove the override for the
customized qcow for rhel-8.
The rhel-8 manifests are now identical to the rhel-87 counterparts.
2023-01-18 11:50:35 +01:00