Commit graph

275 commits

Author SHA1 Message Date
Achilleas Koutsou
5786db9caa tools: update distro-arch-imagetype-map for RHEL 9.0 types
`rhel-90-ga` no longer valid.  Now called `rhel-90`.
2021-11-10 14:54:31 +01:00
Achilleas Koutsou
0c185eca7a tools: update RHEL 9.0 repos for test case generators
No more `rhel-90-ga`.
2021-11-10 14:54:31 +01:00
Achilleas Koutsou
cba73ee9e3 tools: add centos-9 to distro-arch-imagetype-map 2021-11-10 14:54:31 +01:00
Achilleas Koutsou
994dd1e136 tools: copy rhel-90 distro-arch-imagetype-map to -beta and -ga 2021-11-10 14:54:31 +01:00
Achilleas Koutsou
bbf825ef12 test: add repositories for rhel-90-beta and -ga
- Updated repos to use latest nightly (20211105)
- Added rt repository
- Both -ga and -beta are identical
- rhel-90.json repo file is a symlink to rhel-90-beta.json

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-11-10 14:54:31 +01:00
Ondřej Budai
83a35fb9b8 ci: remove 8.5 nightly testing
It no longer makes sense because:

- we don't make any changes to 8.5
- we don't regenerate test manifests for 8.5
- osbuild-composer for 8.5 is in the rhel-8.5.0 branch

Also, the latest-8.5.0 symlink was removed, which broke the CI.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-10-26 09:30:21 +01:00
Achilleas Koutsou
aea77ed3de test: timestamp messages in test scripts
The commonly used 'greenprint' function now adds a date + timestamp to
each message for debugging and tracking the duration of segments of each
scripts.
2021-10-24 16:13:53 +01:00
Achilleas Koutsou
628515a694 tools: add all supported image types for centos-8
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-21 15:16:04 +01:00
Achilleas Koutsou
67e46b888f tools/test-case-generators: fix typo in imagetype-map
image_installer -> image-installer
2021-10-21 15:16:04 +01:00
Ondřej Budai
58423c262b ci: rotate secret names
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-10-21 07:28:46 +02:00
Tomas Hozza
eac6b2c083 RHEL-8.6: add support for official EC2 SAP image
Add support for building the official RHEL EC2 SAP image for RHEL-8.6.
The new image type is available under the `ec2-sap` name. Since the
image type's package set includes RHUI client RPMs, which are not
available publicly, it is by default not exposed through the Weldr API.

Extend the relevant files and definitions used for generating image test
cases. This includes addition of repository definitions needed by the
image type, specifically: `sap`, `saphana` and `ansible`.

Generate image test case for the new image type.

Add News entry for the new image type.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-11 09:49:15 +02:00
Achilleas Koutsou
46fc9b66c5 tools/provision: set up nightly repos for RHEL 8.6
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-11 09:49:15 +02:00
Achilleas Koutsou
1cff6c7dd2 test: add RHEL 8.6 test manifests
Add images to distro-arch-imagetype map and generate all test cases.
2021-10-11 09:49:15 +02:00
Achilleas Koutsou
29ef7efe1b test: add rpmrepo snapshots for RHEL 8.6
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-10-11 09:49:15 +02:00
Tomas Hozza
510bd1d221 Tests/RHEL-9.0: add EC2 SAP image test
Because the RHUI client RPMs for RHEL-9 don't exist yet, this test
case uses the RHUI client RPMs for RHEL-8 to satisfy the required
package set. These RPMs contain the entitlement certificate and repo
definitions for RHEL-8 AWS RHUI repos, but RHUI will not work without
the image being imported with the proper Billing Code to AWS. So these
installed RHUI RPMs are technically useless on the RHEL-9 image, but
this approach is good enough to allow us to at least build and test
the image from configuration point of view.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-08 16:34:55 +01:00
Tomas Hozza
37911599ce Tests/RHEL-9.0: add repos needed for EC2 SAP image tests
Temporarily use the RHEL-8 RHUI client repo, because RHUI for RHEL-9
does not exist yet. Using at least RHEL-8 RHUI repo for EC2 images
allows us to at least successfully build them and inspect them.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-08 16:34:55 +01:00
Tomas Hozza
0e90217fcc RHEL-9.0: add EC2 SAP image definition.
The image is not available via Weldr API, because it requires RHUI
client RPMs.

The content and configuration is based on RHEL-8.6 EC2 SAP image, since
there is no definition for the RHEL-9 SAP image yet.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-08 16:34:55 +01:00
Tomas Hozza
7c6c67ef98 Tests/RHEL-9.0: add EC2 and EC2 HA image tests
Because the RHUI client RPMs for RHEL-9 don't exist yet, these test
cases use the RHUI client RPMs for RHEL-8 to satisfy the required
package set. These RPMs contain the entitlement certificate and repo
definitions for RHEL-8 AWS RHUI repos, but RHUI will not work without
the image being imported with the proper Billing Code to AWS. So these
installed RHUI RPMs are technically useless on the RHEL-9 image, but
this approach is good enough to allow us to at least build and test
these images from configuration point of view.

Regenerate all image test cases because of the updated repositories.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-08 16:34:55 +01:00
Tomas Hozza
d11db7ed83 Tests/RHEL-9.0: add repos needed for EC2 and EC2 HA image tests
Temporarily use the RHEL-8 RHUI client repo, because RHUI for RHEL-9
does not exist yet. Using at least RHEL-8 RHUI repo for EC2 images
allows us to at least successfully build them and inspect them.

Update all RHEL-9 repos to the latest snapshot to ensure the consistency
of package split among repositories.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-08 16:34:55 +01:00
Tomas Hozza
9ec2788ac8 generate-all-test-cases: allow specifying additional DNF repos
Add a new option `--repofrompath` allowing to specify additional DNF
repositories, which will be used on the Runner when installing any
packages (such as osbuild).

Extend the `test/README.md` to mention the new option. In addition,
specify some aspects of the script in more detail, because some of
them were not easy to figure out by users.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-06 15:17:06 +02:00
Tomas Hozza
71cfc35b67 image-info: ensure that directory is analysed as read-only
In some cases, e.g. when analysing an extracted `tar` image, it could
happen that the tools used by `image-info` to analyse the image could
modify its content (e.g. create new files which were originally not on
the image). This is especially an issue with `rpm`, which DB backend
seemed to create files in `/var/lib/rpm/` when run.

Ensure that the analysed directory can not be modified by bind-mounting
it as a read only, before the analysis.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-06 15:17:06 +02:00
Jakub Rusz
1a7858c196 tests: enable koji.sh test on RHEL-9 2021-10-06 15:42:36 +03:00
Tomas Hozza
147cec6baa Image tests: use RPMRepo with released RHEL-8.4 content
Fix #1793

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-04 10:48:21 +02:00
Tomas Hozza
4854b5730c image-info: fix undefined variable in analyse_directory()
One branch of the analyse_directory() function's if statement used
undefined variable `repo`. Copy its existing definition already used in
the function.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-04 10:48:21 +02:00
Tomas Hozza
7b37c92bea image-info: sort partitions list in the report.
Ensure that the list of partitions in the report is always sorted. Use
'partuuid' as the sorting key.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-04 10:48:21 +02:00
Tomas Hozza
02b706b673 image-info: use subprocess_check_output() in read_selinux_ctx_mismatch()
Use the `subprocess_check_output()` wrapper function in the
`read_selinux_ctx_mismatch()` parser function, instead of
directly calling `subprocess.check_output()`.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-04 10:48:21 +02:00
Tomas Hozza
e91c4a114b image-info: check not installed documentation
Extend the report to list also all documentation files, which are
normally part of the installed RPM packages, but were not installed
on the system. This can happen e.g. when '--excludedocs' option
is used when installing packages using rpm.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-04 10:48:21 +02:00
Tomas Hozza
ff2aa771fc image-info: read content of /etc/resolv.conf
Read uncommented lined from /etc/resolv.conf and add them as a list to
the image-info report. The list of lines is present in the report even
if it is empty, so that the report is explicit about the file content
and presence.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-04 10:48:21 +02:00
Tomas Hozza
299bd201e6 image-info: read sysctl.d config files from multiple paths
Extend image-info to read sysctl.d configuration files from multiple
paths:
 - /etc/sysctl.d/*.conf
 - /usr/lib/sysctl.d/*.conf

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-04 10:48:21 +02:00
Tomas Hozza
fea41e9c3a image-info: read security limits config files from multiple paths
Extend image-info to read security limits configuration files from multiple
paths:
 - /etc/security/limits.conf
 - /etc/security/limits.d/*.conf

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-04 10:48:21 +02:00
Tomas Hozza
7c9ecf8a53 image-info: read tmpfiles.d config files from multiple paths
Extend image-info to read tmpfiles.d configuration files from multiple
paths:
 - /etc/tmpfiles.d/*.conf
 - /usr/lib/tmpfiles.d/*.conf

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-04 10:48:21 +02:00
Tomas Hozza
70e1b30347 image-info: read systemd service unit drop-ins from multiple paths
Extend image-info to read systemd service unit drop-ins from
multiple paths:
 - /etc/systemd/system/*.service.d
 - /usr/lib/systemd/system/*.service.d

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-04 10:48:21 +02:00
Tomas Hozza
90a13c3d24 image-info: read cloud-init configs from multiple paths
Extend image-info to read cloud-init configuration files from
multiple paths:
 - /etc/cloud/cloud.cfg
 - /etc/cloud/cloud.cfg.d/*.cfg

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-04 10:48:21 +02:00
Tomas Hozza
9e719a0c33 image-info: read systemd-logind configs from multiple paths
Extend image-info to read systemd-logind configuration files from
multiple paths:
 - /etc/systemd/logind.conf
 - /etc/systemd/logind.conf.d/*.conf
 - /usr/lib/systemd/logind.conf.d/*.conf

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-04 10:48:21 +02:00
Tomas Hozza
fb982b20b9 image-info: read dracut configs from multiple paths
Extend image-info to read dracut configuration files from multiple
paths:
 - /etc/dracut.conf.d/*.conf
 - /usr/lib/dracut/dracut.conf.d/*.conf

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-04 10:48:21 +02:00
Tomas Hozza
ee197346bb image-info: read modprobe configs from multiple paths
Extend image-infro to read modprobe configuration files from multiple
paths:
 - /etc/modprobe.d/*.conf
 - /usr/lib/modprobe.d/*.conf
 - /usr/local/lib/modprobe.d/*.conf

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-10-04 10:48:21 +02:00
Ondřej Budai
2070747876 ci: install ansible-core instead of ansible for EL9
ansible-core no longer provides ansible so we need to require specifically
ansible-core.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-10-01 15:23:53 +02:00
sanne
106e9b3fc2 tools: Push ubi container to quay.io/app-sre/composer 2021-09-29 22:37:25 +02:00
Ondřej Budai
27693b1949 test: use hyphen in DISTRO_CODE instead of underscore
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-24 17:03:53 +02:00
Ondřej Budai
4c5d1ae45a test: move env variables into set-env-variables helper
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-09-24 17:03:53 +02:00
Achilleas Koutsou
ab5d560b1c tools/provision: install community.general ansible collection
Required for the `json_query` which we use in our playbooks.
Also requires python3-jmespath.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-09-24 13:27:21 +02:00
Tomas Hozza
008dfcc5b5 generate-test-cases: drop --with-customizations option
Generating image test case with all possible blueprint customizations is
supported only for `qcow2` image type. Generating `qcow2` images with
customizations was not a lot of fun. One had to use a special CLI option
for this case.

To streamline the generation of image test cases, drop the
`--with-customizations` option from the `generate-test-cases` script and
move the functionality to `format-request-map.json` by defining a new
`qcow2-customize` test case. This is very similar to what `*edge-rt`
test case already uses. This mean that when the test case for `qcow2`
image type is being generated, actually two test cases will be
generated. The filesystem customizations since RHEL-8.5 are handled
through distro-specific `overrides`.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-23 09:20:47 +02:00
Tomas Hozza
6502f527fd format-request-map.json: remove redundant overrides
Some of the `overrides` specified in `format-request-map.json` were
redundant, because the same blueprint customizations were already
specified in the image compose-request. Thus these values had no
additional values.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-23 09:20:47 +02:00
Tomas Hozza
0a3450ea3e generate-test-cases: don't leak "supported_arches" to compose request
Unfortunately, after the addition of `supported_arches` item to some
of the image type definitions in `format-request-map.json`, it started
to leak to the `compose-request` object in the generated image test
case.

Ensure that the item does not leak to the generated image test case,
since it is relevant only for the generator script.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-23 09:20:47 +02:00
Diaa Sami
4ea758a8bb tools: improve deploy-openstack script
Check first parameter and check for a dirty git repo before deploying
2021-09-22 12:24:08 +01:00
Tomas Hozza
50b85abd1c Update distro-arch-imagetype-map.json
Update distro-arch-imagetype-map.json to reflect currently generate
image type test cases.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-22 09:12:57 +02:00
Tomas Hozza
65919615df generate-all-test-cases: add option to keep created workdir on runner
Add a CLI option to keep the workdir created on the runner, after it
finishes its work. The workdir is deleted by default.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-22 09:12:57 +02:00
Tomas Hozza
57b0ed52ae generate-all-test-cases: report results when Runner finishes
Report per-distro list of image test cases which were successfully
generated or failed, after the Runner finishes its work.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-22 09:12:57 +02:00
Tomas Hozza
6203c1e430 generate-all-test-cases: support using existing remote hosts
Add a new command 'remote' allowing to generate image test cases on
existing remote hosts.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-22 09:12:57 +02:00
Tomas Hozza
6e08f0f363 generate-all-test-cases: remove '--keep-image-info' option
Remove the '--keep-image-info' script option. The output directory on
the Runner is always newly created at the beginning and empty, therefore
this option effectively can not be used with the `generate-test-cases`
script.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-22 09:12:57 +02:00