Commit graph

323 commits

Author SHA1 Message Date
Achilleas Koutsou
56839545ea test-case-generators: support generating edge-installer manifests
Adds ostree params to the request to generate a test manifest for
edge-installers and necessary customisations in the blueprint for the
edge-simplified installer.  The manifest is not buildable but works for
checking for changes in the pipeline and packages for the installers.
2022-02-26 15:55:56 +00:00
Xiaofeng Wang
6c4e179171 test: Add work around for bug bz#2057769
bz#2057769 blocked UEFI vm test

Co-Authored-By: Christian Kellner <christian@kellner.me>
2022-02-26 00:27:21 +00:00
Jakub Rusz
3fb80d199d tests/libvirt: add some regression checks
This is just a simple check to verify rhbz#2004401 and
rhbz#2003038
2022-02-25 13:25:06 +02:00
Jakub Rusz
7006a53e71 tools/provision: use upstream EPEL-9
EPEL-9 now exists upstream, let's use it.
2022-02-24 17:54:30 +01:00
Achilleas Koutsou
eda78a9a99 Update CentOS Stream 8 repositories 2022-02-24 17:07:34 +01:00
Achilleas Koutsou
fa209fe747 test: generate aarch64 image installer manifests
Added to imagetype map for test case generators and generate new
manifests.
2022-02-24 16:30:04 +01:00
Xiaofeng Wang
8051ddced5 test: Remove "ansible-galaxy collection install"
Since "ANSIBLE_STDOUT_CALLBACK=yaml" has been removed from all
ostree test script, "ansible-galaxy collection install" should be
removed as well
2022-02-22 19:25:48 +00:00
Achilleas Koutsou
96e09e0322 Update cs9 test repos to latest snapshot 2022-02-18 10:39:08 +00:00
Achilleas Koutsou
f7d8535b16 tools/test-case-generators: add s390x for CS9 to test gen map 2022-02-18 10:39:08 +00:00
Achilleas Koutsou
6bc4afbcbd test,repositories: add cs9 repositories
Add rpmrepo repositories for testing.
Symlink centos-9 to centos-stream-9.
Add rpmrepo snapshot repositories to test and
test-case-generator repos.
Remove unused repositories from cs9 test repositories.
2022-02-18 10:39:08 +00:00
Achilleas Koutsou
e0822bff44 repos: update repos for test case generators
test/data/repositories:
    new snapshots for RHEL 8.6 (20220201) and 9.0 (20220208)

Switching RHEL 8.6 and RHEL 9.0 to the latest rpmrepo snapshots:
    We need updated packages for greenboot, used by edge image types, so
    we technically only need to change the architectures that are
    supported by edge, but let's update all of them for consistency.

test/data/repositories:
    Remove rhel-90 symlink to beta

Keep the rhel-90-beta.json repository definitions, but make rhel-90.json
a file that contains the previously named rhel-90-ga.json definitions.
2022-02-16 10:49:55 +01:00
Tomas Hozza
2182996aae tools/image-info: extend the gathered information
Extend the information gathered by `image-info`. This is needed to
properly inspect GCE images.

Enhancements:
 - read all ssh client configuration files
 - read all sshd configuration files
 - read all YUM / DNF repos
 - read DNF Automatic configuration
 - fix reading of DNF configuration

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-15 15:56:56 +01:00
Tomas Hozza
ece6452e2a Use the latest RPMRepo snapshot for RHEL-9.0 image tests
This is needed to satisfy RPM dependencies of GCP guest tools, which
will be installed on the `gce` image type.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-14 12:39:12 +01:00
Sanne Raymaekers
b3796a7319 tools/appsre-ansible: Explicitly specific mock config 2022-02-09 18:33:14 +01:00
Sanne Raymaekers
a173a3513d tools/appsre-build-worker-packer: Run on subscribed 8.5 machine 2022-02-09 16:54:22 +01:00
Tomas Hozza
f43d9f4744 Add missing F34 and F35 image test cases
Add F34 and F35 image test cases for all remaining image types, which
were previously not tested. With this PR, image test cases are now
generated for all image types on all architectures as supported by the
Fedora distro definition.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-09 15:29:33 +01:00
Tomas Hozza
1401a7a659 test: add image test cases for Fedora 34 and 35
Delete Fedora 33 image test cases.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-08 10:19:52 +01:00
Achilleas Koutsou
93c0485531 tools/provision: remove json_query usage from ansible playbooks
`json_query` requires python3-jmespath which, while available in the
repos, it can sometimes cause issues when the ansible interpreter is
different from the system interpreter.
The `json_query` is only used in a handful of locations that can easily
be served by `jq`, which we use in other places already.
2022-02-03 00:24:59 +01:00
Tom Gundersen
11e2ae4528 cloudapi/v2: add koji-specific tests
These are based on the existing tests for the koji API, making sure the coverage is the same.
2022-02-01 20:28:40 +00:00
sanne
6b716a4bbc tools/appsre-build-worker-packer: Fix branch parsing on jenkins 2022-01-19 12:48:14 +01:00
sanne
a6f40f807d tools/appsre-build-worker-packer: Run entire workflow in container
Avoids the issue with artifacts that jenkins can't clean up.
2022-01-13 15:05:52 +01:00
sanne
2d0bfedc9d tools/appsre-ansible: Correct rpm destination when pulling rpms 2022-01-12 19:47:45 +01:00
sanne
e8b790e8d4 tools/appsre-build-worker-packer: Disable ansible debug output 2022-01-12 19:47:45 +01:00
sanne
6b43594840 tools/appsre-build-worker-packer: Ansible rpmbuild debug output 2022-01-12 17:33:50 +01:00
sanne
e6fd492372 tools/appsre-build-worker-packer: Prevent ssh from timing out 2022-01-12 17:33:50 +01:00
sanne
a21061fc5c tools/appsre-ansible: Don't use /tmp for rpmbuilds 2022-01-12 15:27:27 +01:00
sanne
93ec151c60 tools/appsre-build-worker-packer: Verbose ssh check 2022-01-12 12:33:04 +01:00
sanne
8c514308e7 tools/appsre-build-worker-packer: Correct user in ssh check 2022-01-12 12:13:41 +01:00
sanne
01163a1384 tools/appsre-build-worker-packer: Use ssh instead of ssh-keyscan
`ssh-keyscan` on appsre's infra has a bug where it doesn't exit 1 in
case of failure.
2022-01-12 12:02:58 +01:00
Ondřej Budai
b667df4443 bump the minimum Go version to 1.16
Fedora 34 ships 1.16
RHEL 8.5 also ships 1.16

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-01-12 11:35:06 +01:00
sanne
3c729be3c5 tools/appsre-build-worker-packer: Add image_users variable
packer will share the ami with those users.
2022-01-11 14:30:19 +01:00
sanne
2420504e1d tools/appsre-build-worker-packer: Use WORKSPACE variable
Docker doesn't accept "." as a mountpoint, use the WORKSPACE variable on
jenkins and default to PWD.
2022-01-11 14:30:19 +01:00
sanne
204bf9e176 tools/appsre-build-worker-packer: Don't fast-fail cleanup 2022-01-07 11:42:16 +01:00
sanne
4970e4d788 tools/appsre-build-worker-packer: Use CI_COMMIT_SHA
CI is also defined on appsres infra, and is also true. Use CI_COMMIT_SHA
instead to determine if we're running from a PR or not.
2022-01-07 10:35:34 +01:00
sanne
71da979c81 tools: AppSRE packer build 2022-01-05 22:13:55 +01:00
Tomas Hozza
a5319e78d4 (re)generate image test cases for image-installer on all distros
The existing test cases for `image-installer` image type were not
regenerated as part of PR #2135 which introduced 'image_type_tags'
because regenerating them using `generate-all-test-cases` failed on them
due to trying to analyze them using 'image-info'. These test cases were
most probably originally generated manually with --keep-image-info
option passed to `generate-test-cases`. And I missed that their
regeneration failed at that time.

Add the `image-installer` image type to all distros in
`distro-arch-imagetype-map.json` and (re)generate the test cases for it.

The image test case for CentOS Stream 9 is not generated, because there
are no repos defined yet for the distro. This is part of the CentOS 9
PR #2142.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-21 09:59:17 +01:00
Tomas Hozza
3025bbb775 Tools: add 'no-image-info' option to image test case requests
Add a new option `no-image-info`, which can be specified for an image
type in `format-request-map.json` in case the image can not be analyzed
using `image-info` and thus it should not be run as part of generating
the image test case.

This is the case for e.g. the `image-installer` image type in RHEL-8.6,
which effectively breaks generating all image test cases for RHEL-8.6
due to `image-info` failing to analyze the resulting image.

This change will allow us to consistently generate image test cases
(manifests only) for all image types, which were so far not covered by
the image testing.

Modify the `generate-test-cases` script to recognize the `no-image-info`
option in the `format-request-map.json`

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-21 09:59:17 +01:00
Tomas Hozza
4a2041b38e generate-all-test-cases: use make scratch for building RPMs
Use `make scratch` for building RPMs without tests. This fixes the case,
when the RPM build with test fails to build due to changes in image
manifests. The whole reason of running the script is to regenerate image
test cases when the manifest changed, so this was a chicken and egg
problem.

No CI testing is needed, as this is a development tool.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-20 14:52:45 +01:00
Tomas Hozza
33c7da9dc3 tools: use image_type_tags in repos used for image test cases
Use image_type_tags in repo definitions used for generating image test
cases. Enhance the `generate-test-cases` script to take the
image_type_tags into account when creating a compose request for a
specific image test case.

The motivation for the change is to keep the list of repositories in the
compose request of a image test case as small as possible. Previously,
all of the defined repositories were part of each image test cases for a
specific architecture, even those that were not needed.

Regenerate affected image test cases.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-16 18:07:50 +01:00
Thomas Lavocat
e6a6ed01ea Tests: start dnf-json-socket when needed 2021-12-15 09:41:32 +01:00
Jakub Rusz
6e5642c2d2 tests: add sudo to get_metadata function
With new weldr-client package the metadata tar archive created has
permissions set to 600 instead of 644 which causes permission failures
when interacting with it. Adding sudo to resolve that.
2021-12-08 12:12:53 +01:00
Tomas Hozza
523bcdd98e generate-all-test-cases: add --build-rpms option
Add `--build-rpms` option which will make the script build osbuild-composer
RPMs on the remote runner from the sources which were copied over. These
RPMs are then installed on the system, before any image test cases are
generated.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-07 09:19:41 +01:00
sanne
c43ad2b22a osbuild-service-maintenance: Clean up expired images 2021-12-03 00:14:09 +00:00
Ondřej Budai
fa73586932 nightly: use boston mirror instead of the redirecter
It often takes a lot of time to synchronize the compose across all mirrors,
so it's safer to use just one, otherwise we can get hit by 404s if the
compose isn't available yet by a mirror.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-11-26 12:20:54 +01:00
sanne
ea2812f48f test: Switch api tests to v2 2021-11-18 10:26:17 +01:00
Tomas Hozza
dd0f01edf5 Use RHUI-4 for RHEL-9 EC2 image test cases
Use newly added RHUI-4 repo snapshots for all RHEL-9 EC2* image test
cases. This includes RHEL-9.0 and RHEL-9.0-Beta images. The removal
of installed repos on EC2-HA and EC2-SAP images is expected, as RHUI
client RPMs for these variants are empty for 9.0 Beta. This should
change for GA once there are updated RHUI clients RPMs available.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-11-16 21:01:27 +01:00
Jakub Rusz
0508a7ad1b tests: run ostree.sh tests on fedora and centos
When tools/define-compose-url.sh was sourced the tests were skipped if
they were not running on rhel. Also change the BOOT_LOCATION for
centos-8.
2021-11-16 10:46:50 +01:00
Jakub Rusz
14b7e92eaa tests/ci: split nightly testing based on RHEL major version
Using just the major version from the $VERSION_ID and defining new CI
variable $RHEL_MAJOR.
2021-11-16 10:46:50 +01:00
Achilleas Koutsou
5786db9caa tools: update distro-arch-imagetype-map for RHEL 9.0 types
`rhel-90-ga` no longer valid.  Now called `rhel-90`.
2021-11-10 14:54:31 +01:00
Achilleas Koutsou
0c185eca7a tools: update RHEL 9.0 repos for test case generators
No more `rhel-90-ga`.
2021-11-10 14:54:31 +01:00