Commit graph

387 commits

Author SHA1 Message Date
Juan Abia
c4fa4e4bb7 test: Replace hard coded ssh keys on some test
Generate a new ssh key each time. Only implemented on bash test scripts,
go tests pending.
2021-07-29 09:38:03 +03:00
Lars Karlitski
9c2c92f729 jobqueue: Introduce jobqueue backed by a postgres database
Co-authored-by: sanne <sanne.raymaekers@gmail.com>
2021-07-28 21:52:31 +01:00
Martin Sehnoutka
700afb5af9 test/data: remove rhel-8-beta repository override
It contains beta repositories for RHEL 8.3, which is far outdated and
the file is not used anywhere any more.
2021-07-28 13:31:44 +02:00
Thomas Lavocat
4729990ac0
cloudapi: Add User customization
Cloud api now exposes user customization that let a customer able to add
a new user with a set of groups and a ssh key.

Testing:
* adds 2 users to the AWS image, accessible with a temp ssh key.
* the first one is in the group wheel, the other is not

Fixes #1574
2021-07-28 10:40:25 +02:00
Achilleas Koutsou
df1eddbf07 test/data: update aarch64 manifests
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-07-27 16:53:24 +01:00
Achilleas Koutsou
b113cf0f8a test/data: update x86-64 manifests
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-07-27 16:53:24 +01:00
Achilleas Koutsou
33b7fee194 test/data: generate aarch64 test manifests 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
d3d56dc6e6 test/data: generate x86_64 test manifests 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
e062fe3c44 test/api.sh: suppress shellcheck for sudo redirect 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
359100e1a1 test/api.sh: more robust edge commit validation
Instead of inspecting the tarball directly, extract it and use ostree to
verify the ref and commit ID.

Adds some data to the CI artifacts directory:
- Build manifest
- Tarball file list for s3 edge commit with s3 upload
- Build metadata
2021-07-24 15:54:00 +02:00
Ondřej Budai
025403ad34 tests/rhel8.5: use nightly composes instead of development ones
Nightly composes are more stable because they contain only builds attached
to advisories whereas development composes contain latest brew builds.

Use nightlies because they should be stabler.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-24 15:54:00 +02:00
Ondřej Budai
dc2458fc23 test/libvirt: run the uefi test also for 8.5
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-24 15:54:00 +02:00
Ondřej Budai
fba77534fd test/api: add RHEL 8.5 case
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
47ae5de5c0 test/cases: stop the spread of the tet typo
Hoping this is the end of this viral typo.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
35f0ef1053 test: regression test installing excluded dependency
Unlike the original regression test which explicitly names the excluded
dependency, this test installs a package that depends on the excluded
package without naming it. Based on the new way we depsolve blueprint
packages in RHEL 8.5, this should now work as well.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
2444e633fb test: separate regression from base tests
- Rewrote regression.sh to be like base tests with list of regression
  test scripts to be run and fail counts
- Separate regression tests in CI to have better control of distro
  runners
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
44a22bbb14 test/data: add aarch64 repos for RHEL 8.5 2021-07-24 15:54:00 +02:00
Chloe Kaubisch
4c800f29a7 worker: add metrics
use prometheus to gather metrics
2021-07-23 21:54:28 +02:00
Thomas Lavocat
cfc91a3eb6 CloudAPI: Invalid package set returns 400
Previously a bad error code was returned, fixes #1477.

Testing:

I have two test cases to test the solution. The first is a request that
makes depsolve crash by changing the dnf-json script by an almost empty
one that only throws an exception. The second one fails because it
requests a non existing package. The former ends with a 500 error and
the later with a 400.

----8<-----

HTTP/1.1 500 Internal Server Error

Failed to depsolve base packages for ami/x86_64/centos-8: ailed to
depsolve base packages for ami/x86_64/centos-8: unexpected end of JSON
input

----8<-----

HTTP/1.1 400 Bad Request

Content-Length: 226Failed to depsolve base packages for
ami/x86_64/centos-8: DNF error occured: MarkingErrors: Error occurred
when marking packages for installation: Problems in request:
missing packages: jesuisunpaquetquinexistepas_idonotexist
2021-07-21 16:21:15 +02:00
Achilleas Koutsou
583bfc0b4a test/cases/ostree: save build logs and manifests
Copy log and manifest for each build to the ci-artifacts/ directory to
be stored at the end of the build for troubleshooting.
2021-07-21 11:21:42 +02:00
Yuxin Sun
471353db60 test: Add Fedora 33 ostree test back
Signed-off-by: Yuxin Sun <yuxisun@redhat.com>
2021-07-20 21:18:02 +02:00
Yuxin Sun
c09638efe5 test: Fedora error/fail logs should not be a must
Signed-off-by: Yuxin Sun <yuxisun@redhat.com>
2021-07-20 09:56:38 +03:00
Jakub Rusz
ab3234260a tests: fix wrong default value in azure.sh test
It's not possible to create vm with the single quotes in the name.
Removing it to enable running locally.
2021-07-16 10:59:15 +02:00
Tomas Hozza
3a0540dff0 test/api.sh: randomize used GCP zone from the region
The `api.sh` test currently always defaults to "<REGION>-a" zone when
creating instance using the built image. The resources in a zone may get
exhausted and the solution is to use a different zone. Currently even a
CI job retry won't help with mitigation of such error during a CI run.

Modify `api.sh` to pick random GCP zone for a given region when creating
a compute instance. Use only GCP zones which are "UP".

The `cloud-cleaner` relied on the behavior of `api.sh` to always choose
the "<REGION>-a" zone. Guessing the chosen zone in `cloud-cleaner` is
not viable, but thankfully the instance name is by default unique for
the whole GCP project. Modify `cloud-cleaner` to iterate over all
available zones in the used region and try to delete the specific
instance in each of them.

Make `ComputeZonesInRegion` method from the `internal/cloud/gcp` package
exported and use it in `cloud-cleaner` for getting the list of available
zones in a region.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-16 10:14:30 +02:00
Tomas Hozza
1fd151ce28 test/api.sh: fix generation of predictable TEST_ID in GitLab CI
The `test/cases/api.sh` script relied on environment variables specific
to Jenkins for detecting it if is running in a CI environment. If this
was the case, it used other environment variables to construct a
predictable `TEST_ID` which could be used for names of resources created
in cloud-provider environment as part of the test. This is important to
ensure that `cloud-cleaner` can "guess" resource names and delete them
in case the test script fails to clean up after itself.

With the move from Jenkins to GitLab CI, this stopped to work and the
script started to generate random `TEST_ID`, which can not be guessed by
the `cloud-cleaner` tool.

Modify the `test/cases/api.sh` to detect the CI environment using the
`CI` environment variable, which is always predefined in the GitLab CI
environment [1].

[1] https://docs.gitlab.com/ee/ci/variables/predefined_variables.html

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-16 10:14:30 +02:00
Ondřej Budai
984717628e tests: use rpmrepo for f33 aarch64
This commit also makes aarch64 and x84_64 consistent with each other
and updates the updates repos for x86_64

Fixes #1551

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-14 17:54:24 +02:00
Ondřej Budai
f8fd3d04b8 test/fedora33: use rpmrepo instead of kernel mirrors
mirrors.kernel.org are sometimes desynchronized, and thus causing issues with
the image tests. This commit switches the image tests for F33 to use RPMRepo.

All test cases were regenerated.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-14 12:05:45 +02:00
Jakub Rusz
1b5ca0f163 tests: make azure.sh easier to run locally
We've dropped Jenkins already and this change helps by setting
CI variables to a default value in case they're not present.
2021-07-09 19:52:52 +01:00
sanne
4385c39d66 worker: Introduce heartbeats
An occupied worker checks about every 15 seconds if it's current job was
cancelled. Use this to introduce a heartbeat mechanism, where if
composer hasn't heard from the worker in 2 minutes, the job times out
and is set to fail.
2021-07-08 21:14:38 +01:00
Xiaofeng Wang
c025c60544 test: Allow a failed dmesg imported by new RT kernel
Due to bz#1979272, the failed dmesg is not caused by osbuild, but
RT kernel itself. Allow this failed dmesg to make test happy.
2021-07-06 11:41:46 +01:00
Ondřej Budai
0201822c50 tests: remove all traces of Fedora 32
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-05 11:16:08 +02:00
Xiaofeng Wang
4e4012df28 test: Update ostree.sh to cover bz#1896178
For RHEL 8.5, user and ssh key are added in blueprint.
For Fedora 33, RHEL 8.3 and 8.4, user and ssh key have to be
added in ks file due to bz#1896178.

bz#1896178 has been fixed already on RHEL 8.5 only, but still exists
on Fedora 33, RHEL 8.3 and 8.4. This update covers Fedora 33,
RHEL 8.5, RHEL 8.4 and RHEL 8.3.
2021-07-05 11:15:58 +02:00
Achilleas Koutsou
6e22e1de64 test/cases: fix installers test
COMPOSE_ID shouldn't be set but taken from the build response
2021-07-02 17:57:22 +02:00
Xiaofeng Wang
a0ec753e05 test: Update check_ostree.yaml to work with RHEL 8.5
There's no any error message in dmesg output on RHEL 8.5 Edge image.
Update playbook to support that.
2021-07-01 13:34:12 +02:00
Xiaofeng Wang
35263e5ab4 test: Remove fedora 32 test
GitLab CI dropped Fedora 32 support, fedora 32 test should be
removed.
2021-07-01 13:34:12 +02:00
Achilleas Koutsou
f557ab57bf test/data: update edge test manifests
Core stages were updated to include the fix-bls stage.
Updating existing RHEL 8.5 test manifests.
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
a877326671 test: add installer integration test
Builds a tar-installer, installs the OS, boots it, and checks if the
packages selected in the blueprint are installed.
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
c4aeb256a5 test/data: test case for tar x86_64 2021-07-01 12:48:32 +02:00
Yuxin Sun
6cd6e81e85 Add subscription test case and Azure boot up case
Signed-off-by: Yuxin Sun <yuxisun@redhat.com>
2021-06-30 12:29:33 +02:00
Achilleas Koutsou
f557bf9a8b test/cases/ostree: test new image types
ostree.sh: add case for RHEL 8.5 edge-commit
ostree-ng.sh: conditional container and installer type and file names
2021-06-29 17:26:43 +01:00
Achilleas Koutsou
0b6452bb34 test: new manifests for RHEL 8.5 edge types 2021-06-29 17:26:43 +01:00
Ondřej Budai
cc2c354371 tests: regenerate manifests for rhel 9.0
Previous commits bumped the repo snapshots and changed the manifest generation
for RHEL 9.0.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-06-29 13:56:26 +02:00
Ondřej Budai
39f1d4d485 tests: update snapshot for rhel 9.0
We have a newer one so let's use it.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-06-29 13:56:26 +02:00
Achilleas Koutsou
0ddfc32957 test: verify selected package appears in metadata
Add postgresql to all builds and verify that it's listed in the metadata
2021-06-29 09:33:05 +01:00
Achilleas Koutsou
7e01a9125b test: verify commit ID hash returned from API 2021-06-29 09:33:05 +01:00
Jakub Rusz
b1e40b5ce7 tests: modify image_tests for Gitlab CI 2021-06-28 13:49:57 +03:00
Jakub Rusz
bb2f866470 tests: updating README with info about nightly testing in Gitlab CI 2021-06-25 15:10:40 +03:00
Yuxin Sun
704a3675b9 Update fedora-33 test repo
Signed-off-by: Yuxin Sun <yuxisun@redhat.com>
2021-06-25 08:40:49 +03:00
Alexander Todorov
1eb51b7bab Test Azure images on HyperV Gen1 and Gen2
Related: rhbz#1896264
2021-06-22 13:44:19 +03:00
Aleksandar Todorov
24f297859c Azure: deploy to V2 Hyper-V generation
Related: rhbz#1896264
2021-06-22 13:44:19 +03:00