Commit graph

417 commits

Author SHA1 Message Date
Alexander Todorov
42480f7b26 tests: Cleanup leftover 8.4 references 2021-08-19 12:21:52 +02:00
Jakub Rusz
d927bfbd30 tests: rename Azure resources in API test
Use TEST_ID for any resources created in Azure. Also create all
necessary vm network resources in advance to have predictable names
using TEST_ID as well.
2021-08-19 09:04:28 +02:00
Ondřej Budai
7a298c838b tests: remove all disabling conditions for RHEL 9.0
RHEL 9.0 isn't yet in .gitlab-ci.yml so this actually doesn't change in test
runs but it should make enabling of the tests easier.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-17 16:54:22 +02:00
Ondřej Budai
0336612160 test/ostree: add test case for RHEL 9.0
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-17 16:54:22 +02:00
Ondřej Budai
9ae81cbbeb test/libvirt: enable uefi test for RHEL 9.0
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-17 16:54:22 +02:00
Ondřej Budai
91dd6e3346 test/api: add test case for RHEL 9.0
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-17 16:54:22 +02:00
Tomas Hozza
634dcc0f25 rhel85/90: regenerate aarch64 ami and ec2 image test cases
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-13 08:24:37 +02:00
Ondřej Budai
3f519ace8a test: update repository snapshot
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00
Ondřej Budai
cbe11e280b test/manifests: generate all test manifests for rhel-90
This commit also updates the rpmrepo snapshot for rhel-90 to the latest one.

Note that vmdk and vhd images are currently not buildable, see:
https://bugzilla.redhat.com/show_bug.cgi?id=1989841

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-08-11 11:22:59 +02:00
Achilleas Koutsou
8af0431883 test/casea: trim installer test blueprint 2021-08-11 01:31:47 +02:00
Achilleas Koutsou
ab264abd76 distro/rhel85: tar-installer -> image-installer
Renamed tar-installer to image-installer.
This is a more appropriate name:
- It disassociates the image type from the "tar" image type. The two
  should not be perceived to be connected.
- It's more descriptive. The format of the payload (tar) isn't relevant
  to the purpose of the image type.
2021-08-11 01:31:47 +02:00
Achilleas Koutsou
e171977d09 test/data: add test manifest for ppc64le 2021-08-10 22:38:36 +02:00
Achilleas Koutsou
532ac748cb test/data: add test repos for ppc64le
Also edited s390x to use latest available RPMrepo snapshot.
2021-08-10 22:38:36 +02:00
Martin Sehnoutka
d149d3ab4e test: Make the Satellite test generic over distributions
The test case used to work only on Fedora 33. This patch introduces
support for all future Fedoras and RHEL 8.

It uses a fair bit of magic because it needs to dynamically discover if
a RHEL runner uses the rhel-8.json or the rhel-8-beta.json repository
override. Furthemore it needs to discover a redirection URL, if any, of
the repository override to correctly configure HTTP proxy.
2021-08-09 12:40:23 +02:00
Martin Sehnoutka
4ceb0b41da test/data: update rhel 8.3 repository overrides
Use the latest RPMrepo snapshots available.
2021-08-09 12:40:23 +02:00
Martin Sehnoutka
4d0dcaeff0 test/data: update rhel 8.5 repository overrides
Use the latest RPMrepo snapshots available.
2021-08-09 12:40:23 +02:00
Martin Sehnoutka
d0c0c8791f test/regression: disable excluded pkgs test for Fedora
It is designed for RHEL only. The switch for distros used to live in the
regression.sh script but it was removed so now the tests need to know if
they are running on a compatible system themselves.
2021-08-09 12:40:23 +02:00
Martin Sehnoutka
e6737f71d1 test/cases/ostree*: remove mod_ssl configuration file
It clashes with osbuild-composer already running on port 443.
2021-08-09 12:40:23 +02:00
Martin Sehnoutka
2d75b25ffe test/cases: new case that verifies composer works behind satellite
Introduce a test case where 2 httpd proxies are used to simulate custom
repos with custom subscriptions. That is, every repo requires
client-side tls certificate with its own CA. Make sure both osbuild and
osbuild-composer can use these repositories by issuing an image build.
2021-08-09 12:40:23 +02:00
Achilleas Koutsou
a49d28dfc4 test/cases: install python3 for installer test
Required by Ansible to run checks
2021-08-09 07:57:15 +02:00
Achilleas Koutsou
d96c72e6f4 test: fix host group name in ansible test file
Host group should be 'guest'. The ansible inventory file in the test
uses the group name 'guest'.

Change package check to tmux since zsh isn't installed.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-08-09 07:57:15 +02:00
Thomas Lavocat
fd81593026 Tests: Malformed JSON duplicate customization
Fixes #1612
2021-08-06 10:38:20 +02:00
Achilleas Koutsou
5a4d6bdfe7 test/data: add test repos and manifest for s390x
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-08-06 07:30:05 +02:00
Achilleas Koutsou
7d08535232 test/data: Add tar-installer test manifest
Added to format-request-map and generated manifest.
No image-info since ISOs aren't supported by the tool. The manifest is
useful only for detecting changes in the image type definition.
2021-08-05 16:13:46 +02:00
Achilleas Koutsou
6e8ceb2e41 test/cases: complete installer test
Several changes required to get the installer test working in CI.

Blueprint:
- Add packages for SSH and networking
- Use generated SSH key and username variable for user setup

Kickstart:
- The ISO contains a kickstart file with the `liveimg` command. Instead
  of trying to inject a kickstart during virt-install, we override the
  existing file on the ISO. The `modksiso` function reads a kickstart
  file from an existing ISO, adds commands to make the installation
  fully automatic (partitioning, text --non-interactive) and creates a
  new ISO with the modified kickstart file using mkksiso.
- Simple networking added to kickstart file.
- Kernel command line args for installer added via the mkksiso command
  since the `--cdrom` installation source doesn't support injecting
  command line arguments.

virt-install:
- Using `--cdrom` instead of `--location`: latter was failing to
  install.
- No longer injects a kickstart file (since we handle it separately) and
  doesn't add extra Kernel args.
2021-08-05 16:13:46 +02:00
Tomas Hozza
aed3bf785c image-info: mount partitions in correct order when analysing image
image-info's code which analysed image with multiple partitions was not
correctly working with more than two partitions, which had to be the
root '/' and EFI partition '/boot/efi'. The consequence was that SELinux
labels on paths which were mounted incorrectly could have been reported
as incorrect.

Modify `append_partitions()` to first read the fstab entries and then
mount all partitions using their UUID in the correct order. Only then
analyze the image filesystem tree.

Regenerate affected image test cases.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-05 09:34:07 +02:00
sanne
57555991e0 test/api: Use relevant distributions testing package conflicts 2021-08-03 15:14:22 +02:00
Tomas Hozza
c97420e879 rhel85: add definitions of RHEL ec2 and ec2-ha images
Add new image type definitions `ec2` and `ec2-ha` representing the
official RHEL ec2 image types.

Add a `xzArchivePipeline()`, which returns a pipeline producing a XZ
archive from a file produced by a different pipeline.

Add rpmrepo snapshots for `rhui` and `ha` repositories used to generate
image test cases. `rhui` is used by the `ec2` image and it is available
on x86_64 and aarch64 architectures. `ha` is used by the `ec2-ha` image
and it is available only for x86_64.

The new image type definitions are currently not used by any
API test case.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-02 19:05:09 +02:00
Tomas Hozza
02ff0fc606 rhel85: redefine the ami image based on RHEL ec2 images
Redefine the `ami` image type in RHEL-8.5 to be based on RHEL
ec2 images. The pipeline has different default settings, therefore the
common "os" pipeline is not used. The RHEL ec2 images have a different
default size than the original `ami` image definition. The RHEL ec2
images use a different default partitioning scheme. Their configuration
is slightly different for each architecture and the x86_64 version
of the image does not support UEFI.

Update rpmrepo snapshots used to generate RHEL-8.5 x86_64 and aarch64
image test cases.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-08-02 19:05:09 +02:00
Achilleas Koutsou
248632eaba test/data: update manifests
The previous manifests were generated inside a RHEL VM which means that
the selinux context mismatch check didn't run properly
(see https://bugzilla.redhat.com/show_bug.cgi?id=1973754).

Regenerating them on F34 shows some mismatches for certain image types.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-07-29 16:45:20 +02:00
Juan Abia
c4fa4e4bb7 test: Replace hard coded ssh keys on some test
Generate a new ssh key each time. Only implemented on bash test scripts,
go tests pending.
2021-07-29 09:38:03 +03:00
Lars Karlitski
9c2c92f729 jobqueue: Introduce jobqueue backed by a postgres database
Co-authored-by: sanne <sanne.raymaekers@gmail.com>
2021-07-28 21:52:31 +01:00
Martin Sehnoutka
700afb5af9 test/data: remove rhel-8-beta repository override
It contains beta repositories for RHEL 8.3, which is far outdated and
the file is not used anywhere any more.
2021-07-28 13:31:44 +02:00
Thomas Lavocat
4729990ac0
cloudapi: Add User customization
Cloud api now exposes user customization that let a customer able to add
a new user with a set of groups and a ssh key.

Testing:
* adds 2 users to the AWS image, accessible with a temp ssh key.
* the first one is in the group wheel, the other is not

Fixes #1574
2021-07-28 10:40:25 +02:00
Achilleas Koutsou
df1eddbf07 test/data: update aarch64 manifests
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-07-27 16:53:24 +01:00
Achilleas Koutsou
b113cf0f8a test/data: update x86-64 manifests
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2021-07-27 16:53:24 +01:00
Achilleas Koutsou
33b7fee194 test/data: generate aarch64 test manifests 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
d3d56dc6e6 test/data: generate x86_64 test manifests 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
e062fe3c44 test/api.sh: suppress shellcheck for sudo redirect 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
359100e1a1 test/api.sh: more robust edge commit validation
Instead of inspecting the tarball directly, extract it and use ostree to
verify the ref and commit ID.

Adds some data to the CI artifacts directory:
- Build manifest
- Tarball file list for s3 edge commit with s3 upload
- Build metadata
2021-07-24 15:54:00 +02:00
Ondřej Budai
025403ad34 tests/rhel8.5: use nightly composes instead of development ones
Nightly composes are more stable because they contain only builds attached
to advisories whereas development composes contain latest brew builds.

Use nightlies because they should be stabler.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-24 15:54:00 +02:00
Ondřej Budai
dc2458fc23 test/libvirt: run the uefi test also for 8.5
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-24 15:54:00 +02:00
Ondřej Budai
fba77534fd test/api: add RHEL 8.5 case
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
47ae5de5c0 test/cases: stop the spread of the tet typo
Hoping this is the end of this viral typo.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
35f0ef1053 test: regression test installing excluded dependency
Unlike the original regression test which explicitly names the excluded
dependency, this test installs a package that depends on the excluded
package without naming it. Based on the new way we depsolve blueprint
packages in RHEL 8.5, this should now work as well.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
2444e633fb test: separate regression from base tests
- Rewrote regression.sh to be like base tests with list of regression
  test scripts to be run and fail counts
- Separate regression tests in CI to have better control of distro
  runners
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
44a22bbb14 test/data: add aarch64 repos for RHEL 8.5 2021-07-24 15:54:00 +02:00
Chloe Kaubisch
4c800f29a7 worker: add metrics
use prometheus to gather metrics
2021-07-23 21:54:28 +02:00
Thomas Lavocat
cfc91a3eb6 CloudAPI: Invalid package set returns 400
Previously a bad error code was returned, fixes #1477.

Testing:

I have two test cases to test the solution. The first is a request that
makes depsolve crash by changing the dnf-json script by an almost empty
one that only throws an exception. The second one fails because it
requests a non existing package. The former ends with a 500 error and
the later with a 400.

----8<-----

HTTP/1.1 500 Internal Server Error

Failed to depsolve base packages for ami/x86_64/centos-8: ailed to
depsolve base packages for ami/x86_64/centos-8: unexpected end of JSON
input

----8<-----

HTTP/1.1 400 Bad Request

Content-Length: 226Failed to depsolve base packages for
ami/x86_64/centos-8: DNF error occured: MarkingErrors: Error occurred
when marking packages for installation: Problems in request:
missing packages: jesuisunpaquetquinexistepas_idonotexist
2021-07-21 16:21:15 +02:00
Achilleas Koutsou
583bfc0b4a test/cases/ostree: save build logs and manifests
Copy log and manifest for each build to the ci-artifacts/ directory to
be stored at the end of the build for troubleshooting.
2021-07-21 11:21:42 +02:00