Commit graph

543 commits

Author SHA1 Message Date
Xiaofeng Wang
fc81b89cde test: remove workarounds for fixed bugs
Bugs have been fixed, workarounds are not needed anymore.
2024-02-07 08:52:30 +01:00
Tomáš Hozza
0c0a758600 Packer: default to RHEL-9.3 Schutzfile config for rhel-9
This is necessary to get the proper osbuild SHA to install the latest
required osbuild version into the image.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
2fe5e01d4d Test/api.sh: test distro aliases and names without dot-notation
Extend the `api.sh` to allow testing compose requests with distro name
which does not use dot-notation (specifically when the dot is removed
from the distro name as it used to be in the past). In addition to that,
allow also testing the distro alias using distro name without the minor
version in compose requests.

Enable these two new test variants in the CI.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
66571784d1 Use dot-notation for repo configs and in test cases.
Where applicable, modify all repo config filenames to use a dot
to separate the release major and minor version. Modify test cases
to not remove dot from the distro version any more.

Existing tests will be extended (or new tests added) to explicitly test
backward compatibility and ensure that using old distro names without a
dot still works.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
79d5ab483b tools: delete old Python manifest generator scripts
These are no longer used, let's delete them.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Sanne Raymaekers
589bd5f1a1 tools/fedora-worker-packer: fix packer only/except
It should only build fedora, not everything except fedora.
2024-01-23 13:06:38 +01:00
Sanne Raymaekers
6a0c893a4d tools/appsre-packer: add script to build fedora worker images 2024-01-23 00:18:51 +01:00
Sanne Raymaekers
c7aad1b17d tools/app-sre-build-worker-packer: split out the CI script
To add the fedora workers built from AppSRE's infrastructure, we need a
combination that wasn't possible with the current matrix based on the CI
environment (jenkins vs gitlab).
2024-01-23 00:18:51 +01:00
Sanne Raymaekers
3631316e36 tools/build-rpms: fix getting the osbuild commit from Schutzfile
osbuild/osbuild-composer#3614 removed the rhel-8.6, the centos-stream-9
entry should be there for a while at least.
2024-01-16 16:26:01 +01:00
Brian C. Lane
978c20d36f tools: Add helper script to check Schutzfile and gitlab-ci.yml runners
This script reads the RUNNER entries from .gitlab-ci.yml and the
snapshot repo names from Schutzfile. It then prints which runners don't
have entries in Schutzfile and which Schutzfile entries don't have
runners.

This is slightly complicated by the fact that centos-stream reports
itself as centos. So when checking for missing entries they are
considered to be the same thing. eg. If centos is a RUNNER and
centos-stream is in Schutzfile it doesn't report anything missing.
2024-01-16 12:29:21 +01:00
Michael Vogt
973a8cf285 image-info: update for new "partition" option in mounts.Mount
In the osbuild PR#1501 [0] a new paramter "partition" for
mounts.Mount() was added.

This commit updates the code that it deals cleanly with the new
and the old API for `mount.Mount`.

[0] https://github.com/osbuild/osbuild/pull/1501
2024-01-10 09:38:31 +01:00
Ondřej Budai
7bcf8e5942 test-case-generators/repos: add required repos for azure-sap-rhui
This image requires a different RHUI repository and also the saphana one.

Similarly to the previous commit, we should probably remove the manifest diff
tests altogether, but let's just fix them for now.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-12-22 00:11:19 +00:00
Miguel Martín
991293a897 Generate FIPS compliant SSH keys
Generate FIPS compliant SSH keys required
for testing system FIPS mode support
2023-12-13 10:19:47 +01:00
Sanne Raymaekers
4ee00f7dab tools/provision: disable tracing before manipulating OCI secrets 2023-11-07 10:43:32 +01:00
Jakub Rusz
d68abf3bb5 Generate RHEL-94 and RHEL-810 manifests + update 2023-11-06 13:39:01 +04:00
Tomáš Hozza
ebc256b073 Container: set the composer git revision during the build
When the container with osbuild-composer gets built in our CI or by
AppSRE, we do not set the composer version to any value (as we do when
we built RPMs). As a result, the version reported by composer is always
"devel". This is not useful for debugging and determining the used
version of composer. In addition, this information now gets exposed in
Koji builds, therefore it makes sense to make it useful.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-09 14:06:50 +02:00
Tomáš Hozza
5903aad07b tools/provision.sh: fix unbound variable bug if no OCI secrets are set
This was hit in c9s CI:
https://gitlab.com/redhat/centos-stream/rpms/osbuild-composer/-/merge_requests/88
https://artifacts.dev.testing-farm.io/658332f8-8705-4727-84d3-f27b57775037/

```
/usr/libexec/osbuild-composer-test/provision.sh: line 141: OCI_SECRETS: unbound variable
```

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-09-26 14:12:42 +02:00
Sanne Raymaekers
e81183daa8 tools/appsre-ansible/rpmbuild: retry all tasks 2023-09-25 11:56:42 +02:00
Sanne Raymaekers
522a54188a test/api: add oci test 2023-09-19 22:57:06 +02:00
Achilleas Koutsou
d3ef361dd5 test: update repos and manifests
Summary of changes:
- ova image type uses ustar format
    - https://github.com/osbuild/images/pull/16
- New image type "minimal-raw"
    - https://github.com/osbuild/images/pull/29
- Fedora repositories and test manifests updated
    - https://github.com/osbuild/images/pull/57
- Distro aliases for major version without minor updated
    - https://github.com/osbuild/images/pull/59
2023-08-03 10:35:21 +02:00
Brian C. Lane
8ff4c0c40a tests: Add a check for valid snapshot urls
This pulls the list of snapshots from the rpmrepo API, greps the
codebase for all uses of rpmrepo.osbuild.org that look like a snapshot
name, and then checks to make sure they are still valid.
2023-07-28 11:28:40 +03:00
Xiaofeng Wang
967570a2a6 test: Add workaround for BZ#2213660 and #2225667 2023-07-26 22:23:41 +08:00
Ondřej Budai
cac9327b44 update to go 1.19
UBI and the oldest support Fedora (37) now all have go 1.19, so we are
cleared to switch.

gofmt now reformats comments in certain cases, so that explains the formatting
changes in this commit.
See https://go.dev/doc/go1.19#go-doc

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-07-21 19:18:00 +02:00
Alexander Todorov
38ed9a39be Start collecting logs from virt-install 2023-07-21 12:11:02 +02:00
Brian C. Lane
3cfde5860d tools: Make pylint happier 2023-07-18 23:20:19 +02:00
Brian C. Lane
08e5c53d31 tools: Add support for local go package replacement
Local packages do not have a version number. Fake it with 'HEAD' to
indicate it's using whatever the working directory is pointing at.

The Provides line will then look like:

Provides: bundled(golang(../osbuild-images)) = HEAD

which rpmbuild is happy to process into an rpm.

This is ONLY for test builds on local systems and should never be seen
in the wild.
2023-07-18 23:20:19 +02:00
Brian C. Lane
f7dcc79bd8 tools: Set a+x on rpm_spec_vendor2provides 2023-07-18 23:20:19 +02:00
Achilleas Koutsou
f478323e3a tools: replace spec Provides generator
Replace the generator for the spec file "Provides:" list with the one
from the current go-rpm-macros [1].  The new generator can handle
'replace' commands in vendor/modules.txt, which makes it possible to
build RPMs with temporarily modified dependencies.  This enables making
scratch builds against forks or branches of the new osbuild/images
repository for development and testing.

In the future, we can use the packaged version of the script, but this
isn't yet available in all supported distros (EL8).

Made some minor modifications to the original script to prepend each
dependency with "Provides:" and read input files as arguments instead of
lines from stdin.

https://pagure.io/go-rpm-macros/blob/c32fbbd25bbcedee8c0b898d3653255b18a0d30e/f/rpm/go_mod_vendor.prov
2023-07-18 18:08:49 +02:00
Xiaofeng Wang
2baaa20859 test: Four fixes for RHEL for Edge tests
1. Remove ansible-blocking-io.py workaround. It's not required.
2. Variable should be PROD_REPO_URL, not STAGE_REPO_URL in
ostree-raw-image.sh
3. Use --reboot in rpm-ostree install to reboot VM instead of
a reboot ansible task
4. Wait until config file serviceinfo_api_server.yml exists, to
avoid file not available to use flaky issue
2023-06-30 09:07:39 +08:00
Achilleas Koutsou
78931c0f05 test: update fedora repos
Update Fedora 37, 38, and 39 repositories for the test case generator.
Changes are only package versions and dependencies.

For Fedora 39, a package (lklug-fonts) had to be removed from the
Anaconda package set since it is no longer available.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-27 16:13:25 +02:00
Simon de Vlieger
46f93614f2 fedora: live-installer image type
This image type produces the same artifact as the current workstation
installer live media.

During the implementation of this new installer some names have been
changed to make a bit more sense in the source tree. Installer images
now always mention which installer they are (anaconda, etc).
2023-06-27 13:18:56 +02:00
Ondřej Budai
6e756eb76b tools/libvirt_test.sh: add fedora support
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
6b1f1848e4 tools/libvirt_test.sh: remove hardcoded os-variant
Using rhel8-unknown for all distros was really weird, let's use the proper
variants.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
2c4215a2d0 tools/libvirt_test.sh: fix weird indentation
There were 5 spaces instead of 4.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
89b71d339e tools/libvirt_test.sh: merge two similar switch statements
No need for more switching than just one.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Alexander Todorov
1c3bd128d0 Update rpmrepo snapshots 2023-06-15 17:21:24 +02:00
Alexander Todorov
83cec26a5a Remove cloud-init/network-config file from tests
- not used in oscap.sh test at all
- causes VM in libvirt.sh test to acquire a different IP address instead
  of the hard-coded one inside the test, which causes SSH to fail

- not used anywhere else
2023-06-15 08:54:57 +02:00
Alexander Todorov
1f28e50d21 Adjust tests for new distros
References:
- https://redhat-internal.slack.com/archives/C0235DZB0DT/p1685528139831589
- 1fbd9d975f
2023-06-15 08:54:57 +02:00
Alexander Todorov
4aff9076ea Update rpmrepos & manifest files via command
go run ./cmd/gen-manifests/ -workers 50
2023-06-15 08:54:57 +02:00
Alexander Todorov
22437f7784 Add RPM repository definitions for 8.9 and 9.3 2023-06-15 08:54:57 +02:00
Achilleas Koutsou
403fe74ce4 format-request-map: clean up ostree params
Don't specify ref ID for parent, instead only specify a ref and URL and
let the manifest generator resolve the ID using the sha256 hash.
2023-06-14 11:19:29 +02:00
Irene Diez
a8f7ff487d distro/rhel9: add the edge-ami image type based on edgeRawImage
Creates the 'edge-ami' image type based on edgeRawImage, which generates
a raw image (x86_64, aarch64) ready to upload to AWS EC2.

This 'edge-ami' image type has Ignition support.

Signed-off-by: Irene Diez <idiez@redhat.com>
2023-06-07 13:18:01 +02:00
Ondřej Budai
73c8a482f7 repositories: update fedora 37-39 snapshots
Fedora 37 and 38 still pointed at branched/rawhide, oops. Let's just update
them all.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-06 13:04:10 +02:00
Ondřej Budai
eaa4e9b5fa repositories: fix GPG keys
Oh my, we had the Fedora 37 key even in Fedora 38 and 39 repositories. This
commit fixes that.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-06 13:04:10 +02:00
Ondřej Budai
894e6042c6 tools/test-case-generators/repos: remove fedora-36
It's EOL and already removed from the codebase. This was apparently forgotten
during the purge.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-06 13:04:10 +02:00
Sanne Raymaekers
f5b01125b6 tools/test-case-generators: add ovf image type 2023-05-25 10:14:32 +02:00
Tomáš Hozza
2a50092920 Test: add Fedora iot-raw-image test manifests with customizations
Add Fedora iot-raw-image test manifests with supported customizations,
specifically:
 - custom users and groups
 - custom enabled services
 - custom files and directories

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-22 16:30:14 +02:00
Ondřej Budai
ce5e41f980 tools/run-koji-container: allow passing a custom certificate dir
This helps with running the script locally.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-05-18 09:25:22 +02:00
Ondřej Budai
25161a8316 libvirt_test: stop setting nvram in virt-install on RHEL 9
We use CDN repositories when running tests on GA RHEL. Thus, our RHEL 9.1
machines are actually pulling packages from RHEL 9.2. Therefore, we are
now getting virt-install >= 4. This version brings firmware auto-detection
and doesn't support nvram_template anymore. Let's remove it then.

See 966049ec3c

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-05-12 11:02:49 +02:00
Sanne Raymaekers
c7d2eaa5f7 test: use non-deprecated gitlab CI variables
CI_BUILD_ID was replaced with CI_JOB_ID in gitlab 9+.
2023-05-12 11:02:49 +02:00