Commit graph

559 commits

Author SHA1 Message Date
Tomáš Hozza
71b9c0ad65 Fix pylint issues
Add timeout to `requests.post()` and `requests.get()` calls to make
pylint happy.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-05-30 19:58:34 +02:00
Sanne Raymaekers
a96f1b6d31 templates/packer: switch to fedora-40
Fedora 38 is EOL, and packit no longer builds rpms for it.

The current python3.12 + ansible 2.12 combination which is the default
on fedora 40 doesn't work, so switch to python3.9.
2024-05-29 19:36:31 +02:00
Achilleas Koutsou
4241f5bb39 tools/gen-ssh: upgrade openssh before generating key
openssl gets installed as a dependency of the osbuild-composer-tests but
it might not update openssh at the same time, which can cause a version
mismatch when running ssh-keygen:

  OpenSSL version mismatch. Built against 30000000, you have 30200010

This is currently happening in CentOS Stream 9 so let's make sure
openssh is up to date before running ssh-keygen.
2024-05-29 00:27:31 +02:00
Sanne Raymaekers
13aae7d532 templates/packer: invert tag logic
With the rpmcopy or rpmrepo_osbuild tags, the `Install worker rpm` stage
got skipped on RHEL and CI. Invert the tag logic and use `--tags`
instead of `--skip-tags`.
2024-05-21 09:40:11 +02:00
Sanne Raymaekers
9c047cd8e0 tools/fedora-worker-packer: don't trace statements
It makes the logs extremely verbose, and requests/responses are already
printed when relevant.
2024-05-07 18:38:13 +02:00
Sanne Raymaekers
aadf5dfc9a tools/fedora-worker-packer: increase timeouts further
The first timeout increase didn't take effect. And once the suite is
present the rpm builds can stay pending for a very long time.
2024-05-07 18:38:13 +02:00
Sanne Raymaekers
c9ce5af655 tools/fedora-worker-packer: set the commit sha
Currently the COMMIT_SHA variable is empty.
2024-05-07 14:47:59 +02:00
Sanne Raymaekers
28e4b0aa07 tools/fedora-worker-packer: increase timeout waiting for packit 2024-05-07 14:47:59 +02:00
Sanne Raymaekers
a1476bdfb1 tools/appsre-build-worker-packer: stop writing bogus files 2024-05-07 13:57:48 +02:00
Sanne Raymaekers
61848cb88d tools/appsre-build-fedora-worker-packer: wait for packit
Wait for packit to complete it's runs, after which the rpms should be
available from the @osbuild/osbuild-composer copr repo.
2024-05-07 13:57:48 +02:00
Sanne Raymaekers
616db61105 tools/build-worker-packer: skip rpmcopr for RHEL and on CI
Also skip both rpmrepo tasks on RHEL.
2024-05-07 13:57:48 +02:00
Achilleas Koutsou
f98169c29a go.mod: update to go v1.20
Go 1.20 is included in all currently supported distro versions.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2024-04-11 16:39:21 +02:00
Alexander Todorov
4db594dac7 Skip image_installer tests b/c already tested in installers.sh 2024-03-06 09:09:39 +01:00
Alexander Todorov
d7388fdc66 Remove already tested manifests from test-case-generators
and remove redundant skips from image_tests.sh
2024-03-06 09:09:39 +01:00
Tom Koscielniak
b4dcfc4480 Remove libvirt workaround 2024-03-04 15:39:08 +01:00
Sanne Raymaekers
c138ea6939 tools/appsre-build-fedora: wait until rpms are built upstream
The fedora build script should wait until the rpms are released upstream
before starting the build, otherwise the rpms will come from the fedora
repos.
2024-02-22 16:05:46 +01:00
Xiaofeng Wang
fc81b89cde test: remove workarounds for fixed bugs
Bugs have been fixed, workarounds are not needed anymore.
2024-02-07 08:52:30 +01:00
Tomáš Hozza
0c0a758600 Packer: default to RHEL-9.3 Schutzfile config for rhel-9
This is necessary to get the proper osbuild SHA to install the latest
required osbuild version into the image.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
2fe5e01d4d Test/api.sh: test distro aliases and names without dot-notation
Extend the `api.sh` to allow testing compose requests with distro name
which does not use dot-notation (specifically when the dot is removed
from the distro name as it used to be in the past). In addition to that,
allow also testing the distro alias using distro name without the minor
version in compose requests.

Enable these two new test variants in the CI.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
66571784d1 Use dot-notation for repo configs and in test cases.
Where applicable, modify all repo config filenames to use a dot
to separate the release major and minor version. Modify test cases
to not remove dot from the distro version any more.

Existing tests will be extended (or new tests added) to explicitly test
backward compatibility and ensure that using old distro names without a
dot still works.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
79d5ab483b tools: delete old Python manifest generator scripts
These are no longer used, let's delete them.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Sanne Raymaekers
589bd5f1a1 tools/fedora-worker-packer: fix packer only/except
It should only build fedora, not everything except fedora.
2024-01-23 13:06:38 +01:00
Sanne Raymaekers
6a0c893a4d tools/appsre-packer: add script to build fedora worker images 2024-01-23 00:18:51 +01:00
Sanne Raymaekers
c7aad1b17d tools/app-sre-build-worker-packer: split out the CI script
To add the fedora workers built from AppSRE's infrastructure, we need a
combination that wasn't possible with the current matrix based on the CI
environment (jenkins vs gitlab).
2024-01-23 00:18:51 +01:00
Sanne Raymaekers
3631316e36 tools/build-rpms: fix getting the osbuild commit from Schutzfile
osbuild/osbuild-composer#3614 removed the rhel-8.6, the centos-stream-9
entry should be there for a while at least.
2024-01-16 16:26:01 +01:00
Brian C. Lane
978c20d36f tools: Add helper script to check Schutzfile and gitlab-ci.yml runners
This script reads the RUNNER entries from .gitlab-ci.yml and the
snapshot repo names from Schutzfile. It then prints which runners don't
have entries in Schutzfile and which Schutzfile entries don't have
runners.

This is slightly complicated by the fact that centos-stream reports
itself as centos. So when checking for missing entries they are
considered to be the same thing. eg. If centos is a RUNNER and
centos-stream is in Schutzfile it doesn't report anything missing.
2024-01-16 12:29:21 +01:00
Michael Vogt
973a8cf285 image-info: update for new "partition" option in mounts.Mount
In the osbuild PR#1501 [0] a new paramter "partition" for
mounts.Mount() was added.

This commit updates the code that it deals cleanly with the new
and the old API for `mount.Mount`.

[0] https://github.com/osbuild/osbuild/pull/1501
2024-01-10 09:38:31 +01:00
Ondřej Budai
7bcf8e5942 test-case-generators/repos: add required repos for azure-sap-rhui
This image requires a different RHUI repository and also the saphana one.

Similarly to the previous commit, we should probably remove the manifest diff
tests altogether, but let's just fix them for now.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-12-22 00:11:19 +00:00
Miguel Martín
991293a897 Generate FIPS compliant SSH keys
Generate FIPS compliant SSH keys required
for testing system FIPS mode support
2023-12-13 10:19:47 +01:00
Sanne Raymaekers
4ee00f7dab tools/provision: disable tracing before manipulating OCI secrets 2023-11-07 10:43:32 +01:00
Jakub Rusz
d68abf3bb5 Generate RHEL-94 and RHEL-810 manifests + update 2023-11-06 13:39:01 +04:00
Tomáš Hozza
ebc256b073 Container: set the composer git revision during the build
When the container with osbuild-composer gets built in our CI or by
AppSRE, we do not set the composer version to any value (as we do when
we built RPMs). As a result, the version reported by composer is always
"devel". This is not useful for debugging and determining the used
version of composer. In addition, this information now gets exposed in
Koji builds, therefore it makes sense to make it useful.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-09 14:06:50 +02:00
Tomáš Hozza
5903aad07b tools/provision.sh: fix unbound variable bug if no OCI secrets are set
This was hit in c9s CI:
https://gitlab.com/redhat/centos-stream/rpms/osbuild-composer/-/merge_requests/88
https://artifacts.dev.testing-farm.io/658332f8-8705-4727-84d3-f27b57775037/

```
/usr/libexec/osbuild-composer-test/provision.sh: line 141: OCI_SECRETS: unbound variable
```

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-09-26 14:12:42 +02:00
Sanne Raymaekers
e81183daa8 tools/appsre-ansible/rpmbuild: retry all tasks 2023-09-25 11:56:42 +02:00
Sanne Raymaekers
522a54188a test/api: add oci test 2023-09-19 22:57:06 +02:00
Achilleas Koutsou
d3ef361dd5 test: update repos and manifests
Summary of changes:
- ova image type uses ustar format
    - https://github.com/osbuild/images/pull/16
- New image type "minimal-raw"
    - https://github.com/osbuild/images/pull/29
- Fedora repositories and test manifests updated
    - https://github.com/osbuild/images/pull/57
- Distro aliases for major version without minor updated
    - https://github.com/osbuild/images/pull/59
2023-08-03 10:35:21 +02:00
Brian C. Lane
8ff4c0c40a tests: Add a check for valid snapshot urls
This pulls the list of snapshots from the rpmrepo API, greps the
codebase for all uses of rpmrepo.osbuild.org that look like a snapshot
name, and then checks to make sure they are still valid.
2023-07-28 11:28:40 +03:00
Xiaofeng Wang
967570a2a6 test: Add workaround for BZ#2213660 and #2225667 2023-07-26 22:23:41 +08:00
Ondřej Budai
cac9327b44 update to go 1.19
UBI and the oldest support Fedora (37) now all have go 1.19, so we are
cleared to switch.

gofmt now reformats comments in certain cases, so that explains the formatting
changes in this commit.
See https://go.dev/doc/go1.19#go-doc

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-07-21 19:18:00 +02:00
Alexander Todorov
38ed9a39be Start collecting logs from virt-install 2023-07-21 12:11:02 +02:00
Brian C. Lane
3cfde5860d tools: Make pylint happier 2023-07-18 23:20:19 +02:00
Brian C. Lane
08e5c53d31 tools: Add support for local go package replacement
Local packages do not have a version number. Fake it with 'HEAD' to
indicate it's using whatever the working directory is pointing at.

The Provides line will then look like:

Provides: bundled(golang(../osbuild-images)) = HEAD

which rpmbuild is happy to process into an rpm.

This is ONLY for test builds on local systems and should never be seen
in the wild.
2023-07-18 23:20:19 +02:00
Brian C. Lane
f7dcc79bd8 tools: Set a+x on rpm_spec_vendor2provides 2023-07-18 23:20:19 +02:00
Achilleas Koutsou
f478323e3a tools: replace spec Provides generator
Replace the generator for the spec file "Provides:" list with the one
from the current go-rpm-macros [1].  The new generator can handle
'replace' commands in vendor/modules.txt, which makes it possible to
build RPMs with temporarily modified dependencies.  This enables making
scratch builds against forks or branches of the new osbuild/images
repository for development and testing.

In the future, we can use the packaged version of the script, but this
isn't yet available in all supported distros (EL8).

Made some minor modifications to the original script to prepend each
dependency with "Provides:" and read input files as arguments instead of
lines from stdin.

https://pagure.io/go-rpm-macros/blob/c32fbbd25bbcedee8c0b898d3653255b18a0d30e/f/rpm/go_mod_vendor.prov
2023-07-18 18:08:49 +02:00
Xiaofeng Wang
2baaa20859 test: Four fixes for RHEL for Edge tests
1. Remove ansible-blocking-io.py workaround. It's not required.
2. Variable should be PROD_REPO_URL, not STAGE_REPO_URL in
ostree-raw-image.sh
3. Use --reboot in rpm-ostree install to reboot VM instead of
a reboot ansible task
4. Wait until config file serviceinfo_api_server.yml exists, to
avoid file not available to use flaky issue
2023-06-30 09:07:39 +08:00
Achilleas Koutsou
78931c0f05 test: update fedora repos
Update Fedora 37, 38, and 39 repositories for the test case generator.
Changes are only package versions and dependencies.

For Fedora 39, a package (lklug-fonts) had to be removed from the
Anaconda package set since it is no longer available.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-27 16:13:25 +02:00
Simon de Vlieger
46f93614f2 fedora: live-installer image type
This image type produces the same artifact as the current workstation
installer live media.

During the implementation of this new installer some names have been
changed to make a bit more sense in the source tree. Installer images
now always mention which installer they are (anaconda, etc).
2023-06-27 13:18:56 +02:00
Ondřej Budai
6e756eb76b tools/libvirt_test.sh: add fedora support
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
6b1f1848e4 tools/libvirt_test.sh: remove hardcoded os-variant
Using rhel8-unknown for all distros was really weird, let's use the proper
variants.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
2c4215a2d0 tools/libvirt_test.sh: fix weird indentation
There were 5 spaces instead of 4.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00