Commit graph

530 commits

Author SHA1 Message Date
Sanne Raymaekers
4ee00f7dab tools/provision: disable tracing before manipulating OCI secrets 2023-11-07 10:43:32 +01:00
Jakub Rusz
d68abf3bb5 Generate RHEL-94 and RHEL-810 manifests + update 2023-11-06 13:39:01 +04:00
Tomáš Hozza
ebc256b073 Container: set the composer git revision during the build
When the container with osbuild-composer gets built in our CI or by
AppSRE, we do not set the composer version to any value (as we do when
we built RPMs). As a result, the version reported by composer is always
"devel". This is not useful for debugging and determining the used
version of composer. In addition, this information now gets exposed in
Koji builds, therefore it makes sense to make it useful.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-09 14:06:50 +02:00
Tomáš Hozza
5903aad07b tools/provision.sh: fix unbound variable bug if no OCI secrets are set
This was hit in c9s CI:
https://gitlab.com/redhat/centos-stream/rpms/osbuild-composer/-/merge_requests/88
https://artifacts.dev.testing-farm.io/658332f8-8705-4727-84d3-f27b57775037/

```
/usr/libexec/osbuild-composer-test/provision.sh: line 141: OCI_SECRETS: unbound variable
```

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-09-26 14:12:42 +02:00
Sanne Raymaekers
e81183daa8 tools/appsre-ansible/rpmbuild: retry all tasks 2023-09-25 11:56:42 +02:00
Sanne Raymaekers
522a54188a test/api: add oci test 2023-09-19 22:57:06 +02:00
Achilleas Koutsou
d3ef361dd5 test: update repos and manifests
Summary of changes:
- ova image type uses ustar format
    - https://github.com/osbuild/images/pull/16
- New image type "minimal-raw"
    - https://github.com/osbuild/images/pull/29
- Fedora repositories and test manifests updated
    - https://github.com/osbuild/images/pull/57
- Distro aliases for major version without minor updated
    - https://github.com/osbuild/images/pull/59
2023-08-03 10:35:21 +02:00
Brian C. Lane
8ff4c0c40a tests: Add a check for valid snapshot urls
This pulls the list of snapshots from the rpmrepo API, greps the
codebase for all uses of rpmrepo.osbuild.org that look like a snapshot
name, and then checks to make sure they are still valid.
2023-07-28 11:28:40 +03:00
Xiaofeng Wang
967570a2a6 test: Add workaround for BZ#2213660 and #2225667 2023-07-26 22:23:41 +08:00
Ondřej Budai
cac9327b44 update to go 1.19
UBI and the oldest support Fedora (37) now all have go 1.19, so we are
cleared to switch.

gofmt now reformats comments in certain cases, so that explains the formatting
changes in this commit.
See https://go.dev/doc/go1.19#go-doc

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-07-21 19:18:00 +02:00
Alexander Todorov
38ed9a39be Start collecting logs from virt-install 2023-07-21 12:11:02 +02:00
Brian C. Lane
3cfde5860d tools: Make pylint happier 2023-07-18 23:20:19 +02:00
Brian C. Lane
08e5c53d31 tools: Add support for local go package replacement
Local packages do not have a version number. Fake it with 'HEAD' to
indicate it's using whatever the working directory is pointing at.

The Provides line will then look like:

Provides: bundled(golang(../osbuild-images)) = HEAD

which rpmbuild is happy to process into an rpm.

This is ONLY for test builds on local systems and should never be seen
in the wild.
2023-07-18 23:20:19 +02:00
Brian C. Lane
f7dcc79bd8 tools: Set a+x on rpm_spec_vendor2provides 2023-07-18 23:20:19 +02:00
Achilleas Koutsou
f478323e3a tools: replace spec Provides generator
Replace the generator for the spec file "Provides:" list with the one
from the current go-rpm-macros [1].  The new generator can handle
'replace' commands in vendor/modules.txt, which makes it possible to
build RPMs with temporarily modified dependencies.  This enables making
scratch builds against forks or branches of the new osbuild/images
repository for development and testing.

In the future, we can use the packaged version of the script, but this
isn't yet available in all supported distros (EL8).

Made some minor modifications to the original script to prepend each
dependency with "Provides:" and read input files as arguments instead of
lines from stdin.

https://pagure.io/go-rpm-macros/blob/c32fbbd25bbcedee8c0b898d3653255b18a0d30e/f/rpm/go_mod_vendor.prov
2023-07-18 18:08:49 +02:00
Xiaofeng Wang
2baaa20859 test: Four fixes for RHEL for Edge tests
1. Remove ansible-blocking-io.py workaround. It's not required.
2. Variable should be PROD_REPO_URL, not STAGE_REPO_URL in
ostree-raw-image.sh
3. Use --reboot in rpm-ostree install to reboot VM instead of
a reboot ansible task
4. Wait until config file serviceinfo_api_server.yml exists, to
avoid file not available to use flaky issue
2023-06-30 09:07:39 +08:00
Achilleas Koutsou
78931c0f05 test: update fedora repos
Update Fedora 37, 38, and 39 repositories for the test case generator.
Changes are only package versions and dependencies.

For Fedora 39, a package (lklug-fonts) had to be removed from the
Anaconda package set since it is no longer available.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-27 16:13:25 +02:00
Simon de Vlieger
46f93614f2 fedora: live-installer image type
This image type produces the same artifact as the current workstation
installer live media.

During the implementation of this new installer some names have been
changed to make a bit more sense in the source tree. Installer images
now always mention which installer they are (anaconda, etc).
2023-06-27 13:18:56 +02:00
Ondřej Budai
6e756eb76b tools/libvirt_test.sh: add fedora support
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
6b1f1848e4 tools/libvirt_test.sh: remove hardcoded os-variant
Using rhel8-unknown for all distros was really weird, let's use the proper
variants.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
2c4215a2d0 tools/libvirt_test.sh: fix weird indentation
There were 5 spaces instead of 4.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
89b71d339e tools/libvirt_test.sh: merge two similar switch statements
No need for more switching than just one.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Alexander Todorov
1c3bd128d0 Update rpmrepo snapshots 2023-06-15 17:21:24 +02:00
Alexander Todorov
83cec26a5a Remove cloud-init/network-config file from tests
- not used in oscap.sh test at all
- causes VM in libvirt.sh test to acquire a different IP address instead
  of the hard-coded one inside the test, which causes SSH to fail

- not used anywhere else
2023-06-15 08:54:57 +02:00
Alexander Todorov
1f28e50d21 Adjust tests for new distros
References:
- https://redhat-internal.slack.com/archives/C0235DZB0DT/p1685528139831589
- 1fbd9d975f
2023-06-15 08:54:57 +02:00
Alexander Todorov
4aff9076ea Update rpmrepos & manifest files via command
go run ./cmd/gen-manifests/ -workers 50
2023-06-15 08:54:57 +02:00
Alexander Todorov
22437f7784 Add RPM repository definitions for 8.9 and 9.3 2023-06-15 08:54:57 +02:00
Achilleas Koutsou
403fe74ce4 format-request-map: clean up ostree params
Don't specify ref ID for parent, instead only specify a ref and URL and
let the manifest generator resolve the ID using the sha256 hash.
2023-06-14 11:19:29 +02:00
Irene Diez
a8f7ff487d distro/rhel9: add the edge-ami image type based on edgeRawImage
Creates the 'edge-ami' image type based on edgeRawImage, which generates
a raw image (x86_64, aarch64) ready to upload to AWS EC2.

This 'edge-ami' image type has Ignition support.

Signed-off-by: Irene Diez <idiez@redhat.com>
2023-06-07 13:18:01 +02:00
Ondřej Budai
73c8a482f7 repositories: update fedora 37-39 snapshots
Fedora 37 and 38 still pointed at branched/rawhide, oops. Let's just update
them all.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-06 13:04:10 +02:00
Ondřej Budai
eaa4e9b5fa repositories: fix GPG keys
Oh my, we had the Fedora 37 key even in Fedora 38 and 39 repositories. This
commit fixes that.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-06 13:04:10 +02:00
Ondřej Budai
894e6042c6 tools/test-case-generators/repos: remove fedora-36
It's EOL and already removed from the codebase. This was apparently forgotten
during the purge.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-06 13:04:10 +02:00
Sanne Raymaekers
f5b01125b6 tools/test-case-generators: add ovf image type 2023-05-25 10:14:32 +02:00
Tomáš Hozza
2a50092920 Test: add Fedora iot-raw-image test manifests with customizations
Add Fedora iot-raw-image test manifests with supported customizations,
specifically:
 - custom users and groups
 - custom enabled services
 - custom files and directories

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-05-22 16:30:14 +02:00
Ondřej Budai
ce5e41f980 tools/run-koji-container: allow passing a custom certificate dir
This helps with running the script locally.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-05-18 09:25:22 +02:00
Ondřej Budai
25161a8316 libvirt_test: stop setting nvram in virt-install on RHEL 9
We use CDN repositories when running tests on GA RHEL. Thus, our RHEL 9.1
machines are actually pulling packages from RHEL 9.2. Therefore, we are
now getting virt-install >= 4. This version brings firmware auto-detection
and doesn't support nvram_template anymore. Let's remove it then.

See 966049ec3c

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-05-12 11:02:49 +02:00
Sanne Raymaekers
c7d2eaa5f7 test: use non-deprecated gitlab CI variables
CI_BUILD_ID was replaced with CI_JOB_ID in gitlab 9+.
2023-05-12 11:02:49 +02:00
Gianluca Zuccarelli
1284642ab2 tools/test-case-generator: add custom repos 2023-04-21 17:40:00 +02:00
Achilleas Koutsou
140571bf48 test: add "name" to test manifests with container
Add name field to the manifest-list-test container in the test
request.  The value is the same as the source but with a `v1` tag
added.

In the manifests, the name field for the manifest-list-test is added to
the skopeo stage.  The `name` option of the fedora-minimal container in
the skopeo stage is also changed to reflect the full source reference
including the `latest` tag.
2023-04-19 20:07:40 +02:00
Tomáš Hozza
7a439ba59d Test/manifests: update Azure RHUI client repo snapshots
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-04-19 15:04:42 +02:00
Ondřej Budai
d8342862d5 customizations: create all groups unconditionally
We had this weird condition in code that prevented composer to create groups
with the same name as a user has. This unfortunately means that you are not
able to create a user with a primary group with a certain GID that has the
same name as the user. There's the gid field in the user customization,
but it requires that the group already exists.

In order to allow that, we need to remove the condition. From now on, it's
possible to create groups with the same name as a user has, which can be used
to create primary groups with a custom gid.

Note that the lorax compatibility behaviour was actually wrong. When lorax was
given a custom gid for a user, it didn't require the gid to exist. When it
didn't, the group was just created. Thus, we still don't have full backward
compatibility, but at least we now have support for this.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-04-19 11:50:49 +02:00
Achilleas Koutsou
5a455b4406 test: add container source with manifest-list to manifests
Add a second container to the ostree-commit test manifests that refers
to a manifest list on the osbuild-composer registry on gitlab.
2023-04-17 18:30:41 +02:00
Achilleas Koutsou
4e3d94f9a7 test-case-generators: add request for Azure EAP 7 image type 2023-03-30 18:40:12 +02:00
Tomáš Hozza
58e3716b89 distro: add support for RHEL-8.9 and RHEL-9.3
Add a basic support for building RHEL-8.9 and RHEL-9.3 images with
composer.

Add 8.9 and 9.3 repositories to the multitude of places where we have
them.

Generate image test manifests for 8.9 and 9.3. No functional testing is
added for 8.9 or 9.3 at this moment.

This change unblocks the RHEL Gating (since unit tests are currently
failing on 8.9 and 9.3 as on unknown distribution).

Related to https://issues.redhat.com/browse/COMPOSER-1924

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-03-27 11:39:24 +02:00
Simon de Vlieger
23627e9ce0 distro/fedora: add support for Fedora 39 2023-03-15 09:31:23 +01:00
Sanne Raymaekers
ca8a05bd3a templates/packer: subscribe packer machines
To avoid a mismatch between the RPMs (which are build using CDN content)
and the packer instances (RHUI, which might be older).
2023-03-03 13:00:05 +01:00
Sanne Raymaekers
41d98c85a7 tools/appsre-ansible: revert sub removal on rpmbuild machines
The rhel-9 templates used in mock don't work with RHUI, they require the
CDN. This reintroduces the RHUI-CDN repo sync issue again.

This reverts commit d9f2ac14bc.
2023-03-03 13:00:05 +01:00
yih
266cfdcc44 fix ansible error on centos9 2023-03-02 16:03:11 +01:00
Sanne Raymaekers
75e6333b9e Revert "tools/appsre-ansible: workaround missing authselect-compat"
This reverts commit 4833f99070.
2023-03-01 20:05:38 +01:00
Sanne Raymaekers
0096ff3689 Revert "Packer: workaround missing authselect-compat-1.2.5-2.el9_1 in RHUI repos"
This reverts commit 0a4a75e19e.
2023-03-01 20:05:38 +01:00