These changes enable the use of weldr-client with the test and also
there was a need to change the vm.create command a little bit. Without
the increased CPU and RAM RHEL-9 would not boot and a network adapter
had to be specified to use a newer one because the default one does not
work with RHEL-9.
the prepare-rhel-internal.sh script should be executed on a single
runner and it will loop through all architectures to prepare the
necessary definitions inside .json and .repo files. The resulting
osbuild-composer-tests repository is not multi-arch, containing all
versions of osbuild-composer-tests.rpm downloaded from Brew.
This is needed because jobs from subsequent stages will download
artifacts (.json & .repo files) from the prepare job but there's no
mechanism to control this easily. As a result artifacts are downloaded
from all of the prepare jobs and they overwrite themselves resulting in
test systems being configured with repo files for different CPU
architecture which leads to dnf errors.
We have limited resources in openstack. We can only run about 40 concurrent VMs.
Previously, the rate limiting was kinda stupid:
All (aws and openstack) jobs were run using the same runner. This runner was
globally limited to 60 concurrent jobs. For openstack, the individual
images were also limited to a certain number of concurrent jobs in
the gitlab-ci-terraform repository so we don't hit the quota. This limit
was applied at runtime - the first thing that an openstack job did was to
wait for a slot. This job counted towards the global limit of jobs (60)
and thus was blocking one slot without doing any useful work.
Applying local limits to please global quota is stupid though. We have much
more demand for rhel-8.5 runner than for e.g. Fedora. It would be much
better to just use global limit that would map much nicely to global
quota.
Today, I've introduced a new runner with tag terraform/openstack. It's
currently limited to 20 concurrent jobs. All jobs running on openstack should
run on the new runner. This runner has the local rate limiting for openstack
disabled. This means that we can run 20 concurrent openstack jobs and it
doesn't matter which distribution they run.
To sum it up, this has two benefits:
- no local limits, we can just use the full quota
- no idling jobs waiting for an openstack slot
Note that the openstack global limit is currently set to 20, I will raise
it once all PRs are rebased on top of this change.
Side effect: I moved all libvirt test to openstack. I think this is overall
better because testing guest images on KVM makes more sense than testing them
on TCG.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
#1717 broke the source preparedness check. This was probably caused by the
fact that the PR wasn't rebased on top of the latest main when it was merged.
I guess that some other PR made some incompatible changes in the meantime.
This commit is just a rerun of tools/prepare-source.sh
Merge trains are a nice idea, right?
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2 configurations for the listeners are now possible:
- enableJWT=false with client ssl auth
- enableJWT=true with https
Actual verification of the tokens is handled by
https://github.com/openshift-online/ocm-sdk-go.
An authentication handler is run as the top level handler, before any
routing is done. Routes which do not require authentication should be
listed as exceptions.
Authentication can be restricted using an ACL file which allows
filtering based on JWT claims. For more information see the inline
comments in ocm-sdk/authentication.
As an added quirk the `-v` flag for the osbuild-composer executable was
changed to `-verbose` to avoid flag collision with glog which declares
the `-v` flag in the package `init()` function. The ocm-sdk depends on
glog and pulls it in.
Dependabot pushes branches directly to the upstream repository. This causes
double-triggers of gitlab CI. Prevent it by running gitlab CI only for
the main branch.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
The 'google-cloud-sdk' RPM built by Google for RHEL, which provides
the 'gcloud' command, is built only with Python 2. Since Python 2.7
is already EOL in upstream and not available in CentOS Stream 9, we
can not use 'gcloud' from the 'google-cloud-sdk' RPM.
The 'awscli' is not available in RHEL-9 repositories.
The Azure CLI 'az' available in official upstream repositories has
broken dependencies on RHEL-9 and can not be successfully installed. To
workaround the issue, run the tool from the official container image
provided by Microsoft.
Use the `quay.io/osbuild/cloud-tools` F34-based container image instead
of locally installed cloud CLI tools.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
