Add support for RHSM customizations, which currently allow various
aspects of RHSM. Specifically enabling / disabling DNF plugins shipped
by subscription-manager package and setting a subset of options in the
rhsm.conf.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add support for RPM customizations, which currently allow to import RPM
GPG key from a file installed in the image. This is e.g. done for the
Azure RHUI image type.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
both are defined in CI, but the GOVC_ prefix doesn't match anything
else. Additionally this value is slightly different for vCenter8 and may
lead to failures if not unified.
The image type supports uploading to both s3 and container registries.
Right now we only care about getting it in Koji, so let's just add it to
the s3 upload section.
The container registry test also embeds containers in the image, which
the iot-bootable-container doesn't support (yet).
Specify the resource pool when importing the VMDK to the VCenter.
This should prevent the following error:
govc: default resource pool resolves to multiple instances, please specify
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the `api.sh` to allow testing compose requests with distro name
which does not use dot-notation (specifically when the dot is removed
from the distro name as it used to be in the past). In addition to that,
allow also testing the distro alias using distro name without the minor
version in compose requests.
Enable these two new test variants in the CI.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
When making the upload request for edge commit image types, use the new
upload_targets array to define the aws.s3 upload options.
Leave other upload target definitions as is for now to test the old
options.
Run a basic check to see that the tailoring file has been created when
adding tailoring rules to the OpenSCAP config. Since the OpenSCAP scan
requires more memory and breaks some of the API tests, a full scan is
not run in the cloudapi tests.
Extend the Cloud API AWS test case to verify the uploaded AMI boot mode.
For informational purposes, also log the boot mode of the instance
created using the AMI by the test case.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
- remove `custom-repos.sh` integratoin test
- add custom repositories check to `api` tests for supported
images
- verify custom repositores are added to /etc/yum.repos.d
- verify gpg key is saved to /etc/pki/rpm-gpg (for inline keys)
Use directories and files customization in the compose request for image
types that support this customization (only ostree installer and raw
image do not support it).
Extend the instance verification to check for the custom directories and
files.
Extend the ostree commit verification to check for the custom
directories and files.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Previously, the `--format=json` was always set in the `$GCP_CMD`. This
was working OK in general, but it is creating issues when using
`$GCP_CMD compute ssh` to execute commands on the provisioned VM. In
such case, an extra `[]` is added to the output, which then causes
various checks to fail, since the captured stdout is different than
expected.
Instead remove the `--format=json` by default from `$GCP_CMD` and use it
explicitly only in cases when `jq` is used to parse the command's
output. In all other cases, it should be OK to not set the output format
to JSON and it is very much desirable to do so when SSH-ing to the VM.
Also by default pass the `--quiet` option to `gcloud` when using it from
the container. This was previously done only when using the tool
installed on the system.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Let's default to not providing the Resource Group Location when testing
Cloud API with Azure and leave it up to the implementation to determine
the correct location to use.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
GCP Bucket to use can be now configured in the worker configuration.
Make the `Bucket` optional in the Cloud API when uploading image to GCP.
Adjust the Cloud API test case to configure GCP Bucket on the worker and
not provide it in the API request.
A small test is added to cover this new option. Also, a check for AWS region
in the URL was removed. The public URL doesn't actually contain it and it's
imho useless - S3 buckets are always tied to a certain region.
The elaborate grep command parsing a path from a URI was switched to being
a small python script. The python script can actually handle an URI without
a query (which was always present in the pre-signed URI, but it's not in the
ordinary one)
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Add a common function to check for all environment variables needed to
test VSphere image. Call the function from all S3 implementations of
`checkEnv()` if testing the VSphere image type.
Move the installation of the VSphere client out of the
`api/common/aws.sh` `installClient()` function into a dedicated
`installClientVSphere()` function in `api/common/vsphere.sh`.
Rename the `installClient()` in `api/common/aws.sh` to
`installAWSClient()`. Introduce `installClient()` in both,
`api/aws.sh` and `api/aws.s3.sh`, which calls `installAWSClient()`.
Call `installClientVSphere()` conditionally when testing VMDK image type
in `api/aws.s3.sh` and `api/generic.s3.sh`.
The `verifyEdgeCommit()` and `verifyDisk()` functions are already
defined in the `test/cases/api/common/s3.sh` file sourced by
`test/cases/api/aws.s3.sh`.
Since we're sharing functions between test scripts, move greenprint(),
the most rewritten function in the history of the project, to
shared_lib.sh and source it everywhere.
Support for creating multiple amis from a single compose. It uses the
AWSEC2* jobs to push images to new regions, and share them with new
accounts.
The compose it depends upon has to have succeeded.
Extend the implementation of mock openid server to take the `grant_type`
into consideration for the `/token` endpoint.
In addition to the previously supported `refresh_topen`, the
implementation now supports also `client_credentials`.
This is necessary to make it possible to use the mock server in
the `koji-osbuild` CI, because the builder plugin uses
`client_credentials` to get access token.
The implementation behaves in the following way:
- For `refresh_token` grant type, it takes the `refresh_token` value
from the request and adds it to the `rh-org-id` field in the custom
claim, which is part of the returned token.
- For `client_credentials` grant type, it takes the `client_secret`
value from the request and adds it to the `rh-org-id` field in the
custom claim, which is part of the returned token.
Requests without the supported `grant_type` set are rejected.
Modify affected test cases to specify `grant_type` when fetching a new
access token.
To check embedding containers via the cloud API works, embed a
known test container from our gitlab CI and check that it is
indeed embedded in the image by pulling the commit and poking
into the container storage.
Do not start local worker (mask the unit) and Weldr API socket when
provisioning the SUT with TLS client cert authentication method. This
method is used only in the Service scenario, therefore starting these
units / sockets was not reflecting the intended deployment.
Modify `api.sh` to not rely on local worker.
Modify `base_tests.sh` to provision SUT with TLS for
`osbuild-auth-tests`, while provisioning SUT with no authentication
method for the rest of test cases.
Common integration tests should not need to care about specific ORG ID
configured in the worker, but they should be able to get access token
and check compose status without providing a specific ORG ID. The only
integration test that should care about ORG ID is the
`multi-tenancy.sh`.
Modify the `access_token` and `compose_status` functions to hide the
existence of ORG ID from the user and instead read it from the worker's
configuration, specifically `/etc/osbuild-worker/token`.
The original implementations of the functions mentioned above are now
available under `access_token_with_org_id` and
`compose_status_with_org_id` names.
Modify the `multi-tenancy.sh` to use the new function names.
Move some code related to using JWT tokens from the `multi-tenancy.sh`
test case to `test/cases/api/common/common.sh`, `tools/provision.sh`
and `tools/run-mock-auth-servers.sh`. Move the composer and worker
configuration from the test to new testing configuration files.
The `tools/provision.sh` now accepts an optional argument specifying the
authentication method to use with the provisioned composer and workers.
Valid values are `tls` and `jwt`. If no argument is specified, the `tls`
option is used and the script defaults to its previous behavior.
If the user does not pass a name, use the distribution as a name
A provided tag is used only if name is provided. It
The tag's default is a generated using UUID to avoid collisions
Add a new cloud API test that will build an edge-container,
upload it to the gitlab CI registry, fetch it from there,
run it and compare that the OSTree commit contained in it
is indeed the one we expect.
Co-Developed-By: Christian Kellner <christian@kellner.me>
Extend the `koji.sh` test case to allow also testing the upload to
cloud, in addition to the testing that it supports currently (building
of multiple images in one Koji compose request).
The script now reuses some common functions used by the `api.sh` test
case. Once the Koji compose succeeds, the script verifies that the image
is present in the appropriate cloud environment using a CLI tool. No
additional testing of the image is done, it is not booted.
- Fixed shellcheck errors
- Moved checkEnv from common to individual tests
- Fixed package install section in spec file:
Globs which include a directory fail on el-like distros.
- Use gcloud cli to ssh
- (re)Introduce generic s3 tests
Each cloud now has its own file that's sourced on-demand by the main api.sh
script. The main goal of this commit is to reduce the amount of clutter in
api.sh. I, personally, find 1300 lines of bash overwhelming and I think that
this is a reasonable beginning to start cleaning things up.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
