Expose the ${SUBJ} customization in Cloud API. Weldr API inherited it
automatically with the update of `osbuild/blueprint` to v1.6.0.
Adjust relevant test cases and unit tests to use this new customization.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Unfortunately, `which` does not seem to be installed by default on our
F41 CI images. Instead of doing the dance with rebuilds, which has been
problematic recently, let's not rely on `which` in scripts any more,
since we can replace it with the Bash built-in `type` command.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The reason is that the key that we use to test the customization uses
SHA-1 in its signature. And this algorithm is no longer allowed by
default on el10 / c10s.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add support for RHSM customizations, which currently allow various
aspects of RHSM. Specifically enabling / disabling DNF plugins shipped
by subscription-manager package and setting a subset of options in the
rhsm.conf.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add support for RPM customizations, which currently allow to import RPM
GPG key from a file installed in the image. This is e.g. done for the
Azure RHUI image type.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
When cleaning up api tests, the build job results are dumped from the
database to store in build-result.txt and are printed to the runner log
at the same time. This makes it very difficult to read results as the
database dump prints very long lines that flood the job log. The result
isn't really readable (or useful) unless the file is downloaded
separately.
Silence the `tee` command so that the file is created but no output is
printed from the db dump.
The image type supports uploading to both s3 and container registries.
Right now we only care about getting it in Koji, so let's just add it to
the s3 upload section.
The container registry test also embeds containers in the image, which
the iot-bootable-container doesn't support (yet).
Extend the `api.sh` to allow testing compose requests with distro name
which does not use dot-notation (specifically when the dot is removed
from the distro name as it used to be in the past). In addition to that,
allow also testing the distro alias using distro name without the minor
version in compose requests.
Enable these two new test variants in the CI.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Where applicable, modify all repo config filenames to use a dot
to separate the release major and minor version. Modify test cases
to not remove dot from the distro version any more.
Existing tests will be extended (or new tests added) to explicitly test
backward compatibility and ensure that using old distro names without a
dot still works.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
While I was the one to push for TEST_MODULE_HOTFIXES to be passed as an
argument to `api.sh`, when it was implemented, this is turning our to be
impractical. I would like to extend `api.sh` to test distro names with
and without the dot to separate major and minor version and also to test
the ability to specify distro name explicitly (e.g. a distro alias).
Passing all of these new options and their combinations would make the
CLI go wild.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Run a basic check to see that the tailoring file has been created when
adding tailoring rules to the OpenSCAP config. Since the OpenSCAP scan
requires more memory and breaks some of the API tests, a full scan is
not run in the cloudapi tests.
- remove `custom-repos.sh` integratoin test
- add custom repositories check to `api` tests for supported
images
- verify custom repositores are added to /etc/yum.repos.d
- verify gpg key is saved to /etc/pki/rpm-gpg (for inline keys)
Use directories and files customization in the compose request for image
types that support this customization (only ostree installer and raw
image do not support it).
Extend the instance verification to check for the custom directories and
files.
Extend the ostree commit verification to check for the custom
directories and files.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
If a job is unresponsive the worker has most likely crashed or been shut
down and the in-progress job been lost.
Instead of failing these jobs, requeue them up to two times. Once a job is lost
a third time it fails. This avoids infinite loops.
This is implemented by extending FinishJob to RequeuOrFinish job. It takes a
max number of requeues as an argument, and if that is 0, it has the same
behavior as FinishJob used to have.
If the maximum number of requeues has not yet been reached, then the running
job is returned to pending state to be picked up again.
Since we're sharing functions between test scripts, move greenprint(),
the most rewritten function in the history of the project, to
shared_lib.sh and source it everywhere.
Support for creating multiple amis from a single compose. It uses the
AWSEC2* jobs to push images to new regions, and share them with new
accounts.
The compose it depends upon has to have succeeded.
Extend the implementation of mock openid server to take the `grant_type`
into consideration for the `/token` endpoint.
In addition to the previously supported `refresh_topen`, the
implementation now supports also `client_credentials`.
This is necessary to make it possible to use the mock server in
the `koji-osbuild` CI, because the builder plugin uses
`client_credentials` to get access token.
The implementation behaves in the following way:
- For `refresh_token` grant type, it takes the `refresh_token` value
from the request and adds it to the `rh-org-id` field in the custom
claim, which is part of the returned token.
- For `client_credentials` grant type, it takes the `client_secret`
value from the request and adds it to the `rh-org-id` field in the
custom claim, which is part of the returned token.
Requests without the supported `grant_type` set are rejected.
Modify affected test cases to specify `grant_type` when fetching a new
access token.
Do not start local worker (mask the unit) and Weldr API socket when
provisioning the SUT with TLS client cert authentication method. This
method is used only in the Service scenario, therefore starting these
units / sockets was not reflecting the intended deployment.
Modify `api.sh` to not rely on local worker.
Modify `base_tests.sh` to provision SUT with TLS for
`osbuild-auth-tests`, while provisioning SUT with no authentication
method for the rest of test cases.
Add a new cloud API test that will build an edge-container,
upload it to the gitlab CI registry, fetch it from there,
run it and compare that the OSTree commit contained in it
is indeed the one we expect.
Co-Developed-By: Christian Kellner <christian@kellner.me>
We want to be able to safely gather any artifacts without worrying about
any possible secrets leaking. Every artifacts that we want to upload
will now have to be placed in /tmp/artifacts which will then be uploaded
to S3 by the executor and link to the artifacts will be provided in the
logs. Only people with access to our AWS account can see them.
- Fixed shellcheck errors
- Moved checkEnv from common to individual tests
- Fixed package install section in spec file:
Globs which include a directory fail on el-like distros.
- Use gcloud cli to ssh
- (re)Introduce generic s3 tests
Each cloud now has its own file that's sourced on-demand by the main api.sh
script. The main goal of this commit is to reduce the amount of clutter in
api.sh. I, personally, find 1300 lines of bash overwhelming and I think that
this is a reasonable beginning to start cleaning things up.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
`osbuild-mock-openid-provider`'s `/token` endpoint expects URL-encoded
values in the POST request body. Use the same values as those that would
be used by the worker when refreshing a token.
Use case
--------
If Endpoint is not set and Region is - upload to AWS S3
If both the Endpoint and Region are set - upload the Generic S3 via Weldr API
If neither the Endpoint and Region are set - upload the Generic S3 via Composer API (use configuration)
jobimpl-osbuild
---------------
Add configuration fields for Generic S3 upload
Support S3 upload requests coming from Weldr or Composer API to either AWS or Generic S3
Weldr API for Generic S3 requires that all connection parameters but the credentials be passed in the API call
Composer API for Generic S3 requires that all conneciton parameters are taken from the configuration
Adjust to the consolidation in Target and UploadOptions
Target and UploadOptions
------------------------
Add the fields that were specific to the Generic S3 structures to the AWS S3 one
Remove the structures for Generic S3 and always use the AWS S3 ones
Worker Main
-----------
Add Endpoint, Region, Bucket, CABundle and SkipSSLVerification to the configuration structure
Pass the values to the Server
Weldr API
---------
Keep the generic.s3 provider name to maintain the API, but unmarshel into awsS3UploadSettings
tests - api.sh
--------------
Allow the caller to specifiy either AWS or Generic S3 upload targets for specific image types
Implement the pieces required for testing upload to a Generic S3 service
In some cases generalize the AWS S3 functions for reuse
GitLab CI
---------
Add test case for api.sh tests with edge-commit and generic S3
