Move the installation of the VSphere client out of the
`api/common/aws.sh` `installClient()` function into a dedicated
`installClientVSphere()` function in `api/common/vsphere.sh`.
Rename the `installClient()` in `api/common/aws.sh` to
`installAWSClient()`. Introduce `installClient()` in both,
`api/aws.sh` and `api/aws.s3.sh`, which calls `installAWSClient()`.
Call `installClientVSphere()` conditionally when testing VMDK image type
in `api/aws.s3.sh` and `api/generic.s3.sh`.
The `verifyEdgeCommit()` and `verifyDisk()` functions are already
defined in the `test/cases/api/common/s3.sh` file sourced by
`test/cases/api/aws.s3.sh`.
shared_lib.sh now contains some more helper functions and the
verification script uses them. Copy it over so that we can source it.
Also install all other dependencies before the actual upgrade.
The Fedora IoT Raw image sets the default remote URL to the official
Fedora IoT source at build time. We need to replace it in our tests
with our local test repo.
3013889727
changed the behaviour of --boot uefi. Now, it doesn't pass the loader options
but uses firmware="efi" that uses libvirt's ability to auto-detect the correct
loader.
Therefore, for distributions that ship the new virt-install, we have to change
the --boot argument. --boot uefi should now be able to automatically set the
uefi loader correctly without passing any extra loader options. The only issue
is that CS9 has issues with secure boot and edge, so we need to disable
secure boot using a new (and arcane) syntax (the old loader arguments no
longer work):
firmware.feature0.name=secure-boot,firmware.feature0.enabled=no
(https://libvirt.org/formatdomain.html#bios-bootloader)
See the CS9 bug for secure boot issues on edge:
https://bugzilla.redhat.com/show_bug.cgi?id=2108646
Also, since --os-type is deprecated in libvirt >= 4, let's remove it.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
The image_type variable in the playbook is used for the name of the
deployment, which is 'redhat' by default for RHEL and 'fedora-iot' for
Fedora.
This affects the deployment path checks, for example:
{{ device_name }}[/ostree/deploy/{{ image_type }}/var]
Instead of defining users in the ostree commits, create them during the
deployment and creation of the raw image.
Defining users in commits locks them into the base system: they can't be
removed and their password can't be changed without creating an upgrade
commit and they will always be part of the initial system.
Also, in Fedora 37, password authentication for users in commits is not
working.
Since we're sharing functions between test scripts, move greenprint(),
the most rewritten function in the history of the project, to
shared_lib.sh and source it everywhere.
- Handle the array responses from the new weldr-client (>= 35.6).
- Move the `get_build_info` function to shared_libs.sh to source and
reuse in multiple places.
Distro version branches aren't synced to GitLab, so we will need to
fetch them from GitHub directly.
This is required for any PR made against any branch other than main.
We were using `latest` as tag, this can be dangerous as it's the default
tag, an anyone can accidentally update it. Using `prod` is safer.
Also use dev container image if the test script is running in CIV CI.
Support for creating multiple amis from a single compose. It uses the
AWSEC2* jobs to push images to new regions, and share them with new
accounts.
The compose it depends upon has to have succeeded.
All EXIT traps are cleared on line 280 so the cleanup trap is never run
and VMs are waiting for 4 hours to get cleaned by
scheduled-cloud-cleaner. Run the cleanup at the end and rely on
scheduled-cloud-cleaner only in case of failures before that.
instead of manually updating CIV version every once in a while. Get
always the latest version.
In CIV CI, this test runs before any change can be introduced into the
container image, so no unexpedted errors should come from the CIV side.
some scripts skip the test if it's not supported for that
distro-version. Disable them in gitlab-ci.yml so we don't waste CI
resources.
To disable them, we are using the `rules` on each job with a regex
pattern. Using `=~` (pattern matches) as a WHITELIST and `!~` (pattern
does not match) as a BLACKLIST.
Since the oscap remediation stage in osbuild runs
the oscap package in `chroot`, it is necessary to
install the `openscap-scanner` package to the image
itself rather than the build root.
this is no longer needed b/c the nightly CI jobs are now using the same
json definitions as the regular CI jobs, just changing baseurl. See
previous commit.
Since the LVM support was added to all distros, our disk
related code is adaptive, i.e. we will set the correct BLS
and grub2 prefix if there a `boot` partiton is present in
the layout after all customizations happen, which includes
LVMification.
One thing that was not yet fully working was layouts that
do not yet have a `/boot` partition but allow LVMification.
In that case `NewPartitionTable` and if `/boot` was the
first (or only) customization, would LVMify the partition
which in turn would create the `/boot` partition; but after
`newPT.ensureLVM()` the call to `newPT.createFilesystem`
with `/boot` would try to create another `/boot` mountpoint.
In order to deal with this situation correctly we are now
using a two phase approach: 1) enlarge existing mountpoints
and collect new ones. 2) if there are new ones and LMVify
was allowed, switch to LVM layout. Do a second pass and now
create or enlarge existing partitions, handling `/boot` in
the process.
Fedora is using 'tmpfs' as /tmp and that is based on the size of RAM.
That is not enough in case of medium Openstack machines. Changin to use
/var/tmp which is backed by a drive resolves this.
This test used to spawn two VMs at the same time which requires more
memory than the Openstack ci medium runner can provide. We want to be
using only medium runners so this change is necesasry to allow that.
Extend the implementation of mock openid server to take the `grant_type`
into consideration for the `/token` endpoint.
In addition to the previously supported `refresh_topen`, the
implementation now supports also `client_credentials`.
This is necessary to make it possible to use the mock server in
the `koji-osbuild` CI, because the builder plugin uses
`client_credentials` to get access token.
The implementation behaves in the following way:
- For `refresh_token` grant type, it takes the `refresh_token` value
from the request and adds it to the `rh-org-id` field in the custom
claim, which is part of the returned token.
- For `client_credentials` grant type, it takes the `client_secret`
value from the request and adds it to the `rh-org-id` field in the
custom claim, which is part of the returned token.
Requests without the supported `grant_type` set are rejected.
Modify affected test cases to specify `grant_type` when fetching a new
access token.
Add integration tests for oscap customizations.
This tests only the most basic case of oscap remediation.
Mountpoints and additional packages are not added since
this varies between distros and OpenSCAP profiles
i.e. additional blueprints customizations would need
to be specified for each oscap profile to ensure
best results.
To check embedding containers via the cloud API works, embed a
known test container from our gitlab CI and check that it is
indeed embedded in the image by pulling the commit and poking
into the container storage.
Do not start local worker (mask the unit) and Weldr API socket when
provisioning the SUT with TLS client cert authentication method. This
method is used only in the Service scenario, therefore starting these
units / sockets was not reflecting the intended deployment.
Modify `api.sh` to not rely on local worker.
Modify `base_tests.sh` to provision SUT with TLS for
`osbuild-auth-tests`, while provisioning SUT with no authentication
method for the rest of test cases.
`tools/provision.sh` is provisioning SUT always in the same way for
both, the Service scenario and the on-premise scenario. While this is
not causing any issues, it does not realistically represent how we
expect osbuild-composer and worker to be used in these scenarios.
The script currently supports the following authentication options:
- `none`
- Intended for the on-premise scenario with Weldr API.
- NO certificates are generated.
- NO osbuild-composer configuration file is created.
- NO osbuild-worker configuration file is created. This means that no
cloud provider credentials are configured directly in the worker.
- Only the local worker is started and used.
- Only the Weldr API socker is started.
- Appropriate repository definitions are copied to
`/etc/osbuild-composer/repositories/`.
- `jwt`
- Intended for the Service scenario with Cloud API.
- Should be the only method supported in the Service scenario in the
future.
- Certificates are generated and copied to `/etc/osbuild-composer`.
- osbuild-composer configuration file is created and configured for
JWT authentication.
- osbuild-worker configuration file is created, configured for JWT
authentication and with appropriate cloud provider credentials.
- Local worker unit is masked. Only the remote worker is used (the
socket is started and one remote-worker instance is created).
- Only the Cloud API socket is started (Weldr API socket is stopped).
- NO repository definitions are copied to
`/etc/osbuild-composer/repositories/`.
- `tls`
- Intended for the Service scenario with Cloud API.
- Should eventually go away.
- Certificates are generated and copied to `/etc/osbuild-composer`.
- osbuild-composer configuration file is created and configured for
TLS client cert authentication.
- osbuild-worker configuration file is created, configured for TLS
authentication and with appropriate cloud provider credentials.
- Services and sockets are started as they used to be originally:
- Both local and remote worker sockets are started.
- Both Weldr and Cloud API sockets are started.
- Only the local worker unit will be started automatically.
- NO repository definitions are copied to
`/etc/osbuild-composer/repositories/`.
Common integration tests should not need to care about specific ORG ID
configured in the worker, but they should be able to get access token
and check compose status without providing a specific ORG ID. The only
integration test that should care about ORG ID is the
`multi-tenancy.sh`.
Modify the `access_token` and `compose_status` functions to hide the
existence of ORG ID from the user and instead read it from the worker's
configuration, specifically `/etc/osbuild-worker/token`.
The original implementations of the functions mentioned above are now
available under `access_token_with_org_id` and
`compose_status_with_org_id` names.
Modify the `multi-tenancy.sh` to use the new function names.
Move some code related to using JWT tokens from the `multi-tenancy.sh`
test case to `test/cases/api/common/common.sh`, `tools/provision.sh`
and `tools/run-mock-auth-servers.sh`. Move the composer and worker
configuration from the test to new testing configuration files.
The `tools/provision.sh` now accepts an optional argument specifying the
authentication method to use with the provisioned composer and workers.
Valid values are `tls` and `jwt`. If no argument is specified, the `tls`
option is used and the script defaults to its previous behavior.
