Since we're sharing functions between test scripts, move greenprint(),
the most rewritten function in the history of the project, to
shared_lib.sh and source it everywhere.
- Handle the array responses from the new weldr-client (>= 35.6).
- Move the `get_build_info` function to shared_libs.sh to source and
reuse in multiple places.
We were using `latest` as tag, this can be dangerous as it's the default
tag, an anyone can accidentally update it. Using `prod` is safer.
Also use dev container image if the test script is running in CIV CI.
instead of manually updating CIV version every once in a while. Get
always the latest version.
In CIV CI, this test runs before any change can be introduced into the
container image, so no unexpedted errors should come from the CIV side.
`tools/provision.sh` is provisioning SUT always in the same way for
both, the Service scenario and the on-premise scenario. While this is
not causing any issues, it does not realistically represent how we
expect osbuild-composer and worker to be used in these scenarios.
The script currently supports the following authentication options:
- `none`
- Intended for the on-premise scenario with Weldr API.
- NO certificates are generated.
- NO osbuild-composer configuration file is created.
- NO osbuild-worker configuration file is created. This means that no
cloud provider credentials are configured directly in the worker.
- Only the local worker is started and used.
- Only the Weldr API socker is started.
- Appropriate repository definitions are copied to
`/etc/osbuild-composer/repositories/`.
- `jwt`
- Intended for the Service scenario with Cloud API.
- Should be the only method supported in the Service scenario in the
future.
- Certificates are generated and copied to `/etc/osbuild-composer`.
- osbuild-composer configuration file is created and configured for
JWT authentication.
- osbuild-worker configuration file is created, configured for JWT
authentication and with appropriate cloud provider credentials.
- Local worker unit is masked. Only the remote worker is used (the
socket is started and one remote-worker instance is created).
- Only the Cloud API socket is started (Weldr API socket is stopped).
- NO repository definitions are copied to
`/etc/osbuild-composer/repositories/`.
- `tls`
- Intended for the Service scenario with Cloud API.
- Should eventually go away.
- Certificates are generated and copied to `/etc/osbuild-composer`.
- osbuild-composer configuration file is created and configured for
TLS client cert authentication.
- osbuild-worker configuration file is created, configured for TLS
authentication and with appropriate cloud provider credentials.
- Services and sockets are started as they used to be originally:
- Both local and remote worker sockets are started.
- Both Weldr and Cloud API sockets are started.
- Only the local worker unit will be started automatically.
- NO repository definitions are copied to
`/etc/osbuild-composer/repositories/`.
We want to be able to safely gather any artifacts without worrying about
any possible secrets leaking. Every artifacts that we want to upload
will now have to be placed in /tmp/artifacts which will then be uploaded
to S3 by the executor and link to the artifacts will be provided in the
logs. Only people with access to our AWS account can see them.
When a test script fails in CI, it's often difficult to pinpoint the
exact line in the log where the script failed and the cleanup() function
(trapped on EXIT) begins.
Adding a prominent line (with greenprint where available) at the start
of the cleanup function will make reading logs of failed jobs a lot
easier.
Most test scripts don't have any documentation regarding it's purpose,
although it can be guessed by the code. There's value in adding this
small comment.
[skip-ci]
With new weldr-client package the metadata tar archive created has
permissions set to 600 instead of 644 which causes permission failures
when interacting with it. Adding sudo to resolve that.
The commonly used 'greenprint' function now adds a date + timestamp to
each message for debugging and tracking the duration of segments of each
scripts.
RHEL 9.0 isn't yet in .gitlab-ci.yml so this actually doesn't change in test
runs but it should make enabling of the tests easier.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
3a8c6c8a introduced a new logic for killing journalctl. Unfortunately, it
doesn't work properly. In ostree tests, multiple journalctls are spawned
but there can be only one trap active at a time. This caused all but the last
journalctls to hang indefinitely. Unfortunately, hanging background processes
is something that causes the GitLab CI to hang indefinitely as well.
This commit modifies the logic a bit: The trap is still set. However, there's
also an explicit kill of journalctl after the compose is finished. After the
process is successfully killed, the trap is removed.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
sudo journalctl -af -n 1 -u "${WORKER_UNIT}" &
WORKER_JOURNAL_PID=$!
In this snippet, WORKER_JOURNAL_PID is set to the PID of the sudo process.
Sudo doesn't propagate any signals - therefore the child process of sudo
(journalctl in this case) isn't killed when a signal is sent to the parent.
Use pkill -P instead which kills all processes where sudo is the parent.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Using DISTRO_CODE simplifies test case selection and allows to test
different distro than the one test is running on.
This is used to run tests for RHEL 9.0 on F33 or RHEL 8.4
Adding the tag called `Name` to the AMI ensures that the name appears in
the *Name* column inside AWS' web console.
Fixes#1171.
Signed-off-by: Major Hayden <major@redhat.com>
The integration tests are leaving the composes (which include images) in
osbuild-composer. This can lead to exhausting the disk space we have available
on our tiny testing machines. This commit adds a removal of the composes
after each integration test is finished. This issue is not present in koji.sh
and api.sh as they use different osbuild-composer APIs that doesn't use the
artifact feature.
This issue occurred when I worked on enabling the Fedora 33 tests, see:
https://osbuildci.cloud.paas.psi.redhat.com/blue/organizations/jenkins/osbuild%2Fosbuild-composer/detail/PR-1014/23/pipeline
Attempt to clarify the structure of our tests. Each test case is now
encapsulated in a script in `test/cases`. Each of these scripts should
be runnable on a pristine machine and be independent of each other. It
is up to the test-orchestractor to decide if they should be run
consequtively instance, or in parallel on separate instances. Each
script can execute several tests and call whatever helper binaries
is desired. However, each case should be assumed to always run as one.
