Unfortunately, `which` does not seem to be installed by default on our
F41 CI images. Instead of doing the dance with rebuilds, which has been
problematic recently, let's not rely on `which` in scripts any more,
since we can replace it with the Bash built-in `type` command.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
in many files there was a secondary call to `trap` for the sole purpose
of killing jornalctl (watching worker logs) so that GitLab CI doesn't
hang.
The issue with this is that sometimes the cleared the trap which invokes
the cleanup() function without reinstating it again (not everywhere).
Instead of doing this back-and-forth just make sure we don't leave any
journalctl processes dangling in the background!
NOTES:
- for some scripts, mainly ostree- ones there was no cleanup trap
present, but instead `trap` was configured inside the build_image() function.
The trouble is that this function is executed multiple times and
$WORKER_JOURNAL_PID changes value between these multiple executions.
That's why these scripts introduce the cleanup_on_exit() function where
we make sure to kill any possible dangling journalctl processes.
- The name `cleanup_on_exit()` is chosed because these same scripts
often have a helper function named clean_up() which is sometimes used to remove
virtual machines and other artifacts between calls of build_image().
the conditionals have been introduced in
8960a51d2f
which states (emphasys mine):
> The new partition rules are **in** osbuild-composer v94 and higher.
but the condition was "not v94 or higher" and as a result it fails on
8.10 nightly with osbuild-composer v92 with:
failed to initialize osbuild manifest: The following custom mountpoints are not supported [\"/boot/firmware\" \"/foobar\"]
see https://gitlab.com/redhat/services/products/image-builder/ci/osbuild-composer/-/jobs/5905865402
The mountpoint policy changed as a result of COMPOSER-2030 [1]. Modify
the test case accordingly, to comprehensively test the updated
mountpoint policy.
[1] https://issues.redhat.com/browse/COMPOSER-2030
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
We were using greenprint for failures, which makes it hard to quickly
find where the tests failed. This switches errors to use redprint, and
adds it to places that were simply using echo before doing an exit 1.
Since we're sharing functions between test scripts, move greenprint(),
the most rewritten function in the history of the project, to
shared_lib.sh and source it everywhere.
- Handle the array responses from the new weldr-client (>= 35.6).
- Move the `get_build_info` function to shared_libs.sh to source and
reuse in multiple places.
Since the LVM support was added to all distros, our disk
related code is adaptive, i.e. we will set the correct BLS
and grub2 prefix if there a `boot` partiton is present in
the layout after all customizations happen, which includes
LVMification.
One thing that was not yet fully working was layouts that
do not yet have a `/boot` partition but allow LVMification.
In that case `NewPartitionTable` and if `/boot` was the
first (or only) customization, would LVMify the partition
which in turn would create the `/boot` partition; but after
`newPT.ensureLVM()` the call to `newPT.createFilesystem`
with `/boot` would try to create another `/boot` mountpoint.
In order to deal with this situation correctly we are now
using a two phase approach: 1) enlarge existing mountpoints
and collect new ones. 2) if there are new ones and LMVify
was allowed, switch to LVM layout. Do a second pass and now
create or enlarge existing partitions, handling `/boot` in
the process.
`tools/provision.sh` is provisioning SUT always in the same way for
both, the Service scenario and the on-premise scenario. While this is
not causing any issues, it does not realistically represent how we
expect osbuild-composer and worker to be used in these scenarios.
The script currently supports the following authentication options:
- `none`
- Intended for the on-premise scenario with Weldr API.
- NO certificates are generated.
- NO osbuild-composer configuration file is created.
- NO osbuild-worker configuration file is created. This means that no
cloud provider credentials are configured directly in the worker.
- Only the local worker is started and used.
- Only the Weldr API socker is started.
- Appropriate repository definitions are copied to
`/etc/osbuild-composer/repositories/`.
- `jwt`
- Intended for the Service scenario with Cloud API.
- Should be the only method supported in the Service scenario in the
future.
- Certificates are generated and copied to `/etc/osbuild-composer`.
- osbuild-composer configuration file is created and configured for
JWT authentication.
- osbuild-worker configuration file is created, configured for JWT
authentication and with appropriate cloud provider credentials.
- Local worker unit is masked. Only the remote worker is used (the
socket is started and one remote-worker instance is created).
- Only the Cloud API socket is started (Weldr API socket is stopped).
- NO repository definitions are copied to
`/etc/osbuild-composer/repositories/`.
- `tls`
- Intended for the Service scenario with Cloud API.
- Should eventually go away.
- Certificates are generated and copied to `/etc/osbuild-composer`.
- osbuild-composer configuration file is created and configured for
TLS client cert authentication.
- osbuild-worker configuration file is created, configured for TLS
authentication and with appropriate cloud provider credentials.
- Services and sockets are started as they used to be originally:
- Both local and remote worker sockets are started.
- Both Weldr and Cloud API sockets are started.
- Only the local worker unit will be started automatically.
- NO repository definitions are copied to
`/etc/osbuild-composer/repositories/`.
Don't remove the pre-existing mount-points from the blueprint b/c we
still need to assert on those.
Add `/tmp` and `/var/tmp` to exercise the case where mountpoints have
matching suffixes.
Filesystem mountpoints with matching suffixes, i.e.
`/tmp` and `/var/tmp` was causing image builds to
fail. Since this was fixed with #2141 the integration
tests have been updated to check for this.
- the /var/log/audit mountpoint has been added previously to the
blueprint but wasn't explicitly checked
- reordered the list of mountpoints to match the blueprint
(alphabetically)
- added logging of successfull results we that QE can see what has been
tested in the logs and use it for verification purposes.
Related: rhbz#2002727, rhbz#2001891
It no longer makes sense because:
- we don't make any changes to 8.5
- we don't regenerate test manifests for 8.5
- osbuild-composer for 8.5 is in the rhel-8.5.0 branch
Also, the latest-8.5.0 symlink was removed, which broke the CI.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
The commonly used 'greenprint' function now adds a date + timestamp to
each message for debugging and tracking the duration of segments of each
scripts.
Add test case to check the ability to create partitions for
the /var, /var/log & /var/log/audit mountpoints. This test is
checking that mountpoints with a depth of 3 or greater can be configured.
