Refactor how the 'trap' command is used to avoid double calls

in many files there was a secondary call to `trap` for the sole purpose
of killing jornalctl (watching worker logs) so that GitLab CI doesn't
hang.

The issue with this is that sometimes the cleared the trap which invokes
the cleanup() function without reinstating it again (not everywhere).

Instead of doing this back-and-forth just make sure we don't leave any
journalctl processes dangling in the background!

NOTES:

- for some scripts, mainly ostree- ones there was no cleanup trap
  present, but instead `trap` was configured inside the build_image() function.
  The trouble is that this function is executed multiple times and
  $WORKER_JOURNAL_PID changes value between these multiple executions.
  That's why these scripts introduce the cleanup_on_exit() function where
  we make sure to kill any possible dangling journalctl processes.
- The name `cleanup_on_exit()` is chosed because these same scripts
  often have a helper function named clean_up() which is sometimes used to remove
  virtual machines and other artifacts between calls of build_image().
This commit is contained in:
Alexander Todorov 2024-03-07 12:20:00 +02:00 committed by Alexander Todorov
parent c534689d57
commit 85ce42570d
19 changed files with 110 additions and 83 deletions

View file

@ -32,6 +32,8 @@ fi
TEMPDIR=$(mktemp -d)
function cleanup() {
greenprint "== Script execution stopped or finished - Cleaning up =="
# kill dangling journalctl processes to prevent GitLab CI from hanging
sudo pkill journalctl || echo "Nothing killed"
sudo rm -rf "$TEMPDIR"
}
trap cleanup EXIT
@ -127,8 +129,6 @@ sudo composer-cli blueprints depsolve bash
WORKER_UNIT=$(sudo systemctl list-units | grep -o -E "osbuild.*worker.*\.service")
sudo journalctl -af -n 1 -u "${WORKER_UNIT}" &
WORKER_JOURNAL_PID=$!
# Stop watching the worker journal when exiting.
trap 'sudo pkill -P ${WORKER_JOURNAL_PID}' EXIT
# Start the compose and upload to AWS.
greenprint "🚀 Starting compose"
@ -155,9 +155,8 @@ greenprint "💬 Getting compose log and metadata"
get_compose_log "$COMPOSE_ID"
get_compose_metadata "$COMPOSE_ID"
# Kill the journal monitor immediately and remove the trap
# Kill the journal monitor
sudo pkill -P ${WORKER_JOURNAL_PID}
trap - EXIT
# Did the compose finish with success?
if [[ $COMPOSE_STATUS != FINISHED ]]; then