in many files there was a secondary call to `trap` for the sole purpose
of killing jornalctl (watching worker logs) so that GitLab CI doesn't
hang.
The issue with this is that sometimes the cleared the trap which invokes
the cleanup() function without reinstating it again (not everywhere).
Instead of doing this back-and-forth just make sure we don't leave any
journalctl processes dangling in the background!
NOTES:
- for some scripts, mainly ostree- ones there was no cleanup trap
present, but instead `trap` was configured inside the build_image() function.
The trouble is that this function is executed multiple times and
$WORKER_JOURNAL_PID changes value between these multiple executions.
That's why these scripts introduce the cleanup_on_exit() function where
we make sure to kill any possible dangling journalctl processes.
- The name `cleanup_on_exit()` is chosed because these same scripts
often have a helper function named clean_up() which is sometimes used to remove
virtual machines and other artifacts between calls of build_image().
Do not run `set-env-variables.sh` to set ARCH and OS variables
to avoid the CI failing because of shellcheck lint problems.
Signed-off-by: Miguel Martín <mmartinv@redhat.com>
Make it possible to test if system FIPS mode has been enabled
in the resulting images by running the scripts with a FIPS="true"
environment variable
Signed-off-by: Miguel Martín <mmartinv@redhat.com>
test logs print:
[2023-08-22T10:18:14-04:00] 💿 Install image via installer(ISO) on VM
WARNING --os-type is deprecated and does nothing. Please stop using it.
We were using greenprint for failures, which makes it hard to quickly
find where the tests failed. This switches errors to use redprint, and
adds it to places that were simply using echo before doing an exit 1.
1. Remove ansible-blocking-io.py workaround. It's not required.
2. Variable should be PROD_REPO_URL, not STAGE_REPO_URL in
ostree-raw-image.sh
3. Use --reboot in rpm-ostree install to reboot VM instead of
a reboot ansible task
4. Wait until config file serviceinfo_api_server.yml exists, to
avoid file not available to use flaky issue
ostree-raw-image.sh and ostree-simplified-installer.sh
Don't need dedicated rebase test workflow, move rebase test into
other test script. That will save runner and reduce running time
Add BIOS rebase test in ostree-raw-image.sh
Add UEFI rebase test in ostree-simplified-installer.sh
1. Run RHEL for Edge CI on osbuild/rhel-edge-ci repo
2. Use released RHEL 8.8 and 9.2 boot ISO
3. Extend VM memory to 3072 on ostree.sh to fix error
"Overriding memory to 3072 MiB needed for centos-stream9 network install."
4. Install and start firewalld, configure VM network as trusted zone
Extend the `ostree-raw-image.sh` to use the directories and files
customizations for the raw image and test their effect using the ansible
playbook.
Custom files and directories are currently tested only on Fedora,
because they are allowed by policy only there.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Break the execution of `ansible-playbook` into multiple lines to make it
easier to read.
Add a comment on top of the second `ansible-playbook` execution if
Ignitions is availble, explaining what differs. Otherwise this is hard
to spot.
There are 3 places in the script, where there are always 2 executions of
the `ansible-playbook`. First the user from the BP is used. On the
second run, the user created by Ignition is used. This was not the case
for the last 2 executions of `ansible-playbook` at the end of the
script. Adjust the order for consistency with the rest of the script.
The changes should have no effect on the test case.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Make edge raw images (both the raw image type and the same in the edge
simplified installer) mount the /sysroot as read-only. This was already
done in Fedora 37+ (547f7a66b3).
Copied commit message from 6f89e9d499 to a
comment in all places where the accompanying "rw" kernel option is set
explaining the requirement, since the option is counter-intuitive.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
The Fedora IoT Raw image sets the default remote URL to the official
Fedora IoT source at build time. We need to replace it in our tests
with our local test repo.
3013889727
changed the behaviour of --boot uefi. Now, it doesn't pass the loader options
but uses firmware="efi" that uses libvirt's ability to auto-detect the correct
loader.
Therefore, for distributions that ship the new virt-install, we have to change
the --boot argument. --boot uefi should now be able to automatically set the
uefi loader correctly without passing any extra loader options. The only issue
is that CS9 has issues with secure boot and edge, so we need to disable
secure boot using a new (and arcane) syntax (the old loader arguments no
longer work):
firmware.feature0.name=secure-boot,firmware.feature0.enabled=no
(https://libvirt.org/formatdomain.html#bios-bootloader)
See the CS9 bug for secure boot issues on edge:
https://bugzilla.redhat.com/show_bug.cgi?id=2108646
Also, since --os-type is deprecated in libvirt >= 4, let's remove it.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
The image_type variable in the playbook is used for the name of the
deployment, which is 'redhat' by default for RHEL and 'fedora-iot' for
Fedora.
This affects the deployment path checks, for example:
{{ device_name }}[/ostree/deploy/{{ image_type }}/var]
Instead of defining users in the ostree commits, create them during the
deployment and creation of the raw image.
Defining users in commits locks them into the base system: they can't be
removed and their password can't be changed without creating an upgrade
commit and they will always be part of the initial system.
Also, in Fedora 37, password authentication for users in commits is not
working.
Since we're sharing functions between test scripts, move greenprint(),
the most rewritten function in the history of the project, to
shared_lib.sh and source it everywhere.
- Handle the array responses from the new weldr-client (>= 35.6).
- Move the `get_build_info` function to shared_libs.sh to source and
reuse in multiple places.
`tools/provision.sh` is provisioning SUT always in the same way for
both, the Service scenario and the on-premise scenario. While this is
not causing any issues, it does not realistically represent how we
expect osbuild-composer and worker to be used in these scenarios.
The script currently supports the following authentication options:
- `none`
- Intended for the on-premise scenario with Weldr API.
- NO certificates are generated.
- NO osbuild-composer configuration file is created.
- NO osbuild-worker configuration file is created. This means that no
cloud provider credentials are configured directly in the worker.
- Only the local worker is started and used.
- Only the Weldr API socker is started.
- Appropriate repository definitions are copied to
`/etc/osbuild-composer/repositories/`.
- `jwt`
- Intended for the Service scenario with Cloud API.
- Should be the only method supported in the Service scenario in the
future.
- Certificates are generated and copied to `/etc/osbuild-composer`.
- osbuild-composer configuration file is created and configured for
JWT authentication.
- osbuild-worker configuration file is created, configured for JWT
authentication and with appropriate cloud provider credentials.
- Local worker unit is masked. Only the remote worker is used (the
socket is started and one remote-worker instance is created).
- Only the Cloud API socket is started (Weldr API socket is stopped).
- NO repository definitions are copied to
`/etc/osbuild-composer/repositories/`.
- `tls`
- Intended for the Service scenario with Cloud API.
- Should eventually go away.
- Certificates are generated and copied to `/etc/osbuild-composer`.
- osbuild-composer configuration file is created and configured for
TLS client cert authentication.
- osbuild-worker configuration file is created, configured for TLS
authentication and with appropriate cloud provider credentials.
- Services and sockets are started as they used to be originally:
- Both local and remote worker sockets are started.
- Both Weldr and Cloud API sockets are started.
- Only the local worker unit will be started automatically.
- NO repository definitions are copied to
`/etc/osbuild-composer/repositories/`.
We want to be able to safely gather any artifacts without worrying about
any possible secrets leaking. Every artifacts that we want to upload
will now have to be placed in /tmp/artifacts which will then be uploaded
to S3 by the executor and link to the artifacts will be provided in the
logs. Only people with access to our AWS account can see them.
When deploying an ostree commit, specify a remote, currently hard-
coded to `rhel-edge`, so that updates work automatically, if they
are served from the same location as the initial commit is pulled
from.
NB: now that the remote is specified in the raw image, remove the
corresponding bits form the tests.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
There's conflicting ansible versions in the 86 nightlies and epel. There
should be a correct combination of plugins which fixes the callback on
86. But let's drop it to unblock for now.
With new weldr-client package the metadata tar archive created has
permissions set to 600 instead of 644 which causes permission failures
when interacting with it. Adding sudo to resolve that.
