There is no need for osbuild-composer to wait until the whole multi-user
target is completed. It can be started earlier as it doesn't have any
dependencies in the target.
This can be a problem if there is a unit in the target that is not starting
and still is unrelated to osbuild-composer. There was a bug like this
with Plymouth where the service didn't finish and it was hanging. That
prevented osbuild-composer from starting and the user was left with
working SSH connection and shell, but composer-cli and systemctl start
osbuild-composer.service were both unresponsive.
Replace After=multi-user.target with After=network.target to start
osbuild-composer earlier.
The variables are set to the git revision from which the build is
triggered and rpm version from the spec file, if it is build using RPM.
This can be later used to query exact source version while
running osbuild-composer.
It is necessary to use both, because none of them is available in all
possible scenarios.
Use either git-rev (preferably) or RPM version (NEVRA) instead of the
"devel" build type. It was just a placeholder.
The ostree-ng test was updated to accommodate a change in the
edge-container image type
(see https://github.com/osbuild/osbuild-composer/pull/1595).
RHEL 9.0-beta uses the old edge-container configuration, however RHEL
9.0 GA should eventually adopt the new container.
Copying the old ostree-ng test and running it separately is better than
having several conditions in the single test script to accommodate the
old behaviour and then reverting them when the changes land in RHEL 9.
Modified from the old version to assume we're using 'weldr-client'
(which has a different json output schema) and to use RHEL 9
naming/versioning where necessary.
The ansible checks were also copied to remove the kernel-rt check.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
recover from panics such as out-of-bounds array access & nil
pointer access, print a stack trace and return 5xx error
instead of the service crashing and relying on Execution
framework to handle crashes
The base url for the osbuild-worker container was
missing the `https` protocol in the docker-compose file.
This commit adds this to prevent the worker from exiting
The `nss-altfiles` package is actually needed for OSTree based
systems, since in those the user/groups database is located in
the /usr/lib directory (in addition to /etc). It was removed
because it was not available in RHEL 9 (rhbz#1916260). This has
been fixed, so we need to re-include the package.
Generating image test case with all possible blueprint customizations is
supported only for `qcow2` image type. Generating `qcow2` images with
customizations was not a lot of fun. One had to use a special CLI option
for this case.
To streamline the generation of image test cases, drop the
`--with-customizations` option from the `generate-test-cases` script and
move the functionality to `format-request-map.json` by defining a new
`qcow2-customize` test case. This is very similar to what `*edge-rt`
test case already uses. This mean that when the test case for `qcow2`
image type is being generated, actually two test cases will be
generated. The filesystem customizations since RHEL-8.5 are handled
through distro-specific `overrides`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Some of the `overrides` specified in `format-request-map.json` were
redundant, because the same blueprint customizations were already
specified in the image compose-request. Thus these values had no
additional values.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Unfortunately, after the addition of `supported_arches` item to some
of the image type definitions in `format-request-map.json`, it started
to leak to the `compose-request` object in the generated image test
case.
Ensure that the item does not leak to the generated image test case,
since it is relevant only for the generator script.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add a CLI option to keep the workdir created on the runner, after it
finishes its work. The workdir is deleted by default.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Report per-distro list of image test cases which were successfully
generated or failed, after the Runner finishes its work.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Remove the '--keep-image-info' script option. The output directory on
the Runner is always newly created at the beginning and empty, therefore
this option effectively can not be used with the `generate-test-cases`
script.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The script currently allows generating image test cases only using QEMU
VMs. As part of the process to allow using different approaches for
generating image test cases, introduce CLI commands using subparsers and
move the current functionality under 'qemu' command.
As part of extracting the generic parts from the TestCaseMatrixGenerator
class, rename it to QEMUTestCaseMatrixGenerator and move all generic
parts to a new BaseTestCaseMatrixGenerator class.
Change filtering options such as '--arch', '--distro', '--image-type'
to take only one argument and append the value to a list if used
multiple times, instead of accepting multiple values. This resolves an
issue of 'nargs=*' argument being too greedy and consuming command as
an additional argument.
Modify the script doc text to match the changes.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
When one run the script with debug logging (`-d`), the set log level
in the logger was not preserved in the newly created multiprocessing
processes.
Explicitly set the log level in newly created processes.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Previously the QEMU virtfs devices were used to export osbuild-composer
sources to the QEMU VMs and also to get generated image test cases from
them. This had multiple downsides. The virtfs QEMU option is not
supported on MacOS. In addition, the 9p protocol, which was needed to
mount these devices in the VM, is not supported on RHEL.
Remove all code related to mounting. The data is now copied to and from
the VM using rsync instead.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Don't use paramiko library for SSH connections to the Runner, but
instead execute the `ssh` command using Subprocess.
When one uses SSH ID files protected by password, the paramiko library
can not access them without it, even if the password is stored in the
ssh-agent running in the user session. On the other hand, running the
`ssh` command using Subprocess works just fine in this scenario.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Previously passwords were used to log into provisioned QEMU VMs. This is
not practical if one would like to use e.g. rsync to transfer files from
and to the VM. The script now does not use passwords at all, but instead
configures the most recent SSH key from the system matching
'~/.ssh/id*.pub' as an authorized key on the VM. Alternatively the SSH
key to be used can be provided as an argument to the script.
In addition, the script no longer relies on external files for
cloud-init user-data. If no cloud-init user-data are provided as an
argument to the script creates default user-data file in a temporary
work directory and uses it. The reason for this change is mostly that the
default user-data became very short and need to be always extended with
the authorized SSH key anyway. In addition, this makes the script more
standalone by relying on fewer external file paths.
Delete the `tools/deploy/gen-test-data` which held the cloud-init
user-data previously used by default by the script.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Previously cloud-init was used to install necessary packages on the
runner. This would not be practical in the future with other types of
runners, which would not use cloud-init. Install all necessary RPMs by
directly running DNF command on the runner.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The `BaseRunner` class represented a base QEMU runner. Rename it to
`BaseQEMURunner` and extract parts which are not QEMU specific to a new
`BaseRunner` class. This new base class will be later used as a baseline
for other types of runners which don't rely on QEMU.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Some image type test cases require additional repositories, which are
not available for all architectures. However, when an image type test
case is specified in the `format-request-map.json`, it is generated on any
architecture.
This behavior is creating issues when generating `*edge-commit` image
type test cases. This is because the `format-request-map.json` contains one
additional definition for `*edge-commit-rt`, which includes `kernel-rt`
package. However repositories with this package are available only for
x86_64. Therefore, when generating image test cases for `*edge-commit`,
the `generate-test-cases` script always generates two test cases,
but the generation of `*edge-commit-rt` always fails on non-x86_64
architectures.
Add a new optional member to the image type test case object in
`format-request-map.json`, called `supported_arches`. Its value is a
list of strings, specifying the supported architectures of the image
type test case. In case the member is not specified, the image test
case is supported on any architecture.
Extend the `generate-test-cases` script to skip image type test case
generation in case the case has the `supported_arches` specified and the
requested architecture is not in the list.
Fix#1478
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Instead of keeping a howto on releasing in every repository we move the
documentation to a central place in our "developer guide".
See https://github.com/osbuild/guides/pull/45
