Use the `subprocess_check_output()` wrapper function in the
`read_selinux_ctx_mismatch()` parser function, instead of
directly calling `subprocess.check_output()`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Extend the report to list also all documentation files, which are
normally part of the installed RPM packages, but were not installed
on the system. This can happen e.g. when '--excludedocs' option
is used when installing packages using rpm.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Read uncommented lined from /etc/resolv.conf and add them as a list to
the image-info report. The list of lines is present in the report even
if it is empty, so that the report is explicit about the file content
and presence.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Extend image-info to read systemd service unit drop-ins from
multiple paths:
- /etc/systemd/system/*.service.d
- /usr/lib/systemd/system/*.service.d
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Generating image test case with all possible blueprint customizations is
supported only for `qcow2` image type. Generating `qcow2` images with
customizations was not a lot of fun. One had to use a special CLI option
for this case.
To streamline the generation of image test cases, drop the
`--with-customizations` option from the `generate-test-cases` script and
move the functionality to `format-request-map.json` by defining a new
`qcow2-customize` test case. This is very similar to what `*edge-rt`
test case already uses. This mean that when the test case for `qcow2`
image type is being generated, actually two test cases will be
generated. The filesystem customizations since RHEL-8.5 are handled
through distro-specific `overrides`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Some of the `overrides` specified in `format-request-map.json` were
redundant, because the same blueprint customizations were already
specified in the image compose-request. Thus these values had no
additional values.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Unfortunately, after the addition of `supported_arches` item to some
of the image type definitions in `format-request-map.json`, it started
to leak to the `compose-request` object in the generated image test
case.
Ensure that the item does not leak to the generated image test case,
since it is relevant only for the generator script.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add a CLI option to keep the workdir created on the runner, after it
finishes its work. The workdir is deleted by default.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Report per-distro list of image test cases which were successfully
generated or failed, after the Runner finishes its work.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Remove the '--keep-image-info' script option. The output directory on
the Runner is always newly created at the beginning and empty, therefore
this option effectively can not be used with the `generate-test-cases`
script.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The script currently allows generating image test cases only using QEMU
VMs. As part of the process to allow using different approaches for
generating image test cases, introduce CLI commands using subparsers and
move the current functionality under 'qemu' command.
As part of extracting the generic parts from the TestCaseMatrixGenerator
class, rename it to QEMUTestCaseMatrixGenerator and move all generic
parts to a new BaseTestCaseMatrixGenerator class.
Change filtering options such as '--arch', '--distro', '--image-type'
to take only one argument and append the value to a list if used
multiple times, instead of accepting multiple values. This resolves an
issue of 'nargs=*' argument being too greedy and consuming command as
an additional argument.
Modify the script doc text to match the changes.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
When one run the script with debug logging (`-d`), the set log level
in the logger was not preserved in the newly created multiprocessing
processes.
Explicitly set the log level in newly created processes.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Previously the QEMU virtfs devices were used to export osbuild-composer
sources to the QEMU VMs and also to get generated image test cases from
them. This had multiple downsides. The virtfs QEMU option is not
supported on MacOS. In addition, the 9p protocol, which was needed to
mount these devices in the VM, is not supported on RHEL.
Remove all code related to mounting. The data is now copied to and from
the VM using rsync instead.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Don't use paramiko library for SSH connections to the Runner, but
instead execute the `ssh` command using Subprocess.
When one uses SSH ID files protected by password, the paramiko library
can not access them without it, even if the password is stored in the
ssh-agent running in the user session. On the other hand, running the
`ssh` command using Subprocess works just fine in this scenario.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Previously passwords were used to log into provisioned QEMU VMs. This is
not practical if one would like to use e.g. rsync to transfer files from
and to the VM. The script now does not use passwords at all, but instead
configures the most recent SSH key from the system matching
'~/.ssh/id*.pub' as an authorized key on the VM. Alternatively the SSH
key to be used can be provided as an argument to the script.
In addition, the script no longer relies on external files for
cloud-init user-data. If no cloud-init user-data are provided as an
argument to the script creates default user-data file in a temporary
work directory and uses it. The reason for this change is mostly that the
default user-data became very short and need to be always extended with
the authorized SSH key anyway. In addition, this makes the script more
standalone by relying on fewer external file paths.
Delete the `tools/deploy/gen-test-data` which held the cloud-init
user-data previously used by default by the script.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Previously cloud-init was used to install necessary packages on the
runner. This would not be practical in the future with other types of
runners, which would not use cloud-init. Install all necessary RPMs by
directly running DNF command on the runner.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The `BaseRunner` class represented a base QEMU runner. Rename it to
`BaseQEMURunner` and extract parts which are not QEMU specific to a new
`BaseRunner` class. This new base class will be later used as a baseline
for other types of runners which don't rely on QEMU.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Some image type test cases require additional repositories, which are
not available for all architectures. However, when an image type test
case is specified in the `format-request-map.json`, it is generated on any
architecture.
This behavior is creating issues when generating `*edge-commit` image
type test cases. This is because the `format-request-map.json` contains one
additional definition for `*edge-commit-rt`, which includes `kernel-rt`
package. However repositories with this package are available only for
x86_64. Therefore, when generating image test cases for `*edge-commit`,
the `generate-test-cases` script always generates two test cases,
but the generation of `*edge-commit-rt` always fails on non-x86_64
architectures.
Add a new optional member to the image type test case object in
`format-request-map.json`, called `supported_arches`. Its value is a
list of strings, specifying the supported architectures of the image
type test case. In case the member is not specified, the image test
case is supported on any architecture.
Extend the `generate-test-cases` script to skip image type test case
generation in case the case has the `supported_arches` specified and the
requested architecture is not in the list.
Fix#1478
Signed-off-by: Tomas Hozza <thozza@redhat.com>
This commit also fixes the rpmrepo for 8.5 rt. The previously used one
was wrongly generated and had to be regenerated.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
* Return error when write_files exists in cloud-init
Since the script adds a `write_files` key in cloud-init user-data, it
should return error if this key already exist in the input file.
Co-authored-by: Ondřej Budai <obudai@redhat.com>
This commit also updates the rpmrepo snapshot for rhel-90 to the latest one.
Note that vmdk and vhd images are currently not buildable, see:
https://bugzilla.redhat.com/show_bug.cgi?id=1989841
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Renamed tar-installer to image-installer.
This is a more appropriate name:
- It disassociates the image type from the "tar" image type. The two
should not be perceived to be connected.
- It's more descriptive. The format of the payload (tar) isn't relevant
to the purpose of the image type.
