The script currently allows generating image test cases only using QEMU
VMs. As part of the process to allow using different approaches for
generating image test cases, introduce CLI commands using subparsers and
move the current functionality under 'qemu' command.
As part of extracting the generic parts from the TestCaseMatrixGenerator
class, rename it to QEMUTestCaseMatrixGenerator and move all generic
parts to a new BaseTestCaseMatrixGenerator class.
Change filtering options such as '--arch', '--distro', '--image-type'
to take only one argument and append the value to a list if used
multiple times, instead of accepting multiple values. This resolves an
issue of 'nargs=*' argument being too greedy and consuming command as
an additional argument.
Modify the script doc text to match the changes.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
When one run the script with debug logging (`-d`), the set log level
in the logger was not preserved in the newly created multiprocessing
processes.
Explicitly set the log level in newly created processes.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Previously the QEMU virtfs devices were used to export osbuild-composer
sources to the QEMU VMs and also to get generated image test cases from
them. This had multiple downsides. The virtfs QEMU option is not
supported on MacOS. In addition, the 9p protocol, which was needed to
mount these devices in the VM, is not supported on RHEL.
Remove all code related to mounting. The data is now copied to and from
the VM using rsync instead.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Don't use paramiko library for SSH connections to the Runner, but
instead execute the `ssh` command using Subprocess.
When one uses SSH ID files protected by password, the paramiko library
can not access them without it, even if the password is stored in the
ssh-agent running in the user session. On the other hand, running the
`ssh` command using Subprocess works just fine in this scenario.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Previously passwords were used to log into provisioned QEMU VMs. This is
not practical if one would like to use e.g. rsync to transfer files from
and to the VM. The script now does not use passwords at all, but instead
configures the most recent SSH key from the system matching
'~/.ssh/id*.pub' as an authorized key on the VM. Alternatively the SSH
key to be used can be provided as an argument to the script.
In addition, the script no longer relies on external files for
cloud-init user-data. If no cloud-init user-data are provided as an
argument to the script creates default user-data file in a temporary
work directory and uses it. The reason for this change is mostly that the
default user-data became very short and need to be always extended with
the authorized SSH key anyway. In addition, this makes the script more
standalone by relying on fewer external file paths.
Delete the `tools/deploy/gen-test-data` which held the cloud-init
user-data previously used by default by the script.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Previously cloud-init was used to install necessary packages on the
runner. This would not be practical in the future with other types of
runners, which would not use cloud-init. Install all necessary RPMs by
directly running DNF command on the runner.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The `BaseRunner` class represented a base QEMU runner. Rename it to
`BaseQEMURunner` and extract parts which are not QEMU specific to a new
`BaseRunner` class. This new base class will be later used as a baseline
for other types of runners which don't rely on QEMU.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Some image type test cases require additional repositories, which are
not available for all architectures. However, when an image type test
case is specified in the `format-request-map.json`, it is generated on any
architecture.
This behavior is creating issues when generating `*edge-commit` image
type test cases. This is because the `format-request-map.json` contains one
additional definition for `*edge-commit-rt`, which includes `kernel-rt`
package. However repositories with this package are available only for
x86_64. Therefore, when generating image test cases for `*edge-commit`,
the `generate-test-cases` script always generates two test cases,
but the generation of `*edge-commit-rt` always fails on non-x86_64
architectures.
Add a new optional member to the image type test case object in
`format-request-map.json`, called `supported_arches`. Its value is a
list of strings, specifying the supported architectures of the image
type test case. In case the member is not specified, the image test
case is supported on any architecture.
Extend the `generate-test-cases` script to skip image type test case
generation in case the case has the `supported_arches` specified and the
requested architecture is not in the list.
Fix#1478
Signed-off-by: Tomas Hozza <thozza@redhat.com>
This commit also fixes the rpmrepo for 8.5 rt. The previously used one
was wrongly generated and had to be regenerated.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
This commit also updates the rpmrepo snapshot for rhel-90 to the latest one.
Note that vmdk and vhd images are currently not buildable, see:
https://bugzilla.redhat.com/show_bug.cgi?id=1989841
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Renamed tar-installer to image-installer.
This is a more appropriate name:
- It disassociates the image type from the "tar" image type. The two
should not be perceived to be connected.
- It's more descriptive. The format of the payload (tar) isn't relevant
to the purpose of the image type.
Added to format-request-map and generated manifest.
No image-info since ISOs aren't supported by the tool. The manifest is
useful only for detecting changes in the image type definition.
Add new image type definitions `ec2` and `ec2-ha` representing the
official RHEL ec2 image types.
Add a `xzArchivePipeline()`, which returns a pipeline producing a XZ
archive from a file produced by a different pipeline.
Add rpmrepo snapshots for `rhui` and `ha` repositories used to generate
image test cases. `rhui` is used by the `ec2` image and it is available
on x86_64 and aarch64 architectures. `ha` is used by the `ec2-ha` image
and it is available only for x86_64.
The new image type definitions are currently not used by any
API test case.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Redefine the `ami` image type in RHEL-8.5 to be based on RHEL
ec2 images. The pipeline has different default settings, therefore the
common "os" pipeline is not used. The RHEL ec2 images have a different
default size than the original `ami` image definition. The RHEL ec2
images use a different default partitioning scheme. Their configuration
is slightly different for each architecture and the x86_64 version
of the image does not support UEFI.
Update rpmrepo snapshots used to generate RHEL-8.5 x86_64 and aarch64
image test cases.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
When not flushed, the line often doesn't get printed until after the job
is done. Printing it before the job is useful for knowing the progress
of a multi-job run.
mirrors.kernel.org are sometimes desynchronized, and thus causing issues with
the image tests. This commit switches the image tests for F33 to use RPMRepo.
All test cases were regenerated.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
When `image-info` inspects ostree images, the `/usr/etc` is bind-mounted
to `/etc`. This results in conflicting SELinux policy specification for
these files and makes the outcome dependent on the `setfiles` build.
All the files in `/etc` have mismatch in the expected vs. actual SELinux
context.
Exclude `/etc` from the check of SELinux ctx mismatches in case the
analysed tree is from an ostree-based image.
Sort the list returned `read_selinux_ctx_mismatch()` based on the item's
`filename` key, to make the result consistent across runs.
`image-info` can not read SELinux labels from the images, which are not
known to the host. This makes the report content depend on the host
environment. As a temporary workaround, relabel the image-info script with
osbuild_exec_t label to allow it to read unknown SELinux labels.
Modify documentation in `test/README.md` to explain the issue with
`image-info` and unknown SELinux labels.
Modify the `generate-all-test-cases` to relabel `image-info` before
generating test cases.
Modify the `image_tests.sh` to relabel `image-info` before running image
test cases.
Add 'tar' image for 'rhel-8' on 's390x' back to the matrix of generated
test cases, as it was removed by mistake. Regenerate the image test
case. Remove 'tar' image from 'rhel-84' on 's390x' from the matrix of
generated test cases, as it is not supported.
Regenerate all affected image test cases.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Remove Fedora 32 image test case manifests, because the release
is already EOL. Remove Fedora 32 definitions from
`distro-arch-imagetype-map.json`. Remove Fedora 32 repositories
from `tools/test-case-generators/repos.json`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
This is a preparation for defining EC2 images for RHEL-8.5.0. These
extensions to image-info tool represent modifications done to the
official EC2 images currently produced as RHEL release. It is important
to be able to analyse these aspects of images, before we define them in
osbuild-composer, to ensure that the resulting images will be consistent
with the current state.
- Read non-empty lines from /etc/hosts file and add them to the report.
- Read content of /etc/machine-id and add it to the report.
- Read uncommented key/values from /etc/systemd/logind.conf and add them
to the report.
- Read all ifcfg-* files from /etc/sysconfig/network-scripts/ and add
their values to the report.
- Read content of /etc/locale.conf and add it to the report.
- Read SELinux configuration from /etc/selinux/config and add it to the
report.
- Inspect the filesystem tree for SELinux context mismatches and add
them to the report.
- Read configuration files from /etc/modprobe.d/ and for now report only
all blacklisted kernel modules.
- Read RHSM configuration from /etc/rhsm/rhsm.conf and add it to the
report.
- Read cloud-init configuration from /etc/cloud/cloud.conf and add it to
the report.
- Read all *.conf files from /etc/dracut.conf.d/ and add their content to
the report.
- Read VC and X11 keyboard configuration and add it to the report.
- Read specific configuration directives from Chrony configuration and
add them to the report. Specifically 'server', 'pool', 'peer' and
'leapsectz'.
- Read drop-in configurations for *.service unit files from
/etc/systemd/system/ and add them to the report.
- Read all configuration files from /etc/tmpfiles.d/ and add them to the
report.
- Read all configuration files from /etc/sysctl.d/ and add them to the
report.
- Read the Tuned active profile and profile mode and add them to the
report.
- Read all configuration files from /etc/security/limits.d and add them
to the report.
- Read sudoers configuration from /etc/sudoers and files in
/etc/sudoers.d/ and add uncommented lines to the report. No
sophisticated parsing is done, because the configuration format
grammar is too complicated for the purpose of image-info.
- Read udev rules configuration files from /etc/udev/rules.d/ and add
them to the report.
- Read DNF configuration and defined vars and add them to the report.
- Read profile ID and enabled features used by authselect.
- Enable SELinux, extended attributes and POSIX ACLs support when
unpacking 'tar' image type to prevent potential mismatches
Regenerate all image test cases to reflect changes in the image-info
output. Modify the distro-arch-imagetype-map.json to cover all
combinations currently covered by existing image test cases.
Add doc strings to all read_* functions.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
genisoimage might be removed from RHEL 9. The users are advised to switch
to mkisofs tools from the xorriso package. It should be a drop-in replacement.
The same change was recently done by libguestfs:
efb8a766ca2216ab2e32
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
osbuild requires the export flag otherwise it wont produce an artifact.
For the older manifest format (v1), the export value is always
"assembler". For v2 manifests, it is the name of the last pipeline.
If an unknown version number is read the script now fails. This should
help catch manifest changes that may affect test case generation in the
future.
Afetr a run, a store contains downloaded rpm's and whatever pipelines
were checkpointed. We want to reuse these for subsequent runs, so
don't delet the store.
This risks minimally increasing the disk space usage, but should speed things up significantly.
Use en_US.UTF-8 as default for LANG, which is what previously was
used and is also needed to properly work on non-us/latin setups[1].
In the customization tests, use a different value than the default
one to check that the customization does in fact work.
[1] http://git.app.eng.bos.redhat.com/git/spin-kickstarts.git/tree/rhel8/rhel-8.2-kvm-x86_64.ks#n4
Co-authored-by: Achilleas Koutsou <achilleas@koutsou.net>
We have integration tests that boot most of the image types, let
the image tests focus on the actual content of the images, shaving
off considerable CI time.
The only missing one is OpenStack (which should be ported over) and
tar (which we need to define the integration test for).
Signed-off-by: Tom Gundersen <teg@jklm.no>
Two new test cases added to format-request-map and test cases are
generated.
1. kernel-rt for RHEL images:
Requires new package repositories for RHEL 8.3 and 8.4.
Creates an OSTree commit with the `kernel-rt` as a customization.
2. kernel-debug for Fedora images:
kernel-rt isn't included in the official fedora repositories. Using
kernel-debug at least tests the feature with the fedora-iot-commit type.
The image definition is shared with the latest RHEL 8.y one (8.4 currently).
I expect that we the introduction of 8.5 support, we point the centos 8
distro at it.
The test repositories and manifests use the official CentOS composes. From
what I can tell, they are persistent. This is not guaranteed though, so we
might need to switch to RPMRepo at some point.
The "classic" CentOS 8 should also be buildable but due to the chicken and egg
issue (this commit will get into Centos "8.4" but Centos "8.4" isn't a thing
yet), we cannot test it and therefore it might be broken.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Add RHEL-8.4 image test case manifests for all architectures. The set of
image types per architecture is based on what is currently available for
RHEL-8.3.
The image test case for RHEL-8.4 tar image on s390x is not added yet
because of Issue #1220.
Fixes#1167
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The test case generation script contained a typo in the blueprint
customizations dictionary: "hosname" instead of "hostname".
Fixed typo and regenerated relevant test cases.
Remove 'subscription' customization from 'format-request-map.json',
because it currently has no effect on image test cases. The reason is
that this customization of image is possible only via Cloud API in
composer and setting imageOptions. However image test cases currently
support image customization only via Blueprint customizations.
Update repo links in 'repos.json' to the latest rpmrepo snapshots.
Regenerate all image test cases affected by updates to 'repos.json'
and 'format-request-map.json'. This includes all `rhel-8` and `rhel-84`
image test cases.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Blueprint overrides from `format-request-map.json` were previously not applied
when generating image test cases due to a bug. As a result, there were
multiple "blueprint" sections in the generated image test case, if the
image type had an override configured.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `generate-all-test-cases` tool, which generates all defined image
test cases based on provided "distro x arch x image-type" combination
matrix and CLI arguments. Test cases are generated by running the
`generate-test-cases` tool inside an architecture-specific VM.
The tool requires a qcow2 image to be provided for each requested
architecture. These images are used to create ephemeral VMs used
for test case generation.
There appears to be an issue with entropy on all VMs for other
architecture than the host's one. It makes the generation of the first
test case usually fail due to the `go get` command failing with a "TLS
handshake timeout" error. There is a workaround to retry the generation
of a test case a maximum of 3 times in this case. In the usual case,
the test case generation passes on the second try. All subsequent test
case generations should complete successfully on the first try.
Add the `distro-arch-imagetype-map.json` file as the default source of
the "distro x arch x image-type" combination matrix for generating test
cases. Its content is based on the current content of the
`test/data/manifests` directory.
Update `test/README.md` to mention the `generate-all-test-cases` tool.
Add a note, that `generate-test-cases` must be run on a host with the
same architecture, as the intended test case's.
Known issues:
- The tool does not work with RHEL images because of missing "9p"
filesystem support.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
cockpit-composer can now build rhel 8.4 images. Our distro name for
rhel 8.4 is rhel-84 unlike prior rhel releases which fall
under the umbrella name rhel-8. rhel 8.4 still uses the same
repos as the rest of the rhel 8 releases but points to a different
nightly repo for testing purposes. Test cases are added. The changes
between rhel 8.3 and 8.4 are as follows:
There is now a hybrid boot partition scheme for x86_64. x86_64 images
now use uefi boot and have 3 gpt partitions: a small unformated
partition for mbr compatibility, an efi boot partition of type vfat, and
a root partition of type xfs. The packages grub2-efi-x64 and shim-x64
are added as bootloader packages for all x86_64 images.
For qcow2 images ro is added as a kernel option and the following
packages are added (+) or removed (-):
+ dosfstools
+ efi-filesystem
+ efivar
+ efivar-libs
+ grub2-efi-x64
+ shim-x64
- rhn-client-tools
- rhnlib
- rhnsd
- rhn-setup
This commit does several things:
1) Changes the Fedora 33 repos in the test case generator from development
to release ones.
2) Fixes format-request-map.json so we can generate fedora-iot-commit
"images".
3) Regenerates all the cases.
We no longer release into F31, and the right specfile was anyway not
being tested.
This allows us to remove a workaround that updates the VMs during
deploy, and other fedora-31 specific hacks.
