The org.osbuild.sysconfig stage is now supported. Config updates can be
made to the kernel and network files. Currently, the same values are
used for all image types in rhel84. The image-info script is updated to
allow testing the sysconfig info.
The test case generation script contained a typo in the blueprint
customizations dictionary: "hosname" instead of "hostname".
Fixed typo and regenerated relevant test cases.
Expose a more detailed job status result - specifically, include upload status
alongside image status. Expand openapi.yml accordingly and add an UploadStatus
field to the OSBuildJobResult struct. At the moment, only represent the
"success" and "failure" states of UploadStatus - to differentiate between
"pending" and "running" would involve significant design decisions and should be
addressed in a separate commit.
The algorithm from calculating changed in osbuild 24, thus breaking the
testsuite. As the the inputhash is merely a implementation detail, there's
no need to test it in the image tests. Nevertheless, the inputhash is also
tested in the osbuild's testsuite.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Tom was too quick to merge PR#1178 and as a result, one image test case
was not properly regenerated in that PR. The CI passed for PR#1178,
because RHEL-8.3 images in CI are not tested on aarch64.
Add properly regenerated image test case for rhel-8 aarch64 edge commit.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Remove 'subscription' customization from 'format-request-map.json',
because it currently has no effect on image test cases. The reason is
that this customization of image is possible only via Cloud API in
composer and setting imageOptions. However image test cases currently
support image customization only via Blueprint customizations.
Update repo links in 'repos.json' to the latest rpmrepo snapshots.
Regenerate all image test cases affected by updates to 'repos.json'
and 'format-request-map.json'. This includes all `rhel-8` and `rhel-84`
image test cases.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
`image-info` tools parses output of `systemctl list-unit-files` run on a
different tree (with `--root` option), to determine the list of enabled
and disabled services on the inspected image. However since Fedora 33
(and presumably since systemd v246), the output of `systemctl
list-unit-files` changed. Some units previously reported as "enabled" or
"disabled" are now reported as "alias", which means, that they are just
a symlink to a different unit.
There is no systemd command, that would take an "alias" unit and would
report its state as "enabled" or "disabled" and could run on a different
tree (with "--root" option).
To make the list of reported services in the given state consistent on
systems with older and new (v246+) systemd version, check all "alias"
units and append them to the list of services with a specific status,
if their target is also listed in in the list.
Example of the `systemctl list-unit-files` output change:
~]# rpm -q systemd
systemd-246.6-3.fc33.x86_64
~]# systemctl list-unit-files ctrl-alt-del.target
UNIT FILE STATE VENDOR PRESET
ctrl-alt-del.target alias -
~]# rpm -q systemd
systemd-245.8-2.fc32.x86_64
~]# systemctl list-unit-files ctrl-alt-del.target
UNIT FILE STATE VENDOR PRESET
ctrl-alt-del.target enabled disabled
This change makes it possible to produce consistent output for an
inspected image, regardless if the `image-info` tool is run on Fedora
32, Fedora 33 or RHEL-8.
Also regenerate all Fedora 33 test cases, since this commit changes the
content of produced list of enabled / disabled services since Fedora 33.
The list is now consistent with what would be produced by `image-info`
for an image on older Fedora (e.g. 32) or RHEL-8.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `generate-all-test-cases` tool, which generates all defined image
test cases based on provided "distro x arch x image-type" combination
matrix and CLI arguments. Test cases are generated by running the
`generate-test-cases` tool inside an architecture-specific VM.
The tool requires a qcow2 image to be provided for each requested
architecture. These images are used to create ephemeral VMs used
for test case generation.
There appears to be an issue with entropy on all VMs for other
architecture than the host's one. It makes the generation of the first
test case usually fail due to the `go get` command failing with a "TLS
handshake timeout" error. There is a workaround to retry the generation
of a test case a maximum of 3 times in this case. In the usual case,
the test case generation passes on the second try. All subsequent test
case generations should complete successfully on the first try.
Add the `distro-arch-imagetype-map.json` file as the default source of
the "distro x arch x image-type" combination matrix for generating test
cases. Its content is based on the current content of the
`test/data/manifests` directory.
Update `test/README.md` to mention the `generate-all-test-cases` tool.
Add a note, that `generate-test-cases` must be run on a host with the
same architecture, as the intended test case's.
Known issues:
- The tool does not work with RHEL images because of missing "9p"
filesystem support.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The last imagefactory nightly did not contain dnf-plugin-spacewalk,
fwupd, nss, or udisks2. These packages are now excluded. The
udisks2.service and mdmonitor.service are no longer enabled. Also, the
fwupd-refresh, mdcheck_continue, mdcheck_start, and mdmonitor-oneshot
timers are no longer listed as disabled services.
The packages in the last imagefactory nightly differ from ours. The
following packages are now added:
oddjob
oddjob-mkhomedir
psmisc
authselect-compat
rng-tools
dbxtool
Also, the rngd and nfs-convert services are enabled.
Adding the tag called `Name` to the AMI ensures that the name appears in
the *Name* column inside AWS' web console.
Fixes#1171.
Signed-off-by: Major Hayden <major@redhat.com>
An image only had a systemd stage added if its blueprint contained
services or if its image type contained enabled services. The systemd
stage is now also added if the image type contains disabled services or
a default target.
The RHEL 8.4 qcow2 image type now specifies the multi-user default target.
In order to test this the image-info tool now includes the default
target in its output. Image test manifests are updated to include this
change.
RHEl 8.4 guest images need to have the default timezone of EST/EDT
unless the user specifies one in their blueprint. New York is a major
location for this timezone.
The directory with image-tests test cases has been renamed from `cases`
to `manifests`. This has not been previously reflected in the test/README.md
and osbuild-image-tests code. osbuild-image-tests hardcodes the test
cases directory path and uses it in case no test case are passed
to it on the command line. Since the image_tests.sh CI test case looks
for image-tests test cases in the correct directory and passes the
relevant ones to osbuild-image-tests, the CI didn't detect this issue.
Running osbuild-images-tests without any argument and let it run all
test cases from the default test cases directory as part of CI probably
does not make sense. Due to this reason, I'm not adding any new test.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Imagine this situation: You have a RHEL system booted from an image produced
by osbuild-composer. On this system, you want to use osbuild-composer to
create another image of RHEL.
However, there's currently something funny with partitions:
All RHEL images built by osbuild-composer contain a root xfs partition. The
interesting bit is that they all share the same xfs partition UUID. This might
sound like a good thing for reproducibility but it has a quirk.
The issue appears when osbuild runs the qemu assembler: it needs to mount all
partitions of the future image to copy the OS tree into it.
Imagine that osbuild-composer is running on a system booted from an imaged
produced by osbuild-composer. This means that its root xfs partition has this
uuid:
efe8afea-c0a8-45dc-8e6e-499279f6fa5d
When osbuild-composer builds an image on this system, it runs osbuild that
runs the qemu assembler at some point. As I said previously, it will mount
all partitions of the future image. That means that it will also try to
mount the root xfs partition with this uuid:
efe8afea-c0a8-45dc-8e6e-499279f6fa5d
Do you remember this one? Yeah, it's the same one as before. However, the xfs
kernel driver doesn't like that. It contains a global table[1] of all xfs
partitions that forbids to mount 2 xfs partitions with the same uuid.
I mean... uuids are meant to be unique, right?
This commit changes the way we build RHEL 8.4 images: Each one now has a
unique uuid. It's now literally a unique universally unique identifier. haha
[1]: a349e4c659/fs/xfs/xfs_mount.c (L51)
The kernel options are updated to remove the read only option "ro" from
the qcow2 and amazon image types. Also, the qcow2's kernel options are
updated to only set console=ttyS0 once. It was declared twice which is
redundant so now it is set for both tty0 and ttyS0.
rng-tools is no longer included as a package in the RHEL 8.4 image. This
package is both removed from being an included package and also
specifically declared as excluded. The test manifests are updated.
The RHEL 8.4 specific runner was introduced in osbuild 22, released
on the 8th of October 2020. It should by now be in relevant Fedora
releases and RHEL 8.4.
Adapt the corresponding test cases.
All tests in /usr/libexec/tests/osbuild-composer should be able to run without
any arguments. This was not a case of libvirt.sh - it required two arguments
set by some Jenkinsfile logic.
This commit moves test/cases/libvirt.sh to tools/libvirt_test.sh and extracts
the logic controlling the test case from Jenkinsfile to test/cases/libvirt.sh.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Our current testing environment cannot run VMs for aarch64, therefore we
cannot test the qcow2 image. Also, we don't have an openstack account with
aarch64 so also we cannot test openstack image type. Additionaly, we cannot
selectively disable boot testing for now.
Let's do the smallest change possible and just add ami test case - we can boot
test these images in AWS.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
x86_64 and aarch64 rhel 8.4 images support hybrid boot. qcow2 images are
now tested with both BIOS and UEFI boot. The Jenkinsfile now specifies
which boot type is supported when calling run_tests. For the qemu tests,
this boot type can be passed as an option so the test can boot using
UEFI.
In order to boot using uefi from a rhel machine, we must pass the --boot
uefi flag as well as tell libvirt where the variable store template is.
Since we are not testing secureboot, we must pass
nvram_template=/usr/share/edk2/ovmf/OVMF_VARS.fd to the --boot flag.
This is the location of our non-secureboot varstore template.
rhel 8.4 tests are added. The configs are based off of those used for
rhel 8.3. The Schutzbot Mockbuild, Base, Image, Integration, and OSTree
tests are added for 8.4. Repo overrides are added for the rhel 8.4 tests
so that the tests use rpmrepo snapshots.
The mockbuild uses the jenkins rhel84-nightly-repo credential to
override the rhel mock template's repos with rhel 8.4 nightly repos.
These repos are stored in a credential because they are internal links.
The image tests and koji tests need a special distro selector since the
rhel-8 test cases are only for rhel 8 versions less than 8.4. The rhel
8.4 tests are named with the rhel-84 pattern whereas the other rhel 8
versions have the rhel-8 pattern.
Also, instead of having only rhel-8 and rhel-8-beta repo configs for the
tests, we now have a specific repo config for each rhel release we test.
The repo is also now pulled from an rpmrepo snapshot. For whichever
distro is being tested, the approriate repo config will be copied to
/etc/osbuild-composer/repositories as rhel-8 and rhel-8-beta since this
is the naming osbuild-composer looks for. For testing purposes, the
rhel-8 and rhel-8-beta repo should be the same since eventually all rhel
releases will go from beta to not beta. The fedora repo overrides are
already done in tools/provision.sh so the rhel override is set there as
well. Currently, only rhel 8.4 requires an override.
cockpit-composer can now build rhel 8.4 images. Our distro name for
rhel 8.4 is rhel-84 unlike prior rhel releases which fall
under the umbrella name rhel-8. rhel 8.4 still uses the same
repos as the rest of the rhel 8 releases but points to a different
nightly repo for testing purposes. Test cases are added. The changes
between rhel 8.3 and 8.4 are as follows:
There is now a hybrid boot partition scheme for x86_64. x86_64 images
now use uefi boot and have 3 gpt partitions: a small unformated
partition for mbr compatibility, an efi boot partition of type vfat, and
a root partition of type xfs. The packages grub2-efi-x64 and shim-x64
are added as bootloader packages for all x86_64 images.
For qcow2 images ro is added as a kernel option and the following
packages are added (+) or removed (-):
+ dosfstools
+ efi-filesystem
+ efivar
+ efivar-libs
+ grub2-efi-x64
+ shim-x64
- rhn-client-tools
- rhnlib
- rhnsd
- rhn-setup
Now that all interaciton with the koji API happens in the workers
we can drop koji configuration from composer itself. This means
that composer no longer needs to be provisioned with kerberos
credentials, and does not need to know about which koji servers
the workers support.
This commit does several things:
1) Changes the Fedora 33 repos in the test case generator from development
to release ones.
2) Fixes format-request-map.json so we can generate fedora-iot-commit
"images".
3) Regenerates all the cases.
If the ostree test was run on an unsupported distro, it failed but with a
very weird error message. This commit makes the test fail fast and with a
nice message.
The downloaded image may not fit inside tmpfs, especially when testing
on a constrained VM. This commit makes the test script use a different
temporary directory while handling the possibly big image.
The integration tests are leaving the composes (which include images) in
osbuild-composer. This can lead to exhausting the disk space we have available
on our tiny testing machines. This commit adds a removal of the composes
after each integration test is finished. This issue is not present in koji.sh
and api.sh as they use different osbuild-composer APIs that doesn't use the
artifact feature.
This issue occurred when I worked on enabling the Fedora 33 tests, see:
https://osbuildci.cloud.paas.psi.redhat.com/blue/organizations/jenkins/osbuild%2Fosbuild-composer/detail/PR-1014/23/pipeline
We claim to have self-contained test cases, but the base_tests.sh script
still requires the WORKSPACE environment variable to be set outside of
the script, which is what Jenkins does.
This patch replaces WORKSPACE with a temporary directory and modifies
Jenkinsfile to use it when collecting logs.
We have several repository definitions across the tests which is quite messy.
This commit switches the Koji test to use the "central" repository configs defined in test/data/repositories/
Don't use common.State anymore, because it has different values from
what's defined in openapi.yml. It makes sense to have these strings
defined in the same package as the spec — ideally, the code generator
would make them for us.
While at it, add a "running" status.
Fix the api.sh test to use these new statuses. Thanks to Ondřej Budai
for an additional fix there.
We don't install any packages in test cases anymore, therefore we don't need
to install EPEL there.
A slightly different explanation:
osbuild-composer-tests depends on packages from EPEL on RHEL. Therefore, you
cannot run the test cases without EPEL installed. Therefore, there's no
point in installing EPEL there.
RHEL 8.3 is GA so we can run the ostree test here. This is a first step
to remove the RHEL 8.3 testing pipeline.
8.3 GA also ships a different version of composer-cli, therefore a slight
adjustment was needed.
The certificate generation is based on work by Lars Karlitski in our osbuild
CA. The server and client certs now contains Subject Alternative Name making
Python's request module and Go 1.15 happy (they deprecated certificates
without SAN).
Several reasons why we want to switch to the certificate generation:
1) The pre-generated certificates are not documented. If someone wants
to inspect them, he must know the right openssl incantation. This way,
you are able to see what's inside the certificates in a plain text.
2) The pre-generated certificates are going to expire at one point and
someone will be surprised.
3) Shipping private keys in RPMs is iffy. I know, it's just for testing but
still...
4) Auth tests are generating their own certificates. To achieve consistency,
we have two options:
a) Ship also all certificates for auth tests. That's extra 8 ones or
something like that.
b) Generate all certificates on fly. This commit does that.
5) The setup introduced by this commit is very similar to the one in our CA
making the test environment very similar to what's running in production.
tl;dr: I think this is a good step forward.
The Koji test in Github actions was always a bit quick and dirty solution.
I think it's much nicer solution to run it on Schutzbot.
Therefore, this commit moves the koji_test.go to a new osbuild-koji-tests
executable. This new test isn't run in the base test suite as one would
anticipate but inside the koji.sh test. This is needed because
osbuild-koji-tests requires a running koji instance. This might change
in the future but I think it works for now.
The test package should be self-contained and contain all the configuration
necessary for a known-good test run (minus secrets). This moves repo
overrides from the test orchestration into the test package.
We want all the external sources (including boot isos and repositories) to
be pinned by their content hash, and never use anything that is not strictly
defined. This moves us in the right direction, but we still have some tests
to update to use these shipped repos rather than official mirrors.
One remaining challenge is that we must make sure all our test runners have
access to the RHEL snapshots, which is not currently the case for the runners
in EC2, but a solution is in the works.
Signed-off-by: Tom Gundersen <teg@jklm.no>
