Commit graph

214 commits

Author SHA1 Message Date
Tom Gundersen
3cff028c84 test/cases: drop rhel-8.2 test cases
These cases are pointing to internal repos that have since changed. Drop them
until we have a better long-term story.

Our CI currently does not verify these cases, so this is not a behavioural
change.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-13 15:00:24 +01:00
Alexander Todorov
972b7c1181 Add documentation around assert/require. Refs #309 2020-03-13 14:01:18 +01:00
Ondřej Budai
714f64bf22 tests: fail the tests if no manifest is given
All the tests have a manifest, therefore there's no need for skipping a test.
2020-03-10 14:40:52 +01:00
Ondřej Budai
e5d501c9a3 tests: fix test/run script
Commit fc47ae8 changed the test case format but did not change the test
runner. This commit fixes that.
2020-03-10 14:40:52 +01:00
Tom Gundersen
fc47ae8c39 test/cases: test manifests rather than pipelines
A manifest is struct made up of a pipeline and a sources object. So
far all our sources objects are empty, but we have moved from
using pipelines to manifests everywhere, in preparation for
generating pipelines that require sources.

Make the same change in the test cases.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-08 21:05:01 +01:00
Tom Gundersen
caca553a8f test/cases: rename compose -> compose-request
This is to avoid any confusion with the Compose struct in the store,
which contains the pinned rpmmd data and the pipeline, among other
things.

The struct in the test cases represent the user input to the compose
route, so rename it 'compose-request', to make that clearer.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-08 21:05:01 +01:00
Tom Gundersen
fbf137a1b7 test/cases: move checksum out of the compose struct
The intention is that the compose struct fully specifies the test
case, and pipeline and image-info specifies the expected outputs.
Lastly, the boot struct specifies how to boot-test the image.

The checksum does not fit into this scheme, as it is computed from
the compose by querying rpmmd, and it is then passed as an input to
distro.Pipeline in order to compute the pipeline.

Introduce a new struct, rpmmd, which will eventually contain all
the data returned from rpmmd.Depsolve and later passed to
distro.Pipeline. For now it only contains the checksum.

This is not a functional change.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-08 21:05:01 +01:00
Alexander Todorov
44889842bd Add instructions how to install build-deps in test/README
serves as a reference and will also help less experienced folks
2020-03-04 19:56:25 +01:00
Ondřej Budai
b2f360da46 update osbuild submodule
Due to the change in commit b3ae34b07 in osbuild repository, the tests
had to be slightly modified.
2020-02-24 20:17:50 +01:00
Alexander Todorov
defc34b7f0 Add quick README about testing 2020-02-19 23:41:58 +01:00
Alexander Todorov
895add8539 Replace local download URL with global mirror URL
which will be resolved to the closes file server
2020-02-14 15:37:24 +01:00
Jacob Kozol
a86e697171 distro: add custom image size
When creating a pipeline the assembler includes an image size. This
image size can be set when creating the pipeline but if it is 0 then a
default image size will be used. The default is 2 GB except for ami
images which are 6 GB.
2020-01-23 00:57:31 +01:00
Tom Gundersen
4919ef1271 repositories: install the base repositories in the filesystem
During development of a new distro, we need to test composer against
nightly or beta repositories, but we cannot ship composer itself
with the nightly repository information hardcoded in. At the same
time, we want to distinguish between the system repositories of the
host and the repositories we use to generate images (the host may not
use the same distro/version/architecture as the target, and it may
include custom repositories that the target should not).

We therefore ship per distro repository information that can be
overriden (typically in testing) by dropping files in /etc.

For now use the latest nightlies for RHEL-8.2, we may want to
replace these with the official mirrors for GA eventually.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-01-03 22:35:52 +01:00
Martin Sehnoutka
029cd9260f Tweak the Fedora vhd output format to boot in Azure 2020-01-02 12:57:27 +01:00
Tom Gundersen
51f1691ab6 test/cases/rhel82: add image info to the test cases
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-17 14:19:51 +01:00
Tom Gundersen
93b13e3fa7 blueprints: fix group handling
When group names are passed on to dnf, they must be prefixed with an
ampersand, or they are treated as a regular package, potentially
causing the build to fail.

Add a testcase to verify this behavior.

This resolves rhbz#1784035.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-17 09:24:15 +01:00
Lars Karlitski
73641ea904 distro: rename openstack and vmdk images to match lorax 2019-12-16 01:32:41 +01:00
Tom Gundersen
ba32f58ac8 distro/rhel82/qcow2: bring package set in line with official guest image
Also hook up the tast case to verify that the image boots.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-15 21:41:33 +01:00
Tom Gundersen
b18c66f9fb distro/rhel82: add UEFI support
This follows the style used in fedor-30.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
db300cb481 distro/fedora-30: add UEFI support
On architectures that require EFI, we must create the ESP partition
and use a GPT partition table. We must also install either the UEFI
or the legacy version of GRUB2 in the image.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
8c1f194fc1 distro/fedora-30: set the module_platform_id
Make sure we don't fallback to looking at /etc/os-release.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
18971bd8ef pipeline/stage/grub2: support explicit legacy/uefi API
This allows legacy bootloader and UEFI bootloaders to be
enabled/disabled independently of each other.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
7e4c4e4f6d pipeline/stage/grub2: make the boot uuid optional
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
4c892d6d32 pipeline/assembler/qemu: rework partition table handling
Move to the new options format, allowing more flexible partition
tables. The pipeline changes, but the result should be the same.

This requires a yet-to-be-released version of osbuild.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
506b012f08 distro: take architecture as an argument
Allow bootloader specific packages to be defined per architecture,
and allow repositories to depend on the architecture.

This does not altert he pipelines we produce, part from the ami
image now contains the grub2-pc package, rather than the grub2
package. This should make no difference.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
b034117e62 distro: sort the packages passed to the dnf stage
This should not have any effect apart from hiding any internal changes
to the way we compile the lists.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
caf72fd97f test/cases/rhel82/qcow2: include a user and the ssh daemon
We don't currently boot-test this, but at least it should work
in principle with these changes.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
9f05e0b7c4 test/run: print more helpful output
Make sure we catch expected errors and fail gracefully. Also, make
sure the output id is printed on successufl osbuild run so it can
be introspected from outside the test suite.

Handle the case where osbuild suceeds but does not return an
output_id.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
028adf375a test/cases/rhel82: add basic test-cases
These are used to verify that our pipeline generation is stable, and
that the piplines can generate images that boot.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Tom Gundersen
8255d6ff9c test/run: fix ssh test
This was broken in a rework. The correct status was printed, but
the test failures were not passed on.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Tom Gundersen
9b7ad82773 test/run/ssh: keep trying if the '--wait' switch does not work
When we get an ssh connection, before the image is fully booted,
systemctl is-system-started returnse "starting", treat this as a
failed connection, and keep retrying.

Some distros may not support the wait switch correctly.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Lars Karlitski
dcc9cdedee distro: rename output of qcow2 image to disk.qcow2
Integration tests currently expect that and they need to be able to run
against lorax-composer (which outputs disk.qcow2) and osbuild-composer.
2019-12-10 20:21:47 +00:00
Lars Karlitski
d3a0b788a2 distro: set the repository checksum dynamically
Instead of having a static repository checksum, set it dynamically from
the metadata that osbuild-composer last saw. This is implemented in
dnf-json, which returns the checksums for each repository on every call.

This enables the use of repositories that change over time, such as
fedora-updates. Note that the osbuild pipeline will break when such a
repository changes. This is intentional: pipelines have to be
reproducible.
2019-12-10 20:38:22 +01:00
Lars Karlitski
75218ad2d9 distro/fedora30: don't include variables in repo URL
Replace those variables with their contents everywhere the repository
URLs are used is error-prone.
2019-12-10 20:38:22 +01:00
Tom Gundersen
0d091decb5 test: add provisional multi-arch support
Currently we still only build for x86_64, but now the test suite is
prepared for hooking up other architectures.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-10 02:47:35 +01:00
Tom Gundersen
9bedc0c393 test/run: allow filter on distro
This way we can test the distros on their respective CI, as not
all distros can be built in all environment. In particular RHEL
needs to be on a subscribed host.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-10 02:47:35 +01:00
Tom Gundersen
34a4f24141 test/run: print a bit more useful info on test-runs
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-10 02:47:35 +01:00
Tom Gundersen
8c68018d62 test/run: make relative OSBUILD_TEST_STORE consistent
Make it relative to the directory invoking ./run, rather than passing it along.
2019-12-10 02:47:35 +01:00
Tom Gundersen
3adcac0123 test/run: merge boot and non-boot test handling
We can now select specific cases, but whether or not to check image-info
or boot the image is determined purely by the contents of the json test
case.

We still run the tests as two travis workers just to avoid the timeout,
this should clearly be reworked.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-10 02:47:35 +01:00
Tom Gundersen
c090ab3812 test/run: drop support for non-pipeline test cases
We were supporting downloading an image and checking its image info. We don't
want to rely on external resources, and we should not test images made by others.

Drop it.
2019-12-10 02:47:35 +01:00
Tom Gundersen
cc04f92997 test/cases: rename json sections
Rename to fit more closely with their use before we add too many more
of these.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-10 02:47:35 +01:00
Tom Gundersen
6c44d231ee test/run/nspawn: name the instances differently
We were failing on name reuse. We should be able to not name these
at all, but nspawn is not happy with that.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-10 02:47:35 +01:00
Tom Gundersen
82c156ef1b test/cases: boot test the empty blueprint ami image
There should be no need to create a custom blueprint, even the empty one
should boot.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-10 02:47:35 +01:00
Tom Gundersen
a7203db963 test/run/qemu: support compressed images
Uncompress, strip the extension and pass to qemu as usual.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-10 02:47:35 +01:00
Tom Gundersen
404ad2023d test/run: wait on is-system-running
Once we are connected, wait for the command to return a definitive
result, rather than retry.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-10 02:47:35 +01:00
Tom Gundersen
ad7c5bc735 test/run: improve output on ssh retry
We expect to have to retry the ssh connection, so let's print something
less scary when that happens.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-10 02:47:35 +01:00
Tom Gundersen
4ec7ac1ecd test/cases: start to unify boot tests with other image tests
We now have three top-level maps, that can be combined in any way:

boot-test: information about how to boot the image
compose: information about how to generate the pipeline
pipeline: the pipeline to generate the image
expected: the expected image-info

This creates compose entries for all the boot tests, but the blueprints
are named 'blueprint-draft', as we are not yet verifynig that the pipeline
is correct.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-10 02:47:35 +01:00
Tom Gundersen
49387604e4 test/run: pass cloud-init data to qemu
This passes the redhat user with ssh key as an ISO image to our
qemu instances, making sure images relying on cloud-init rather than
hardcoded user credentials can be used in our tests.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-10 02:47:35 +01:00
Tom Gundersen
59ab975fea test: use the standard port for sshd
For the qemu tests this makes no difference as we are anyway forwarding
the ports. But the nspawn tests share the same network namespace between
the image and the ssh client running the test without any forwarding. In
order for that to work we had to modify the image to use a non-standard
port.

We don't want this for two reasons: we want to make sure we test our images
unmodified, and also this meant that when we changed our pipeline generation
we were not verifying that the boot test cases were updated accordingly. As
a result they have now drifted.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-10 02:47:35 +01:00
Tom Gundersen
79b868d423 test/run: run each test in its own network namespace
This makes sure we can use any port to connect to sshd, without
worrynig about clashes with parallel tests, or an sshd instance
running on the host.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-10 02:47:35 +01:00