Commit graph

136 commits

Author SHA1 Message Date
Major Hayden
4b0409df5b Update Fedora 31/32 aarch test cases
Add Fedora 32 test cases with the base OS + updates (to work around
aarch64 bugs in the original release). Add updates for Fedora 31 for
consistency.

Fixes #861.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-15 09:46:06 +02:00
Major Hayden
41e255bc8c Make qemu integration tests more reliable
Set a static network configuration using cloud-init so that we don't
have to wait a long time for DHCP to work. Also, since we know the IP
and it is set earlier, this reduces the delay in testing on systems
without nested virt (like AWS).

Fixes #844.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-13 11:19:07 -05:00
Major Hayden
ef0826599d Fix valk > talk typo 2020-07-09 08:53:23 -05:00
Major Hayden
3113ba02d8 Install awscli version 2 for AWS tests
Signed-off-by: Major Hayden <major@redhat.com>
2020-07-09 08:53:23 -05:00
Major Hayden
e88b9c4bd7 🛑 Stop using internal repositories
Remove the F31/F32 internal repositories since they are causing dnf
download errors from time to time.

The 8.2 repository override should have never been there in the first
place since we're using the CDN.

The 8.3 repository is no longer needed since we can register the RHEL
8.3 instances against the staging CDN and get beta content from there.

Fixes #822.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-09 08:53:23 -05:00
Ondřej Budai
843f7701cf test/azure: don't tag the resources
The tag is not used anymore, see the previous commits.
2020-07-09 12:50:06 +02:00
Ondřej Budai
57b1788c9d test/azure: specify resource names as parameters
Prior this commit the resource names were generated in the deployment
template, so the Go code actually didn't know them. This commit generates
all names in the Go code, so they can be used in the future commits.
2020-07-09 12:50:06 +02:00
Major Hayden
0f32f35756 📦 Use raw image format for AWS
Although the vhdx format is smaller than raw, AWS often has some issues
booting instances from an imported vhdx snapshot. The raw format seems
to boot consistently at AWS.

Fixes #754.

Signed-off-by: Major Hayden <major@redhat.com>
2020-07-02 13:11:11 -05:00
Tom Gundersen
fe9f2c55b8 repositories/rhel-8: default to beta repos
The next release we are targeting is RHEL 8.3 Beta, switch the
repositories over.

In the future, we should distinguish between beta and non-beta based on
/etc/os-release so we don't have to hard-code this upstream.

The beta repositories are not actually available yet, so in our tests,
we have to override this again to point at the GA ones (currently 8.2).

Needless to say, this situation leaves much to be desired.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-29 16:04:38 +02:00
Tom Gundersen
b32f0feeae test/cases: add test-cases to verify customizations
Add one test-case per distro that tries to cover all the customizations
we support.

For now omit firewall customizations, as there are open questions about
how to support that on all image types.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-29 16:04:38 +02:00
Ondřej Budai
0b74ebd2ff test: bump 8.3 repo
The previously used snapshot was deleted. This commit changes the override to
use Internal Snapshot 2.0, which should not be deleted in the near future.
2020-06-29 12:29:20 +02:00
Tom Gundersen
18b17c87fa test/cases/rhel/edge: update with dropped fstab
The previous commit removes fstab, update the test-cases accordingly:

`sudo ./tools/test-case-generators/generate-test-cases --distro rhel-8
--arch x86_64 --image-types rhel-edge-commit --output test/cases/
--store /mnt/osbuild`

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-21 16:16:14 +02:00
Major Hayden
cf2392681a Remove 8.2 test case symlinks
These symlinks are no longer needed now that the testing scripts have
been adjusted.

Signed-off-by: Major Hayden <major@redhat.com>
2020-06-17 11:47:50 +03:00
Major Hayden
5107115e8b Set consistent ssh-keyscan timeout
Fedora's timeout for ssh-keyscan is short by default, but RHEL's is much
longer. Set the timeout to 5 seconds for consistency.

Signed-off-by: Major Hayden <major@redhat.com>
2020-06-16 11:48:45 +03:00
Major Hayden
989e15e2fb CI: Fix copy/paste errors in log gathering
Perhaps my ability to copy and paste code should be taken away. 🤦🏻‍♂️

Signed-off-by: Major Hayden <major@redhat.com>
2020-06-15 15:50:40 -05:00
Ondřej Budai
b759f4e0be Fix RHEL 8.3 repo override
BaseOS -> AppStream, see the diff
2020-06-12 09:59:25 +02:00
Major Hayden
4ca535c673 Add RHEL 8.3 repository override
Override the repository for RHEL 8.3 since they can't be registered yet.

Signed-off-by: Major Hayden <major@redhat.com>
2020-06-12 09:59:25 +02:00
Martin Sehnoutka
1e5c31a7ad distro/rhel8: introduce s390x support
This patch introduces support for s390x architecture. It has some
specific requirements like using kernel-cmdline stage before the RPM
stage because RPM scriptlets require these values to be present in the
system already. It also uses zipl stage and bootloader as opposed to all
the other architectures.
2020-06-11 21:50:09 +02:00
Tom Gundersen
fcd5ff76ac tools/image-info/ostree: don't record the commit sha
This is not stable between builds, instead, record the inputhash, which
is.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-11 20:44:17 +02:00
Tom Gundersen
3925b3f6eb test/cases/rhel8: add RHEL for Edge image test case
We currently don't have a boot-test for this, it must be performed
manually.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-11 20:44:17 +02:00
Tom Gundersen
4fe8bb6baf test/cases/rhel8/tar: add image info
image-info now supports tar-balls, so add this to the test-case.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-11 20:44:17 +02:00
Major Hayden
3d488385ff Gather compose log/metadata for AWS/qemu test
Gather logs directly for AWS + qemu tests using `composer-cli compose
log` and also gather the metadata. This will hopefully help diagnost the
problems we are seeing in #754.

Signed-off-by: Major Hayden <major@redhat.com>
2020-06-10 16:26:43 -05:00
Major Hayden
71c4dd2c20 Cleanup integration tests
* Ensure we get the output of `composer-cli compose start` for
  diagnostics
* Hide the annoying `genisoimage` output
* Simplify the `IMAGE_EXTENSION` check
* Make it more clear what is happening while we wait for ssh

Signed-off-by: Major Hayden <major@redhat.com>
2020-06-10 13:45:38 -05:00
Tom Gundersen
130c4b80a2 test/case: update test-cases
This updates the test-cases to reflect the changes in the previous few
commits.

The osbuild rpm stage options format was changed, and the build root on
RHEL8 now includes SELinux policy, and is labelled correctly.

None of these changes affect our output images, so the image info is
unchanged.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-10 19:25:54 +02:00
Major Hayden
7b562d943c Expand native tests to include VHD/VMDK
Add VHD/VMDK support to the QEMU native compose/boot test.

Signed-off-by: Major Hayden <major@redhat.com>
2020-06-10 07:53:33 -05:00
Major Hayden
bd8168adbd
CI hotfix: Remove cpu passthrough
Some hypervisors really dislike cpu passthrough and it causes tests to fail.
2020-06-08 20:06:46 -05:00
Major Hayden
bc2a91a688 AWS: Fix console screenshot retrieval
The JSON from AWS was not being passed to `jq` and the console
screenshots were empty. 🤦

Signed-off-by: Major Hayden <major@redhat.com>
2020-06-08 19:39:47 -05:00
Major Hayden
904335cc94 Add retries to AWS smoke test check
Signed-off-by: Major Hayden <major@redhat.com>
2020-06-08 19:38:54 -05:00
Major Hayden
860bb219af Add native qcow2 boot test
Add an end-to-end qcow2 test that follows a customer's steps with
`composer-cli`. The image is booted with libvirt to allow the best
virtualization options to be chosen by libvirt. It also uses libvirt's
default network.

Signed-off-by: Major Hayden <major@redhat.com>
2020-06-08 19:38:54 -05:00
Major Hayden
5903cc42fe Run AWS test using osbuild worker code
The `osbuild-image-tests` code does a good job testing image builds for
AWS, but it doesn't use the same code paths that a customer would when
using `composer-cli`.

Add a test that builds a compose and uploads the image using
osbuild-composer and its worker. Also, build an instance in AWS based on
the image we imported and check to see if the smoke test file is
present.

Signed-off-by: Major Hayden <major@redhat.com>
2020-06-08 10:31:27 -05:00
Tom Gundersen
83a63aaf17 distro/rhel8: add tar image type for all architectures
This is a very minimal image type, which is not expected to do anything
with an empty blueprint. It is documented as a way to install RHEL for
Satellite.

This fixes #720.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-08 13:32:59 +02:00
Tom Gundersen
253379a3b9 tests: use more reliable mirrors for Fedora tests
Move away from the round-robin fedora mirrors, and use the most reliable
one we have found (kernel.org).

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-06-07 13:33:32 +02:00
Martin Sehnoutka
385df479f3 distro/rhe8: introduce ppc64le qcow2 image type
This patch introduces a qcow2 image type for RHEL8 distro and associated
test case.
2020-06-06 16:56:39 +02:00
Martin Sehnoutka
65bc635c30 osbuild: change legacy type from bool to string
This work is based on previous PRs, namely:
https://github.com/osbuild/osbuild-composer/pull/501
and
https://github.com/osbuild/osbuild/pull/327

The problem here is that we used to treat legacy as a boolean before we
started introducing support for alternative architectures, but now we
need to specify exact strings for the grub2 stage, for example for
ppc64le the legacy parameter looks like this:

```
"legacy": "powerpc-ieee1275"
```

This patch will allow us to introduce support for ppc64le and fix
associated issues:
https://github.com/osbuild/osbuild-composer/issues/693
2020-06-03 12:29:47 +02:00
Jacob Kozol
2faf8b3861 test/cases: update distro tests
The distro tests will continue to use the internal repos. But, the
manifest should use the new urls format and have the url inside a
map.
2020-05-28 00:23:54 +02:00
Tom Gundersen
e66368b099 distro: drop fedora-30
We never shipped into fedora-30 and it is now EOL.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-27 16:20:35 +02:00
Tom Gundersen
e2c14b9fe6 distro: drop legacy image types
We do not properly test, and do not have properly defined use-cases for
the ext4-filesystem, partitioned-disk, nor tar image types. Drop them to
focus on delivering the things we car properly test.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-27 16:20:35 +02:00
Tom Gundersen
cc231e0ed1 distro/rhel-8: merge all the RHEL-8 minor releases into one
As it turns out, the default expectation is not to distinguish between
these. We will now produce whatever is the most recent minor release by
default, and image tests will still be pinned at a given snapshot to be
reproducible.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-05-21 17:57:28 +02:00
Ondřej Budai
8b58d2c91a test: document the Azure setup process
The image tests are now able to upload images to Azure and boot them there.
However, the documentation on how to set up the required resources in Azure
was missing. This commit adds it.
2020-05-19 10:00:48 -05:00
Ondřej Budai
7f7769cae0 test/cases: fix expected values for the image info sub-test
osbuild removed GRUB2_ROOT_FS_UUID from grubenv in 22d131a5. This broke
the Jenkins CI because it runs against osbuild master.

This commit fixes all the testcases and bumps the osbuild submodule
to version 13 that changed the GRUB2_ROOT_FS_UUID behaviour.
2020-04-30 15:55:34 +02:00
Jacob Kozol
0165a97761 test/cases: remove tar size
When building a tar image the image size should not be specified. All
image size fields are removed from the assembler options in the tar test
cases.
2020-04-29 13:44:44 +02:00
Ondřej Budai
ef92dd7d7c test: use azure for all vhd images
rhel 8.2 and fedora 32 images are missing the new boot type because of a bad
merge timing.
2020-04-28 13:29:43 -05:00
Tom Gundersen
046d08b8ef test/cases: add rhel8.2 tests for x86_65
Generated with

$ sudo ./tools/test-case-generators/generate-test-cases --distro rhel-82 --arch x86_64 --store .osbuild/ --output test/cases/

(Partially) fixes #542.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-04-28 11:10:12 +02:00
Tom Gundersen
f3ad87ea0b test/cases: add f32 tests for x86_64
Generated with

$ sudo ./tools/test-case-generators/generate-test-cases --distro fedora-32 --arch x86_64 --store .osbuild/ --output test/cases/

Fixes #544.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-04-28 11:10:12 +02:00
Tom Gundersen
b7268b2948 test/README: mention the tests cases and how to generate them
Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-04-28 11:10:12 +02:00
Ondřej Budai
e235fdedb3 tests/image: boot the vhd images on Azure
Previously, vhd images were tested using QEMU. This commit changes that to
boot them in the actual Azure infrastructure.

Azure VMs have quite a lot of dependencies - a network interface, a virtual
network, a network security group, a public ip address and a disk. Azure CLI
and Azure Portal handle the creation of all these resources internally.
However, when using the API, the caller is responsible to create all these
resources before creating an actual VM.

To handle the creation of all the resources in the right order, a deployment
is used. A deployment is a set of resources defined in a JSON document.
It can optionally take parameters to customize each deployment. After the
deployment is finished, the VM is up and ready to be tested using SSH.

Sadly, the deployments are a bit hard to clean-up. One would expect that
deleting a deployment removes all the deployed resources. However, it doesn't
work this way and therefore it's needed to clean up all resources "manually".
For this reason, our deployment sets a unique tag on all the resources created
by the deployment. After this test is finished, the API is queried for all
the resources with the tag and then, they're deleted in the right order.
2020-04-27 20:34:20 +02:00
Major Hayden
8dd6fd24b5 Use download.fp.o for downloading packages
Requests to download.fedoraproject.org are redirected to a regional
mirror that is fairly close to the system that is making the request.
This could speed up downloads slightly since it avoids using the
Fedora proxies.

Signed-off-by: Major Hayden <major@redhat.com>
2020-04-27 11:57:17 +02:00
Ondřej Budai
83a3a8d821 change the format of the ami image type to vhdx
Prior this commit the ami image type produced raw.xz images. This was bad for
two reasons:

- The upload was broken because AWS doesn't support tar.xz format
- XZ compression is terribly slow

This commit changes the format to vhdx, which is supported by AWS and also
quite quick. See https://github.com/osbuild/osbuild-composer/issues/257
why vhdx was chosen.

Fixes #257
2020-04-24 15:27:39 +02:00
Martin Sehnoutka
e34607b955 travis: plug in arm image test cases
The test generation script we use outputs the test cases prefixed with
"fedora_$RELEASE" so I renamed all the current test cases to follow this
convention and also changed the travis job names so that it can run both
x86_64 test cases and aarch64 test cases.
2020-04-15 15:22:57 +02:00
Martin Sehnoutka
58b550061b test/cases: include fedora test cases for aarch64
These tests were generated by the script in tools/ directory and are not
modified in any way.
2020-04-15 15:22:57 +02:00