Test the full set of locally-bootable image tests. Cloud based tests
are excluded for now until the infrastructure to boot them is ready.
Signed-off-by: Major Hayden <major@redhat.com>
The .* before the digit group is greedy and it catches the first version
digit when the version >= 10. This commit changes the .* to [^[:digit:]]*
so it doesn't catch the first digit.
Hiding logs makes the output prettier, but it is sometimes easier to see
the problem right there in the Ansible output.
Signed-off-by: Major Hayden <major@redhat.com>
Image size should not be specified for a tar image. When the tar
assembler was being created image size was not passed to it. However,
the TarAssemblerOptions contained a Size field which would default to 0.
This field is removed so that the tar assembler options no longer
specify a size. The assembler tests are also updated to reflect this
change.
Currently, removing osbuild-composer produces these warnings:
Failed to stop osbuild-worker@.service: Unit name
osbuild-worker@.service is missing the instance name.
See system logs and 'systemctl status osbuild-worker@.service' for details.
Failed to stop osbuild-remote-worker@.service: Unit name
osbuild-remote-worker@.service is missing the instance name.
See system logs and 'systemctl status osbuild-remote-worker@.service' for
details.
%systemd_preun calls on a package upgrade this:
systemctl --no-reload disable --now [ARGS...]
I tried using a wildcard:
systemctl --no-reload disable --now "osbuild-worker@*.service"
But this gives:
Invalid unit name "osbuild-worker@*.service" was escaped as
"osbuild-worker@\x2a.service" (maybe you should use systemd-escape?)
Failed to disable unit: Unit file osbuild-worker@\x2a.service does not exist.
In the end I decided to use two commands - disable and stop. Disable works
without a wildcard, stop requires one. This solution should mimic systemctl
disable --now pretty well.
Currently, upgrading osbuild-composer produces these warnings:
Failed to try-restart osbuild-worker@.service: Unit name
osbuild-worker@.service is missing the instance name.
See system logs and 'systemctl status osbuild-worker@.service' for details.
Failed to try-restart osbuild-remote-worker@.service: Unit name
osbuild-remote-worker@.service is missing the instance name.
See system logs and 'systemctl status osbuild-remote-worker@.service'
for details.
%systemd_postun_with_restart calls on a package upgrade this:
systemctl try-restart [ARGS...]
However, try-restart requires using a wildcard when restarting all
the services created from a template unit. This commit therefore adds the
wildcards.
Enable image tests for Fedora 31 and RHEL 8.2 since we now have the JSON
test case files for all distributions.
Signed-off-by: Major Hayden <major@redhat.com>
Previously, vhd images were tested using QEMU. This commit changes that to
boot them in the actual Azure infrastructure.
Azure VMs have quite a lot of dependencies - a network interface, a virtual
network, a network security group, a public ip address and a disk. Azure CLI
and Azure Portal handle the creation of all these resources internally.
However, when using the API, the caller is responsible to create all these
resources before creating an actual VM.
To handle the creation of all the resources in the right order, a deployment
is used. A deployment is a set of resources defined in a JSON document.
It can optionally take parameters to customize each deployment. After the
deployment is finished, the VM is up and ready to be tested using SSH.
Sadly, the deployments are a bit hard to clean-up. One would expect that
deleting a deployment removes all the deployed resources. However, it doesn't
work this way and therefore it's needed to clean up all resources "manually".
For this reason, our deployment sets a unique tag on all the resources created
by the deployment. After this test is finished, the API is queried for all
the resources with the tag and then, they're deleted in the right order.
The cmd/osbuild-image-tests package is becoming bigger than I would like to.
It will be nice to split it to some smaller pieces at some point.
This commit does the first step - splits off the first subpackage containing
all the constants.
The commit 2435163f broke sending the logs to osbuild-composer. This was
partly because of unusual error handling in the RunOSBuild function.
This commit fixes that by creating a custom error and properly propagating
the result from it.
Requests to download.fedoraproject.org are redirected to a regional
mirror that is fairly close to the system that is making the request.
This could speed up downloads slightly since it avoids using the
Fedora proxies.
Signed-off-by: Major Hayden <major@redhat.com>
os.Exit() doesn't execute defer-ed functions, OTOH we call panic()
in case of setup errors, which does run defer-ed functions. So move
the actual test setup and execution in a separate function which will
execute all defer-ed functions and return the exit code from the
test suite to TestMain.
Prior this commit the ami image type produced raw.xz images. This was bad for
two reasons:
- The upload was broken because AWS doesn't support tar.xz format
- XZ compression is terribly slow
This commit changes the format to vhdx, which is supported by AWS and also
quite quick. See https://github.com/osbuild/osbuild-composer/issues/257
why vhdx was chosen.
Fixes#257
We will soon need the support for vhdx format, which is supported since
osbuild 12.
This commit bumps the dependency in the spec file and also updates the
submodule.
This is the same fix as in 7ff15e1f, but for the old package (and therefore
for f31).
Copy of the 7ff15e1f commit message:
The source of issue here is that our release cycles are synchronized and
if we want to use the latest released osbuild we need to wait for the
bodhi process. It makes sense that we don't want to use latest osbuild
from git master but we should be confident enough in the version we
submit to bodhi.
Using the version from updates-testing therefore fixes the issue with
synchronization and avoids using unreleased versions.
Speed up test run times by putting base tests and image tests into two
separate jobs in Jenkins. Add support to the testing playbook for both
types of tests and make it easy to add more types later, especially
for image that require booting in the cloud.
Signed-off-by: Major Hayden <major@redhat.com>
This commits enables the parallelism for the image tests. However, there's
a catch. Osbuild cannot be reliably run in parallel, so the code uses
a mutex to ensure there's always only one osbuild instance for now. Even
with this limitation, there's a significant speed-up of the tests:
Prior this commit, the image tests run in 40 minutes on Travis. After this
commit, the time is reduced to 32 minutes.
The speed-up will have an even bigger effect when more cloud-upload tests are
added to the test suite.
Run multiple dnf-json tests in parallel to speed up execution. The
total number of parallel tests is limited by the number of CPUs in
the machine that is running the tests.
Signed-off-by: Major Hayden <major@redhat.com>
