Commit graph

916 commits

Author SHA1 Message Date
Lars Karlitski
2b7adb3200 jsondb: add List method
And add some additional tests.
2020-05-08 14:53:00 +02:00
Lars Karlitski
d74a63f4f2 weldr: inline composesToComposeEntries
This function had two modes (`uuids == nil` and `uuids != nil`). Instead
of splitting it up into one function for each mode, inline it at call
sites. It was only used three times.
2020-05-08 14:53:00 +02:00
Lars Karlitski
cbf0ec05b4 weldr: simplify filtering in composeStatusHandler
Instead of iterating over all composes, iterate over the passed UUIDs
(which amount to all composes when `*` is passed). This is more
straight-forward to read and more efficient in the case of requesting
only one (or a few) statuses.
2020-05-08 14:53:00 +02:00
Lars Karlitski
8e9f725301 weldr: fix minor inefficiency in composeStatusHandler
`uuids` was initialized with a length, but only appended to. This lead
to a slice with lots of empty uuids, followed by the actual ones.

Fix this by initializing to an empty slice.

This happened to work because there's a non-fatal check for non-existing
composes further down, and the empty string fails that test.
2020-05-08 14:53:00 +02:00
Lars Karlitski
35173cae82 weldr: return fewer errors in composeDeleteHandler
Don't try deleting a compose when it's already known that the uuid is
invalid.
2020-05-08 14:53:00 +02:00
Lars Karlitski
cf53e6a9e2 test/helpers: include error return when route fails 2020-05-08 14:53:00 +02:00
Lars Karlitski
5c019cd092 test/helpers: mark TestRoute() as a helper function
This makes the calling function/location appear when a test fails.
2020-05-08 14:53:00 +02:00
Jiri Kortus
9315ccb88e Assert existing pipelines in TestDistro_Manifest
Without the assertion some tests can just pass without actually doing
anything if no matching pipeline jsons exist.

Refs: #442
2020-05-08 10:12:28 +03:00
Jiri Kortus
3cea1f27cf Fix paths in TestDistro_Manifest
It turned out the paths contained more ../s than appropriate.

Refs: #442
2020-05-08 10:12:28 +03:00
Major Hayden
2e34a94980
Disable Jenkins CI for RHEL temporarily 2020-05-06 17:04:47 +00:00
Major Hayden
08594bd66d 🚽 Use Jenkins pipeline for better flow
By using a small Jenkins pipeline in the repository, we can define
almost all of our testing parameters in the repo itself and not inside
Jenkins.

This also allows us to use the GitHub Branch Source plugin and
auto-discover new repositories without `ok to test` bombs in
pull requests.

Signed-off-by: Major Hayden <major@redhat.com>
2020-05-05 18:33:04 -05:00
Ondřej Budai
3ba8c79e5e test/image: don't fallback to hvf/tcg if kvm is not available
-cpu host cannot be used with anything else than kvm. This commit removes
hvf and tcg because it doesn't make any sense with -cpu host.

If this causes some issues for anyone, we can revert back and remove -cpu
host.
2020-05-04 10:12:56 +02:00
Ondřej Budai
0c354e34e4 test/image: fix qemu args on qemu>=4.2
Running qemu with -accel accel= results in the following error:

qemu-system-x86_64: -accel accel=kvm:hvf:tcg: Don't use ':' with -accel,
use -M accel=... for now instead

Qemu 4.2 deprecated the -accel accel= argument. When the arg is passed in,
qemu exists status code of 1.

This commit changes the qemu command to use the recommended way of specifying
the acceleration options.

See:
3d5e90a50b
2020-05-04 10:12:56 +02:00
Major Hayden
b725ac24d2 Use sudo/become for tests
Jenkins now uses a non-root user for its agent, so we will need to use
sudo for some commands and `become` within the Ansible playbooks.

Signed-off-by: Major Hayden <major@redhat.com>
2020-05-01 11:03:50 -05:00
Ondřej Budai
7f7769cae0 test/cases: fix expected values for the image info sub-test
osbuild removed GRUB2_ROOT_FS_UUID from grubenv in 22d131a5. This broke
the Jenkins CI because it runs against osbuild master.

This commit fixes all the testcases and bumps the osbuild submodule
to version 13 that changed the GRUB2_ROOT_FS_UUID behaviour.
2020-04-30 15:55:34 +02:00
Jiri Kortus
50df564caf Improve coverage for distro packages - F32
Refs: #442
2020-04-30 11:26:27 +02:00
Jiri Kortus
72192a0f7c Improve coverage for distro packages - F31
Refs: #442
2020-04-30 11:26:27 +02:00
Jiri Kortus
0150adae7a Use assert.NoError in F30 distro test
Refs: #442
2020-04-30 10:58:47 +02:00
Major Hayden
9223036249 Add remaining local image tests
Test the full set of locally-bootable image tests. Cloud based tests
are excluded for now until the infrastructure to boot them is ready.

Signed-off-by: Major Hayden <major@redhat.com>
2020-04-30 08:26:13 +02:00
Major Hayden
4f86db8ea4 🐞 Fix worker https logic
Use https scheme for workers with TLS configuration set.

Fixes #568.

Signed-off-by: Major Hayden <major@redhat.com>
2020-04-29 21:07:28 +02:00
Ondřej Budai
169f54ed34 11
Release osbuild-composer version 11
2020-04-29 20:42:37 +02:00
Alexander Todorov
02482a0eea tests: Distro is fedora-31 update the repo URL to match 2020-04-29 18:48:51 +02:00
Ondřej Budai
6eb43c3d97 worker: add a support for uploads to azure
Everything else is already implemented, this commit just connects the bits
and pieces in worker.
2020-04-29 18:15:13 +02:00
Ondřej Budai
6513263a14 upload/azure: force .vhd extension
Azure cannot create an image from a storage blob without .vhd extension.
This commit ensures that image always has the right extension.
2020-04-29 18:15:13 +02:00
Ondřej Budai
1d52dfcc2b upload/azure: fix typos 2020-04-29 18:15:13 +02:00
Ondřej Budai
5415be465f upload/azure: remove an unneeded custom error type
The errorString was not exported and used anywhere else, this commit replaces
it with a standard method of creating simple errors.
2020-04-29 18:15:13 +02:00
Ondřej Budai
e52a35d84e upload/azure: add a context to errors
Debugging azure upload fails is currently pretty hard, this commit adds
a context to all errors Azure upload can emit.
2020-04-29 18:15:13 +02:00
Ondřej Budai
771f9bf849 azure: normalize the terms for Azure credentials
The account field is actually a storage account and the secret key is actually
a storage secret key. This is clearer to the user.
2020-04-29 18:15:13 +02:00
Ondřej Budai
3ce3a2ad80 makefile: add the old specfile to the release instructions 2020-04-29 18:15:04 +02:00
Ondřej Budai
3f9e5e1e35 makefile: fix parsing the version from the spec file
The .* before the digit group is greedy and it catches the first version
digit when the version >= 10. This commit changes the .* to [^[:digit:]]*
so it doesn't catch the first digit.
2020-04-29 18:15:04 +02:00
Ondřej Budai
75dd7886e4 makefile: do not print the comments in the release target
Prior this commit `make release` printed the comments. This commit fixes that.
2020-04-29 18:15:04 +02:00
Major Hayden
e21a8095b1 📔 Remove the no_log for image tests
Hiding logs makes the output prettier, but it is sometimes easier to see
the problem right there in the Ansible output.

Signed-off-by: Major Hayden <major@redhat.com>
2020-04-29 17:50:25 +02:00
Jacob Kozol
0165a97761 test/cases: remove tar size
When building a tar image the image size should not be specified. All
image size fields are removed from the assembler options in the tar test
cases.
2020-04-29 13:44:44 +02:00
Jacob Kozol
51c610e1ae osbuild: remove tar image size
Image size should not be specified for a tar image. When the tar
assembler was being created image size was not passed to it. However,
the TarAssemblerOptions contained a Size field which would default to 0.
This field is removed so that the tar assembler options no longer
specify a size. The assembler tests are also updated to reflect this
change.
2020-04-29 12:36:43 +02:00
Ondřej Budai
b916a88242 worker: fix passing the result from osbuild when it fails
I tried fixing this in 181128c5 and forgot to pass the right error in one
place. This commit fixes it.
2020-04-29 11:40:36 +02:00
Ondřej Budai
c6b9bf4545 spec: fix warnings during the package removal
Currently, removing osbuild-composer produces these warnings:

Failed to stop osbuild-worker@.service: Unit name
osbuild-worker@.service is missing the instance name.
See system logs and 'systemctl status osbuild-worker@.service' for details.
Failed to stop osbuild-remote-worker@.service: Unit name
osbuild-remote-worker@.service is missing the instance name.
See system logs and 'systemctl status osbuild-remote-worker@.service' for
details.

%systemd_preun calls on a package upgrade this:
systemctl --no-reload disable --now [ARGS...]

I tried using a wildcard:

systemctl --no-reload disable --now "osbuild-worker@*.service"

But this gives:

Invalid unit name "osbuild-worker@*.service" was escaped as
"osbuild-worker@\x2a.service" (maybe you should use systemd-escape?)
Failed to disable unit: Unit file osbuild-worker@\x2a.service does not exist.

In the end I decided to use two commands - disable and stop. Disable works
without a wildcard, stop requires one. This solution should mimic systemctl
disable --now pretty well.
2020-04-29 10:50:58 +02:00
Ondřej Budai
e3acf45588 spec: fix warnings during the package upgrade
Currently, upgrading osbuild-composer produces these warnings:

Failed to try-restart osbuild-worker@.service: Unit name
osbuild-worker@.service is missing the instance name.
See system logs and 'systemctl status osbuild-worker@.service' for details.
Failed to try-restart osbuild-remote-worker@.service: Unit name
osbuild-remote-worker@.service is missing the instance name.
See system logs and 'systemctl status osbuild-remote-worker@.service'
for details.

%systemd_postun_with_restart calls on a package upgrade this:
systemctl try-restart [ARGS...]

However, try-restart requires using a wildcard when restarting all
the services created from a template unit. This commit therefore adds the
wildcards.
2020-04-29 10:50:58 +02:00
Major Hayden
37ff79229b 🤗 Image tests for Fedora 32 + RHEL 8.2
Enable image tests for Fedora 31 and RHEL 8.2 since we now have the JSON
test case files for all distributions.

Signed-off-by: Major Hayden <major@redhat.com>
2020-04-29 06:49:17 +02:00
Ondřej Budai
d9af3bf6ed ci/travis: enable fedora 32 2020-04-28 13:29:43 -05:00
Ondřej Budai
ef92dd7d7c test: use azure for all vhd images
rhel 8.2 and fedora 32 images are missing the new boot type because of a bad
merge timing.
2020-04-28 13:29:43 -05:00
Tom Gundersen
046d08b8ef test/cases: add rhel8.2 tests for x86_65
Generated with

$ sudo ./tools/test-case-generators/generate-test-cases --distro rhel-82 --arch x86_64 --store .osbuild/ --output test/cases/

(Partially) fixes #542.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-04-28 11:10:12 +02:00
Tom Gundersen
f3ad87ea0b test/cases: add f32 tests for x86_64
Generated with

$ sudo ./tools/test-case-generators/generate-test-cases --distro fedora-32 --arch x86_64 --store .osbuild/ --output test/cases/

Fixes #544.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-04-28 11:10:12 +02:00
Tom Gundersen
1d1dcfefb6 tools/generate-test-cases/repos: add GPG keys and update to most recent
Update RHEL-8.2 to the most recent RC now that it has been released.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-04-28 11:10:12 +02:00
Tom Gundersen
b7268b2948 test/README: mention the tests cases and how to generate them
Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-04-28 11:10:12 +02:00
Ondřej Budai
e235fdedb3 tests/image: boot the vhd images on Azure
Previously, vhd images were tested using QEMU. This commit changes that to
boot them in the actual Azure infrastructure.

Azure VMs have quite a lot of dependencies - a network interface, a virtual
network, a network security group, a public ip address and a disk. Azure CLI
and Azure Portal handle the creation of all these resources internally.
However, when using the API, the caller is responsible to create all these
resources before creating an actual VM.

To handle the creation of all the resources in the right order, a deployment
is used. A deployment is a set of resources defined in a JSON document.
It can optionally take parameters to customize each deployment. After the
deployment is finished, the VM is up and ready to be tested using SSH.

Sadly, the deployments are a bit hard to clean-up. One would expect that
deleting a deployment removes all the deployed resources. However, it doesn't
work this way and therefore it's needed to clean up all resources "manually".
For this reason, our deployment sets a unique tag on all the resources created
by the deployment. After this test is finished, the API is queried for all
the resources with the tag and then, they're deleted in the right order.
2020-04-27 20:34:20 +02:00
Ondřej Budai
10c016edca vendor Azure SDK
This should actually be a part of the next commit, but that would make the
review hard, so I decided to create a separate commit just for vendoring.
2020-04-27 20:34:20 +02:00
Ondřej Budai
0041ae5655 tests/image: move constants to a subpackage
The cmd/osbuild-image-tests package is becoming bigger than I would like to.
It will be nice to split it to some smaller pieces at some point.
This commit does the first step - splits off the first subpackage containing
all the constants.
2020-04-27 20:34:20 +02:00
Ondřej Budai
b109ec878e cmd/osbuild-upload-azure: improve error message on wrong arguments
After this commit osbuild-upload-azure tool returns the help text when wrong
arguments were passed to it.
2020-04-27 20:34:20 +02:00
Ondřej Budai
181128c5b9 worker: fix missing logs when osbuild fails
The commit 2435163f broke sending the logs to osbuild-composer. This was
partly because of unusual error handling in the RunOSBuild function.

This commit fixes that by creating a custom error and properly propagating
the result from it.
2020-04-27 19:36:22 +02:00
Major Hayden
a613f778c2 Remove incomplete comment
Signed-off-by: Major Hayden <major@redhat.com>
2020-04-27 11:57:17 +02:00