We're only using code coverage as additional information. Put in a 5%
threshold so that small fixes that decrease coverage slightly don't
block pull requests.
Attempt osbuild-composer testing on the internal Jenkins deployment with
nodes that are destroyed after each use. The internal Jenkins looks for
a Jenkinsfile inside the `schutzbot` directory.
Let's not remove the `jenkins` directory (used by jenkins.osbuild.org)
yet until we know the internal Jenkins is stable and performs well.
Signed-off-by: Major Hayden <major@redhat.com>
The store is responsible for two things: user state and the compose queue. This
is problematic, because the rcm API has slightly different semantics from weldr
and only used the queue part of the store. Also, the store is simply too
complex.
This commit splits the queue part out, using the new jobqueue package in both
the weldr and the rcm package. The queue is saved to a new directory `queue/`.
The weldr package now also has access to a worker server to enqueue and list
jobs. Its store continues to track composes, but the `QueueStatus` for each
compose (and image build) is deprecated. The field in `ImageBuild` is kept for
backwards compatibility for composes which finished before this change, but a
lot of code dealing with it in package compose is dropped.
store.PushCompose() is degraded to storing a new compose. It should probably be
renamed in the future. store.PopJob() is removed.
Job ids are now independent of compose ids. Because of that, the local
target gains ComposeId and ImageBuildId fields, because a worker cannot
infer those from a job anymore. This also necessitates a change in the
worker API: the job routes are changed to expect that instead of a
(compose id, image build id) pair. The route that accepts built images
keeps that pair, because it reports the image back to weldr.
worker.Server() now interacts with a job queue instead of the store. It gains
public functions that allow enqueuing an osbuild job and getting its status,
because only it knows about the specific argument and result types in the job
queue (OSBuildJob and OSBuildJobResult). One oddity remains: it needs to report
an uploaded image to weldr. Do this with a function that's passed in for now,
so that the dependency to the store can be dropped completely.
The rcm API drops its dependencies to package blueprint and store, because it
too interacts only with the worker server now.
Fixes#342
Now that the "old" `jobqueue` package was renamed to `worker`, add a new
package that contains an interface to an actual job queue. Also add two
implementations: fsjobqueue, a job queue backed by the file system, and
testjobqueue, which can be used as a mock implementation for testing.
These packages are not yet used.
Code that's calling PushCompose() had to depsolve packages and fetch the
right ImageType from a distro, but not create the osbuild manifest. That
was left for PushCompose to do. Move it out of there to the callers, so
that the store is mainly concerned with storing things.
This also simplifies the argument list of PushCompose().
This function had two modes (`uuids == nil` and `uuids != nil`). Instead
of splitting it up into one function for each mode, inline it at call
sites. It was only used three times.
Instead of iterating over all composes, iterate over the passed UUIDs
(which amount to all composes when `*` is passed). This is more
straight-forward to read and more efficient in the case of requesting
only one (or a few) statuses.
`uuids` was initialized with a length, but only appended to. This lead
to a slice with lots of empty uuids, followed by the actual ones.
Fix this by initializing to an empty slice.
This happened to work because there's a non-fatal check for non-existing
composes further down, and the empty string fails that test.
By using a small Jenkins pipeline in the repository, we can define
almost all of our testing parameters in the repo itself and not inside
Jenkins.
This also allows us to use the GitHub Branch Source plugin and
auto-discover new repositories without `ok to test` bombs in
pull requests.
Signed-off-by: Major Hayden <major@redhat.com>
-cpu host cannot be used with anything else than kvm. This commit removes
hvf and tcg because it doesn't make any sense with -cpu host.
If this causes some issues for anyone, we can revert back and remove -cpu
host.
Running qemu with -accel accel= results in the following error:
qemu-system-x86_64: -accel accel=kvm:hvf:tcg: Don't use ':' with -accel,
use -M accel=... for now instead
Qemu 4.2 deprecated the -accel accel= argument. When the arg is passed in,
qemu exists status code of 1.
This commit changes the qemu command to use the recommended way of specifying
the acceleration options.
See:
3d5e90a50b
Jenkins now uses a non-root user for its agent, so we will need to use
sudo for some commands and `become` within the Ansible playbooks.
Signed-off-by: Major Hayden <major@redhat.com>
osbuild removed GRUB2_ROOT_FS_UUID from grubenv in 22d131a5. This broke
the Jenkins CI because it runs against osbuild master.
This commit fixes all the testcases and bumps the osbuild submodule
to version 13 that changed the GRUB2_ROOT_FS_UUID behaviour.
Test the full set of locally-bootable image tests. Cloud based tests
are excluded for now until the infrastructure to boot them is ready.
Signed-off-by: Major Hayden <major@redhat.com>
The .* before the digit group is greedy and it catches the first version
digit when the version >= 10. This commit changes the .* to [^[:digit:]]*
so it doesn't catch the first digit.
Hiding logs makes the output prettier, but it is sometimes easier to see
the problem right there in the Ansible output.
Signed-off-by: Major Hayden <major@redhat.com>
Image size should not be specified for a tar image. When the tar
assembler was being created image size was not passed to it. However,
the TarAssemblerOptions contained a Size field which would default to 0.
This field is removed so that the tar assembler options no longer
specify a size. The assembler tests are also updated to reflect this
change.
Currently, removing osbuild-composer produces these warnings:
Failed to stop osbuild-worker@.service: Unit name
osbuild-worker@.service is missing the instance name.
See system logs and 'systemctl status osbuild-worker@.service' for details.
Failed to stop osbuild-remote-worker@.service: Unit name
osbuild-remote-worker@.service is missing the instance name.
See system logs and 'systemctl status osbuild-remote-worker@.service' for
details.
%systemd_preun calls on a package upgrade this:
systemctl --no-reload disable --now [ARGS...]
I tried using a wildcard:
systemctl --no-reload disable --now "osbuild-worker@*.service"
But this gives:
Invalid unit name "osbuild-worker@*.service" was escaped as
"osbuild-worker@\x2a.service" (maybe you should use systemd-escape?)
Failed to disable unit: Unit file osbuild-worker@\x2a.service does not exist.
In the end I decided to use two commands - disable and stop. Disable works
without a wildcard, stop requires one. This solution should mimic systemctl
disable --now pretty well.
Currently, upgrading osbuild-composer produces these warnings:
Failed to try-restart osbuild-worker@.service: Unit name
osbuild-worker@.service is missing the instance name.
See system logs and 'systemctl status osbuild-worker@.service' for details.
Failed to try-restart osbuild-remote-worker@.service: Unit name
osbuild-remote-worker@.service is missing the instance name.
See system logs and 'systemctl status osbuild-remote-worker@.service'
for details.
%systemd_postun_with_restart calls on a package upgrade this:
systemctl try-restart [ARGS...]
However, try-restart requires using a wildcard when restarting all
the services created from a template unit. This commit therefore adds the
wildcards.
Enable image tests for Fedora 31 and RHEL 8.2 since we now have the JSON
test case files for all distributions.
Signed-off-by: Major Hayden <major@redhat.com>
