We're currently rewriting all the integration tests to use the Go
testing framework. This commit does the switch for the image tests.
I decided not to use the testing framework in functions which are
not directly tight to testing (booting images, running osbuild). I think
it's reasonable to use classic error handling there and propagate the errors
to places directly tight to testing and use the testing library.
This enables us to reuse the code in different part of projects if needed.
Convert weldrcheck to use the standard go testing framework along with
the github.com/stretchr/testify/require assert package.
This also removes the cmd/osbuild-weldr-tests and builds the test binary
directly from the weldrcheck package. This makes it easier to organize
the code instead of putting it all into a single main_test.go file.
This commit makes the osbuild-image-tests binary doing the same set of tests
like the old test/run script.
Changes from test/run:
- qemu/nspawn are now killed gracefully. Firstly, SIGTERM is sent.
If the process doesn't exit till the timeout, SIGKILL is sent.
I changed this because nspawn leaves some artifacts behind when killed
by SIGKILL.
- the unsharing of network namespace now works differently because of
systemd issue #15079
This allows us to take advantage of the `testing` package. It also gives
the resulting test binary common command line arguments (same as `go
test`).
Tests need to be compiled with `go test -c`, which injects a `Main()`
that calls the Test* functions.
This is not supported by the golang rpm macros. Thus, build this binary
by calling `go test -c` directly, but taking care to pass the same
linker flags as the `%gobuild` macro.
Mark the test binary with the `integration` build constraint, so that
`go test ./...` doesn't pick them up. That's only for unit tests.
The idea is to move all other test binaries to this scheme as well.
Spec file changes by Lars Karlitski <lars@karlitski.net>
Since the recent commit (04db4fa) we are no longer provider
of lorax-composer. This was needed because when installing cockpit-composer,
which depends on lorax-composer, dnf chose us as the lorax-composer provider
instead of the original lorax-composer package. As we are not yet ready to
fully replace lorax-composer, this broke the cockpit-composer tests.
This commit is the first part of long term fix. Osbuild-composer is now
a provider of weldr. Soon, lorax-composer will also be a provider of
weldr. After that, cockpit-composer will be switched to be depending
on weldr. Also, it will suggest lorax-composer, which will force dnf
to use lorax-composer as the default weldr backend. If someone wants
to experiment with osbuild-composer, it will be possible, because
it will also be a provider of weldr and there will be no need to
install lorax-composer to satisfy cockpit-composer dependencies.
We're not fully compatible with lorax-composer's API yet, and we don't
provide lorax-composer's systemd unit files.
As a result, cockpit-composer's integration tests fail, because `dnf
install lorax-composer` on Fedora can result in installing
osbuild-composer in some cases.
This runs tests against a running API server, either lorax-composer or
osbuild-composer, and reports the results to stdout. It uses the
/run/weldr/api.socket to communicate with the server.
Prior this commit installing the worker sub-packages shows the following
warning:
Failed to preset unit: Unit file osbuild-worker@.service does not exist.
Moving the unit file to the sub-package fixes it.
./test/run test suite has served us well over the last months. However,
there is currently a major effort to run the better defined integration
test suite on a CI. Nonetheless, two very important parts are still missing
from the integration test suite: inspecting the image with image-info
and booting the image. This commit begins the work on this matter by porting
a part of ./test/run suite to Go. Currently, only image-info tests work, the
rest will come in the following commits.
This changes osbuild-composer's behavior to match lorax-composer when
encountering invalid versions. Instead of leaving them as-is it will
return a BlueprintError explaining the problem. eg.
"errors": [
{
"id": "BlueprintsError",
"msg": "Invalid 'version', must use Semantic Versioning: is not in dotted-tri format"
}
]
This is enforced on new blueprints (including the workspace). If a
previously stored blueprint has an invalid version and a new one is
pushed it will use the new version number instead of trying to bump the
invalid one.
This also moves the version bump logic into blueprint instead of store,
and adds an Initialize function that will make sure that the blueprint
has sane default values for any missing fields.
This includes tests for the Initialize and BumpVersion functions.
As the worker can now be running on a different machine than the composer
it makes sense to install only worker binary on some machines. This commit
does exactly that - worker is now its own subpackage with the beautiful name
of golang-github-osbuild-composer-worker.
The main osbuild-composer package requires the worker subpackage, therefore
there will always be worker installed with composer. When composer is started
one local worker process will be spawned. If you don't want the default
worker process you need to mask its unit file:
systemctl mask osbuild-worker@1.service
Introduce a new osbuild-tests command and ship it in the -tests
sub-package.
The intended usecase is to install the -tests subpackage into an
otherwise pristine VM, and call the osbuild-tests binary over ssh
from the outside of the booted VM. If the binary exits with a return
code of 0, the tests passed, otherwise they failed. The VM should
not be reused after running the tests.
Signed-off-by: Tom Gundersen <teg@jklm.no>
During development of a new distro, we need to test composer against
nightly or beta repositories, but we cannot ship composer itself
with the nightly repository information hardcoded in. At the same
time, we want to distinguish between the system repositories of the
host and the repositories we use to generate images (the host may not
use the same distro/version/architecture as the target, and it may
include custom repositories that the target should not).
We therefore ship per distro repository information that can be
overriden (typically in testing) by dropping files in /etc.
For now use the latest nightlies for RHEL-8.2, we may want to
replace these with the official mirrors for GA eventually.
Signed-off-by: Tom Gundersen <teg@jklm.no>
In RHEL golang dependencies must be vendored, whereas on Fedora they
must be packaged separately. Add conditionals accordingly.
Some macros are not yet available in RHEL, so fall back to older
versions. We may want to just use macros avilable everywhere
unconditionally, but I left that for a follow-up.
Signed-off-by: Tom Gundersen <teg@jklm.no>
this is a small helper to enable easy switch between backends
in the existing test suite where we reference everything by name.
The current working variant is:
yum install $BACKEND
systemctl enable $BACKEND.socket
where BACKEND is either lorax-composer or osbuild-composer!
This commandline tools uploads a file to S3, as a proof of concept.
All options are mandatory. Credentials are only read from the
commandline and not from the environment or configuration files.
The next step is to add support for importing from S3 to EC2,
currently the images we produce cannot be imported as-is, so this
requires more research.
To try this out: create an S3 bucket, get your credentials and
call the tool, passing any value as `key`. Note that if the key
already exists, it will be overwritten.
Signed-off-by: Tom Gundersen <teg@jklm.no>
It uses Azure SDK to connect to Azure storage, creates a container there
and uploads the image. Unfortunately the API for page blobs does not
include some thread pool for upload so I implemented one myself. The
performance can be tweaked using the upload chunk size and number of
parallel threads.
The package is prepared to be refactored into common module within
internals package as soon as we agree on the of these common packages
for image upload.
Add azure-blob-storage rpm package as a dependency
It didn't work for me using the `golang(package)` syntax. Using the
package name explicitly works.
According to Fedora packaging guidelines, the rpm scriptlets must
call into systemd to make sure the services are enabled/disabled
corerctly on install and uninstall.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This makes no difference, so let's just put them where the Fedora
guidelines say they should be.
Also, make sure to own the containing directory.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Addressing https://bugzilla.redhat.com/show_bug.cgi?id=1768774#c1
The review also points out that the Source0 URL is wrong, I fixed
this by pushing a new tag: `v1`, rather than just `1`.
Signed-off-by: Tom Gundersen <teg@jklm.no>
