For one, this allows us to use `require` instead of `assert` and
`continue`, which was awkward to read. That works because it's ok to
fail a subtest: the remaining subtests are executed after it.
Also, this shows which test was executed, making debugging easier:
=== RUN TestCrossArchDepsolve
=== RUN TestCrossArchDepsolve/fedora-30
=== RUN TestCrossArchDepsolve/fedora-30/x86_64
=== RUN TestCrossArchDepsolve/fedora-30/x86_64/ami
...
You can now also run those sub tests in isolation:
osbuild-dnf-json-tests -test.run TestCrossArchDepsolve/fedora-30/x86_64/ami
Lastly, it enables running those tests in parallel (not part of this
patch) by calling `t.Parallel()`.
This function prints the formatted unexpected error message, instead of
the error struct in golang syntax.
It also allows to remove the error in the assertion message.
Prior this commit the ami images were tested locally using qemu. This does
not reflect at all how they're used in practice. This commit introduces
the support for running them in the actual AWS. Yay!
The structure of code reflects that we want to switch to osbuild-composer
to build the images soon.
It's not very clear that the constants are indeed constants. This commit moves
them to a new struct. This way it should be more clear that those values are
constants.
Soon, images will be run non-locally (AWS, Azure). For the remote ones it's
potentially dangerous to use the publicly available key-pair. This change
prepares the codebase for specifying different keys than the pre-generated
one.
Soon, images will be run non-locally (AWS, Azure). For those boot types
there's no need to have an unshared network namespace. This commit prepares
the code for that.
Soon, images will be run non-locally (AWS, Azure). For this, we cannot
use ssh in a namespaced environment. This commit makes the namespace for ssh
optional.
This is not a functional change (except for the slightly different output).
NoError() is more understandable (both in the code and in the output) than
Nil() when asserting errors. Also the rest of the tests uses mostly NoError()
when asserting errors, this change should lead to a better consistency.
testify library cannot deal with error messages with length > 64k. Sadly,
osbuild output is very long one line. This commits formats the output before
making the error from it.
This makes the jobqueue package independent of forking osbuild, the
choices for which (exact invocation, location of the cache directory)
should be made in the worker.
In PR#395 we discussed the spelling of archs vs. arches and we agreed to
use arches. This patch only renames the public method `ListArchs`in the
`Distro` interface.
Require repository informaiton to be passed as input, rather than
read from the current directory.
Reading from the repository informaiton meant to be used by weldr
has several drawbacks.
- it makes it impractical to use the tool outside a git checkout
- it makes it awkward to adapt the repositories to different use
cases
- it means that the shipped repositories cannot be extended with
update repos, as the same repos are used for testing, and that
would render our tests non-reproducible.
Overall, we are moving towards making repositories something the
caller must always pass in, rather than something that composer
maintains. For the weldr API we need to keep working as before,
but for new APIs we are avoiding that.
Signed-off-by: Tom Gundersen <teg@jklm.no>
The API allowed composes to have multiple architectures, image
types and repositories. Turns out that's not exactly what we want
it is not clear how to combine the lits of each. Each architecture
might not combine with each image type, and it is not clear which
repositories are need for each image build. Lastly, while allowing
different image builds to have different distros in the same
compose does not appear immediately useful, there is no particular
reason to disallow that. This patch reworks the way composes are
specified.
The intention remains the same, to be able to submit several image
builds as one compose. But rather than taking arrays of image types
and architectures, take one array of image builds instead, each of
which consists of one distro, one architecture, one image build and
an array of repositories.
In a follow-up patch they will also each contain an array of upload
targets.
This means that each image build will have the same sort of structure
as a compose request in the weldr API. The reason we want to submit
an array of them rather than have them as individual composes, is that
in a follow-up patch we will introduce the concept of a "finalizer",
or "call-back" or something to that effect, which will be triggered
when all the images have been built successfully. The use-case is,
as always, koji, which requires this.
Signed-off-by: Tom Gundersen <teg@jklm.no>
We require passing the address from the unit file. Do the same for the
socket, using host:port syntax.
Overriding the port was broken before, because we unconditionally
appended ":8700" to every address.
Introduce a mandatory argument `address`, which is interpreted as a path
to a unix socket when `-unix` is given or a network address otherwise.
Move the default path to the service file.
Add a more useful usage message when passing `-help` or no arguments.
This moves the client code into the same package as the server code,
which makes it easier to change (and version) the two in sync. Also, it
will allow to make some structs private to the jobqueue package and to
test `Client`.
Also rename it to jobqueue.Client.
Use the default dialing functions for tcp connections and set the tls
config on the transport directly. This makes the code easier to follow,
because the only special case is overriding the DialContext() for unix
connections.
According to the new guidelines in docs/errors.md.
Note that this does not include code that marshals to a writer that
might fail (when a connection drops, for example).
This test makes sure we can run depsolve for all build packages sets and
base packages sets for all image types for all architectures for all
defined Fedora versions. In could run the same for RHEL, but I haven't
yet implemented it, because such tests cannot run in public Internet.
The following commit will introduce support for forced architecture in
dnf-json. The APIs already have this kind of information, so we can
simply pass it to the Depsolve and FetchMetadata functions.
weldr needs to know the host architecture. Rather than pinning
a string, pin a real Arch object, and query its name when we
need it.
This verifies the validitiy of the architecture for the given
distro before it is passed to weldr, rather than lazily on
demand.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Require the caller to pass in the required distros explicitly. This
would allow us to easily add distros in osbuild-pipeline and tests
before exposing them in composer itself, for instance.
This means there is no longer a dependency from the distro package
to each of the individual distros, so the distros are now able
to depend on the distro packag for types and interfaces.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Build constraints must be added to every file. We only want to build
this package when the `integration` tag is set.
Without this, every build prints this warning:
# github.com/osbuild/osbuild-composer/cmd/osbuild-image-tests
runtime.main_main·f: function main is undeclared in the main package
When using %#v instead of %v, printing an error gives a slightly more
informative error message, therefore this commit switches to %#v in all
error prints.
The distro argument and restrictions to run only tests for the same distro
as the host's one were confusing. This commit removes them. Now the behaviour
is following:
By default all the test cases in test case directory are run.
If test cases are given by arguments, they are all run, and test case
directory is ignored.
Previously we had only tests for Fedora 30, therefore I added a hack to treat
every Fedora as Fedora 30, so the tests would run. Now, we have tests for
multiple Fedora versions, therefore the hack is no longer needed.
We're currently rewriting all the integration tests to use the Go
testing framework. This commit does the switch for the image tests.
I decided not to use the testing framework in functions which are
not directly tight to testing (booting images, running osbuild). I think
it's reasonable to use classic error handling there and propagate the errors
to places directly tight to testing and use the testing library.
This enables us to reuse the code in different part of projects if needed.
Mixing the way to build a distribution with where to get the source
packages from is wrong: it breaks pre-release repos, local mirrors, and
other use cases. To accommodate those, we introduced
`/etc/osbuild-composer/repositories`.
However, that doesn't work for the RCM API, which receives repository
URLs to use from outside requests. This API has been wrongly using the
`additionalRepos` parameter to inject those repos. That's broken,
because the resulting manifests contained both the installed repos and
the repos from the request.
To fix this, stop exposing repositories from the distros, but require
passing them on every call to `Manifest()`. This makes `additionalRepos`
redundant.
Fixes#341
