Attempt to clarify the structure of our tests. Each test case is now
encapsulated in a script in `test/cases`. Each of these scripts should
be runnable on a pristine machine and be independent of each other. It
is up to the test-orchestractor to decide if they should be run
consequtively instance, or in parallel on separate instances. Each
script can execute several tests and call whatever helper binaries
is desired. However, each case should be assumed to always run as one.
No tests should be run directly from git, but should rather be installed
onto the test system using rpm and run from there. This moves towards
unifying our two types of test cases.
The new structure of is now:
`test/cmd`: the executors, one for each test-case. This is installed
into `/usr/libexec/test/osbuild-composer`.
`test/data`: data and config used by the tests. This is installed into
`/usr/share/tests/osbuild-composer`.
`schutzbot`: configuration of the actual test run. In particular, this
is where the distros and repositories to test against are
configured.
This is very much still work-in-progress, and is only the first step
towards simplifying schutzbot. Apart from moving files around, this
should be a noop.
Signed-off-by: Tom Gundersen <teg@jklm.no>
These had the generic name osbulid-tests, rename the binary to more
closely describe what is under test: weldr API as composed by
composer-cli.
Signed-off-by: Tom Gundersen <teg@jklm.no>
The image tests are now able to upload images to Azure and boot them there.
However, the documentation on how to set up the required resources in Azure
was missing. This commit adds it.
coveralls doesn't work from GitHub actions. Its "github" service type
uses the GITHUB_TOKEN from the action, which only has read access when
invoked from a forked repository.
codecov gets this right: it validates that an uploaded coverage file
originated from a GitHub action run by asking GitHub, and then uses its
OAuth credentials (through the Marketplace App) to comment and set
status.
Also, coveralls requires a third-party package to convert go's coverage
report to a format it understands. codecov detects the format
server-side. It also handles go's coverage format better: it highlights
lines with "some coverage" in yellow (coveralls has no concept of this).
