Use the images provided by `osbuild/containers` tagged as GHCI (GitHub
CI). These images are fully under our control, cached on the GitHub
infrastructure, and prepared to run `systemd-nspawn` and friends in a
docker container.
The GHCI infrastructure is versioned. New updates to the CI
infrastructure are not automatically picked up. Instead, the `v1` tag
has to be explicitly redirected to new image builds to deploy them. If
a new deployment causes CI failures, we can simply redirect the `v1` tag
back to the previous image builds and get the previous behavior back.
The `osbuild/containers` repository contains the required
infrastructure for this logic. If new dependencies are required in the
CI environment, the respective Dockerfiles must be updated. As a
temporary workaround (e.g., as part of a PR that introduces this), you
can simply add `dnf install -y <package>` to the required entries in
`.github/workflows/*`.
Introduce a third test-group called `src` alongside `mod` and `run.
This will contain tests that run against the source code of osbuild.
This initial commit introduces `test/src/test_pylint.py` which will run
the python linter against all our sources.
This new module contains utilities that help to introspect parts
that constitute the inner parts of osbuild, i.e. its stages
and assembler (which is also considered a type of stage in
this context). It contains the `StageInfo` class that can that
contains meta-information about the individual stage, such as
a short information (`info`), a longer description (`desc`) and
its JSON schema. A new Schema class represents schema data and
has a `validation` method that can be used to validate that json
data conforms to said schema.
A `Index` class can be used to obtain `StageInfo` and `Schema`
for entities identified via `klass` and `name`.
A top level `validate` method is introduced that can validate
manifest data.
Internally it uses the `jsonschema` package so add that as a
requirement and Install this dependency in the CI.
Move the 'test_util_selinux.py' test into the module-unittest
subdirectory.
Drop the '__main__' hookup while at it. `python -m unittest --help`
explains how you can run individual tests.
Add convenience targets to `Makefile` which can run common sets of
tests. For now, add a target for pylint, module-unittests,
pipeline-runtime-tests, as well as all tests.
Currently, it is quite cumbersome to run a reasonable test-setup
locally. Pylint invokation is rather complex, the unittests and runtime
tests in ./test are mixed, and not all tests in ./test can necessarily
be run from a development system.
This commit prepares for a simpler setup:
* Add `make test-pylint` to run pylint as it is run by CI.
* Add `make test-module` to run all module-unittests. This is meant to
be fast (preferably close to instant) and easy to run during
development to do a short check whether there are obvious typos or
other errors in local changes.
If we can keep these tests to machine-local requirements, if we
avoid any sleeps or heavy computations, then this will remain a
convenient test-suite to run locally without having to wait for
30min. In other words: We should be able to keep this under 10s (and
for the long term under 1min) easily.
* Add `make test-runtime` to run all osbuild pipeline executions. This
is not meant to be fast, but thorough. This will require external
sources (preferably limited to a suitable container image with
everything embedded). This will very likely not be run during
development, but rather by the CI.
* Add `make test-all` to run all tests. Very handy for shy people when
the chance of embarrassing copy-paste mistakes is too high to push
publicly.
Additionally to these new targets, this PR introduces 2 new directories
in ./test: ./test/mod/ and ./test/run/
These are meant as equivalent to `test-module` and `test-runtime`. The
reason is that preferably we stick to the auto-discovery of `unittest`
to enumerate tests, rather than enrolling our own or having to enumerate
them explicitly somewhere.
However, we need some way to tell `unittest` which test belongs into
which group. The easiest setup is likely to just use sub-directories.
Note that `test-all` picks all tests independently of where they are
put, even if they are in further different sub-modules under ./test.
For now, no tests are moved into the new directories. I expect this to
take a bit, since there are several out-standing PRs that modify ./test.
I intend to do the final move once we agreed on this and we synchronized
our test-modifications.
Make sure we always pylint all python sources. We currently skip tests
as well as osbuild/util. Fix this by always recursively looking for all
python sources and then linting them.
Now that containers are being built daily in GitHub Actions from
the container[0] repository, we can use those containers. That will
speed up the GitHub Actions for osbuild and reduce errors from
unreleased Fedora versions.
[0] https://github.com/osbuild/containers
Signed-off-by: Major Hayden <major@redhat.com>
Continue our effort to move to Github-actions. This imports the runtime
tests from Travis into Github-actions. The `test_boot` test is still
left on travis, since it requires stacked KVM, which is not yet
available on github-actions.
Drop the filter on `master`. This prevents us from running the CI on
forked repositories. In particular, it prevents us from pushing
branches to github-forks and have the CI run on them. This is very nice
to have as development tool, as it allows running the real CI without
opening a PR.
This adds another job to the CI runner. This builds and tests the
documentation. The tests are currently reduced to just verifying the
respective man-pages are actually generated. This can be extended on in
the future.
Verify the copy on write semantics of `objectstore.Object`, i.e.
content will only be copied at the moment a client wants to write
to `Object`. This also checks that `Object.base` works.
Modify the CI to execute the unit tests in a privileged container
because `Object.read()` works internally by bind mounting a path.
The mount operation needs at least CAP_SYS_ADMIN and overwriting
the file permissions CAP_DAC_OVERRIDE.
A single workflow allows us to re-use artifacts throughout all jobs.
Also, we can fail the tests early if there's a linting issue before
we spend time waiting for RPMs to build.
Signed-off-by: Major Hayden <major@redhat.com>
By default, GitHub Actions stops running all RPM builds if one fails.
This means that a temporary repository issue (like what is happening
with F32 today) stops all RPM builds.
Set the `fail-fast` option to `false` to disable this behavior.
Signed-off-by: Major Hayden <major@redhat.com>
Make RPM building more similar to how RPMs will be built in koji.
This downloads the specfile from github at the given commit, appends
the commit sha to the specfile and uses spectools to download the
correct sources from github.
This should be equivalent to what is done in the makefile, the only
behavioral difference is that the rpms are now versioned based on
the git sha they are built from.
The main purpose of this change is to avoid any differences between
the CI and the real RPMs due to bugs in the Makefile. Correctly
versioned RPMs will also be handy for testing/debugging/deployment.
Signed-off-by: Tom Gundersen <teg@jklm.no>
