We need a privileged / admin user doing the post-release version bump as
this is a direct commit to main (i.e. without a PR) so switch to using
schutzbot with a scoped personal access token (only public_repo).
This is necessary for the new simplified release process and is done
ahead of time once for the upcoming release now.
After osbuild 40 this will be done by the GitHub composite action.
This commit changes our release process from the model of having a
release commit (and pull request) which also updated the NEWS.md file
and bumped the versions in the osbuild.spec and setup.py files to simply
pushing a tag.
After the tag (containing the release notes) is pushed, a GitHub
composite action is triggered that creates a GitHub release with the
contents of the git release tag. Furthermore the bumping of the version
number now always has to happen directly after a release to avoid having
to push a(n untested) commit to main for the release and this is also
handled by the GitHub composite action.
Finally packit pushes directly to dist-git now on pushing the release
tag, so no pull-request needs to be reviewed and merged anymore.
Use the "Checks" workflow to trigger gitlab; this workflow should
be much quicker to complete and thus the gitlab ci will trigger
earlier leading to a more parallel ci run.
Sadly `github.event.workflow_run.pull_requests` is empty if the pull
request was opened from another fork. Use the sha to find an open PR,
otherwise assume it's a branch.
This workflow doesn't have access to the original pull request event
that resulted in this workflow being triggered.
Simply use `head_sha` which will contain the PR sha if it was triggered
by a PR's workflow, or the branch sha if it was triggered from a
branch's workflow.
The `workflow_run` event is triggered either when a workflow was
requested or complete (see `types`). We can use this event as a
trigger for the gitlab ci conditioning on a successful workflow
run of the main tests ("Tests" workflow). This will ensure that,
with outside contributor protection turned o, no secrets are
leaked via PRs from non-contributors, but also that gitlab ci is
run for those PRs once they were manually allowed to run.
The only downside is that now the gitlab ci will only run after
the main workflow ("Tests) has completed and thus serializing
both CI runs. OTOH gitlab CI is quite intense so maybe this is
not so bad after all. If in the future we want to parallelize
both CI runs we could have a third "precheck" condition with
maybe the spell checker and the pylint tests that the main tests
as well as the gitlab ci run depend on.
Instead of maintaining a separate set of samples that by now are very
much outdated (using Fedora 31 or older), make the samples directory
a symlink to the test data. Manifests in there are indeed tested and
maintained.
The error and noop samples are also covered in unit tests, so no need
for extra samples there either.
CI: remove the sample validation since all the test data manifests
are actually built.
Add new `org.osbuild.cloud-init` stage, which currently allows to create
configuration files for cloud-init under `/etc/cloud/cloud.cfg.d`. The
stage supports only a very limited subset of cloud-init configuration
options, which is covering needs of RHEL AMI images.
The schema mandates that if the 'configuration_files' option is
specified, then at least one configuration file must be defined. In
addition each section of the configuration must contain at least one
property (section or configuration option).
Add `python3-pyyaml` package to the `F34-build` testing manifest,
because it is required for running and testing the new stage.
Regenerate all affected manifests.
Add test for the new stage.
Update the `osbuild-ci` container image used for testing to a new tag,
which includes python3-pyyaml, the dependency of the new stage.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
We explicitly pinned the F32 CI images in the past due to update issues
in F33. However, those have been resolved and we should switch back to
the most recent Fedora CI images.
This commits switches all instances of the osbuild-ci image back to the
latest stream, snapshot taken on 2021-02-19 13:11 (latest-202102191311).
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
Use the new immutable image infrastructure from `osbuild/containers`.
While at it, also switch over to the new github-actions helper, now that
we no longer run `systemd-nspawn` in our tests.
The old image was renamed from `ghci-osbuild` to `osbuild-ci` to avoid
accidentally replacing old images. The new infrastructure uses immutable
images, so downstream will no longer get automatic updates, unless the
`latest` tags are used.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
Previously, we had a webhook relay. It received a notification from Github
and sent it to AWS SQS. Now, the webhook is dead. The new method (already used
in osbuild-composer and image-builder) is to send the notification directly
from a github action to AWS SQS.
`make test-data` always regenerates test data, without the need to pass
the `--always-make` option to make.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Integrate with codecov. Define a threshold of 5% to pass. Coverage
is cumulative, i.e. all the tests send their coverage to codecov,
which will integrate them all into a total.
Move the last remaining test into the correct subdir. With this done,
all our tests run in one of the 3 groups:
* `make test-src`
Run tests against the source-code, including linters.
* `make test-mod`
Run unit-tests on the individual python modules. This needs no
special permissions (unless noted in each test) or runtime
environments. It is meant to be fast and easy to run in all
circumstances.
* `make test-run`
Run tests that execute the osbuild pipeline. This requires
superuser privileges and will likely take a while. Furthermore,
this might produce large artifacts.
Align the makefile targets with the test-targets (`module` -> `mod`,
etc.). This way, we have consistent names everywhere.
While at it, move the `make test-run` invocation closer to the others.
Move the stage-tests over to the new test-infrastructure. This moves
the test invocation into `./test/run/test_stages.py`, so it is invoked
as part of the runtime-tests. Secondly, the test-data is stored in
./test/data/stages/ so the path is relative to
TestBase.locate_test_data().
While at it, this also drops the dynamic class modifications and instead
uses subTest(). This simplifies the code quite a bit and avoids
dynamically creating python code.
Move the `test_osbuild.py` test into the module-test directory. This
test contains just a bunch of basic functionality tests for a selection
of osbuild modules. Hence, it can be run together with the other module
tests.
Move `test_objectstore` into the module-level tests. This allows us to
run it as part of `make test-module.
Make sure to properly guard it as root-only module.
Run the MPP tools in the CI and verify the committed test-data did not
change and is up-to-date.
This runs `make test-data` and then simply uses `git diff --exit-code`
to trigger a CI failure if there are any differences in ./test/data.
Add a new trivial runtime-test which simply runs a no-op pipeline. This
is a fast, trivial test that simply verifies osbuild is properly setup
and accessible.
Remove the explicit no-op test from the CI, now that the test-suite has
it as well.
We are now building RPMs via mock in Jenkins, so we don't need these RPM
build jobs in GitHub Workflows.
Signed-off-by: Major Hayden <major@redhat.com>
Work around the quay.io issues by using the standard Fedora containers.
Also, make most of the dnf operations a little quieter to make it easier
to find problems.
Signed-off-by: Major Hayden <major@redhat.com>
Move the `test_boot` test from Travis over to the Github-Actions based
CI. This is the last test on Travis, and the Travis CI can now be
disabled, if we so wish.
This test leaves a valid `travis.yml` file around, since Travis will
still be enabled on the repository. We should first disable Travis and
then drop this file, if we want to get rid of it.
Use the images provided by `osbuild/containers` tagged as GHCI (GitHub
CI). These images are fully under our control, cached on the GitHub
infrastructure, and prepared to run `systemd-nspawn` and friends in a
docker container.
The GHCI infrastructure is versioned. New updates to the CI
infrastructure are not automatically picked up. Instead, the `v1` tag
has to be explicitly redirected to new image builds to deploy them. If
a new deployment causes CI failures, we can simply redirect the `v1` tag
back to the previous image builds and get the previous behavior back.
The `osbuild/containers` repository contains the required
infrastructure for this logic. If new dependencies are required in the
CI environment, the respective Dockerfiles must be updated. As a
temporary workaround (e.g., as part of a PR that introduces this), you
can simply add `dnf install -y <package>` to the required entries in
`.github/workflows/*`.
Introduce a third test-group called `src` alongside `mod` and `run.
This will contain tests that run against the source code of osbuild.
This initial commit introduces `test/src/test_pylint.py` which will run
the python linter against all our sources.
This new module contains utilities that help to introspect parts
that constitute the inner parts of osbuild, i.e. its stages
and assembler (which is also considered a type of stage in
this context). It contains the `StageInfo` class that can that
contains meta-information about the individual stage, such as
a short information (`info`), a longer description (`desc`) and
its JSON schema. A new Schema class represents schema data and
has a `validation` method that can be used to validate that json
data conforms to said schema.
A `Index` class can be used to obtain `StageInfo` and `Schema`
for entities identified via `klass` and `name`.
A top level `validate` method is introduced that can validate
manifest data.
Internally it uses the `jsonschema` package so add that as a
requirement and Install this dependency in the CI.
Move the 'test_util_selinux.py' test into the module-unittest
subdirectory.
Drop the '__main__' hookup while at it. `python -m unittest --help`
explains how you can run individual tests.
Add convenience targets to `Makefile` which can run common sets of
tests. For now, add a target for pylint, module-unittests,
pipeline-runtime-tests, as well as all tests.
Currently, it is quite cumbersome to run a reasonable test-setup
locally. Pylint invokation is rather complex, the unittests and runtime
tests in ./test are mixed, and not all tests in ./test can necessarily
be run from a development system.
This commit prepares for a simpler setup:
* Add `make test-pylint` to run pylint as it is run by CI.
* Add `make test-module` to run all module-unittests. This is meant to
be fast (preferably close to instant) and easy to run during
development to do a short check whether there are obvious typos or
other errors in local changes.
If we can keep these tests to machine-local requirements, if we
avoid any sleeps or heavy computations, then this will remain a
convenient test-suite to run locally without having to wait for
30min. In other words: We should be able to keep this under 10s (and
for the long term under 1min) easily.
* Add `make test-runtime` to run all osbuild pipeline executions. This
is not meant to be fast, but thorough. This will require external
sources (preferably limited to a suitable container image with
everything embedded). This will very likely not be run during
development, but rather by the CI.
* Add `make test-all` to run all tests. Very handy for shy people when
the chance of embarrassing copy-paste mistakes is too high to push
publicly.
Additionally to these new targets, this PR introduces 2 new directories
in ./test: ./test/mod/ and ./test/run/
These are meant as equivalent to `test-module` and `test-runtime`. The
reason is that preferably we stick to the auto-discovery of `unittest`
to enumerate tests, rather than enrolling our own or having to enumerate
them explicitly somewhere.
However, we need some way to tell `unittest` which test belongs into
which group. The easiest setup is likely to just use sub-directories.
Note that `test-all` picks all tests independently of where they are
put, even if they are in further different sub-modules under ./test.
For now, no tests are moved into the new directories. I expect this to
take a bit, since there are several out-standing PRs that modify ./test.
I intend to do the final move once we agreed on this and we synchronized
our test-modifications.
Make sure we always pylint all python sources. We currently skip tests
as well as osbuild/util. Fix this by always recursively looking for all
python sources and then linting them.
