The spinner is mostly distracting IMHO and it makes the CI logs
harder to read as it generates several dozens of lines like:
```
...
⠋ [3/5] pylint | autopep8 | mypy-strict
...
```
I was considering to make it conditional on GH but decided to
just disable globally. If someone feels attached to the spinner
I only disable it in GH.
The current `make test-all` will only run tests under `test/`. This
is no longer the only place we have tests so update the code to
just run `pytest` to collect all tests.
Instead of using the F34-based manifests, let's switch to F38. I tried my
best to import the vars into the new manifest, but I don't think that's
currently supported.
This commit migrates the test to a brand new V2, F38-based manifest. It's
actually based on osbuild-composer interpretation of the Fedora Cloud Base
image.
This commit changes this manifest to use the new fedora-vars.ipp and
fedora-build-v2.ipp infrastructure to remove all hardcoded Fedora versions.
We are also able to remove the container manifests from the Makefile,
because they are no longer depending on the old f34-build-v2.json.
osbuild-mpp supports yaml files. The advantage of yaml is that it's less
verbose and allows comments. This way, we can start documenting all the
test manifests in this repository.
This is the first step: This commit just renames the files, so we keep the
history intact. Remember: YAML is a super-set(*) of JSON, so this is fine.
There is no functional change in this commit.
(*) There are some caveats, but we don't need to care in this case.
This adds a `osbuild --version` command that prints the current osbuild
version in use. Allows users to confirm their osbuild is up to date
enough to use newer features.
This commit changes our release process from the model of having a
release commit (and pull request) which also updated the NEWS.md file
and bumped the versions in the osbuild.spec and setup.py files to simply
pushing a tag.
After the tag (containing the release notes) is pushed, a GitHub
composite action is triggered that creates a GitHub release with the
contents of the git release tag. Furthermore the bumping of the version
number now always has to happen directly after a release to avoid having
to push a(n untested) commit to main for the release and this is also
handled by the GitHub composite action.
Finally packit pushes directly to dist-git now on pushing the release
tag, so no pull-request needs to be reviewed and merged anymore.
Include a build manifest that is itself not have tied to a specified
version and thus is meant to be included with the following vars
pre-defined as .mpp file:
- arch architecture (x86_64)
- releasever release version (f34)
- snapshot rpmrepo snapshot (20210326)
Move from using 'zram' to 'zram-generator-defaults' in the ostree bootiso
testing manifest. More information is available in Fedora 33 Change
document [1].
Add org.osbuild.kernel-cmdline stage to fedora-boot.json manifest
because of change in how grub handles the kernel command line arguments
[2].
GRUB2 Stage 2 checksums in assemblers test are updated. The change have
been verified by building the fedora-boot.json manifest with each checked
filesystem and booting the image in QEMU with legacy mode.
[1] https://fedoraproject.org/wiki/Changes/SwapOnZRAM
[2] https://github.com/osbuild/osbuild-composer/pull/982#issuecomment-697356929
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add a v2 manifest to builds an ostree commit, then builds a small
container with a webserver and puts that commit into it.
Start a new test suit where this manifest is built and then checked
that the artifact exists. In the future the artifact should also be
inspected and run by a container engine and the commit be pulled
via ostree.
Add an example that builds a small container, based on the minimal
container fedora kickstart (see [1, 2]). There is a deviation
because osbuild currently does not support removing
random files like it is done in [2].
The build pipeline "build" in f32-build-v2.json) is a port of the
existing build pipeline in f32-build.json.
This manifest is currently not used in any automated tests (yet),
but is there as a showcase for the new format. It does however
test the format version 2 support for mpp, since that is called
in CI.
[1] https://pagure.io/fedora-kickstarts/blob/main/f/fedora-container-common.ks
[2] https://pagure.io/fedora-kickstarts/blob/main/f/fedora-container-base-minimal.ks
Mark '$(TEST_MANIFESTS_GEN)' target as .PHONY.
Currently the `test-data` make target does not always trigger
regenerating of manifests used for testing various osbuild parts,
although it is marked as .PHONY. The reason that its dependency
'$(TEST_MANIFESTS_GEN)' is not marked as .PHONY and therefore it is not
run if the files already exist in repository.
Due to the above reason, CI is actually running `make --always-make
test-data` to always regenerate manifests.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Stage tests (test/run/test_stage.py) currently use hard-coded pipelines
from `test/data/stages`. These could benefit from the mpp tools to make
them easier to understand and update to newer versions of Fedora.
Future commits will do so.
This simplifies the `test-data` rules in `Makefile` considerably. Also,
it allows adding `*.mpp.json` files in other directories without needing
to copy rules. (make's pattern-based rules only allow a single `%`).
Adjust test/data/README.md accordingly.
#471 extends the assembler test suite to also test xfs and btrfs filesystems
in raw and qemu assemblers. However, this change leads to long running times
of this suite.
The running time of these test consist of 3 main steps:
1) Building the build pipeline
2) Building the stages
3) Running the assembler
There are two optimization approaches:
1) Caching
OSBuild supports caching, therefore it's possible to cache results of first
two steps.
2) Minimizing the operating system tree
Assemblers don't care about the image contents. Therefore, it's possible
to create just a small tree which would be used to test the assemblers.
This should lead to speed up in the step 2 (smaller tree should be built
quicker) and in step 3 (big part of assembling is just copying files over
to the image).
This commit implements the second approach. A new test manifest is now added,
which just installs the filesystem package and its dependencies and this tree
is then labeled. This solution was chosen, so that the assemblers get
something that looks as a proper filesystem tree but also can be built pretty
quickly.
Before this change, the test_rawfs method with #471 merged ran for 842 seconds.
After this change, it ran for 391 seconds.
Add the scaffolding and various targets to run a coverity analysis,
including downloading the tool, running the "build", archiving
the results and submitting it to coverity for analysis.
The downloading and submitting need COVERITY_{EMAIL, TOKEN} to be
defined in the environment.
Co-authored-by: David Rheinsberg <david.rheinsberg@gmail.com>
Align the makefile targets with the test-targets (`module` -> `mod`,
etc.). This way, we have consistent names everywhere.
While at it, move the `make test-run` invocation closer to the others.
This adds F32 manifests in ./test/data/. To avoid magically deducing the
package list out of the void, this adds a ManifestPreProcessor (MPP)
called `./tools/mpp-depsolve.py`. What this does is it takes a manifest
on stdin, modifies it, and produces a manifest on stdout.
The `mpp-depsolve.py` preprocessor takes a manifest and modifies all the
`org.osbuild.rpm` stages. It parses a new option to that stage called
`mpp-depsolve`, which contains a package-list, a repo-list, and dnf
metadata. It then drops this `mpp-depsolve` option (since it would be an
invalid manifest otherwise), depsolves the packages, inserts a proper
"packages" option as well as appends the correct paths to the sources
entry.
With this in place, this adds `mpp-f32-base.json` and
`mpp-f32-build.json` in ./test/data/manifests/. These will then be used
as base F32 manifests for our test-suite.
Lastly, this adds `./test/data/README.md` as a place to document the
files we place in `./test/data/`, since most of the files do not allow
for comments.
The NEWS file currently uses non-markdown indentation, but uses the
markdown file-format. Since both GitHub-Repository-Viewer and the
Release Pages can display proper markdown, lets just re-indent the file
according to the markdown specs.
While at it, also switch out the list-indicator in front of the
timestamp to a proper hyphen, to avoid it being rendered as list.
Introduce a third test-group called `src` alongside `mod` and `run.
This will contain tests that run against the source code of osbuild.
This initial commit introduces `test/src/test_pylint.py` which will run
the python linter against all our sources.
Add convenience targets to `Makefile` which can run common sets of
tests. For now, add a target for pylint, module-unittests,
pipeline-runtime-tests, as well as all tests.
Currently, it is quite cumbersome to run a reasonable test-setup
locally. Pylint invokation is rather complex, the unittests and runtime
tests in ./test are mixed, and not all tests in ./test can necessarily
be run from a development system.
This commit prepares for a simpler setup:
* Add `make test-pylint` to run pylint as it is run by CI.
* Add `make test-module` to run all module-unittests. This is meant to
be fast (preferably close to instant) and easy to run during
development to do a short check whether there are obvious typos or
other errors in local changes.
If we can keep these tests to machine-local requirements, if we
avoid any sleeps or heavy computations, then this will remain a
convenient test-suite to run locally without having to wait for
30min. In other words: We should be able to keep this under 10s (and
for the long term under 1min) easily.
* Add `make test-runtime` to run all osbuild pipeline executions. This
is not meant to be fast, but thorough. This will require external
sources (preferably limited to a suitable container image with
everything embedded). This will very likely not be run during
development, but rather by the CI.
* Add `make test-all` to run all tests. Very handy for shy people when
the chance of embarrassing copy-paste mistakes is too high to push
publicly.
Additionally to these new targets, this PR introduces 2 new directories
in ./test: ./test/mod/ and ./test/run/
These are meant as equivalent to `test-module` and `test-runtime`. The
reason is that preferably we stick to the auto-discovery of `unittest`
to enumerate tests, rather than enrolling our own or having to enumerate
them explicitly somewhere.
However, we need some way to tell `unittest` which test belongs into
which group. The easiest setup is likely to just use sub-directories.
Note that `test-all` picks all tests independently of where they are
put, even if they are in further different sub-modules under ./test.
For now, no tests are moved into the new directories. I expect this to
take a bit, since there are several out-standing PRs that modify ./test.
I intend to do the final move once we agreed on this and we synchronized
our test-modifications.
Comments must not be indented in makefiles, otherwise they might end up
being interpreted as rules. Simply drop the indentation and move it into
the comments itself.
If the local checkout has no git tags, the contributor list cannot be
generated. Just omit the information in that case. We can try to be
more clever here, but lets fix this first. I could not come up with any
reasonable solution to improve on this easily. Hence, the source
includes a comment explaining that you might just have to fetch the git
tags first.
This extends on our model how we do releases. It introduces `NEWS.md`
as the authoritative source of our release-notes. It is pre-populated
with the release-notes from the previous 'v9' release, and contains a
suggestion for the upcoming 'v10'.
Furthermore, this adds `make release` as a simple checklist target that
contains instructions how to create a new release. Note that it is a
passive make-target which has no side-effects at all. It only prints
release information.
With this in place, we can drop `RELEASE.md`, as all information is now
combined in `make release`.
This improves `make man` in the following ways:
* The recently added `osbuild-manifest.5` man-page is now generated
as well.
* The target now honors `SRCDIR` and `BUILDDIR` variables.
* Any newly added man-page is now automatically picked up and
generated as well.
* The output directory structure now mirrors the input directory
structure.
This documents the structure of `Makefile` as well as its supported
targets. Furthermore, we add support for SRCDIR and BUILDDIR so we can
use this makefile to ultimately deploy generated documentation to our
website and more.
This commit does not convert all of the makefile to honor SRCDIR. It
merely sets up the infrastructure to support SRCDIR. Follow-ups will
improve the different targets.
It is not used anywhere and might be confusing to newcomers, because it
only contains the (private) osbuild library, without the command line
tool and runners/stages/assemblers.
Prior to this patch, `make rpm` would produce rpms that have the latest
tag as their versions. This was confusing, because one could never know
which contents are in a locally built rpm.
Change this so that the is version always based on the commit hash of
HEAD. This is easy: the golang macros read a `%commit` macro when it
exists and do this for us.
To simplify more, only define `%_topdir` to ./rpmbuild and use
rpmbuild's known directory structure (SPEC, SOURCES, RPMS, ...)
otherwise, to make it easier to find build results.
Build the specfile, tarball, source rpms, and rpms with `make rpm`,
without separate sub-targets. We can reintroduce them if they're needed
somewhere.
