Mark '$(TEST_MANIFESTS_GEN)' target as .PHONY.
Currently the `test-data` make target does not always trigger
regenerating of manifests used for testing various osbuild parts,
although it is marked as .PHONY. The reason that its dependency
'$(TEST_MANIFESTS_GEN)' is not marked as .PHONY and therefore it is not
run if the files already exist in repository.
Due to the above reason, CI is actually running `make --always-make
test-data` to always regenerate manifests.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Stage tests (test/run/test_stage.py) currently use hard-coded pipelines
from `test/data/stages`. These could benefit from the mpp tools to make
them easier to understand and update to newer versions of Fedora.
Future commits will do so.
This simplifies the `test-data` rules in `Makefile` considerably. Also,
it allows adding `*.mpp.json` files in other directories without needing
to copy rules. (make's pattern-based rules only allow a single `%`).
Adjust test/data/README.md accordingly.
#471 extends the assembler test suite to also test xfs and btrfs filesystems
in raw and qemu assemblers. However, this change leads to long running times
of this suite.
The running time of these test consist of 3 main steps:
1) Building the build pipeline
2) Building the stages
3) Running the assembler
There are two optimization approaches:
1) Caching
OSBuild supports caching, therefore it's possible to cache results of first
two steps.
2) Minimizing the operating system tree
Assemblers don't care about the image contents. Therefore, it's possible
to create just a small tree which would be used to test the assemblers.
This should lead to speed up in the step 2 (smaller tree should be built
quicker) and in step 3 (big part of assembling is just copying files over
to the image).
This commit implements the second approach. A new test manifest is now added,
which just installs the filesystem package and its dependencies and this tree
is then labeled. This solution was chosen, so that the assemblers get
something that looks as a proper filesystem tree but also can be built pretty
quickly.
Before this change, the test_rawfs method with #471 merged ran for 842 seconds.
After this change, it ran for 391 seconds.
Add the scaffolding and various targets to run a coverity analysis,
including downloading the tool, running the "build", archiving
the results and submitting it to coverity for analysis.
The downloading and submitting need COVERITY_{EMAIL, TOKEN} to be
defined in the environment.
Co-authored-by: David Rheinsberg <david.rheinsberg@gmail.com>
Align the makefile targets with the test-targets (`module` -> `mod`,
etc.). This way, we have consistent names everywhere.
While at it, move the `make test-run` invocation closer to the others.
This adds F32 manifests in ./test/data/. To avoid magically deducing the
package list out of the void, this adds a ManifestPreProcessor (MPP)
called `./tools/mpp-depsolve.py`. What this does is it takes a manifest
on stdin, modifies it, and produces a manifest on stdout.
The `mpp-depsolve.py` preprocessor takes a manifest and modifies all the
`org.osbuild.rpm` stages. It parses a new option to that stage called
`mpp-depsolve`, which contains a package-list, a repo-list, and dnf
metadata. It then drops this `mpp-depsolve` option (since it would be an
invalid manifest otherwise), depsolves the packages, inserts a proper
"packages" option as well as appends the correct paths to the sources
entry.
With this in place, this adds `mpp-f32-base.json` and
`mpp-f32-build.json` in ./test/data/manifests/. These will then be used
as base F32 manifests for our test-suite.
Lastly, this adds `./test/data/README.md` as a place to document the
files we place in `./test/data/`, since most of the files do not allow
for comments.
The NEWS file currently uses non-markdown indentation, but uses the
markdown file-format. Since both GitHub-Repository-Viewer and the
Release Pages can display proper markdown, lets just re-indent the file
according to the markdown specs.
While at it, also switch out the list-indicator in front of the
timestamp to a proper hyphen, to avoid it being rendered as list.
Introduce a third test-group called `src` alongside `mod` and `run.
This will contain tests that run against the source code of osbuild.
This initial commit introduces `test/src/test_pylint.py` which will run
the python linter against all our sources.
Add convenience targets to `Makefile` which can run common sets of
tests. For now, add a target for pylint, module-unittests,
pipeline-runtime-tests, as well as all tests.
Currently, it is quite cumbersome to run a reasonable test-setup
locally. Pylint invokation is rather complex, the unittests and runtime
tests in ./test are mixed, and not all tests in ./test can necessarily
be run from a development system.
This commit prepares for a simpler setup:
* Add `make test-pylint` to run pylint as it is run by CI.
* Add `make test-module` to run all module-unittests. This is meant to
be fast (preferably close to instant) and easy to run during
development to do a short check whether there are obvious typos or
other errors in local changes.
If we can keep these tests to machine-local requirements, if we
avoid any sleeps or heavy computations, then this will remain a
convenient test-suite to run locally without having to wait for
30min. In other words: We should be able to keep this under 10s (and
for the long term under 1min) easily.
* Add `make test-runtime` to run all osbuild pipeline executions. This
is not meant to be fast, but thorough. This will require external
sources (preferably limited to a suitable container image with
everything embedded). This will very likely not be run during
development, but rather by the CI.
* Add `make test-all` to run all tests. Very handy for shy people when
the chance of embarrassing copy-paste mistakes is too high to push
publicly.
Additionally to these new targets, this PR introduces 2 new directories
in ./test: ./test/mod/ and ./test/run/
These are meant as equivalent to `test-module` and `test-runtime`. The
reason is that preferably we stick to the auto-discovery of `unittest`
to enumerate tests, rather than enrolling our own or having to enumerate
them explicitly somewhere.
However, we need some way to tell `unittest` which test belongs into
which group. The easiest setup is likely to just use sub-directories.
Note that `test-all` picks all tests independently of where they are
put, even if they are in further different sub-modules under ./test.
For now, no tests are moved into the new directories. I expect this to
take a bit, since there are several out-standing PRs that modify ./test.
I intend to do the final move once we agreed on this and we synchronized
our test-modifications.
Comments must not be indented in makefiles, otherwise they might end up
being interpreted as rules. Simply drop the indentation and move it into
the comments itself.
If the local checkout has no git tags, the contributor list cannot be
generated. Just omit the information in that case. We can try to be
more clever here, but lets fix this first. I could not come up with any
reasonable solution to improve on this easily. Hence, the source
includes a comment explaining that you might just have to fetch the git
tags first.
This extends on our model how we do releases. It introduces `NEWS.md`
as the authoritative source of our release-notes. It is pre-populated
with the release-notes from the previous 'v9' release, and contains a
suggestion for the upcoming 'v10'.
Furthermore, this adds `make release` as a simple checklist target that
contains instructions how to create a new release. Note that it is a
passive make-target which has no side-effects at all. It only prints
release information.
With this in place, we can drop `RELEASE.md`, as all information is now
combined in `make release`.
This improves `make man` in the following ways:
* The recently added `osbuild-manifest.5` man-page is now generated
as well.
* The target now honors `SRCDIR` and `BUILDDIR` variables.
* Any newly added man-page is now automatically picked up and
generated as well.
* The output directory structure now mirrors the input directory
structure.
This documents the structure of `Makefile` as well as its supported
targets. Furthermore, we add support for SRCDIR and BUILDDIR so we can
use this makefile to ultimately deploy generated documentation to our
website and more.
This commit does not convert all of the makefile to honor SRCDIR. It
merely sets up the infrastructure to support SRCDIR. Follow-ups will
improve the different targets.
It is not used anywhere and might be confusing to newcomers, because it
only contains the (private) osbuild library, without the command line
tool and runners/stages/assemblers.
Prior to this patch, `make rpm` would produce rpms that have the latest
tag as their versions. This was confusing, because one could never know
which contents are in a locally built rpm.
Change this so that the is version always based on the commit hash of
HEAD. This is easy: the golang macros read a `%commit` macro when it
exists and do this for us.
To simplify more, only define `%_topdir` to ./rpmbuild and use
rpmbuild's known directory structure (SPEC, SOURCES, RPMS, ...)
otherwise, to make it easier to find build results.
Build the specfile, tarball, source rpms, and rpms with `make rpm`,
without separate sub-targets. We can reintroduce them if they're needed
somewhere.
We already have a makefile for maintenance work. Merge the
`bump-version.sh` script into `Makefile` as a new target so we avoid
placing scripts spread around the repository.
While at it, this commit also uses immediate assignment for
shell-evaluated make variables. This ensures the shell commands are
only executed once, and then are guaranteed to be of the same value for
the remainder of the execution.
This changes the name of the source tarball, but apart from that has
no effect as-is.
However, if the `commit` global is set, then the source tarball is
expected to be the standard git-archive at that commit, and the
release field is set to contain the date and the git sha. This is
useful for when we want to build rpms between releases.
This works analogously to how osbulid-composer currently works.
See <https://fedoraproject.org/wiki/Forge-hosted_projects_packaging_automation>
for details.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Building rpm directly on Ubuntu is somewhat possible but it's fragile and
it cannot cover all quirks of rpm build. This commit switches the rpm build
tests to using fedora containers. This isn't also the best solution but
imho it's much better than the previous one.
This is achieved using "jobs" from packit-as-a-service, more
specifically the propose_downstream job. Furthermore
sync_from_downstream job is configured to keep the spec file
synchronized and prevent merge conflicts for new releases.
Also a small change in Makefile was necessary as it does not reflect the
current state of the spec file in Fedora dist-git (tarball name is
different). The spec file itself is not modified in this commit, because
it will be synchronized automatically using Packit.
RPM packages are now kept in output directory after build so that we
know exactly which packages to copy to the test. The test directory now
contains special directory for RPMs. Fedora developer portal is
referenced from README file.
The repository now contains a Vagrantfile for running the testing script
against an RPM package created locally using `make rpm`. To run this
test use `make vagrant-test`. setup.py was also modified to adhere to
packaging guidelines and not to install system-level executables.
The lincense is now included in the Python package using the MANIFEST.in
file.
