Split out the part of `api.API` that is responsible for providing
the server infrastructure for the API; i.e. setting up the server
and the corresponding context manager and asynchronous event
handling. This leaves `API` itself which just the implementation
of the high level protocol and makes the API-server part re-usable.
NB: pylint, for some reason, confuses `API` and `BaseAPI`, like in
`test_monitor`. Annotate that accordingly.
Create small test cases that check the execution of Stages and
Assembler. This ensure that path handling, the sandbox, as well
as basic result reporting works as expected.
Instead of using string interpolation and concatenation to build
file system paths, use `os.path.join` or directly the constructor
for `pathlib.Path`, which can take path segments.
Create a new monitor that records all the invocations of the
monitoring (virtual) functions and use that to check that when
running (i.e. building) a pipeline all of them are executed
the excepted number of times (and with the correct arguments).
Add a basic test that will set up an 'API' endpoint, then spawn a
child process that uses that 'API' endpoint to setup its stdio in
very much the same way as runners do. This is used to verify that
the API itself works properly as well as the new LogMonitor class
by comparing the inputs and outputs.
#471 extends the assembler test suite to also test xfs and btrfs filesystems
in raw and qemu assemblers. However, this change leads to long running times
of this suite.
The running time of these test consist of 3 main steps:
1) Building the build pipeline
2) Building the stages
3) Running the assembler
There are two optimization approaches:
1) Caching
OSBuild supports caching, therefore it's possible to cache results of first
two steps.
2) Minimizing the operating system tree
Assemblers don't care about the image contents. Therefore, it's possible
to create just a small tree which would be used to test the assemblers.
This should lead to speed up in the step 2 (smaller tree should be built
quicker) and in step 3 (big part of assembling is just copying files over
to the image).
This commit implements the second approach. A new test manifest is now added,
which just installs the filesystem package and its dependencies and this tree
is then labeled. This solution was chosen, so that the assemblers get
something that looks as a proper filesystem tree but also can be built pretty
quickly.
Before this change, the test_rawfs method with #471 merged ran for 842 seconds.
After this change, it ran for 391 seconds.
Add the ability to mask services, which is done e.g. when building
installers. See systemctl(1) for more information about masked
services.
Modify the existing stage test to include a test for it.
Add a simple check for the selinux check by building the f32-base
image with an added selinux stage. Use the options from a test
json file and verify the labels against a set of labels given in
the aforementioned test file.
Only generate stage tests for sub-directories in stages_tests
that contain a diff.json. This should allow us to have specialized
stage tests that don't use the current {a, b}.json & diff.json
pattern.
Verify the rpm-ostree.input hash is set correctly for the repository
itself as well. This will in turn also verify that the repository
is existent and can be accessed.
Install rpm-ostree in the (mpp-)f32-build.json build manifest, so
ostree commits can be created using the same build root. Update
all affected manifests.
Using the network block device (nbd) kernel module to test all
the non-raw image formats often caused tests to fail due to nbd
not being stable itself (see below).
Instead convert non-raw images to the raw format via qemu-img
convert and mount those with loop-back devices. All the testing
code itself stays the same.
Example nbd error messages:
kernel: block nbd15: NBD_DISCONNECT
kernel: block nbd15: Disconnected due to user request.
kernel: print_req_error: 89 callbacks suppressed
kernel: blk_update_request: I/O error, dev nbd15, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
kernel: buffer_io_error: 134 callbacks suppressed
kernel: Buffer I/O error on dev nbd15, logical block 0, async page read
kernel: blk_update_request: I/O error, dev nbd15, sector 1 op 0x0:(READ) flags 0x0 phys_seg 7 prio class 0
Split up the partition table test into reading the partition table
and then asserting it has the correct entries. Prepares the usage
of the partition information later.
The `capture_output` argument for subprocess.run was added in 3.7,
but want to support 3.6 as well. Change all the usages of it with
`stdout=subprocess.PIPE` that will have the same effect, at least
for stdout.
This replaces the round-robin mirror at fedoraproject.org, as that was
proving to be quite unreliable.
This is a short-term fix before add metalink support.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Move the last remaining test into the correct subdir. With this done,
all our tests run in one of the 3 groups:
* `make test-src`
Run tests against the source-code, including linters.
* `make test-mod`
Run unit-tests on the individual python modules. This needs no
special permissions (unless noted in each test) or runtime
environments. It is meant to be fast and easy to run in all
circumstances.
* `make test-run`
Run tests that execute the osbuild pipeline. This requires
superuser privileges and will likely take a while. Furthermore,
this might produce large artifacts.
Move the stage-tests over to the new test-infrastructure. This moves
the test invocation into `./test/run/test_stages.py`, so it is invoked
as part of the runtime-tests. Secondly, the test-data is stored in
./test/data/stages/ so the path is relative to
TestBase.locate_test_data().
While at it, this also drops the dynamic class modifications and instead
uses subTest(). This simplifies the code quite a bit and avoids
dynamically creating python code.
Move the `test_osbuild.py` test into the module-test directory. This
test contains just a bunch of basic functionality tests for a selection
of osbuild modules. Hence, it can be run together with the other module
tests.
Move `test_objectstore` into the module-level tests. This allows us to
run it as part of `make test-module.
Make sure to properly guard it as root-only module.
This moves the `sources_tests` into ./test/data/sources/ and makes the
entire test use `locate_test_data()` to get relative paths for their
accesses.
This further improves our test cases to support running from
installments rather than local checkouts. We need access to ./test/data
guarded, so we can install packages and still have the tests access the
correct paths.
This also adjusts the HTTP-Server we use in the test to serve data
relative to a path it is handed. I now chose `./test/data`, which will
easily allow us to re-use the same HTTP-Server in the future for other
tests that require it.
We want to extend our base-class to support extensions to
unittest.TestCase, so make sure we inherit from it.
Adjust all callers to no longer inherit from TestCase, since this is now
done automatically by TestBase.
Move the `tree-diff` tool into ./tools, which is our new place for tools
used by the test-suite or during development.
The only hard-coded user is the TestBase, so fix its path to the tool
so the test-suite will continue to find it.
Add a new trivial runtime-test which simply runs a no-op pipeline. This
is a fast, trivial test that simply verifies osbuild is properly setup
and accessible.
Remove the explicit no-op test from the CI, now that the test-suite has
it as well.
Recently downloading rpms from the fau.de mirror is failing quite
often with timeouts. Replace its only usage in the rpm stages
test with the kernel.org mirror.
We currently run pylint on all files we find in the checkout. This is
particularly annoying when we have osbuild-stores in the checkout, which
then contain lots of python files in their stored trees.
Change the pylint test to use `git ls-tree` to find all files in the
index and then only run pylint against the subset that we are interested
in.
Convert our custom code to list modules to the new ModuleInfo
method list_modules_for_class that does the same thing. This
is then indeed also testing that new function.
This was superseded by test_osbuild's test_moduleinfo. It also
seems to be non-functional do to assuming `properties` in all
the STAGE_OPTS. Removing this.
The are converging on a nomenclature where the sum of Stages,
Assemblers, Sources (and future entities like those) together
are called 'Modules'.
Thus rename StageInfo to ModuleInfo and the corresponding
variables and methods.
Using `[]` as default value for arguments makes `pylint` complain. The
reason is that it creates an array statically at the time the function
is parsed, rather than dynamically on invocation of the function. This
means, when you append to this array, you change the global instance and
every further invocation of that function works on this modified array.
While our use-cases are safe, this is indeed a common pitfall. Lets
avoid using this and resort to `None` instead.
This silences a lot of warnings from pylint about "dangerous use of []".
We use comments in all other tests, rather than doc-strings. Convert the
os-release test to do the same. If we wanted doc-strings, we can convert
all tests over. This commit just tries to keep the tests in-sync.
Note that doc-strings cause `unittest` to print the doc-strings to
stdout during test-execution, making it overly verbose (especially for
multiline docs). By converting it to comments, this behavior is
suppressed.
This adds F32 manifests in ./test/data/. To avoid magically deducing the
package list out of the void, this adds a ManifestPreProcessor (MPP)
called `./tools/mpp-depsolve.py`. What this does is it takes a manifest
on stdin, modifies it, and produces a manifest on stdout.
The `mpp-depsolve.py` preprocessor takes a manifest and modifies all the
`org.osbuild.rpm` stages. It parses a new option to that stage called
`mpp-depsolve`, which contains a package-list, a repo-list, and dnf
metadata. It then drops this `mpp-depsolve` option (since it would be an
invalid manifest otherwise), depsolves the packages, inserts a proper
"packages" option as well as appends the correct paths to the sources
entry.
With this in place, this adds `mpp-f32-base.json` and
`mpp-f32-build.json` in ./test/data/manifests/. These will then be used
as base F32 manifests for our test-suite.
Lastly, this adds `./test/data/README.md` as a place to document the
files we place in `./test/data/`, since most of the files do not allow
for comments.
The `tree-diff` tool currently requires access to our local checkout,
since we do not install the tool. Provide accessors in `TestBase` so we
do not hard-code the path everywhere.
Add a new OSBuild class to `./test/test.py`. This class is an extension
of `./test/osbuildtest.py`, but no longer requires the `output_id` and
`tree_id` identifiers of osbuild.
Furthermore, this new executor uses context-managers to make sure any
temporary object is only accessed for a contained time-frame.
