Most modules do not support both schema versions. This is masked by
module type code in get_schema() in most cases, but really should not be
tested. This skips running check_moduleinfo if the module doesn't
support the version. eg. org.osbuild.librepo only supports v2.
This commit moves the joining of path fragements from f-strings
to pathlib and simplifies some of the map/filter/lambda expressions
into more standard list comprehensions.
Create a proper `ObjectStore.Object` to use as tree for the `run`
method of `Stage`, since that is what is also normally passed to
it from `Pipeline.run`. It prepares for a future where `Object`
is not just used as `os.PathLike`.
Use the new `Index.detect_runner` method that will give us the best
available runner for a requested one. To do so a new `pipeline.Runner`
class is introduced that stores the `meta.RunnerInfo` class for the
specific runner and the original name that was requested.
In the manifest loading and describing functions of the formats, use
`Index.detect_runner` to get the `RunnerInfo` for a requested runner
and then wrap it in a `pipeline.Runner` object, which is then passed
to the `Manifest.add_pipeline` method.
See also commit "meta: ability to auto-detect runner".
Adjust all test.
Newer warning from pylint, also consistent with how we do things
elsewhere. Note that this only applies to one file in the tests but
disabling it would be very weird for such a small fix.
New function that take a list of pipelines and return the list of
pipelines that need to be build, i.e. the pipelines and all their
dependencies that are not already present in the store.
Add corresponding test.
Change the `ModuleInfo.schema` propertly into a `get_schema`
method call. This is in preparation to allow for different
schemata versions to be supported.
Add a new `add_source` method that will add an individual `Source`
to a `Manifest` give its `ModuleInfo` and options. The dictionary
of source options in the manifest is replaced with a list of such
`Sources` and `add_source` will append to it. Adap the version 1
format code to use `add_source` and reconstruct the source options
from the list of source on `describe`.
Remove the `sources_options` constructor parameter for `Manifest`
and adapt all the source base for this.
Instead of using the `Assemblers` class to represent assemblers,
use the `Stage` class: The `Pipeline.add_assembler` method will
now instantiate and `Stage` instead of an `Assembler`. The tree
that the pipeline built is converted to an Input (while loading
the manifest description in `format/v1.py`) and all existing
assemblers are converted to use that input as the tree input.
The assembler run test is removed as the Assembler class itself
is not used (i.e. run) anymore.
Inputs are modules like Stages, Assemblers and Sources. Add them
as a new module klass to the various functions. Include them in
the schema test, so the schema of all inputs is validated.
Also sort the module classes alphabetically in the class mapping
and class list.
Add a new `info` property that holds the `meta.ModuleInfo` info
for the stage. This gives each instance of a stage access to
meta (or class) information about it, i.e. its schema, docs but,
more importantly, also its name and path to the executable.
Thefore the `name` property is coverted into a transient property
which access the `name` member of `info`.
Change the `formats/v1` load mechanism to carry a new `index`
argument which is used to load the `ModuleInfo` for each stage.
Adapt all tests to load the info as well when creating stages.
All tests and invocations of `add_stage` actually pass a valid
options dictionary. Thefore move the `options` args before
the `sources` arg and remove the default value (`None`).
Instead of having build pipelines nested within the pipeline it is
the build pipeline for, the nested structure is transferred into a
flat list of pipelines. As a result the recursion is gone and all
the pipelines and trees are build one after the other. This is now
possible since floating objects are kept alive by the store itself
and all trees that are being built are transparently via them.
The immediate result dictionary changed accordingly. To keep the
JSON output of osbuild the same, the result is now routed through
a format specific converter.
Additionally, the v1 format module gained a function to retrieve
the global tree_id and output_id. With the new models those global
ids will go away eventually and thus need to go through the format
specific code.
Instead of a pipeline, describe now takes a Manifest instance.
The reason is that a manifest fully describes the build, which
includes the sources. Now that the describe function takes the
manifest, the sources can be included as well.
Adapt the tests to refelect that change.
The 'Manifest' class represents what to build and the necessary
sources to do so. For now thus it is just a combination of the
pipeline the source options.
The description of a pipeline is format dependent and thus needs
to be located at the specific format module.
Temporarily remove two tests; they should be added back to a format
specific test suit.
Instead of having the pipeline and the source option as separate
arguments, the load function now takes the full manifest, which
has those two items combined.
Instead of importing the load, load_build functions into the osbuild
namespace and using it via that, use the load function via the module
that provides them, i.e. the formats.v1 module.
The validation of the manifest descritpion is eo ipso format
specific and thus belongs into the format specific module.
Adapt all usages throughout the codebase to directly use the
version 1 specific function.
Instead of hard-coding the use of the "org.osbuild.linux" runner,
use the new `osbuild.pipeline.detect_host_runner` function to
dynamically detect the runner for the host system. That should fix
the tests on RHEL systems, where python3 is by default not present
and even if it is manually installed, is an indirection via
alternatives (i.e. a link to /etc/alternatives), which must be
explicitly configured in the build root container for the host.
Change the default of libdir to /usr/lib/osbuild and
remove redundant logic. Additionally, change how the
python package is detected.
Instead of checking if libdir is None, check if
/usr/lib/osbuild is empty - i.e. if the user has specified
a different directory than the default.
Create small test cases that check the execution of Stages and
Assembler. This ensure that path handling, the sandbox, as well
as basic result reporting works as expected.
Move the `test_osbuild.py` test into the module-test directory. This
test contains just a bunch of basic functionality tests for a selection
of osbuild modules. Hence, it can be run together with the other module
tests.
