Also, upgrade the test to fedora-32 from https://osbuild.org/rpmrepo.
Changed a.json to not include a kernel-cmdline stage with empty options.
This breaks the test, because the osbuild run of b.json is relying on a
checkpoint of the result of a.json. If that's not present, the rpm
database (and other generated files) are marked as differing, because
the whole pipeline is generated again. This might have worked before,
because the previous package list happened to be deterministic.
Stage tests (test/run/test_stage.py) currently use hard-coded pipelines
from `test/data/stages`. These could benefit from the mpp tools to make
them easier to understand and update to newer versions of Fedora.
Future commits will do so.
Use dnf's per-user cache when `--dnf-cache` is not given. This speeds up
repeated invocations of `mpp-depsolve` considerably, without having to
specify a cache location.
Technically, a few extra "/"s don't matter, but in practice, it
sometimes does. In particular, rpmrepo returns `500 Internal Server
Error` when a path contains additional "/"s.
This simplifies the `test-data` rules in `Makefile` considerably. Also,
it allows adding `*.mpp.json` files in other directories without needing
to copy rules. (make's pattern-based rules only allow a single `%`).
Adjust test/data/README.md accordingly.
Create a cache directory with the scope of the stage test suit and
after each test is run cache the downloaded files for the `files`
source via the new `OSBuild.copy_source_data` method. Initialize
OSBuild with that cache directory, so previously downloaded files
get used between different stage tests.
Add a new method that will copy the downloaded data for a specified
source to a directory, creating a directory structure so that the
target directory can then be used to initialize the osbuild cache,
i.e. the target directory be used for the `cache_from` parameter in
the `test.OSBuild` constructor.
The main reason why this is done per source, and not for all sources,
is that not all source can be copied via a plain `cp` operation or
easily added to, in case that the `target` directory already contains
data. The `org.osbuild.ostree` source is such an example.
Wrap all calls to the various setup functions in the new exception
handler provided by `osbuild.api`. This will make sure that any
exception is properly printed to stderr, as well as communicated
to osbuild in a structured and machine readable way.
Use `traceback.print_tb()` to serialize the exceptions' backtrace.
The previously used expression `str(e.__traceback__)` will just
give `<traceback object at 0x…>`, which is not very helpful.
Add a test to check that the method name that raises the exception,
also called `exception`, is in the traceback.
When using `str(type(exception))` this ends up to be something like
`<class 'ValueError'>` for a `ValueError` exception. Get the vanilla
name of the exception type via `type(exception).__name__`.
Add a test to ensure that we encode this properly.
Rename the `API.exception` member to `API.error`, to make it more
generic, so it can also be used for other sort of errors in the
future. Also add a layer of additional structure with `type` and
`data` members so different types of errors apart. Currently only
`exception` is used.
Adapt the tests in test/mod/test_api.py to check for the new
structure and its content.
The `api.exception` method, which is internally also used by
`api.exception_handler` helper, calls `sys.exit` and this no
statement after a call to either method should be reached.
Add an assertion to make sure of that.
The osbuild-composer-tests package recently started to list its actual
dependencies, which include packages from EPEL. Enable EPEL in
deploy.sh.
Based on this patch by Ondřej Budai <obudai@redhat.com>:
https://github.com/osbuild/osbuild-composer/pull/1022
Create a new api endpoint called exception, that communicates
exception backtraces separately back to osbuild, as opposed to
dumping them into the normal log. Additionally, add a corresponding
test to check that a call to api.exception correctly sets
API.exception.
Access metadata.api only after `api` has exited the context and
thus the event loop has stopped and all incoming messages, like
the one setting the metadata, have been processed.
See commit 803433fb62 for a lecture
about the internals and all the details involved.
Now that the `org.osbuild.linux` runner does not use `api.setup_stdio`
anymore, the output of the binary run from the BuildRoot must end up
in `BuildRoot.output`. Check for that.
Now that the BuildRoot is capable of capturing the output of the
runner and modules (stages, assemblers), there is no need for
using `api.setup_stdio`. Therefore, drop it from all runners and
replace `api.output` with `BuildRoot.output`, which will contain
the output if `api.setup_stdio` is not called from the runners.
Create a new CompletedBuild object that wraps and is very similar
to the subprocess.CompletedProcess, i.e. it has a process member
but also has shortcuts for returncode. Additionally, the output
of the process is not only forwarded to the monitor, but also
captured and then handed to CompletedBuild, so its output member
will actually contain the full build output. To be compatible
with the previously returned CompletedProcess, `stderr`, `stdout`
members exist on CompletedBuild that also return `output`.
Instead of reading the arguments from sys.stdin, which requires
that stdin is setup properly for that in the runner, use the new
api.arguments() method to directly fetch the arguments.
Also fix missing newlines between imports and methods to be more
PEP-8 complaint, where needed.
In case that bubblewrap fails to, e.g. because it fails to execute
the runner, it will print an error message to stderr. Currently,
this output is not capture and thus not logged. To fix that, the
`BuildRoot.run` method now takes a monitor object and will stream
stdout/stderr to the log via the monitor.
Instead of reading the arguments from sys.stdin, which requires
that stdin is setup properly for that in the runner, use the new
api.arguments() method to directly fetch the arguments.
Also fix missing newlines between imports and methods to be more
PEP-8 complaint, where needed.
Instead of reading the arguments from sys.stdin, which requires
that stdin is setup properly for that in the runner, use the new
api.arguments() method to directly fetch the arguments.
Also fix missing newlines between imports and methods to be more
PEP-8 complaint, where needed.
Simple check for the new server side method, `get-arguments`, and
client side counterpart, `api.arguments`, that compares that using
the later we get the supplied input (arguments) to API.
