Extend the `org.osbuild.systemd` stage to create drop-in configuration
files for Systemd `.service` units under `/usr/lib/systemd/system`.
Currently only the `Environment` option in the `Service` section can be
configured.
Update the `org.osbuild.systemd` stage test case to create drop-in
configuration `10-rh-enable-for-ec2.conf` for `nm-cloud-setup.service`
unit, as used in RHEL AMI images.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Extend the `org.osbuild.rhsm` stage to configure selected options in the
subscription-manager configuration (in `/etc/rhsm/rhsm.conf`). It is
possible to set only values currently set in RHEL AMI images,
specifically:
- `manage_repos` option in `rhsm` section
- `auto_registration` option in `rhsmcertd` section
Ensure that the stage does not "touch" any configuration files, unless
it actually changes them. This prevents changing the file modification
time.
Update the `org.osbuild.rhsm` stage test case to set the additional
configuration options.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add a new manifest that creates an ostree commit, deploys that,
creates a raw image and copies the deployment into it. The
resulting artefact is a bootlabel qcow2 image.
Add the ability to only read a sub-tree of a tree via `Object.read_at`.
Expose the functionality via the `Store{Server,Client}.read_tree_at`.
Extend the tests to check this new functionality.
Often, a message is being sent and followed by a call to `recv`
to wait for a reply. Create a simple helper `send_and_recv` that
does both in one method.
Add a simple check for that helper to the tests.
Add a new constructor method that allows creating a `Socket` from
an existing file-descriptor of a socket. This might be need when
the socket was passed to a child process.
Add a simple test for the new constructor method.
Add a new constructor method, `Socket.new_pair`, to create a pair
of connected sockets (via `socketpair`) and wrap both sides via
`jsoncomm.Socket`.
Add a simple test to check it.
Move from using 'zram' to 'zram-generator-defaults' in the ostree bootiso
testing manifest. More information is available in Fedora 33 Change
document [1].
Add org.osbuild.kernel-cmdline stage to fedora-boot.json manifest
because of change in how grub handles the kernel command line arguments
[2].
GRUB2 Stage 2 checksums in assemblers test are updated. The change have
been verified by building the fedora-boot.json manifest with each checked
filesystem and booting the image in QEMU with legacy mode.
[1] https://fedoraproject.org/wiki/Changes/SwapOnZRAM
[2] https://github.com/osbuild/osbuild-composer/pull/982#issuecomment-697356929
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The previous version covered too few use cases, more specifically a
single subscription. That is of course not the case for many hosts, so
osbuild needs to understand subscriptions.
When running org.osbuild.curl source, read the
/etc/yum.repos.d/redhat.repo file and load the system subscriptions from
there. While processing each url, guess which subscription is tied to
the url and use the CA certificate, client certificate, and client key
associated with this subscription. It must be done this way because the
depsolving and fetching of RPMs may be performed on different hosts and
the subscription credentials are different in such case.
More detailed description of why this approach was chosen is available
in osbuild-composer git: https://github.com/osbuild/osbuild-composer/pull/1405
Test that `checksum.verify_file` works correctly, which internally
uses the only other utility function `checksum.hexdigest_file`.
Check all algorithms currently supported by the `org.osbuild.curl`
source.
Add basic checks for the ostree source, which includes a successful
pull of a commit, an empty source entry and one where the specified
commit is non-existant. For this create a simple commit in a ostree
repo is checked in. The commit was created via:
mkdir "/tmp/data"
echo "Hello World" > /tmp/data/hello.txt
ostree init --repo test/data/sources/org.osbuild.ostree/data/repo \
--mode=archive
ostree commit --repo test/data/sources/org.osbuild.ostree/data/ \
--branch "test/ostree" /tmp/data \
--timestamp="1995-05-13 12:34:56 +0000"
This should give an commit with the following commit id:
d6243b0d0ca3dc2aaef2e0eb3e9f1f4836512c2921007f124b285f7c466464d8
Parameterize `test_sources` via `pytest.mark.parameterize`, so that
now the product of source and test cases for that source is visible
to pytest and thus also the caller.
Instead of testing the legacy `SourcesServer` and `sources.get`,
test the `Source.download` method, which is the new and exclusive
way how sources are used in osbuild. For this, the complete info
for a specific source is now included the specific test case.
For the `org.osbuild.curl` source this means that the respective
information is moved from `sources.json`, which is then unused
and thus removed. The test case that checks for an unknown
checksum is also removed because `Source.download` just fetches
everything instead of a subset.
Use `pytest`, which is a dependency of osbuild since a few versions,
instead of `unittest` for the source tests. The main reason is to be
able to use `pytest.mark.parameterize` in the near future to easily
generate a test matrix.
In order to have a more stable package metadata representation,
sort the generated metadata by name. Adapt the tests' metadata
file to reflect that change.
When the expected metadata does not match the produced metadata,
write the latter to `/tmp` for closer inspection; additionally
this should help update the metadata file in case the changes
are expected.
This is a partial revert of commit d584a1e, which converted the
dynamically generate stage tests to sub tests. The problem with
the latter is that they can't be individually run, which makes
testing changes to specific stages cumbersome to develop.
Therefore switch back to a model where the stage tests are
dynamically generated via a class decorator.
In the output test, check that for a successful pipeline run all
the elements are present: main result, assembler result, stage
result.
NB: Build result is hard to test because we would need to actually
build a valid build root.
Extend the current ostree container test case to use the new
bootiso manifest to build a bootable iso. This is done in the
existing container test case in order to share the build root
and ostree commit results and speed up testing. In the future,
the test infrastructure should be extended so that the cache
can be optionally shared between test cases.
This new manifests creates an boot iso which contains an ostree
commit that is build in the same manifest. The boot iso can then
be installed via the ISO.
This is to test the functionality of the new `saved_entry` grub2
config file. Ideally an integration test would install a new non-
default kernel and check that it does not get selected. Something
for the future.
In addition to the required base layer, provided via the the input
of the same name, the oci-archive stage now accepts up to nine
additional layers that get added on top of each other, sorted in
ascending order, i.e. `layer.1` to `layer.9`.
Adapt the `fedora-ostree-container` example manifest so that the
ostree commit is now in a separate layer, which makes it possible
to share the base layer between different commits container.
Add a new basic test for the dracut stage. It uses a osbuild
pipeline to build an initrd and inspects it via the initrd.py
module. The content is compared to a reference located in the
same directory as the pipeline (test/data/stages/dracut/).
Add a new module that contains a utilities to inspect an initrd.
It contains a `Initrd` class that loads a given initrd and can
then be used get the list of files via Initrd.filelist, the dracut
modules via `Initrd.modules` and the kernel modules via the
`Initrd.kmods` properties. Another top level function `read_initrd`
returns a dict that contains the name of the initrd as key and as
value another dictionary with `modules`, `kmods` as well as the
`compression` & `early_cpio` metadata.
The initrd.py can also use as a script that will print the dict
from `read_initrd` as JSON.
The implementation is a heavily based on dracut's `lsinitrd` bash
script and additionally contains a python port of the skipcpio.c
utility.
Change the test manifests that use containers to not include
docs when installing. Also don't install docs in the build root
for those manifests. Since the fedora-ostree-container.mpp is
being built in CI, this also tests that the new option.
The `org.osbuild.files` source provides files, but might in the
future not be the only one that does. Therefore rename it to
match the internal tool that is being used to fetch the files.
This is done for most other osbuild modules that target tools.
The format v1 loader is adapted to make this change transparent
for users of the v1 format, so we are backwards compatible.
Change the MPP depsolve preprocessor so that for format v2 based
manifest `org.osbuild.curl` source is used. Also rename the
corresponding source test. Adapt the format v2 mod test to use
the curl source.
Add a v2 manifest to builds an ostree commit, then builds a small
container with a webserver and puts that commit into it.
Start a new test suit where this manifest is built and then checked
that the artifact exists. In the future the artifact should also be
inspected and run by a container engine and the commit be pulled
via ostree.
Add a basic check to verify that loading and then describing the
pipeline results in the same description that was put in. This
test is esp. valuable because it checks the runner mapping and
name, id mappings.
Add a new test to check that validation works for the basic test
pipeline. This needs to be extended in the future to check that
invalid data is being caught properly, but it is a start.
Add a simple noop pipeline, with a noop stage and a noop input,
which all in all does nothing much; but it will validate, load
exectue the pipeline, stage and inputs. So maybe not really
"nothing" in the strictest sense.
Add an example that builds a small container, based on the minimal
container fedora kickstart (see [1, 2]). There is a deviation
because osbuild currently does not support removing
random files like it is done in [2].
The build pipeline "build" in f32-build-v2.json) is a port of the
existing build pipeline in f32-build.json.
This manifest is currently not used in any automated tests (yet),
but is there as a showcase for the new format. It does however
test the format version 2 support for mpp, since that is called
in CI.
[1] https://pagure.io/fedora-kickstarts/blob/main/f/fedora-container-common.ks
[2] https://pagure.io/fedora-kickstarts/blob/main/f/fedora-container-base-minimal.ks
Change the `ModuleInfo.schema` propertly into a `get_schema`
method call. This is in preparation to allow for different
schemata versions to be supported.
Commit d028ea5b16 introduced bug when introducing the `store`
argument to `Stage.run`, instead of passing `var=var`, i.e.
`var` is being passed as keyword argument, it is now being
passed as a positional one. Since the `path=/run/osbuild`
keyword argument comes before the `var=/var/tmp` argument,
`var` is now being passed as `path` instead of var.
Since `var` is always being passed in throughout the entire
codebase, make it a positional argument, and move it before
`path`.
Adapt the tests to pass `var` as positional argument.
