Some platforms like the TI AM62 require a particular FAT geometry for
their CPU to read the file system (and thus the bootloader). Failing
that the CPU will simply not boot and keep looking for a bootloader.
Let's add some options to enforce a particular filesystem geometry
through the -g option of mkfs.fat.
Signed-off-by: Maxime Ripard <mripard@kernel.org>
A simple test to make sure that our btrfs stages work.
Updating the testing container is needed, because the old version
didn't have btrfs-progs in it.
This adjustment allows the definition of the mark with the RPMs and runs
DNF after installing the RPMs to put the proper markings in the DNF
state database. See #455.
This ensures that packages don't get removed during `autoremove` leading
to broken systems.
This commit adds support for a new field in tree diffs fed to the stage tests.
This is useful when you care that a directory is added, but you don't care
about its content.
Since this is useful only for the expected tree diff, it's supported only
there. The actual tree diff doesn't support the new field.
Refactor unit test implementation for `parted`, `sfdisk` and `sgdisk`
stages by extracting the common parts into a helper function. Each stage
now implements only its own function for filtering `sfdisk --json`
output and calls the common helper function.
In addition, flip the oder when comparing the expected and actual output
from `sfdisk --json`, to make it more easier to comprehend.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The `sfdisk` tool on RHEL-8 does not include the `sectorsize` in its
output when we are testing the `sgdisk` stage. As a result, the test
case fails, because the expected and actual output differs. Modify the
test to delete the `sectorsize` key from the expected output if it is
not present in the actual output.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The `sfdisk` tool on RHEL-8 does not include the `sectorsize` in its
output when we are testing the `parted` stage. As a result, the test
case fails, because the expected and actual output differs. Modify the
test to delete the `sectorsize` key from the expected output if it is
not present in the actual output.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
LOOP_CONFIGURE allows to atomically configure the decive when opening
it. This avoid the possibility of a race condition where between set_fd
and set_status some operations are already accepted by the loopback
device. See https://lwn.net/Articles/820408/
This feature was included in the linux kernel 5.8 however it is safe to
not include any kind of fallback to the previous method as @obudai
points out that:
LOOP_CONFIGURE was backported into RHEL 8 kernel in RHEL 8.4 as a part
of https://bugzilla.redhat.com/show_bug.cgi?id=1881760 (block layer:
update to upstream v5.8).
Since RHEL 8.4 is currently the oldest supported release that we support
running osbuild on, it might be just fine implementing this without the
fallback.
From a centos stream 8 container:
kernel-4.18.0-448.el8.x86_64
- loop: Fix missing discard support when using LOOP_CONFIGURE (Ming Lei) [1997338]
- [block] loop: Set correct device size when using LOOP_CONFIGURE (Ming Lei) [1881760]
- [block] loop: unset GENHD_FL_NO_PART_SCAN on LOOP_CONFIGURE (Ming Lei) [1881760]
- [block] loop: Add LOOP_CONFIGURE ioctl (Ming Lei) [1881760]
Add conditional skip to some tests that depend on rpm-ostree
availability, but were not checking for its presence. These tests would
previously fail if rpm-ostree is not available. They will be skipped
now.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The `directory` argument has been added only since Python 3.7, which
breaks the unit test on Python 3.6.
Reimplement the intended behavior by overriding the `translate_path()`
method, which takes the `directory` value into account on newer Python
versions.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Do not specify the default value for 'expected_size' argument in
assertImageFile() function declaration. Previously, it was set to
`None`, which was never taken into account. Moreover, all callers of the
function always provide an explicit value, so the default was never
really used.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add a new optional pytest CLI argument `--unsupported-fs` allowing to
specify file-systems which should be treated as unsupported in the
platform where running tests. Any test cases dependent on such
file-system support will be sipped.
This will allow to run unit tests and selectively skipping test cases
for unsupported file-systems.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Port assembler tests from unittest to pytest. In addition, use
parametrized tests when testing various filesystems and various
combinations.
This is important to be able to selectively skip the test for if a
specific filesystem is not supported by the kernel (e.g. btrfs is not
supported on RHEL). Skipping a unittest subtest is not possible, which
is the motivation to move away from it and use only pytest.
Test output is now also much nicer for parametrized test cases.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Previously, the unit test depended on osbuild modules being installed on
the system. As a result, this made the test not work in CI where we do
not install osbuild when running unit tests. In addition, the stage
executed by the unit test would use different version of osbuild
internals than the version that is being tests, which could result in
issues or not testing the intended code.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The directory does not exist when the unit test is run in CI. Handle
this case by ensuring that parent directories are created as needed.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Before we could only ask OSBuild to mount a device as readonly. But
devices can have more mount options than this. Supporting more options
is necessary for the new version of image-info that is using OSBuild's
internals in order to mount the image it wants to work on. Otherwise,
for instance, some umasks aren't applied properly and we can get
differences in rpm-verify results, thus corrupting the DB.
Mount is now accepting:
* readonly
* uid
* gid
* umask
* shortname
This modification will allow a user to ask to mount the system as read
only for instance. Which would be super useful for image-info who is
progressively using more of OSbuild internals to mount partitions.
Convert the manifest to use version 2 format. Version 1 is really
not used much anymore because composer was completely ported to
using v2. Welcome to the future, ostree commit.
Can be used to create partition tables via GPT laypout via `sgdisk(8)`.
The schema of `partitions` is intentionally kept identical to the one
in `org.osbuild.sfdisk`.
Add corresponding tests.
The unit test consists of a manifest creating an empty file, which
is then converted to various formats using the `org.osbuild.qemu` stage
in separate pipelines.
The unit test then builds and exports each pipeline with qemu stage and
inspects the resulting image file using `qemu-img info` command and checks
that the test data specified in `checks.json` is a subset of the data
returned by the command.
During the rework done in commit "use and require explicit exports"
with commit id 7ae4a7e78, the test got overlooked. Add an empty
list of checkpoints to the `obs.compile` invocation as to actually
trigger the osbuild invocation.
Reported-By: Thomas Lavocat <tlavocat@redhat.com>
OSTree tests, especially the fedora-ostree-image one, will soon
need the tight integration with the host for LVM2/LUKS support.
This we cannot run them in github action containers. Move them
to Schutzbot.
Explicitly install the new sub-package until composer gains the
needed requirement.
Instead of checkpointing the tree and then accessing the generated
tree inside the store via the `map_object` function we not just
export the tree and use that. This better hides the internals of
the store and also allows us to activate on-demand building that
does not rely on checkpoints being implicitly built like exports.
Require all the tests that compile a manifest to either specify
checkpoints or exports. Convert all the tests that were relying
on implicit exports with v1 manifests to use explicit exports.
Introduce a new class to manage devices, `DeviceManger` and move the
code to open devices from the `Device` here. The main insight of why
the logic should be place here is that certain information is needed
to open the devices, independently of specific type: the path to the
device node directory, `devpath`, the actual `tree` and the service
manager instance to start the actual service. Instead of passing all
this information again and again to the `Device` class, we now have
a specialized (service) manager class for devices that has all the
needed information all the time. Additionally, the special handling
of parent devices is moved from the pipeline to the service manager,
which is where it belongs.
This will make even more sense for mounts, where the `DeviceManger`
can then be passed to access the individual devices.
Port the test to use the `DeviceManager`.
Include a new test that writes a partition table to a disk and
then reads it back via `sfdisk` compares it against an layout
that was generated.
NB: This test needs `sfdisk` with `--json` support on the use host.
Port sources to also use the host services infrastructure that is
used by inputs, devices and mounts. Sources are a bit different
from the other services that they don't run for the duration of
the stage but are run before anything is built. By using the same
infrastructure we re-use the process management and inter process
communcation. Additionally, this will forward all messages from
sources to the existing monitoring framework.
Adapt all existing sources and tests.
Include a simple test for the `tar` stage that does basic checks,
i.e. that a tarball was successfully created, but also checks
that we do not create entries with a `./` prefix.
Convert the test to use `pytest` and split out the individual
tests. The temp-directory fixture has the session scope so
that checkpoints can be shared between the individual tests.
Remove the dependency on unittest for the `OSBuild` class which
used the `unittest` instance only for `assertEqual`, which can
easily also be done via a plain `assert`.
Move from using 'zram' to 'zram-generator-defaults' in the ostree bootiso
testing manifest. More information is available in Fedora 33 Change
document [1].
Add org.osbuild.kernel-cmdline stage to fedora-boot.json manifest
because of change in how grub handles the kernel command line arguments
[2].
GRUB2 Stage 2 checksums in assemblers test are updated. The change have
been verified by building the fedora-boot.json manifest with each checked
filesystem and booting the image in QEMU with legacy mode.
[1] https://fedoraproject.org/wiki/Changes/SwapOnZRAM
[2] https://github.com/osbuild/osbuild-composer/pull/982#issuecomment-697356929
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Parameterize `test_sources` via `pytest.mark.parameterize`, so that
now the product of source and test cases for that source is visible
to pytest and thus also the caller.
Instead of testing the legacy `SourcesServer` and `sources.get`,
test the `Source.download` method, which is the new and exclusive
way how sources are used in osbuild. For this, the complete info
for a specific source is now included the specific test case.
For the `org.osbuild.curl` source this means that the respective
information is moved from `sources.json`, which is then unused
and thus removed. The test case that checks for an unknown
checksum is also removed because `Source.download` just fetches
everything instead of a subset.
