Extract the opening of LVM LV devices from `discover_lvm()` to
`OSBuildDeviceManager` class as `open_lvm_lv()` method.
`open_lvm_lv()` returns the path to the opened device in the devpath set
in the underlying `DeviceManager`. The `org.osbuild.lvm2.lv`
implementation takes the responsibility for creating and managing
device nodes. This means that we don't need to be creating any device
nodes directly in `osbuild-image-info`, especially in the current
working directory. This was previously causing issues when inspecting
two images with different LVM layout in a sequence.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add a new class `OSBuildDeviceManager`, which wraps
`devices.DeviceManager`, so that we can consolidate all code that is
opening devices using osbuild, in it. As the fist step, move the
`loop_open()` function to the class.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Rename the function for naming consistency and always include the actual
error from `pvdisplay` when raising RuntimeError.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extract the code that discovers LVM LV names for a given VG, from
`discover_lvm()` into a separate function `lvm_lvs_for_vg()`. This
improves the readability of the code. In addition, some values returned
by the `lvdisplay` invocation were never used. Don't request them and
simplify the code. Rename variables that hold LV names to clearly
express that.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This commit adds a small test that ensures that we notice when
the solver API adds new top-level keys. When this happens the
images library breaks and we need to increase the
`Provides: osbuild-dnf-json-api` version in the `osbuild.spec`.
See e.g. https://github.com/osbuild/osbuild/pull/1992
Add two unit tests for the read_default_target() function:
1. When default target should be found.
2. When there should be no default target.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add a new `enabled_repos` field on the testcases which explicitly lists
which repositories are passed into a certain testcase. This allows us to
pass appstream only to the module testcase.
Re-adjust the package lists again since we're now not using appstream in
all depsolve tests.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
With the enabling of AppStream some more (perhaps optional) packages are
included.
This also adds a test case which installs a module and verifies that
that module is returned.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
The CentOS Stream 9 repository metadata contains modules; these are
necessary for testing modularity depsolving.
Note that the filelists metadata is kept empty to keep repository size
down.
Co-authored-by: Michael Vogt <michael.vogt@gmail.com>
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
The mount ID must be unique. So far, we were using the device as the ID
for the mount because that was unique to each mount. With btrfs
subvolumes however, the device and partition are the same for all, so we
need another way to differentiate.
Btrfs volumes typically only contain subvolumes instead of (parts of)
the OS tree directly. In our images in particular, this is always the
case. When searching for root to find /etc/fstab, search through the
subvolumes on a btrfs volume for the file and return the path to the
root subvolume.
Co-authored-by: Michael Vogt <michael.vogt@gmail.com>
Always set partition=None for the kwargs of the Mount() constructor.
The previous code was added for backwards compatibility with older
versions of the Mount() constructor that didn't include the 'partition
argument. It's safe to remove now because:
1. It's been long enough that we wont run osbuild-image-info with an old
version of osbuild.
2. The tool is packaged with osbuild so there is no version drift and no
compatibility issues.
When the fstab file isn't found, the root_tree will never be set after
being initialised to "" and an exception is raised "The root filesystem
tree is not mounted". It's a lot clearer if the failure happens closer
to the root cause, which is that fstab wasn't found and there are no
fstab entries to iterate through and find the root filesystem.
When iterating partitions to mount, skip any with filesystem type
"swap". This is done in two places:
1. When mounting partitions to find /etc/fstab.
2. When mounting partitions and volumes to analyse the tree.
When iterating through partitions, store the fstype along with the other
information. This will be useful for identifying btrfs partitions,
which we will need to scan for subvolumes, and for identifying swap
partitions, so we can avoid trying to mount them.
Run isort for imports.
Pylint: wrong-import-order / C0411
Solves the following linter warnings:
- standard import "pathlib" should be placed before third party import
"yaml"
- standard import "collections.OrderedDict" should be placed before
third party imports "yaml", "jsonschema"
- standard import "typing.Dict" should be placed before third party
imports "yaml", "jsonschema"
Fix default arg values.
Pylint: dangerous-default-value / W0102
- Using mutable default values ([]) for function arguments is considered
dangerous.
Rename format variable.
Pylint: redefined-builtin / W0622
- 'format' is a built-in function.
Use f-strings instead of formatting where possible.
Pylint: consider-using-f-string / C0209
Remove unnecessary else after returns.
Pylint: no-else-return / R1705
Remove unnecessary else after continue.
Pylint: no-else-continue / R1724
Set the encoding (utf-8) for all calls to open().
Pylint: unspecified-encoding / W1514
Disable the too-many-branches and too-many-statements warnings for
append_partitions() and append_filesystem(). We can refactor the
functions to make them smaller later, but for now we're addressing only
the simpler issues.
Initialise with dict literal instead of call to function.
Pylint: use-dict-literal / R1735
Use implicit truthiness for glob instead of len().
Pylint: use-implicit-booleaness-not-len / C1802
Rename ambiguous variable 'l' to 'line'.
pycodestyle: ambiguous-variable-name (E741)
Merge comparisons with 'in'.
Pylint: consider-using-in / R1714
`read_boot_entries()` could previously fail when trying to split lines
in bootloader entries, which contained only "\n" and became empty
string after stripping whitespace characters. This is the case e.g. on
F41 images.
Moreover, bootloader entries can contain comments as lines starting with
"#", which were previously not ignored by the function and would end up
in the parsed entry and could potentially fail to be split.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add unit test for osbuild-image-info's `read_boot_entries()` function,
to ensure that it can handle various situations that can happen in the
real world.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add the image-info tool as osbuild-image-info from the manifest-db
repository [0]. This is an exact copy without any changes.
[0] 8e05a898d4/tools/image-info
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Enable generating SBOM documents for depsolved transactions when using
DNF5. Enable SBOM testing with DNF5 in unit tests.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The change in commit ed33869430 to
use hashlib.file_digest breaks with older python, because
it was added in 3.11.
This change reverts back to hashing all the data in the case
where file_digest doesn't exist.
This allows using e.g. mpp-eval in the resolve-image operation, similar
to how it is now possible in the mpp-depsolve handling.
We want this so we can inject the list of images from a list variable
(that can then be mpp-join:ed, etc).
For the DNF4 version, we actually use 'dnf' package and not 'libdnf'.
Fix the SPEC file dependencies and also the check in unit test.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Currently if one uses `mpp-embed` with URLs, osbuild-mpp still wants to
download the full file just so it can hash it. Make this more efficient
by hashing from the stream instead, which `hashlib` natively supports.
This also makes osbuild-mpp work with large artifacts in environments
that may not have enough space to temporarily save the data.
Extend osbuild-depsolve-dnf, to return JSON with SPDX SBOM that
corresponds to the depsolved package set, if it has been requested.
For now, only DNF4 is supported.
Cover the new functionality with unit test.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This will allow validating request arguments in the solver method in a
different way for dnf4 and dnf5 and raising an exception if needed.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The DNF5 implementation has known issues, which are not yet fixed and in
some cases, they can't be fixed due to the limitations and state of the
DNF5 implementation itself. Skip them for now.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add a test case for installation of a package, which is excluded in a
different transaction. This is a common scenario in osbuild-composer,
where the image base package set is depsolved in the first transaction
and can contain package excludes. The user must be able to install these
excluded packages when specified explicitly in the Blueprint.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add a test case for depsolving a package group with specific optional
packages of the group being excluded. This is a common scenario in
many image definitions.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add a test case, that the repositories not enabled for a transaction are
not really used to depsolve its package specs. This is a common use case
for osbuild-composer, where the custom repositories specified by the
user are enabled only for the second transaction and not for the first
one (when depsolving the image base package set). This is important, so
that the user can't possibly replace important packages shipped by
the "system" repositories (e.g. kernel, systemd, etc.) with versions
from their custom repositories.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Fix a bug in the `osbuild-depsolve-dnf`, which would cause the
`RepoError` to not be handled and producing a traceback.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Implement the generator for repo config combinations, instead of
hard-coding it. The motivation is to be able to optionally add e.g.
q broken repo to the list to test `RepoError`.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Porting the latests osbuild version in `osbuild/images` revealed a
regression in error messages returned by the tool in case a non-existent
package is requested in the depsolve request.
Test that requesting a non-existent package results in `MarkingErrors`,
which was the original behavior of the DNF4-based
`osbuild-depsolve-dnf`.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Remove one test case, which does not really add value or extend the test
coverage, since the same thing is already tested by previous test case.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
