Add the option to specify the SELinux type label and file type label for
the process.
Add the option to specify a tmpfs mount to create into the container.
Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
The test_host.py:test_signals_on_separate_fd (and more) runs
itself but that run will happen without the tox env so a pip/tox
installed pytest will not be found, install the pytest package
as a workaround.
This commit adds a tiny smoke test that runs the unit tests as
a normal user on a regular github runner. This should ensure that
we catch issues like in osbuild#1936.
This will bring the latest image builds with new dependencies for unit
tests.
In addition, pin down the osbuild-ci-c*s container to a specific tag, to
ensure stability of the CI.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Update the fedora manifest template to F41. This is a preparation for
testing a new `org.osbuild.dnf5.sbom.spdx` stage. In addition to that,
F39 is already EOL and unsupported, so moving to a newer version is
desirable.
Regenerate all testing manifests.
Notes about specific changes:
- Remove `pcmciautils` package from ostree manifest, because it has
been deprecated and it is not available on F41 any more.
- Add `python3-dnf` package to the Fedora build pipeline. The reason
is that DNF5 is installed by default since F41 and the module is
needed to test e.g. `org.osbuild.dnf4.sbom.spdx` or
`org.osbuild.dnf4.mark` stages.
- Add `python3-libdnf5` package to the Fedora build pipeline. This is a
preparation for testing a new `org.osbuild.dnf5.sbom.spdx` stage.
- Install `pam` in `authselect` stage test manifest, because otherwise
authselect fails on the nonexistence of /etc/pam.d.
- Adjust the `dnf4.mark` test case to also accept `dnf5` as the marked
package, because on F41, `dnf` is a virtual provide of it.
- Adjust stage tests and their diffs as needed.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The change in commit ed33869430 to
use hashlib.file_digest breaks with older python, because
it was added in 3.11.
This change reverts back to hashing all the data in the case
where file_digest doesn't exist.
The useradd, usermod, and passwd commands support a `--root` option that
handles chroot-ing for the command. In general, we prefer using this
option for commands that provide it and relying on the utility itself to
know how to set up the chroot in the way it needs.
The option has been available for these commands since 2011 [1] and it's
unclear why they weren't used originally.
The `mkhomedir_helper` command is still run using our Chroot context, so
the fix introduced in 9071cd0abb is
unaffected.
[1] 365279ea95/ChangeLog (L1339)
Since 9071cd0abb the commands in the
org.osbuild.users stage are run using the osbuild.util.chroot.Chroot
context manager. This includes mounting /proc into the chroot.
The `usermod` command now fails with
usermod: user root is currently used by process N
when modifying any property of the root user that requires the user to
not be logged in (like the home directory).
Change the home directory of the root user in the stage test to check
for this issue and catch regressions.
Use the chroot utility module for all cases where we need to chroot
during a stage's execution.
The advantage is that all stages use the same tested code path for
setting up a chroot and all chrooted commands run in the same
environment, with the /proc, /dev, and /sys filesystems mounted.
Not all distros ship `/var/empty` so just create an empty dir
on demand as needed.
This also tweaks `test_ostree_source_exists()` into calling
`make_repo()` instead of duplicating that code.
Use Chroot class from osbuild.util.chroot module, instead of calling
`chroot` directly. The class handles mounting of various paths in the
chroot to make us more usable. This resolves new failure when running
the stage test on F41 results in `mkhomedir_heper` failing with `6`
return code, meaning permissions denied.
Adjust the stage unit tests, because `chroot.Chroot` can't work with
`pathlib.Path`.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Since v2.23.0, TuneD changed the default directory under which it
looks for profiles. The profiles are newly nested under `profiles/`
directory. More information in [1].
Modify the stage implementation to check if the default profile
directories contain `profiles/` directory. If yes, then look for
profiles in it. If not, use the original behavior.
[1] https://github.com/redhat-performance/tuned/releases/tag/v2.23.0
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
On Fedora 41 with DNF5, the dnf-automatic plugin by default does not
install any configuration file. This means that the stage would fail in
such case.
Previously, the full config file was placed in /etc and its purpose was
also to document all possible options. The example config file is now
installed only in /usr/share/dnf5/dnf5-plugins/automatic.conf.
Relax the stage implementation to not fail when the configuration file
does not exist. Just log a warning and create the configuration file.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Disable the newly reported pylint issue W0135
(contextmanager-generator-missing-cleanup), because as far as I was able
to understand the motivation behind it, it should not apply to the code
and it should be a false positive. We do not use context manager inside
a generator, but inside another context manager. So the execution flow
should return sequentially through the stack and all context manager
cleanups should be executed as expected.
The reported issue:
osbuild/util/containers.py:184:4: W0135: The context used in function 'container_source' will not be exited. (contextmanager-generator-missing-cleanup)
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Fix:
assemblers/org.osbuild.qemu:310:36: E0606: Possibly using variable 'prep_type' before assignment (possibly-used-before-assignment)
inputs/org.osbuild.tree:85:15: E0606: Possibly using variable 'path' before assignment (possibly-used-before-assignment)
stages/org.osbuild.sfdisk:58:36: E0606: Possibly using variable 'prep_type' before assignment (possibly-used-before-assignment)
stages/org.osbuild.systemd.unit:23:16: E0606: Possibly using variable 'unit_dropins_dir' before assignment (possibly-used-before-assignment)
test/mod/test_meta.py:219:29: E0606: Possibly using variable 'schema_part' before assignment (possibly-used-before-assignment)
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Fix:
osbuild/pipeline.py:101:12: R1737: Use 'yield from' directly instead of yielding each element one by one (use-yield-from)
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The new version of Pylint reports this issue in too many places. It is
unlikely that we would change all occurrences of it just to make
Pylint happy. Disable the check.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Update pylint to the latest upstream version, to fix a false positive
with Python 3.13, that was resolved in `astroid` dependency in version
`3.3.5`. In order to update the dependency, the `pylint` version itself
needs to be bumped to at least version `3.0.0`.
The error was:
test/run/test_stages.py:21:0: E0611: No name 'Mapping' in module 'collections.abc' (no-name-in-module)
More details: https://github.com/pylint-dev/pylint/issues/10000
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This is needed to resolve fialure to find `lib2to3` module on F41, which
was removed since Python 3.13 used by default there.
Update autopep8 and pycodestyle to the latest upstream releases.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The test case is skipped in the upstream CI, because the `autotailor`
executable is not installed in the `osbuild-ci` image. This will not
be the case in the future and the CI run will reveal a Python 3.6
incompatibility in the test implementation. Fix it.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This adds support for specifying paths to operate on,
rather than just the root of the target:
```
- type: org.osbuild.selinux
options:
file_contexts: etc/selinux/targeted/contexts/files/file_contexts
target: mount://root/path/to/dir
mounts:
- name: root
source: disk
target: /
```
or
```
- type: org.osbuild.selinux
options:
labels:
mount://root/path/to/file: system_u:object_r:boot_t:s0
mount://root/path/to/other/file: system_u:object_r:var_t:s0
mounts:
- name: root
source: disk
target: /
```
New fucntion returns tuple of 'root' and relative 'file path', which could be
useful in contexts, where knowing 'root' is required, for example setting
selinux labels.
This allows using e.g. mpp-eval in the resolve-image operation, similar
to how it is now possible in the mpp-depsolve handling.
We want this so we can inject the list of images from a list variable
(that can then be mpp-join:ed, etc).
The dnf5 library in Fedora 41 still has some issues that prevents us
from using it in osbuild-composer. Switch to using dnf4 on Fedora 41 as
well until these issues are resolved.
Add remove-signatures option to container-deploy stage.
The option will be translated to --remove-signatures
skopeo option and passed to skopeo when copying the container.
This option must be set when deploying signed containers.
Signed-off-by: Miguel Martín <mmartinv@redhat.com>
Michael Vogt pointed out that testcases start failing when we describe
the new format. Let's add a test case and fix the describe to include
the metadata.
Metadata is freeform in the `Manifest` instance but it is stored on it
during loading (at which time its properties are validated) and returned
as-is on describe.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
Allows a new object under the top-level key `metadata` which contains a
`generators` property. This property is a list of all generators involved
in the creation of a manifest.
Each generator can add its name and version to this list.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
For usecases where for example selinux is not supported,
we should expect more errors from tar so we should also accept this
when matching the string.
Kudos go to Achilleas Koutsou <achilleas@koutsou.net> for this hint
The full list of packages is also listed in terraform
containers/blob/main/docker-bake.hcl#L240 ("BASE_PACKAGES")
so this README and the package list should somewhat stay in sync
This change allows use the more advanced features of bash like
array operations (e.g. `${@:2}` to drop the first two arguments
or similar. On fedora/rhel this is a no-op as it is already using
sh -> bash (afaik).
We currently use the absolute path of these binaries in the
helper. This has some advantages but given that we control the
inputs for PATH in general it seems unnecessary.
We are also slightly inconsistent about this in the codebase but
favor the non absolute path version. A quick count:
```
$ git grep '"chroot"'|wc -l
13
$ git grep '"/usr/sbin/chroot"'|grep -v test_|wc -l
8
```
for `mount` and `umount` it seems this is the only place that uses
the absolute path.
It's not an important change but it has the nice property that it
allows us to use e.g. `testutil.mock_command()` in our tests and
it would be nice to be consistent.
