The useradd, usermod, and passwd commands support a `--root` option that
handles chroot-ing for the command. In general, we prefer using this
option for commands that provide it and relying on the utility itself to
know how to set up the chroot in the way it needs.
The option has been available for these commands since 2011 [1] and it's
unclear why they weren't used originally.
The `mkhomedir_helper` command is still run using our Chroot context, so
the fix introduced in 9071cd0abb is
unaffected.
[1] 365279ea95/ChangeLog (L1339)
Since 9071cd0abb the commands in the
org.osbuild.users stage are run using the osbuild.util.chroot.Chroot
context manager. This includes mounting /proc into the chroot.
The `usermod` command now fails with
usermod: user root is currently used by process N
when modifying any property of the root user that requires the user to
not be logged in (like the home directory).
Change the home directory of the root user in the stage test to check
for this issue and catch regressions.
Use the chroot utility module for all cases where we need to chroot
during a stage's execution.
The advantage is that all stages use the same tested code path for
setting up a chroot and all chrooted commands run in the same
environment, with the /proc, /dev, and /sys filesystems mounted.
Not all distros ship `/var/empty` so just create an empty dir
on demand as needed.
This also tweaks `test_ostree_source_exists()` into calling
`make_repo()` instead of duplicating that code.
Use Chroot class from osbuild.util.chroot module, instead of calling
`chroot` directly. The class handles mounting of various paths in the
chroot to make us more usable. This resolves new failure when running
the stage test on F41 results in `mkhomedir_heper` failing with `6`
return code, meaning permissions denied.
Adjust the stage unit tests, because `chroot.Chroot` can't work with
`pathlib.Path`.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Since v2.23.0, TuneD changed the default directory under which it
looks for profiles. The profiles are newly nested under `profiles/`
directory. More information in [1].
Modify the stage implementation to check if the default profile
directories contain `profiles/` directory. If yes, then look for
profiles in it. If not, use the original behavior.
[1] https://github.com/redhat-performance/tuned/releases/tag/v2.23.0
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
On Fedora 41 with DNF5, the dnf-automatic plugin by default does not
install any configuration file. This means that the stage would fail in
such case.
Previously, the full config file was placed in /etc and its purpose was
also to document all possible options. The example config file is now
installed only in /usr/share/dnf5/dnf5-plugins/automatic.conf.
Relax the stage implementation to not fail when the configuration file
does not exist. Just log a warning and create the configuration file.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Disable the newly reported pylint issue W0135
(contextmanager-generator-missing-cleanup), because as far as I was able
to understand the motivation behind it, it should not apply to the code
and it should be a false positive. We do not use context manager inside
a generator, but inside another context manager. So the execution flow
should return sequentially through the stack and all context manager
cleanups should be executed as expected.
The reported issue:
osbuild/util/containers.py:184:4: W0135: The context used in function 'container_source' will not be exited. (contextmanager-generator-missing-cleanup)
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Fix:
assemblers/org.osbuild.qemu:310:36: E0606: Possibly using variable 'prep_type' before assignment (possibly-used-before-assignment)
inputs/org.osbuild.tree:85:15: E0606: Possibly using variable 'path' before assignment (possibly-used-before-assignment)
stages/org.osbuild.sfdisk:58:36: E0606: Possibly using variable 'prep_type' before assignment (possibly-used-before-assignment)
stages/org.osbuild.systemd.unit:23:16: E0606: Possibly using variable 'unit_dropins_dir' before assignment (possibly-used-before-assignment)
test/mod/test_meta.py:219:29: E0606: Possibly using variable 'schema_part' before assignment (possibly-used-before-assignment)
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Fix:
osbuild/pipeline.py:101:12: R1737: Use 'yield from' directly instead of yielding each element one by one (use-yield-from)
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The new version of Pylint reports this issue in too many places. It is
unlikely that we would change all occurrences of it just to make
Pylint happy. Disable the check.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Update pylint to the latest upstream version, to fix a false positive
with Python 3.13, that was resolved in `astroid` dependency in version
`3.3.5`. In order to update the dependency, the `pylint` version itself
needs to be bumped to at least version `3.0.0`.
The error was:
test/run/test_stages.py:21:0: E0611: No name 'Mapping' in module 'collections.abc' (no-name-in-module)
More details: https://github.com/pylint-dev/pylint/issues/10000
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This is needed to resolve fialure to find `lib2to3` module on F41, which
was removed since Python 3.13 used by default there.
Update autopep8 and pycodestyle to the latest upstream releases.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The test case is skipped in the upstream CI, because the `autotailor`
executable is not installed in the `osbuild-ci` image. This will not
be the case in the future and the CI run will reveal a Python 3.6
incompatibility in the test implementation. Fix it.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This adds support for specifying paths to operate on,
rather than just the root of the target:
```
- type: org.osbuild.selinux
options:
file_contexts: etc/selinux/targeted/contexts/files/file_contexts
target: mount://root/path/to/dir
mounts:
- name: root
source: disk
target: /
```
or
```
- type: org.osbuild.selinux
options:
labels:
mount://root/path/to/file: system_u:object_r:boot_t:s0
mount://root/path/to/other/file: system_u:object_r:var_t:s0
mounts:
- name: root
source: disk
target: /
```
New fucntion returns tuple of 'root' and relative 'file path', which could be
useful in contexts, where knowing 'root' is required, for example setting
selinux labels.
This allows using e.g. mpp-eval in the resolve-image operation, similar
to how it is now possible in the mpp-depsolve handling.
We want this so we can inject the list of images from a list variable
(that can then be mpp-join:ed, etc).
The dnf5 library in Fedora 41 still has some issues that prevents us
from using it in osbuild-composer. Switch to using dnf4 on Fedora 41 as
well until these issues are resolved.
Add remove-signatures option to container-deploy stage.
The option will be translated to --remove-signatures
skopeo option and passed to skopeo when copying the container.
This option must be set when deploying signed containers.
Signed-off-by: Miguel Martín <mmartinv@redhat.com>
Michael Vogt pointed out that testcases start failing when we describe
the new format. Let's add a test case and fix the describe to include
the metadata.
Metadata is freeform in the `Manifest` instance but it is stored on it
during loading (at which time its properties are validated) and returned
as-is on describe.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
Allows a new object under the top-level key `metadata` which contains a
`generators` property. This property is a list of all generators involved
in the creation of a manifest.
Each generator can add its name and version to this list.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
For usecases where for example selinux is not supported,
we should expect more errors from tar so we should also accept this
when matching the string.
Kudos go to Achilleas Koutsou <achilleas@koutsou.net> for this hint
The full list of packages is also listed in terraform
containers/blob/main/docker-bake.hcl#L240 ("BASE_PACKAGES")
so this README and the package list should somewhat stay in sync
This change allows use the more advanced features of bash like
array operations (e.g. `${@:2}` to drop the first two arguments
or similar. On fedora/rhel this is a no-op as it is already using
sh -> bash (afaik).
We currently use the absolute path of these binaries in the
helper. This has some advantages but given that we control the
inputs for PATH in general it seems unnecessary.
We are also slightly inconsistent about this in the codebase but
favor the non absolute path version. A quick count:
```
$ git grep '"chroot"'|wc -l
13
$ git grep '"/usr/sbin/chroot"'|grep -v test_|wc -l
8
```
for `mount` and `umount` it seems this is the only place that uses
the absolute path.
It's not an important change but it has the nice property that it
allows us to use e.g. `testutil.mock_command()` in our tests and
it would be nice to be consistent.
The recent change in https://github.com/osbuild/osbuild/pull/1896
removed the dnf dependency, leading to failures in osbuild-mpp with the
following error:
ModuleNotFoundError: No module named 'dnf'
To fix it, add the python3-dnf dependency for the
osbuild-tools package.
Signed-off-by: Renata Ravanelli <rravanel@redhat.com>
I am keeping mailing list link, however, nobody was able to tell me how one can subscribe to it. I think it is Google Groups list now and there is no join option.
This commit moves the joining of path fragements from f-strings
to pathlib and simplifies some of the map/filter/lambda expressions
into more standard list comprehensions.
This was added years ago in 3c19420b3e.
Back then, we were constructing the initial buildroot using dnf directly
(stages had network access at that point). We reworked this with the
introduction of the rpm stage, sources and an external depsolver. The
dnf stage itself was actually removed in
892342b978. Thus, I think we don't need
to pull dnf anymore, so this commit removes the dependency.
This commit adds a new `transform` option to the tar stages that
maps directly to the `--transform=` comamndline argument of tar(1).
This allows to transform the names while files/dirs are added to
a tarfile. This is useful for the `gcp` pipeline for
bootc-image-builder where we want to create a gcp tar file that
expects the disk image filename in the tar to be exactly `disk.raw`.
Note that tar allows only a single `--transform` and we leave it
to the user to construct `sed` expressions if multiple renames
are required.
With the labels option the user is specifying the exact context
they want to set on the path so it's not necessary to supply a
context here. This can be also useful in the case where you want
to set some labels and you haven't yet populated the tree yet.
For the DNF4 version, we actually use 'dnf' package and not 'libdnf'.
Fix the SPEC file dependencies and also the check in unit test.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
