This adds a new key masked_generators, similar to masked_services,
which masks systemd generators from running at boot, by creating
symlinks to /dev/null in /etc/systemd/systemd-generators, as
described in:
https://www.freedesktop.org/software/systemd/man/latest/systemd.generator.html#Description
This will be useful for the automotive project, as it allows disabling
of unsupported things like sysv or rc.local legacy support, while
improving boot performance.
Adjust the source path to just have /dev in front (i.e. /dev/loop0)
and not be a path to the temporary dev directory that was prepared.
We do this because some tools (like grub2-install) consult
mountinfo to try to canonicalize paths for mounts.
Fixes https://github.com/osbuild/osbuild/issues/1492
Update the skopeo sources stage to check other container-transports [1]
for downloading container images. This commit adds a transport for
`containers-storage` in addition to the existing `docker://`
transport.
Jira: https://issues.redhat.com/browse/HMS-3235
[1] CONTAINERS-TRANSPORTS(5)
The skopeo stage is updated to accommodate two types of
destinations: 'containers-storage' and 'oci'. Now, it can
copy a container image to either a container store or
a directory as per 'Open Container Image Layout
Specification'.
Pytoml is no longer being maintained: https://github.com/avakar/pytoml
The author suggest switching to toml.
We already use the
```
try:
import toml
except ModuleNotFoundError:
import pytoml as toml
```
pattern in stages/org.osbuild.containers.storage.conf so use it in the tests too to prefer "toml" instead of pytoml.
This helper can be used to implement a strategy to find the oldest
cache entries and evict them when the cache is full.
The implementation uses the `atime` of the per object `cache.lock`
file and ensures in `load()` that it's actually updated.
This commit adds mount output to the error raised by
FileSystemMountService.mount(). This is useful when running into
mount failures during osbuild runs.
The issue was discovered while debugging a mount failure for
osbuild-composer PR#3820. Initially osbuild PR#1490 was meant
to fix it but it turned out there is a third mount helper in
the code that was originally overlooked (sorry for that!).
This is a convenient way for tests to assert that some nested dicts
(like a parsed json) has a particular key/value somewhere in it.
For example:
assert_dict_has(config, "toplevel.subitem.key", True)
While debugging a failure of osbuild-composer [0] on fc39 it was
noticed that a mount failure does not include the output of
the mount command:
```
File "/usr/lib/python3.12/site-packages/osbuild/mounts.py", line 78, in mount
path = client.call("mount", args)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/osbuild/host.py", line 348, in call
ret, _ = self.call_with_fds(method, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/osbuild/host.py", line 384, in call_with_fds
raise error
osbuild.host.RemoteError: CalledProcessError: Command '['mount', '-t', 'xfs', '-o', 'ro,norecovery', '--source', '/dev/rootvg/applv', '--target', '/tmp/tmpjtfmth56/app']' returned non-zero exit status 32.
File "/usr/lib/python3.12/site-packages/osbuild/host.py", line 268, in serve
reply, reply_fds = self._handle_message(msg, fds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/osbuild/host.py", line 301, in _handle_message
ret, fds = self.dispatch(name, args, fds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/osbuild/mounts.py", line 111, in dispatch
r = self.mount(args)
^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/osbuild/mounts.py", line 160, in mount
subprocess.run(
File "/usr/lib64/python3.12/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
```
which makes diagnostic errors harder of course. This commit adds
a test that ensures that mount output is visbile and also changes
the code to include it.
[0] https://github.com/osbuild/osbuild-composer/pull/3820
osbuild-composer project has build related section in the main README.
For the osbuild it was missing. Even if this is script-based project
external developers may not be aware of this just by looking at README.
Missing build related section has been added for consistency.
This commit extends the current support for OpenSCAP
tailoring by accepting an array of key/value overrides.
Users will be able to specify override values for specific
rules that will update the value when remediating the
image.
The next commit will add a stage test that requires erofs-utils. Let's add it
into the buildroot in a separate commit, so the history is more readable.
Add a simple stage test for the erofs stage. It uses dump.erofs
instead of mounting the file because the kernel in the GH runners
do not support mounting erofs just yet.
This adds tests for the erofs stage. The tests are slightly different
from the existing tests that run the filesystem utils inside the
stages. Depending on what exactly we want to test we may still need
a run inside the stages. However running this inside a container
should be good enough if we just want to validate that the options
are passed correctly and the file is created.
Erofs is "a lightweight read-only file system"[1]. Imagine squashfs, but with
faster reads.
This commit adds support for creating it. The new stage is heavily inspired by
the squashfs one. I've decided to add all features of mkfs.erofs that looked
useful: All compression types and most of extended options (excluding the
compatibility ones, we can always add them later).
[1]: https://en.wikipedia.org/wiki/EROFS
Add a new Packit "copr_build" job, which will build new upstream
releases in a dedicated COPR project "osbuild-stable". This will allow
people to consume stable builds of osbuild ASAP (including some
sub-packages on CS which are not part of the official distro builds).
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
`mpp-define-images` can create an image file, using `losetup` to deal
with non-standard sector sizes requires root. Not all users run
`osbuild-mpp` as root.
While I am not a fan of "suddenly sudo" based on the input manifest this
does alleviate builds breaking for manifests with default sector sizes
when non-root.
This is a variation of PR https://github.com/osbuild/osbuild/pull/960
that put the machine-id handling into it's own stage and adds
explicit handling what should happen with it.
For machine-id(5) we essentially want the following three states
implemented:
1. `first-boot: yes` will ensure that /etc/machine-id is
in the "uninitialized" state. This means on boot the systemd
`ConditionFirstBoot` is triggered and a new id in `/etc/machine-id`
is created. This will work for systemd v247+.
2. `first-boot: no` will ensure that /etc/machine-id exists but
is empty. This will trigger the creation of a new machine-id but
will *not* trigger `ConditionFirstBoot`.
3. `first-boot: preserve` will just keep the existing machine-id.
Note that it will error if there is no /etc/machine-id
Note that the `org.osbuild.rpm` will also create a
`{tree}/etc/machine-id` while it runs to ensure that postinst
scripts will not fail that rely on this file. This is an
implementation detail but unfortunately the rpm stage will
leave an empty machine-id file if it was missing. So we cannot
just remove /etc/machine-id because any following rpm stage
would re-create it again (and we cannot change that without
breaking backward compatiblity). Thanks to the special semantic
that a missing /etc/machine-id and an /etc/machine-id with
the `uninitialized` string are equivalent we don't care.
To support systemd versions below v247 we could offer an option
to remove /etc/machine-id. But the downside of this is that
it would only work if the org.osbuild.machine-id stage is after
the rpm stage.
See also the discussion in PR#960.
Thanks to Tom, Christian for the PR and the background.
For our Fedora CoreOS disk images we set the partition labels (name)
for the partitions. This is also supported using the primitives here
in OSBuild, but it wasn't obvious that I needed to set the name in
the mpp-define-images definition. Let's set the name there, but let's
also allow osbuild-mpp to set the `id`, which is what is used later
to access that partition from the `name` too if `id` isn't set.
This means we allow something like:
- name: BIOS-BOOT
type: 21686148-6449-6E6F-744E-656564454649
bootable: true
uuid: FAC7F1FB-3E8D-4137-A512-961DE09A5549
size: 100
rather than requiring something like:
- id: BIOS-BOOT
name: BIOS-BOOT
type: 21686148-6449-6E6F-744E-656564454649
bootable: true
uuid: FAC7F1FB-3E8D-4137-A512-961DE09A5549
size: 100
Now you can specify a sector_size in `mpp-define-images` to support
creating a 4k native disk image (sector_size=4096).
This does use a loopback device, which means osbuild-mpp also needs
to run as root, when previously that wasn't necessary.
If you do math in mpp-format-int it could end up getting converted
to a float. Of course if you end up with a decimal value that isn't
`.0` that's a problem for an int, but if it is `.0` let's handle it
gracefully.
For example, math like this could end up with a value with `.0`:
mpp-format-int: "{bios_boot_size_mb * 1024 * 1024 / sector_size_bytes}"
For the org.osbuild.loopback the user can set the sector size, but
it had no effect on the underlying loopback device. Let's make it
meaningful by passing along the given value to the underlying code.
I'm trying to debug some failures, and having no feedback as to
what file we're parsing or what code is evaluated when something
fails makes it hard to debug manifests.
This adds some nice error messages that will help.
