Partitions by default are indexed starting at 1, but in
some cases, such as CoreOS for IBM Z, it may be usefull
to set the 'partnum' for GPT disks explicitly, without
creating dummy partitions.
Now user can define an image:
```
mpp-define-images:
- id: image
size: 10737418240
table:
uuid: 00000000-0000-4000-a000-000000000001
label: gpt
partitions:
- name: boot
type: 0FC63DAF-8483-4772-8E79-3D69D8477DE4
partnum: 3
size: 786432
- name: root
type: 0FC63DAF-8483-4772-8E79-3D69D8477DE4
partnum: 4
size: 4194304
```
So target disk would look like:
```
Disklabel type: gpt
Disk identifier: 00000000-0000-4000-A000-000000000001
Device Start End Sectors Size Type
/dev/loop0p3 2048 788479 786432 384M Linux filesystem
/dev/loop0p4 788480 4982783 4194304 2G Linux filesystem
```
This patch updates the osbuild-mpp tool and the sgdisk and sfdisk
stages to support this.
Co-authored-by: Dusty Mabe <dusty@dustymabe.com>
This commit adds code that will remove the least recently used
entries when a store() operation does not succeeds because the
cache is full. To be more efficient it will try to free
twice the requested size (this can be configured in the code).
Add or remove the immutable bit to the specified mount directory.
The need we have for this right now is for the CoreOS builds where
the immutable bit being set on an OSTree deployment root doesn't
survive the `cp -a --reflink=auto` in the org.osbuild.copy stage when
being copied from the directory tree into the mounted XFS filesystem
we created on the disk image. Thus we have to workaround this loss
of attribute by applying the attribute directly on the mounted
filesystem from the disk.
This is a drive-by change after spending some quality time with the
mount code. The `id` field of `Mount` is calculated only once and
only when creating a `Mount`. This seems slightly dangerous as
any change to an attribute after creation will not update the
id. This means two options:
1. dynamically update the `id` on changes
2. forbid changes after the `id` is calculcated
I went with (2) but happy to discuss of course but it seems more
the spirit of the class.
It also does the same change for "devices.Device"
The test_libc_futimes_works() is failing under RHEL/Centos right
now. To make it more robust a tiny sleep and rounding of the
timestamps is introduced to ensure that we are not run into
floating point comaparison funnines.
The second part of the fix is to open the stamp_file in read-only
mode to ensure that the mtime is not modified by the open itself
which is what lead to the actual test failure.
Older versions of RHEL/Centos do not have `mkfs.fat -g` yet so
this test will fail. Detect this and skip the test if mkfs.fat
is too old (see 7af2f1a for the original commit).
When constructing a minimal environment for osbuild the exported
dirs lacked "schemas" so the test_exports.py test failed on RHEL8.
This commit adds it (and also "assemblers" for good measure). With
that the test will pass.
Older version of `lsblk --json` will not have the plural `mountpoints`
but only a singular `mountpoint`. But newer version lost the
singular `mountpoint` in the json. Adjust the test accordingly.
The new `create_image_with_partitions()` helper fails under rhel8
currently. The reason is that `mkfs.ext4 -E offset=` will warn
in older versions about a partition table and require user input.
This got fixed `e2fsprogs` 1.46.3 in Jul 2021 but RHEL8 still
has 1.45.
[0] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=989612
The stable stream currently doesn't have a new enough bootupd to pass
the tests for the bootupd stage. Let's update to `:testing` for now
and we'll switch back to `:stable` later.
Add the bootupd stage to install GRUB on both BIOS and UEFI systems,
ensuring that your bootloader stays up-to-date.
Signed-off-by: Renata Ravanelli <rravanel@redhat.com>
Extract a new helper `make_fake_tree()` that generalizes the existing
helper `make_fake_input_tree()`. The later will always create the
content under `{basedir}/tree` which is convinient for input tree
based tests but too specialized when using it in different contexts.
The existing `make_fake_input_tree()` is preserved unchanged and
becomes just a tiny wrapper.
This commit allows to exclude preserving ownership from an object
export. This is required to fix the issue that on macOS the an
podman based workflow cannot export objects with preserving
ownerships.
Originally this was a `no_preserve: Optional[List[str]] = None)`
to be super flexible in what we pass to `cp` but then I felt like
YAGNI - if we need more we can trivially change this (internal)
API again :)
Similar to the aleph file created for builds of FCOS based on ostree
commit inputs, this adds an aleph file that contains information about
the initial deployment of data when the disk image was built
A new stage is preferred here as both the org.osbuild.ostree.deploy
and org.osbuild.ostree.deploy.container stages need an aleph file and
use of the aleph file may depend on the project/product. For example,
right now CoreOS is the only project that uses an aleph file, but others
may want it in the future.
Diffs for stage tests have changed after the sources were updated.
Update them to match expected behaviour. This was mostly done with
some form of:
```
foo=update-crypto-policies;
sudo tools/gen-stage-test-diff --libdir . --store /var/osbuild/store/ test/data/stages/$foo > test/data/stages/$foo/diff.json
```
For the dracut one I had to figure out what new kernel was used
and the new modules and update the vanilla.json file to get the
test to pass.
For the rpm one I had to also update the metadata.json with something
like:
```
sudo python3 -m osbuild --libdir . --store /var/osbuild/store/ --export tree \
--output-directory /var/osbuild/out/ test/data/stages/rpm/b.json --json \
| jq .metadata >test/data/stages/rpm/metadata.json
```
This adds a new key masked_generators, similar to masked_services,
which masks systemd generators from running at boot, by creating
symlinks to /dev/null in /etc/systemd/systemd-generators, as
described in:
https://www.freedesktop.org/software/systemd/man/latest/systemd.generator.html#Description
This will be useful for the automotive project, as it allows disabling
of unsupported things like sysv or rc.local legacy support, while
improving boot performance.
This helper can be used to implement a strategy to find the oldest
cache entries and evict them when the cache is full.
The implementation uses the `atime` of the per object `cache.lock`
file and ensures in `load()` that it's actually updated.
This commit adds mount output to the error raised by
FileSystemMountService.mount(). This is useful when running into
mount failures during osbuild runs.
The issue was discovered while debugging a mount failure for
osbuild-composer PR#3820. Initially osbuild PR#1490 was meant
to fix it but it turned out there is a third mount helper in
the code that was originally overlooked (sorry for that!).
While debugging a failure of osbuild-composer [0] on fc39 it was
noticed that a mount failure does not include the output of
the mount command:
```
File "/usr/lib/python3.12/site-packages/osbuild/mounts.py", line 78, in mount
path = client.call("mount", args)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/osbuild/host.py", line 348, in call
ret, _ = self.call_with_fds(method, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/osbuild/host.py", line 384, in call_with_fds
raise error
osbuild.host.RemoteError: CalledProcessError: Command '['mount', '-t', 'xfs', '-o', 'ro,norecovery', '--source', '/dev/rootvg/applv', '--target', '/tmp/tmpjtfmth56/app']' returned non-zero exit status 32.
File "/usr/lib/python3.12/site-packages/osbuild/host.py", line 268, in serve
reply, reply_fds = self._handle_message(msg, fds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/osbuild/host.py", line 301, in _handle_message
ret, fds = self.dispatch(name, args, fds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/osbuild/mounts.py", line 111, in dispatch
r = self.mount(args)
^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/osbuild/mounts.py", line 160, in mount
subprocess.run(
File "/usr/lib64/python3.12/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
```
which makes diagnostic errors harder of course. This commit adds
a test that ensures that mount output is visbile and also changes
the code to include it.
[0] https://github.com/osbuild/osbuild-composer/pull/3820
This commit extends the current support for OpenSCAP
tailoring by accepting an array of key/value overrides.
Users will be able to specify override values for specific
rules that will update the value when remediating the
image.
The next commit will add a stage test that requires erofs-utils. Let's add it
into the buildroot in a separate commit, so the history is more readable.
Add a simple stage test for the erofs stage. It uses dump.erofs
instead of mounting the file because the kernel in the GH runners
do not support mounting erofs just yet.
For our Fedora CoreOS disk images we set the partition labels (name)
for the partitions. This is also supported using the primitives here
in OSBuild, but it wasn't obvious that I needed to set the name in
the mpp-define-images definition. Let's set the name there, but let's
also allow osbuild-mpp to set the `id`, which is what is used later
to access that partition from the `name` too if `id` isn't set.
This means we allow something like:
- name: BIOS-BOOT
type: 21686148-6449-6E6F-744E-656564454649
bootable: true
uuid: FAC7F1FB-3E8D-4137-A512-961DE09A5549
size: 100
rather than requiring something like:
- id: BIOS-BOOT
name: BIOS-BOOT
type: 21686148-6449-6E6F-744E-656564454649
bootable: true
uuid: FAC7F1FB-3E8D-4137-A512-961DE09A5549
size: 100
