Commit graph

802 commits

Author SHA1 Message Date
Michael Vogt
530ce4f80c stages: ensure dirs for dracut-conf stage are available
If the dracut config stage runs early the directory for the
dracut config may not be available yet. So just create it.
2025-02-24 13:18:33 +01:00
Dusty Mabe
a08736be49 stages/coreos.live-artifacts: use /var/tmp for mkfs.erofs
This helps reduce the memory requirements of mkfs.erofs since
it won't be writing to tmpfs (memory backed) storage, which is
what is mounted on /tmp/.
2025-02-17 20:39:38 +01:00
Michael Vogt
0c1a8c802f stages: add basic qemu cmdline test
This commit adds a basic fake commandline test for the qemu
stage.
2025-02-17 13:21:40 +01:00
Dusty Mabe
4e033c305e stages: add more options to qemu vmdk disk type
The CoreOS team uses the compat6 and adapter_type options when creating
a VMDK for AWS.

e1943d6adb/src/cosalib/qemuvariants.py (L48)
2025-02-17 03:41:11 -05:00
Sanne Raymaekers
441fbf70d6 org.osbuild.cloud-init: add new datasources and network section
Adds WSL and NoCloud datasources. The network section only allows you to
disable network configuration by cloud-init for now.
2025-02-15 10:38:12 +01:00
Nikita Dubrovskii
035ced543b stages/coreos.live-artifacts: update comments/names to reflect erofs related changes 2025-02-13 13:40:20 +01:00
Michael Vogt
1a2637b1a3 stages: replace "logger" binary in the dracut chroot
This commit replaces the `/usr/bin/logger` binary in the dracut
chroot with a bind mount to `/usr/bin/true` to silence the spam
that we get from dracut during initramfs generation:
```
logger: socket /dev/log: No such file or directory
```

Unfortunately I could not find a nicer way, it seems it is
not possible to simply pass `sysloglvl=0` via the commandline
or an environment.

The extra complication here is that the dracut stage mounts
`devtmpfs` which will likely include:
```
/dev/log -> /run/systemd/journal/dev-log
```
but of course inside this chroot there is no `/run` which
leads to these messages.

Closes: https://github.com/osbuild/osbuild/issues/1976
2025-02-12 15:44:04 +00:00
Paweł Poławski
ed732b36d8 stages: add tests to org.osbuild.ovf
Add some tests for org.osbuild.ovf.

Extracted from https://github.com/osbuild/osbuild/pull/1751
2025-02-12 11:24:47 +01:00
Nikita Dubrovskii
ab1f48795b stages/coreos.live-artifacts: add erofs support 2025-02-11 17:41:35 -05:00
Albert Esteve
a6b64860b3 stages/kernel-cmdline: add max cmdline option
Add an additional option called `kernel_line_size`
to allow setting a maximum cmdline size check
value for custom kernels or other restrictions.

This will override the arch defaults, if not set,
then the size map is checked, and if the current
architecture is not in the map, fallback to
4096, which is the max value allowed for
COMMAND_LINE_SIZE.

Signed-off-by: Albert Esteve <aesteve@redhat.com>
2025-02-11 09:16:17 +00:00
Albert Esteve
7645d717e6 stages/kernel-cmdline: fix link in comment
Signed-off-by: Albert Esteve <aesteve@redhat.com>
2025-02-11 09:16:17 +00:00
Albert Esteve
bd316ddb8f kernel-cmdline: add size check
Add check to ensure that the size of
the parameters does not exceed the
maximum kernel cmdline size.
Otherwise, the parameters will
be truncated and the command line
will fail.

The size is arch-dependant. In
order to not to over-complicate
the search of the value in the
kernel files (which will probably
not be installed in most cases),
it uses a map with some values
for common architectures.

If architecture is not found in
the map, defaults to 4096, which
is the maximum posible size for
COMMAND_LINE_SIZE.

Signed-off-by: Albert Esteve <aesteve@redhat.com>
2025-02-07 15:18:45 +01:00
Achilleas Koutsou
45c6006cfc stages: Where is mandatory in systemd.unit.create
`systemd.mount(5)` describes both `What` and `Where` as mandatory
options for the `Mount` section of a unit.
2025-02-04 22:24:02 +01:00
Michael Vogt
752f4af6b3 stages: add test for the new org.grub2.iso.legacy stage
This commit adds a small smoke test that ensures our new
org.osbuild.grub2.iso.legacy stages does not regress easily.
2025-01-30 08:29:43 +01:00
Simon de Vlieger
d92be415f8 stage/dnf.module-config: remove path
Instead of passing in the path we name the file according to the module
name. Path can be reintroduced later if absolutely necessary.

Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
2025-01-22 18:03:21 +01:00
Simon de Vlieger
0d25c845f8 stages: introduce dnf.module-config
A new stage that allows writing a DNF module configuration file to a
given path.

Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
2025-01-22 18:03:21 +01:00
Albert Esteve
55d53f58fd stages/mkfs.ext4: disable lazy_init
This helps to clear a risk concern for
automotive.

Add optional flag to org.osbuild.mkfs.ext4 stage
to enable/disable both lazy_itable_init and
lazy_journal_init extended options. Both mke2fs
options are controled by the same flag, namely
`lazy_init`.

Signed-off-by: Albert Esteve <aesteve@redhat.com>
2025-01-17 19:11:38 +01:00
Michael Vogt
1eb8138ab5 test: tweak test_grub2_inst.py to support running on Debian/Ubuntu
This commit adds support to run the tests on Debian/Ubuntu systems.
Here the binary is called `grub-mkimage` instead of `grub2-mkimage`,
in addition the "grub-pc-bin" package must be installed there.
2025-01-13 22:53:02 +01:00
Brian C. Lane
3bed7c7ace grub2.inst: Add support for creating iso9660 boot image
Don't include the "location" offset, and use just a path for the prefix
section to set the path of the grub2 modules on the iso. eg.

{
    "filename": "eltorito.img",
    "platform": "i386-pc",
    "core": {
        "type": "mkimage",
        "partlabel": "gpt",
        "filesystem": "iso9660"
    },
    "prefix": {
        "path": "/boot/grub2/"
    }
}
2025-01-13 22:53:02 +01:00
Brian C. Lane
41c0550a1f tests: Add simple test for org.osuild.grub2.inst stage
Tests that grub2-mkimage ran without errors, and that the image file
actually has something written to it.
2025-01-13 22:53:02 +01:00
Brian C. Lane
fd19ab41fb org.osbuild.xorriso: Add support for grub2 bootable iso
This creates an iso matching the current method used in Fedora where it
uses grub2 for BIOS and UEFI booting. Pass the path to the grub2 hybrid
mbr to the stage in the 'grub2mbr' field. eg.
    "grub2mbr": "/usr/lib/grub/i386-pc/boot_hybrid.img"
2025-01-06 21:23:23 +01:00
Tomáš Hozza
6a4a7c31c6 Stages/test/tuned: handle "improved" jsonschema messages
python-jsonschema improved the error messages for some special cases
since v4.21.0 [0], which means that we need to handle multiple versions
of the error message in unit tests.

[0] 8875c2ecb1 (diff-c21226b904760a669a70785494cd8ecf5fb1e7415fd25765dd02f0ad00394099R231)

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-06 11:53:46 -08:00
Tomáš Hozza
806f949e8c Stages/dnf5.sbom.spdx: don't use format strings for constructing paths
Use the new helper function `osbuild.util.path.join_abs()` for
constructing paths.

Fix #1964

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-06 11:53:46 -08:00
Tomáš Hozza
b6dd45af4d Stages/dnf4.sbom.spdx: don't use format strings for constructing paths
Use the new helper function `osbuild.util.path.join_abs()` for
constructing paths.

Fix #1964

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-06 11:53:46 -08:00
Tomáš Hozza
89f94af60d Stages/tuned: don't use format strings for constructing paths
Use the new helper function `osbuild.util.path.join_abs()` for
constructing paths.

Fix #1941

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-06 11:53:46 -08:00
Tomáš Hozza
67d9663c83 Stages/tuned: skip non-existing profile directories
Previously, the stage would try to call `os.scandir()` on non-existing
profile directories, which results in an exception. While the
directories that it scans are currently created by the TuneD RPM package
when installed, it it much nicer to gracefully handle their potential
non-existence, than to fail with an exception.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-06 11:53:46 -08:00
Tomáš Hozza
350ae9ed40 Stages/tuned: require profile to be non-empty string
The stage schema previously accepted an empty string as a profile name.
This would not work in practice, therefore extend the schema to require
non-empty strings.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-06 11:53:46 -08:00
Tomáš Hozza
d7d41b477b Stages/tuned: add unit tests for the stage.
Fix #1941

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-06 11:53:46 -08:00
Tomáš Hozza
f173efea56 Stages/dnf*.sbom.spdx: explicitly mention DNF version in the summary
Explicitly mention the used DNF version in the stage summary.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-06 11:53:46 -08:00
Brian C. Lane
902c4f7d71 erofs: Add ability to set the cluster size
`cluster-size` sets the maximum size of compress physical cluster size
in bytes.
2025-01-03 12:05:56 +01:00
Brian C. Lane
1ab098fbf9 erofs: Add zstd to the list of supported compression algorithms
This is supported in erofs-utils 1.8 and later.
2025-01-03 12:05:56 +01:00
Colin Walters
fcb93bde01 kickstart: Drop ostree signature verification flag
It's a no-op nowadays, we dropped support for the custom
ostree-container signature verification status.

Signed-off-by: Colin Walters <walters@verbum.org>
2024-12-18 16:34:30 -08:00
Dusty Mabe
cd19587464 stages/coreos.live-artifacts: move rename of vendor dir
Seems more appropriate in copy_configs_and_init_kargs_json() since this
is where files were originally copied in.
2024-12-18 11:09:29 -05:00
Dusty Mabe
d5aff7b1af stages/coreos.live-artifacts: drop usage of dir fd
Rework rename of vendor directory to not use dfd APIs.

This was requested in code review.

Also added comments since I now understand it better.
2024-12-18 11:09:29 -05:00
Dusty Mabe
45583720af stages/coreos.live-artifacts: stop chmod initrd.img
The initramfs.img in Fedora and RHEL CoreOS images is already 644
so this isn't needed.
2024-12-18 11:09:29 -05:00
Dusty Mabe
b8d986b51c stages/coreos.live-artifacts: drop kernel rename for s390x
The rename to get file names + suffix with < 8 characters
was done in [1] when our initramfs was named initramfs.img.
It was subsequently renamed to initrd.img in [2] and the
rename of the initramfs was dropped but the rename of the
kernel was never dropped. Since vmlinuz is already < 8
characters let's just drop the rename here too.

[1] 60400910bb
[2] 6f533ef55e
2024-12-18 11:09:29 -05:00
Michael Vogt
af710ee740 stage: add test for core live-artifacts stage
This commit adds some unit tests around the coreos live-artifcats
mono stage.

- test/coreos_live_artifacts: add test for align_initrd()
- test/coreos_live_artifacts: add test for extend_initramfs()
    - This actually tests the mkinitrd_pipe() function, which
      extend_initramfs() calls after opening the file.
- test/coreos_live_artifacts: add test for make_stream_hash()
- test/coreos_live_artifacts: add test for make_efi_bootfile()

Co-authored-by: Achilleas Koutsou <achilleas@koutsou.net>
2024-12-18 11:09:29 -05:00
Achilleas Koutsou
4e93a66311 stages/coreos.live-artifacts: refactor into more functions
This improves the readability and maintainability of the code to
have it split into smaller pieces.

Co-Authored-by: Dusty Mabe <dusty@dustymabe.com>
2024-12-18 11:09:29 -05:00
Jonathan Lebon
0331e6f313 stages: add coreos.live-artifacts.mono stage
This adds a new `org.osbuild.coreos.live-artifacts.mono` stage to build
CoreOS Live ISO/PXE artifacts. The code is heavily based on the
`cmd-buildextend-live` script from coreos-assembler [1], but a lot of
things had to be adapted:
- the stage is provided the deployed oscontainer tree, metal, and
  metal4k images as inputs
- we use chroot instead of supermin to execute some commands in the
  context of the target oscontainer
- a bunch of calls that were wrapped by libguestfs for us (e.g.
  mkfs.vfat, mksquashfs), we now have to call ourselves; to retain
  maximum compatibility, we ensured that we still effectively use the
  same args that libguestfs passed

And various other minor adjustments.

Of course, this is not really in line with the OSBuild philosophy
of having smaller-scoped stages. We have labeled this with a .mono
suffix to denote it is monolithic, similar to the existing
`org.osbuild.bootiso.mono` stage today.

Eventually we may be able to break this stage down if we find it worth
the effort. Alternatively the need for it may go away as we align more
with Image Mode.

[1] 43a9c80e1f/src/cmd-buildextend-live

Co-authored-by: Dusty Mabe <dusty@dustymabe.com>
Co-authored-by: Renata Ravanelli <renata.ravanelli@gmail.com>
2024-12-18 11:09:29 -05:00
Tomáš Hozza
7699f8134f Add org.osbuild.dnf5.sbom.spdx stage
Add DNF5 version of the `sbom.spdx` stage, which generates an SPDX SBOM
document from a given FS tree using libdnf5 API.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-12-18 05:46:45 +01:00
Brian C. Lane
41658da9cf org.osbuild.grub2.iso.legacy: Add grub2 setup for booting BIOS ISO
This is the BIOS version of the grub2 iso stage. It installs the config
file and copies over the grub2 modules to /boot/grub2/
2024-12-12 09:29:25 +01:00
Brian C. Lane
5a8b2edaa0 typo: Fix incorrect 'tar archive' descriptions in stage metadata
SSIA
2024-12-10 15:16:33 +01:00
Tomáš Hozza
f1c43ae5bd util/sbom/spdx: rename {,s}bom_pkgset_to_spdx2_doc()
Rename the function for consistency reason. The parent package is named
SBOM (originally BOM).

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-12-02 23:24:39 +01:00
Achilleas Koutsou
3bf6ca399f test: add new containers.unit.create options in test 2024-12-02 12:55:30 +01:00
Pierre-Yves Chibon
17fda9e318 Expand the options available in containers.unit.create
Add the option to specify the SELinux type label and file type label for
the process.
Add the option to specify a tmpfs mount to create into the container.

Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
2024-12-02 12:55:30 +01:00
Achilleas Koutsou
af7b7db66f test: update users stage unit tests
Update the unit tests to match new calls and options.
2024-11-26 23:35:04 +01:00
Achilleas Koutsou
97007b9e3d stages: run useradd, usermod, and passwd with --root
The useradd, usermod, and passwd commands support a `--root` option that
handles chroot-ing for the command.  In general, we prefer using this
option for commands that provide it and relying on the utility itself to
know how to set up the chroot in the way it needs.

The option has been available for these commands since 2011 [1] and it's
unclear why they weren't used originally.

The `mkhomedir_helper` command is still run using our Chroot context, so
the fix introduced in 9071cd0abb is
unaffected.

[1] 365279ea95/ChangeLog (L1339)
2024-11-26 23:35:04 +01:00
Achilleas Koutsou
8e3d054099 stages: use util.chroot in all stages that call "chroot"
Use the chroot utility module for all cases where we need to chroot
during a stage's execution.

The advantage is that all stages use the same tested code path for
setting up a chroot and all chrooted commands run in the same
environment, with the /proc, /dev, and /sys filesystems mounted.
2024-11-26 12:13:44 +01:00
Tomáš Hozza
9071cd0abb Stages/users: use Chroot from osbuild.util.chroot
Use Chroot class from osbuild.util.chroot module, instead of calling
`chroot` directly. The class handles mounting of various paths in the
chroot to make us more usable. This resolves new failure when running
the stage test on F41 results in `mkhomedir_heper` failing with `6`
return code, meaning permissions denied.

Adjust the stage unit tests, because `chroot.Chroot` can't work with
`pathlib.Path`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-11-25 18:05:54 +01:00
Tomáš Hozza
a8aa6361b1 Stages/tuned: handle changed profiles dir with new TuneD versions
Since v2.23.0, TuneD changed the default directory under which it
looks for profiles. The profiles are newly nested under `profiles/`
directory. More information in [1].

Modify the stage implementation to check if the default profile
directories contain `profiles/` directory. If yes, then look for
profiles in it. If not, use the original behavior.

[1] https://github.com/redhat-performance/tuned/releases/tag/v2.23.0

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-11-25 10:22:21 +01:00