Commit graph

789 commits

Author SHA1 Message Date
Michael Vogt
752f4af6b3 stages: add test for the new org.grub2.iso.legacy stage
This commit adds a small smoke test that ensures our new
org.osbuild.grub2.iso.legacy stages does not regress easily.
2025-01-30 08:29:43 +01:00
Simon de Vlieger
d92be415f8 stage/dnf.module-config: remove path
Instead of passing in the path we name the file according to the module
name. Path can be reintroduced later if absolutely necessary.

Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
2025-01-22 18:03:21 +01:00
Simon de Vlieger
0d25c845f8 stages: introduce dnf.module-config
A new stage that allows writing a DNF module configuration file to a
given path.

Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
2025-01-22 18:03:21 +01:00
Albert Esteve
55d53f58fd stages/mkfs.ext4: disable lazy_init
This helps to clear a risk concern for
automotive.

Add optional flag to org.osbuild.mkfs.ext4 stage
to enable/disable both lazy_itable_init and
lazy_journal_init extended options. Both mke2fs
options are controled by the same flag, namely
`lazy_init`.

Signed-off-by: Albert Esteve <aesteve@redhat.com>
2025-01-17 19:11:38 +01:00
Michael Vogt
1eb8138ab5 test: tweak test_grub2_inst.py to support running on Debian/Ubuntu
This commit adds support to run the tests on Debian/Ubuntu systems.
Here the binary is called `grub-mkimage` instead of `grub2-mkimage`,
in addition the "grub-pc-bin" package must be installed there.
2025-01-13 22:53:02 +01:00
Brian C. Lane
3bed7c7ace grub2.inst: Add support for creating iso9660 boot image
Don't include the "location" offset, and use just a path for the prefix
section to set the path of the grub2 modules on the iso. eg.

{
    "filename": "eltorito.img",
    "platform": "i386-pc",
    "core": {
        "type": "mkimage",
        "partlabel": "gpt",
        "filesystem": "iso9660"
    },
    "prefix": {
        "path": "/boot/grub2/"
    }
}
2025-01-13 22:53:02 +01:00
Brian C. Lane
41c0550a1f tests: Add simple test for org.osuild.grub2.inst stage
Tests that grub2-mkimage ran without errors, and that the image file
actually has something written to it.
2025-01-13 22:53:02 +01:00
Brian C. Lane
fd19ab41fb org.osbuild.xorriso: Add support for grub2 bootable iso
This creates an iso matching the current method used in Fedora where it
uses grub2 for BIOS and UEFI booting. Pass the path to the grub2 hybrid
mbr to the stage in the 'grub2mbr' field. eg.
    "grub2mbr": "/usr/lib/grub/i386-pc/boot_hybrid.img"
2025-01-06 21:23:23 +01:00
Tomáš Hozza
6a4a7c31c6 Stages/test/tuned: handle "improved" jsonschema messages
python-jsonschema improved the error messages for some special cases
since v4.21.0 [0], which means that we need to handle multiple versions
of the error message in unit tests.

[0] 8875c2ecb1 (diff-c21226b904760a669a70785494cd8ecf5fb1e7415fd25765dd02f0ad00394099R231)

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-06 11:53:46 -08:00
Tomáš Hozza
806f949e8c Stages/dnf5.sbom.spdx: don't use format strings for constructing paths
Use the new helper function `osbuild.util.path.join_abs()` for
constructing paths.

Fix #1964

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-06 11:53:46 -08:00
Tomáš Hozza
b6dd45af4d Stages/dnf4.sbom.spdx: don't use format strings for constructing paths
Use the new helper function `osbuild.util.path.join_abs()` for
constructing paths.

Fix #1964

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-06 11:53:46 -08:00
Tomáš Hozza
89f94af60d Stages/tuned: don't use format strings for constructing paths
Use the new helper function `osbuild.util.path.join_abs()` for
constructing paths.

Fix #1941

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-06 11:53:46 -08:00
Tomáš Hozza
67d9663c83 Stages/tuned: skip non-existing profile directories
Previously, the stage would try to call `os.scandir()` on non-existing
profile directories, which results in an exception. While the
directories that it scans are currently created by the TuneD RPM package
when installed, it it much nicer to gracefully handle their potential
non-existence, than to fail with an exception.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-06 11:53:46 -08:00
Tomáš Hozza
350ae9ed40 Stages/tuned: require profile to be non-empty string
The stage schema previously accepted an empty string as a profile name.
This would not work in practice, therefore extend the schema to require
non-empty strings.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-06 11:53:46 -08:00
Tomáš Hozza
d7d41b477b Stages/tuned: add unit tests for the stage.
Fix #1941

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-06 11:53:46 -08:00
Tomáš Hozza
f173efea56 Stages/dnf*.sbom.spdx: explicitly mention DNF version in the summary
Explicitly mention the used DNF version in the stage summary.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-01-06 11:53:46 -08:00
Brian C. Lane
902c4f7d71 erofs: Add ability to set the cluster size
`cluster-size` sets the maximum size of compress physical cluster size
in bytes.
2025-01-03 12:05:56 +01:00
Brian C. Lane
1ab098fbf9 erofs: Add zstd to the list of supported compression algorithms
This is supported in erofs-utils 1.8 and later.
2025-01-03 12:05:56 +01:00
Colin Walters
fcb93bde01 kickstart: Drop ostree signature verification flag
It's a no-op nowadays, we dropped support for the custom
ostree-container signature verification status.

Signed-off-by: Colin Walters <walters@verbum.org>
2024-12-18 16:34:30 -08:00
Dusty Mabe
cd19587464 stages/coreos.live-artifacts: move rename of vendor dir
Seems more appropriate in copy_configs_and_init_kargs_json() since this
is where files were originally copied in.
2024-12-18 11:09:29 -05:00
Dusty Mabe
d5aff7b1af stages/coreos.live-artifacts: drop usage of dir fd
Rework rename of vendor directory to not use dfd APIs.

This was requested in code review.

Also added comments since I now understand it better.
2024-12-18 11:09:29 -05:00
Dusty Mabe
45583720af stages/coreos.live-artifacts: stop chmod initrd.img
The initramfs.img in Fedora and RHEL CoreOS images is already 644
so this isn't needed.
2024-12-18 11:09:29 -05:00
Dusty Mabe
b8d986b51c stages/coreos.live-artifacts: drop kernel rename for s390x
The rename to get file names + suffix with < 8 characters
was done in [1] when our initramfs was named initramfs.img.
It was subsequently renamed to initrd.img in [2] and the
rename of the initramfs was dropped but the rename of the
kernel was never dropped. Since vmlinuz is already < 8
characters let's just drop the rename here too.

[1] 60400910bb
[2] 6f533ef55e
2024-12-18 11:09:29 -05:00
Michael Vogt
af710ee740 stage: add test for core live-artifacts stage
This commit adds some unit tests around the coreos live-artifcats
mono stage.

- test/coreos_live_artifacts: add test for align_initrd()
- test/coreos_live_artifacts: add test for extend_initramfs()
    - This actually tests the mkinitrd_pipe() function, which
      extend_initramfs() calls after opening the file.
- test/coreos_live_artifacts: add test for make_stream_hash()
- test/coreos_live_artifacts: add test for make_efi_bootfile()

Co-authored-by: Achilleas Koutsou <achilleas@koutsou.net>
2024-12-18 11:09:29 -05:00
Achilleas Koutsou
4e93a66311 stages/coreos.live-artifacts: refactor into more functions
This improves the readability and maintainability of the code to
have it split into smaller pieces.

Co-Authored-by: Dusty Mabe <dusty@dustymabe.com>
2024-12-18 11:09:29 -05:00
Jonathan Lebon
0331e6f313 stages: add coreos.live-artifacts.mono stage
This adds a new `org.osbuild.coreos.live-artifacts.mono` stage to build
CoreOS Live ISO/PXE artifacts. The code is heavily based on the
`cmd-buildextend-live` script from coreos-assembler [1], but a lot of
things had to be adapted:
- the stage is provided the deployed oscontainer tree, metal, and
  metal4k images as inputs
- we use chroot instead of supermin to execute some commands in the
  context of the target oscontainer
- a bunch of calls that were wrapped by libguestfs for us (e.g.
  mkfs.vfat, mksquashfs), we now have to call ourselves; to retain
  maximum compatibility, we ensured that we still effectively use the
  same args that libguestfs passed

And various other minor adjustments.

Of course, this is not really in line with the OSBuild philosophy
of having smaller-scoped stages. We have labeled this with a .mono
suffix to denote it is monolithic, similar to the existing
`org.osbuild.bootiso.mono` stage today.

Eventually we may be able to break this stage down if we find it worth
the effort. Alternatively the need for it may go away as we align more
with Image Mode.

[1] 43a9c80e1f/src/cmd-buildextend-live

Co-authored-by: Dusty Mabe <dusty@dustymabe.com>
Co-authored-by: Renata Ravanelli <renata.ravanelli@gmail.com>
2024-12-18 11:09:29 -05:00
Tomáš Hozza
7699f8134f Add org.osbuild.dnf5.sbom.spdx stage
Add DNF5 version of the `sbom.spdx` stage, which generates an SPDX SBOM
document from a given FS tree using libdnf5 API.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-12-18 05:46:45 +01:00
Brian C. Lane
41658da9cf org.osbuild.grub2.iso.legacy: Add grub2 setup for booting BIOS ISO
This is the BIOS version of the grub2 iso stage. It installs the config
file and copies over the grub2 modules to /boot/grub2/
2024-12-12 09:29:25 +01:00
Brian C. Lane
5a8b2edaa0 typo: Fix incorrect 'tar archive' descriptions in stage metadata
SSIA
2024-12-10 15:16:33 +01:00
Tomáš Hozza
f1c43ae5bd util/sbom/spdx: rename {,s}bom_pkgset_to_spdx2_doc()
Rename the function for consistency reason. The parent package is named
SBOM (originally BOM).

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-12-02 23:24:39 +01:00
Achilleas Koutsou
3bf6ca399f test: add new containers.unit.create options in test 2024-12-02 12:55:30 +01:00
Pierre-Yves Chibon
17fda9e318 Expand the options available in containers.unit.create
Add the option to specify the SELinux type label and file type label for
the process.
Add the option to specify a tmpfs mount to create into the container.

Signed-off-by: Pierre-Yves Chibon <pingou@pingoured.fr>
2024-12-02 12:55:30 +01:00
Achilleas Koutsou
af7b7db66f test: update users stage unit tests
Update the unit tests to match new calls and options.
2024-11-26 23:35:04 +01:00
Achilleas Koutsou
97007b9e3d stages: run useradd, usermod, and passwd with --root
The useradd, usermod, and passwd commands support a `--root` option that
handles chroot-ing for the command.  In general, we prefer using this
option for commands that provide it and relying on the utility itself to
know how to set up the chroot in the way it needs.

The option has been available for these commands since 2011 [1] and it's
unclear why they weren't used originally.

The `mkhomedir_helper` command is still run using our Chroot context, so
the fix introduced in 9071cd0abb is
unaffected.

[1] 365279ea95/ChangeLog (L1339)
2024-11-26 23:35:04 +01:00
Achilleas Koutsou
8e3d054099 stages: use util.chroot in all stages that call "chroot"
Use the chroot utility module for all cases where we need to chroot
during a stage's execution.

The advantage is that all stages use the same tested code path for
setting up a chroot and all chrooted commands run in the same
environment, with the /proc, /dev, and /sys filesystems mounted.
2024-11-26 12:13:44 +01:00
Tomáš Hozza
9071cd0abb Stages/users: use Chroot from osbuild.util.chroot
Use Chroot class from osbuild.util.chroot module, instead of calling
`chroot` directly. The class handles mounting of various paths in the
chroot to make us more usable. This resolves new failure when running
the stage test on F41 results in `mkhomedir_heper` failing with `6`
return code, meaning permissions denied.

Adjust the stage unit tests, because `chroot.Chroot` can't work with
`pathlib.Path`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-11-25 18:05:54 +01:00
Tomáš Hozza
a8aa6361b1 Stages/tuned: handle changed profiles dir with new TuneD versions
Since v2.23.0, TuneD changed the default directory under which it
looks for profiles. The profiles are newly nested under `profiles/`
directory. More information in [1].

Modify the stage implementation to check if the default profile
directories contain `profiles/` directory. If yes, then look for
profiles in it. If not, use the original behavior.

[1] https://github.com/redhat-performance/tuned/releases/tag/v2.23.0

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-11-25 10:22:21 +01:00
Tomáš Hozza
062feda60a Stages/dnf-automatic.config: don't fail on non-existent config file
On Fedora 41 with DNF5, the dnf-automatic plugin by default does not
install any configuration file. This means that the stage would fail in
such case.

Previously, the full config file was placed in /etc and its purpose was
also to document all possible options. The example config file is now
installed only in /usr/share/dnf5/dnf5-plugins/automatic.conf.

Relax the stage implementation to not fail when the configuration file
does not exist. Just log a warning and create the configuration file.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-11-25 10:15:17 +01:00
Tomáš Hozza
3ac6d405b5 Fix pylint issue E0606: possibly-used-before-assignment
Fix:
assemblers/org.osbuild.qemu:310:36: E0606: Possibly using variable 'prep_type' before assignment (possibly-used-before-assignment)
inputs/org.osbuild.tree:85:15: E0606: Possibly using variable 'path' before assignment (possibly-used-before-assignment)
stages/org.osbuild.sfdisk:58:36: E0606: Possibly using variable 'prep_type' before assignment (possibly-used-before-assignment)
stages/org.osbuild.systemd.unit:23:16: E0606: Possibly using variable 'unit_dropins_dir' before assignment (possibly-used-before-assignment)
test/mod/test_meta.py:219:29: E0606: Possibly using variable 'schema_part' before assignment (possibly-used-before-assignment)

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-11-25 10:09:18 +01:00
Tomáš Hozza
36a60d1c29 Test/autotailor: fix Python 3.6 compatibility
The test case is skipped in the upstream CI, because the `autotailor`
executable is not installed in the `osbuild-ci` image. This will not
be the case in the future and the CI run will reveal a Python 3.6
incompatibility in the test implementation. Fix it.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-11-25 09:12:19 +01:00
Nikita Dubrovskii
a8e8ebde44 org.osbuild.selinux: support for specifying where file_contexts comes from
file_context now can come from
- tree (current default)
- mount
- input

Example:
```
- type: org.osbuild.selinux
  inputs:
    tree:
      type: org.osbuild.tree
      origin: org.osbuild.pipeline
      references:
        - name:tree
  options:
    file_contexts: input://tree/etc/selinux/targeted/contexts/files/file_contexts
```
2024-11-14 17:49:26 +01:00
Nikita Dubrovskii
84d4de5770 org.osbuild.selinux: support operating on mounts
This adds support for specifying paths to operate on,
rather than just the root of the target:
```
- type: org.osbuild.selinux
  options:
    file_contexts: etc/selinux/targeted/contexts/files/file_contexts
    target: mount://root/path/to/dir
  mounts:
    - name: root
      source: disk
      target: /
```

or

```
- type: org.osbuild.selinux
  options:
    labels:
      mount://root/path/to/file: system_u:object_r:boot_t:s0
      mount://root/path/to/other/file: system_u:object_r:var_t:s0
  mounts:
    - name: root
      source: disk
      target: /

```
2024-11-14 17:49:26 +01:00
Nikita Dubrovskii
6a59e740e4 parsing: treat locations without scheme as belonging to 'tree://' 2024-11-14 17:49:26 +01:00
Miguel Martín
dd16c2b769 feat: add remove-signatures option to container-deploy stage
Add remove-signatures option to container-deploy stage.
The option will be translated to --remove-signatures
skopeo option and passed to skopeo when copying the container.
This option must be set when deploying signed containers.

Signed-off-by: Miguel Martín <mmartinv@redhat.com>
2024-10-29 14:23:02 +01:00
Florian Schüller
2d1e8553d7 stages/test/test_tar: fix regex to be more robust
For usecases where for example selinux is not supported,
we should expect more errors from tar so we should also accept this
when matching the string.

Kudos go to Achilleas Koutsou <achilleas@koutsou.net> for this hint
2024-10-26 20:04:37 +02:00
Florian Schüller
a1f02113cd stages/org.osbuild.tar: implement disk-full test
this should be an example environment
for more stages to test if they return a proper error
in a "disk full scenario"
2024-10-26 20:04:37 +02:00
Nikita Dubrovskii
23f01307b2 org.osbuild.mkdir: support creating dirs on mounts
This allows creating new directories on mounts:
```
- type: org.osbuild.mkdir
  options:
    paths:
      - path: mount:///boot/efi
  devices:
    disk: ...
  mounts:
    - name: boot
      target: /boot
      ...
```
2024-10-25 16:22:19 +02:00
Michael Vogt
a3e32f3823 util: drop absolute path from Chroot.run() calls
We currently use the absolute path of these binaries in the
helper. This has some advantages but given that we control the
inputs for PATH in general it seems unnecessary.

We are also slightly inconsistent about this in the codebase but
favor the non absolute path version. A quick count:
```
$ git grep '"chroot"'|wc -l
13
$ git grep '"/usr/sbin/chroot"'|grep -v test_|wc -l
8
```
for `mount` and `umount` it seems this is the only place that uses
the absolute path.

It's not an important change but it has the nice property that it
allows us to use e.g. `testutil.mock_command()` in our tests and
it would be nice to be consistent.
2024-10-24 10:06:46 +02:00
Michael Vogt
6cc0e584ba stages(tar): expose new transform option to tar stage
This commit adds a new `transform` option to the tar stages that
maps directly to the `--transform=` comamndline argument of tar(1).

This allows to transform the names while files/dirs are added to
a tarfile. This is useful for the `gcp` pipeline for
bootc-image-builder where we want to create a gcp tar file that
expects the disk image filename in the tar to be exactly `disk.raw`.

Note that tar allows only a single `--transform` and we leave it
to the user to construct `sed` expressions if multiple renames
are required.
2024-09-25 12:57:00 +02:00
Dusty Mabe
f01a3d3c01 stages/selinux: don't require file_contexts if labels passed
With the labels option the user is specifying the exact context
they want to set on the path so it's not necessary to supply a
context here. This can be also useful in the case where you want
to set some labels and you haven't yet populated the tree yet.
2024-09-25 09:17:34 +02:00