On Python 3.6 we're getting a *different* error message from other
versions in CI. Let's accept both versions.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
For the terminal_input, terminal_output, and serial variables in the
GRUB_CFG_TEMPLATE, the keys were not set when those variables were not
defined in the options. This caused the template variables to show up
in the final file itself, which could break the config. For example,
the following line was being generated for one of our images:
serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1${terminal_input}${terminal_output}
Setting the keys to an empty string when they're not defined solves the
issue.
Since this is a stage behaviour change, the new behaviour is toggled by
an option that maintains backwards compatibility when not set.
This introduces a new convention of adding a `compat_version` number to
the stage options to control the behaviour. The value defaults to 1,
which is the old (broken) behaviour. Manifest generators should always
set it to 2 to get the correct behaviour.
In Python 3.14 configparser started validating keys. We use hacky bits
to write duplicate keys which means we have delimiters in our keys which
is now no longer allowed.
Let's explicitly turn off key validation for now as suggested by
mhroncok in [1].
[1]: https://github.com/osbuild/osbuild/issues/2109#issuecomment-2964585958
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
In Python 3.14 configparser started validating keys. We use hacky bits
to write duplicate keys which means we have delimiters in our keys which
is now no longer allowed.
Let's explicitly turn off key validation for now as suggested by
mhroncok in [1].
[1]: https://github.com/osbuild/osbuild/issues/2109#issuecomment-2964585958
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
Add the ability to exclude files and directories from the erofs
image. This uses the mkfs.erofs --exclude-regex feature, so simple
matches or regexes can be used.
Add the ability to exclude files and directories from the squashfs
image. This uses the mksquashfs -regex -e FILES... feature, so simple
matches or regexes can be used.
This also adds a new test for squashfs, based on the existing test for
erofs.
The `org.osbuild.tar` stage only supports auto compression which is
based on the filename. For several (newer) artifacts such as Vagrant
(`.box`) and WSL (`.wsl`) we want to explicitly give the compression
algorithm.
I've chosen a (few) commonly used compression algorithms. If others are
needed they are one-line followups away.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
Add a new %post option to the kickstart stage that supports adding
multiple post blocks to a kickstart file, with all the options supported
by the directive.
The `generic.ins` template uses:
images/kernel.img
images/initrd.img
However, kernel and initrd are currently stored as:
images/pxeboot/vmlinuz
images/pxeboot/initrd.img
This change ensures correct paths are used.
Extend the stage to support setting new options:
- Provisioning.UseCloudInit
- Provisioning.Enabled
Extend the stage test to use them and add a simple stage unit test for
the schema.
Related to https://github.com/osbuild/images/issues/1416
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The refclock directive can be used to specify one or more hardware
reference clocks to be used as a time source. Each refclock line must
specify a driver and a mandatory parameter, in the form:
refclock driver parameter
Drivers can have driver-specific options:
refclock driver parameter:[driver-option,...]
General refclock options can also be specified:
refclock driver parameter:[driver-option,...] [general-option]...
The stage options schema is written so that the "driver" property is an
object that must match one of four schemas corresponding to the four
drivers, each with a "name" property matching the driver name.
Each driver defines its required property and any optional
driver-specific options.
There are more general refclock options supported than the ones defined
in this commit, but we can add them if and when we need them in the
future.
Note that the restriction on the top-level stage options schema is now
lifted and any set of options can be specified. Servers are not
required. However, at least one top-level property is required still.
Docs: https://chrony-project.org/doc/3.4/chrony.conf.html
Support setting the `auto_enable_yum_plugins` option in the `rhsm`
section of `rhsm.conf`.
Add a unit test for the stage schema and also adjust the stage test
manifest.
Related to: https://github.com/osbuild/images/issues/1408
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
We originally required there to be only one kernel modules directory
since we typically only have one kernel installed. Change this check
up a bit since some kernel module RPMs aren't delivering a new kernel
but do deliver kmods that were built against a different kernel.
For example v4l2loopback from https://elrepo.org/
```
bash-5.1# rpm -ql kmod-v4l2loopback
/etc/depmod.d/kmod-v4l2loopback.conf
/lib/modules/5.14.0-427.13.1.el9_4.x86_64
/lib/modules/5.14.0-427.13.1.el9_4.x86_64/extra
/lib/modules/5.14.0-427.13.1.el9_4.x86_64/extra/v4l2loopback
/lib/modules/5.14.0-427.13.1.el9_4.x86_64/extra/v4l2loopback/v4l2loopback.ko
/usr/share/doc/kmod-v4l2loopback-0.12.7
/usr/share/doc/kmod-v4l2loopback-0.12.7/GPL-v2.0.txt
/usr/share/doc/kmod-v4l2loopback-0.12.7/greylist.txt
```
This patch now allows an image built from a Containerfile like the
following to work as input to the live artifacts stage.
```
FROM rhcos
RUN <<EOF
set -eux -o pipefail
dnf install -y https://www.elrepo.org/elrepo-release-9.el9.elrepo.noarch.rpm
dnf install -y v4l2loopback
dnf clean all
ostree container commit
EOF
```
Add support for setting environment variables for kernel-install and its
plugins. These can be used to control the behaviour of kernel-install
during the kernel rpm installation. Currently, we only add support for
$BOOT_ROOT, which we need to control the path where the kernel should be
installed.
Normally, kernel-install scripts will detect the correct path based on
the filesystem layout, specifically the ESP path. However, during our
installation the filesystem is not yet set up, so with this option, we
can control the installation path based on the location we know the ESP
will be mounted in the final image.
This is a very similar override to what we do with the
org.osbuild.fix-bls stage.
The stage option is added under a kernel_install_env object so we can
potentially support more of the known env vars in the future.
Printing 'owner' in an fstring triggers the mypy warning 'str-bytes-safe'
Error: If x = b'abc' then f"{x}" or "{}".format(x) produces "b'abc'",
not "abc".
We can't yet use the nicer run() argument, 'text=True', because we still
target the RHEL 7 build root which only has Python 3.6 and 'text' was
introduced in Python 3.7.
Use a temporary directory for the output of grub2-mkimage.
This makes the stage clean up the grub2-core.img from the build root
after its done. It also has the nice side-effect that unit tests that
call the stage are independent. Previously, a bug in the stage *might*
have been missed if a certain configuration of the stage was not
creating the grub2-core.img. One unit test could create an image at the fixed path
(/var/tmp/grub2-core.img) and then another one could call the stage with
the buggy configuration but the `shutil.copyfile()` call at the end of
the stage would succeed because it would find the image from the
previous stage run.
To accommodate for this change, the unit test with the mocked run call
is adjusted to intercept the random tmp output path and use it to create
a fake file for the stage to succeed.
This commit adds a test that mocks subprocess.run() so that we
can regression test the arguments passed to "grub-mkimage".
This is a regression test for
https://github.com/osbuild/osbuild/issues/2037
When writing the grub2 core image, the format should be set based on the
platform. This was accidentally changed to be `i386-pc` for all
platforms except when building ISOs.
Set the format to the platform and only change it to `i386-pc-eltorito`
when building ISOs.
This commit replaces the `/usr/bin/logger` binary in the dracut
chroot with a bind mount to `/usr/bin/true` to silence the spam
that we get from dracut during initramfs generation:
```
logger: socket /dev/log: No such file or directory
```
Unfortunately I could not find a nicer way, it seems it is
not possible to simply pass `sysloglvl=0` via the commandline
or an environment.
The extra complication here is that the dracut stage mounts
`devtmpfs` which will likely include:
```
/dev/log -> /run/systemd/journal/dev-log
```
but of course inside this chroot there is no `/run` which
leads to these messages.
Closes: https://github.com/osbuild/osbuild/issues/1976
Add an additional option called `kernel_line_size`
to allow setting a maximum cmdline size check
value for custom kernels or other restrictions.
This will override the arch defaults, if not set,
then the size map is checked, and if the current
architecture is not in the map, fallback to
4096, which is the max value allowed for
COMMAND_LINE_SIZE.
Signed-off-by: Albert Esteve <aesteve@redhat.com>
Add check to ensure that the size of
the parameters does not exceed the
maximum kernel cmdline size.
Otherwise, the parameters will
be truncated and the command line
will fail.
The size is arch-dependant. In
order to not to over-complicate
the search of the value in the
kernel files (which will probably
not be installed in most cases),
it uses a map with some values
for common architectures.
If architecture is not found in
the map, defaults to 4096, which
is the maximum posible size for
COMMAND_LINE_SIZE.
Signed-off-by: Albert Esteve <aesteve@redhat.com>
Instead of passing in the path we name the file according to the module
name. Path can be reintroduced later if absolutely necessary.
Signed-off-by: Simon de Vlieger <supakeen@redhat.com>
This helps to clear a risk concern for
automotive.
Add optional flag to org.osbuild.mkfs.ext4 stage
to enable/disable both lazy_itable_init and
lazy_journal_init extended options. Both mke2fs
options are controled by the same flag, namely
`lazy_init`.
Signed-off-by: Albert Esteve <aesteve@redhat.com>
This commit adds support to run the tests on Debian/Ubuntu systems.
Here the binary is called `grub-mkimage` instead of `grub2-mkimage`,
in addition the "grub-pc-bin" package must be installed there.
Don't include the "location" offset, and use just a path for the prefix
section to set the path of the grub2 modules on the iso. eg.
{
"filename": "eltorito.img",
"platform": "i386-pc",
"core": {
"type": "mkimage",
"partlabel": "gpt",
"filesystem": "iso9660"
},
"prefix": {
"path": "/boot/grub2/"
}
}
This creates an iso matching the current method used in Fedora where it
uses grub2 for BIOS and UEFI booting. Pass the path to the grub2 hybrid
mbr to the stage in the 'grub2mbr' field. eg.
"grub2mbr": "/usr/lib/grub/i386-pc/boot_hybrid.img"
