This makes sure all disk access is backed by the same disk. We may
want this for performance reasons (avoiding moving across disks), but
also to experiment with different backing stores for all disk access.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Currently /var was always backed by /var/tmp, but we may want to
control exactly what it is backed by. The default is the same, so
this is not a behavioral change.
The dnf stage wants to import `osbuild.sources` but currently the
osbuild module is not available in the stages. Apply the same hack
done in the Assembler also in for the stages, i.e. bind mount the
osbuild module to the stages/osbuild.
The Fedora 31 and Fedora 32 runners are symlinks but the spec file
only looked for files and not symlinks. Fix that.
Follow up by 6a14ba40f7. Found by
Lars.
Add a new sample that supports hybrid boot, i.e. legacy boot and
UEFI, like f30-hybrid-qcow2, but with a separate boot partition.
The latter means that 'fix-bls' as well as the 'grub2' stage need
to be adapted for it: 'fix-bls' needs have the prefix set to "/"
and grub2 needs to have a 'boot_fs_uuid' option pointing to the
uuid of the filesystem for the boot partition.
The grub prefix ("/boot/grub2") should be defined as relative to the
mountpoint of the filesystem containing it, i.e. /boot/grub2 if it is
on the root filesystem or /grub2 if boot is on a separate partition.
The paths for the kernel and the initrd in the BLS snippets are
meant to be relative to the root of the filesystem they are on.
The current code assumes that kernel and initrd are installed
under '/boot' and that '/boot' is on the root file system and
thus all paths get fixed up to start with '/boot/…'. But the
'/boot' directory can be on a separate partition and thus file
system, and then paths need to be relative to that and should
be fixed up with '/…'. Introduce a new option 'prefix' that
can be used to manually specify the prefix after the fixup,
defaulting to '/boot' for backwards compatibility.
NB: The canonical Boot Loader Specification[1] requires that
a separate partition is used boot related files and it will
be mounted at '/boot' (or '/efi').
[1] https://systemd.io/BOOT_LOADER_SPECIFICATION/
Up until now the grub config theoretically supported having different
values for 'root' (via grubenv's $GRUB2_ROOT_FS_UUID) and 'boot' (via
grubenv's $GRUB2_BOOT_FS_UUID). 'boot' is a leftover from the initial
implementation when grub was looking for BLS snippets also in the ESP,
and will be removed in the future.
In our and also the canonical Fedora/RHEL grub configurations the BLS
are in the very same location for UEFI and legacy and thus 'boot' is
always 'root'.
Therefore we get rid of the extra grubenv variable refering to 'boot'
and just set 'boot' to 'root' after that was discovered.
Add a new `boot_fs_uuid` option for when a separate partition is
being used for '/boot' with the indicated uuid. This will then be
used for the grub2 "root" and "boot" variables. Additionally, in
the redirect config, need to refer to files and directories paths
relative to the partition they are contained, i.e. /boot/grub2 if
/boot is on root ('/') or /grub2 if /boot is on a extra dedicated
partition.
This happens rarely when the same loop device is used in rapid
succession. The kernel flushes the page cache asynchronously, which
means that it might not be cleared yet when a new file is bound.
`set_status` checks if the cache is clear (`set_fd` doesn't).
Handle this by trying a different device when `set_status` returns
`EBUSY`.
Fixes#177
Don't wait until python's garbage collector closes the file descriptors
to loop devices. Close them when the `LoopServer` context manager exits,
after an assembler has finished running.
Instead of directly encoding the repo configuration make use of the
new sources api. The corresponding entry in samples/sources.json
which is referenced here ("sha256:450d4c0…") was added with the
previous commit ("92cfc57d720…").
"it's the future! 🛸" - Lars Karlitski
Bare bones example for image creation on s390x with MBR partition
layout and zipl bootloader. The kernel command line is configured
via the new org.osbuild.kernel-cmdline stage which needs to be run
before the dnf stage.
Support the s390x bootloader zipl (z Initial Program Loader). We
supply the parameters for the kernel+initrd as well es the target,
i.e. the boot partition where the bootmap is creating, the device,
here called 'targetbase', to install the bootloader on, including
parameters describing the device (type, blocksize) and also the
offset of the partition containing the target from the start of
device (in sectors).
The kernel and initrd are found via the bootloader entry, ignoring
the rescue kernel.
Since zipl needs the device as well as access to the boot partition
the image is bound to a loopback device. Also keep the filesystem
tree mounted during the execution of the zipl installation.
Add a stage test to check that the new kopts stage is creating the
target file /etc/kernel/cmdline with the right content. Since tree
diff currently seems to lack support for content hashing new files
we work around this by creating first an empty /etc/kernel/cmdline
file and then get a content diff with the desired options set.
The canonical way to set the kernel commandline, which is used by
the kernel post install scripts, see kernel-install(8), is the
file /etc/kernel/cmdline, or in the case this does not exist,
/proc/cmdline. The new stages offers a way to write this file in a
more "type-safe" way, by providing explicit options for certain
well known params (for now only `root_fs_uuid`). Additional params
are specified via `kernel_opts`. This follows the grub2 stage name
convention.
The zipl stage is a fairly simple stage that just creates a file
in /etc called zipl.conf with a single configurable option, which
is called `timeout`. Check the file gets properly created with
the desired hash and verify that setting the timeout works.
The z Initial Program Loader (zipl) when creating the bootmap in
bootmap_creat (src/zipl/bootmap.c) wants to create a device node
via misc_temp_dev (bootmap_create:1141) for the device that it
is installing the bootloader to[1]. Currently access to loopback
devices is allowed from within the container (it is used to mount
the image), but only read/write access. On s390x also allow the
creation of device nodes, so zipl can do its work and install
the bootloader stages on the "disk".
[1] zipl source at commit dcce14923c3e9615df53773d1d8a3a22cbb23b96
Include the `bootloader` options in the STAGE_OPTS json schema.
Commit 8fcf7d5c4… introduce the `bootloader` option but the
corresponding schema entry was omitted.
Update the command line options help text as well as the sample
command line to build the `base-qcow2.json` to include the new
sources command line option.
Add a example demonstrating hybrid boot. The main ingredients are:
- grub2 stage has `uefi` and `legacy` options
- GPT partition layout
- BIOS boot partition so that core image for grub2 legacy can
be stored (small, 1MB, type 21686148-6449-6E6F-744E-656564454649)
- ESP partition so UEFI can load grub*.efi (via the shim)
In the case that the image should support booting via EFI *and*
legacy grub, i.e. hybrid booting, the canonical grub config is
stored in /boot/grub2 just as for normal legacy booting. The
config file for efi grub is a very small one that will just look
for the partition containing the /boot/grub2/grub.cfg file and use
then set the prefix accordingly and load that file. In the hybrid
case grubenv file also will just be located in /boot/grub2 and
not in the ESP therefore the symlink that was created by the
package needs to be removed.
With the introduction of the `bootloader` option, grub2 legacy
installation setting changed. Before, grub2 legacy installation
was dependent on the partition scheme, i.e. only when dos/mbr
layout was used grub2 got installed. After the change the default
is to install it unless `bootloader.type" is explicitly set, even
if the partition layout is GPT. But a legacy grub2 installation
on GPT requires a BIOS boot partition, so the new default is not
right for the case of pure (non-hyrid) UEFI images.
Therefore revert to the old behavior of only defaulting to grub2
legacy if the option is not explicitly set *and* the partition
layout is "dos"/"mbr".
Adapt the f30-qcow2-gpt sample, which is non-uefi grub2 legacy
but with GPT and a bios boot partition, to explicitly request
the grub2 bootloader.
In the case that legacy is of type bool it is automatically converted
to the platform string ("i386-legacy"). This is mainly done to keep
backwards comparability as it was just a boolean before. But the auto
conversion did not take the actual *value* of the boolean into account
meaning "legacy: False" would be turned into "legacy: i386-pc" and
thus effectively changing the value from False to True.
dnf skips unavailable repositories by default, which only leads to
harder understand errors later. Configure it to fail when any of the
passed repositories cannot be reached.
Add support for dnf's sslcacert, sslclientcert, and sslclientkey
options. The latter two are passed as secrets (clientcert as well
because it might be a pem file that also includes the private key).
Sources run on the host, so their options may contain paths to the host
file system. Make use of that by accepting only paths in those options,
because it allows using tools to deal with certificate files.
Also make sure that the dnf source only returns options it knows about.
Add a new command line option `--secrets`, which accepts a JSON file
that is structured similarly to a source file. It is should contain data
that is necessary to fetch content, but shouldn't appear in any logs.
As noted in earlier commits the grub2 boot image needs to be patched
to contain the position of the grub2 core. By default, the location
in the boot image is hard-coded to be the mbr gap (sector 1) but for
GPT partition schemes a separate BIOS boot partition is used that is
located at a "random" location. Refactor the code to generalize the
boot image patching, where the default mbr gap location is just a
special case of the general.
