Based on the UEFI sample (f30-base-uefi.json). NB: the inclusion
of the dracut-config-generic is needed to disable "host-only" for
dracut so the initramfs will include the virtio_blk block device
driver that is needed to mount the root file system when running
the image in qemu.
Add mkfs_vfat and hook it up into the generic mkfs_for_type()
dispatcher function. Install grub2 to the MBR only if the partition
table is of type "MBR".
Introduce two new assembler options `pttype` and `partitions` to
allow fine grained control over how the partition table is created.
The first one controls the partition type, either `mbr` (default,
when the key is missing) or `gpt`; if specified the `partitions`
key must contain a list of objects describing the individual
partitions (`start`, `size`, `type`) together with a `filesystem`
object describing the filesystem (`type`, `uuid`, `mountpoint`) to
be created on that partition.
In the case the `pttype` option is missing, the legacy mode is used
where `root_fs_uuid` and `root_fs_type` need to be specified.
Use the newly available partition information in the install_grub2
method: detect which module to use for the root filesystem and
assert the second stage fits between the MBR and the first partition.
Introduce a generic mkfs_for_type() function that will dispatch
to the correct mkfs function depending on the type. Additionally
refactor the partition creation and mounting code to handle more
than one partition.
Part of the refactoring to support uefi/gpt: the method that creates
the partition table now returns an array of dictionaries corresponding
to the individual partitions that have been created together with the
information for the filesystem that this partition should end up with.
Prepare the stage for uefi/gpt support by extracting the code that
installs GRUB and creates the partitions into its own functions.
Should not have any effect on the actual data written to the image.
Introduce two new configuration options: `legacy` and `uefi`. The
first one being a boolean (default: True) that controls if GRUB
modules, fonts and the configuration is installed in the right
locations to support legacy boot mode.
The `uefi` option (of type object with a single `vendor` property)
enables UEFI support by writing the configuration into the correct
EFI directory, "/boot/efi/EFI/<vendor>/grub.cfg", where vendor is
taken from said `vendor` property.
The workaround of manually linking /lib64 -> /usr/lib64 inside the
container that is needed on s390 is also required on ppc64 because
here the dynamic linker is set to /lib64/ld64.so.2 and the /lib64
link is not created.
Work around a combination of systemd not creating the link from
/lib64 -> /usr/lib64 (see systemd issue #14311) and the dynamic
linker is being set to (/lib/ld64.so.1 -> /lib64/ld64.so.1)
Therefore we manually create the link before calling nspawn
Commit 283281f broke compression by appending the argument last to the
tar command line. It needs to appear before the file.
Fix that and add a test.
[teg: add minor fix]
osbuild currently throws an error when not passing a build environment
on the command line, because the runner is unset. This is annoying on
hosts which only need a runner set, but no build pipeline.
To simplify running osbuild in this common case, introduce
`org.osbuild.host`, which is a runner that is defined to work on the
host that osbuild is installed on. Use this runner by default and
include a symlink to the right runner in the Fedora and RHEL packages.
Also add `runners/org.osbuild.host` to `.gitignore`, so that developers
can set the symlink when running osbuild from the source directory.
Fixes#171
We've been using a generic `osbuild-run`, which sets up the build
environment (and works around bugs) for all build roots. It is already
getting unwieldy, because it tries to detect the OS for some things it
configures. It's also about to cause problems for RHEL, which doesn't
currently support a python3 shebang without having /etc around.
This patch changes the `build` key in a pipeline to not be a pipeline
itself, but an object with `runner` and `pipeline` keys. `pipeline` is
the build pipeline, as before. `runner` is the name of the runner to
use. Runners are programs in the `runners` subdirectory.
Three runners are included in this patch. They're copies of osbuild-run
for now (except some additions for rhel82). The idea is that each of
them only contains the minimal setup code necessary for an OS, and that
we can review what's needed when updating a build root.
Also modify the `--build-pipeline` command line switch to accept such a
build object (instead of a pipeline) and rename it accordingly, to
`--build-env`.
Correspondingly, `OSBUILD_TEST_BUILD_PIPELINE` → `OSBUILD_TEST_BUILD_ENV`.
`osbuild-run` sets up the build root so that programs can be run
correctly in it. It should be run for all programs, not just stages and
assemblers (even though they're the only consumers right now).
Also, conceptually, `osbuild-run` belongs to the build root. We'll
change its implementation based on the build root in a future commit.
The buildroot already sets up `/run/osbuild/api`. It makes sense to have
it manage libdir as well.
A nice side benefit of this is a simplification of the Stage and
Assembler classes, which grew quite complex and contained duplicate
code.
This introduces the `root_fs_type` option on the org.osbuild.rawfs
assembler. It only accepts "ext4" and "xfs" values right now and
defaults to "ext4" to preserve backwards compatibility.
This introduces the `root_fs_type` option on the org.osbuild.qemu
assembler. It only accepts "ext4" and "xfs" values right now and
defaults to "ext4" to preserve backwards compatibility.
This commit adds semi-structured documentation to all osbuild stages and
assemblers. The variables added work like this:
* STAGE_DESC: Short description of the stage.
* STAGE_INFO: Longer documentation of the stage, including expected
behavior, required binaries, etc.
* STAGE_OPTS: A JSON Schema describing the stage's expected/allowed
options. (see https://json-schema.org/ for details)
It also has a little unittest to check stageinfo - specifically:
1. All (executable) stages in stages/* and assemblers/ must define strings named
STAGE_DESC, STAGE_INFO, and STAGE_OPTS
2. The contents of STAGE_OPTS must be valid JSON (if you put '{' '}'
around it)
3. STAGE_OPTS, if non-empty, should have a "properties" object
4. if STAGE_OPTS lists "required" properties, those need to be present
in the "properties" object.
The test is *not* included in .travis.yml because I'm not sure we want
to fail the build for this, but it's still helpful as a lint-style
check.
It turns out that rpm will happily check signatures on `--install`,
that's just not the default behavior, because of Historical Reasons.
This commit enables RPM's signature checking and drops our manual check,
which will probably speed up the RPM stage a little bit. Fun!
Oh, also there's two bonus code cleanups: one to use f-strings harder,
and one to make sure we ignore whitespace in package checksum strings.
As a general rule, using temporary files with predictable names is a
security risk. It probably isn't _actually_ a security risk inside
osbuild stages, since they're usually running in some kind of isolated
container environment, but it's still a better idea to use tempfiles.
This makes the dnf and yum stages put their temporary files into a
temporary directory that gets deleted after dnf/yum finishes.
Use the new the osbuild API to setup the standard input/output
inside the container, i.e. replace stdin, stdout, and stderr with
sockets provided by the host.
Introduce an osbuild API that can be used by the container to talk
to the osbuild host. It currently supports one method 'setup-stdio'
which should be used by the container to setup its standard input/
output so the stages can transparently do i/o with the osbuild host
via stdio.
The input data (args) is written to a temp-file backed buffer. The
output is either the host's stdout directly or another temp-file
backed buffer; the latter is re-opened (via /proc/self/fd) to get
another file-descriptor for the container, so in theory the host
and the container could do i/o to the same buffer independently.
Expose the flags, address parameter of the underlying sock.sendmsg
method, in order to be able to explicitly specify the recipient of
the message; as needed in connection-less mode.
Python 3.2 renamed array.fromstring to array.frombytes, but kept
the former as an, now deprecated, alias. Use the canonical form
which indeed better describes what is going on.
In case osbuild is invoked without libdir parameter, the osbuild files
are not propagated into the buildroot container and therefore all
pipelines containing buildroot fail.
Example:
```
$ sudo osbuild --store /var/osbuild/ qcow2-pipeline.json
...
execv(/usr/lib/osbuild/osbuild-run) failed: No such file or directory
```
Unfortunately this is only the first error. Once you fix it, you realize
that also the symlink from "assemblers" directory is missing and
therefore you cannot import osbuild because it is not available anywhere
in the path. This is why I had to bind the osbuild module from host to
the build container.
lorax-composer supports modifying timeservers, this stage implements it.
I was concerned if I should name this stage timeservers or chrony, but
I've decided to go with chrony. If some day in future Fedora/RHEL
changes the ntp client, we can easily introduce new stage named after
the new ntp client. Additionally, this solution enables us to create
systemd-timesyncd stage, which can change timeservers when chrony is not
installed (in that case systemd-timesyncd takes over the ntp
synchronization).
