Instead of maintaining a separate set of samples that by now are very
much outdated (using Fedora 31 or older), make the samples directory
a symlink to the test data. Manifests in there are indeed tested and
maintained.
The error and noop samples are also covered in unit tests, so no need
for extra samples there either.
CI: remove the sample validation since all the test data manifests
are actually built.
The documentation states that the stage accepts any options and
dumps them but the schema was not allowing any. Fix that and also
change the sample to include a random option.
The mirror used in f30-ppc64le example did contain the RPM packages but
there were two issues
* f32 is not in development any more
* ppc64le is available as a secondary arch
The patch also renames the file as it contains packages for fedora 32.
Introduce a new `tar` option, which when given together with the
required `tar.filename` option, will result in the output of the
assembler being a tarball that contains the repo and the compose
information (`compose.json`).
Requires the `tar` command to be present in the build root. Modify
the sample to use that option and include the tar for the build
pipeline.
This converts all fedora links in our samples to `mirrors.kernel.org`.
This mirror works best from around the world, so lets avoid the wild
mix of local mirrors and instead use kernel.org.
This mirror is also well-managed and properly funded, so we should not
run into too many problems with it.
That was mostly prepared already, the one last missing piece was
to use the default grub2 stage (previous commit) and now, with
this commit, to actually install the grub2 legacy bootloader via
the qemu assembler.
An example that takes an ostree commit and creates a qcow2 image
that is bootable. NB: the commit ids and the corresponding source
will need to be adjusted after having run the pipeline that will
create the ostree commit.
This is a Fedora 31 based example that results in a ostree repo
with a ostree commit. The latter was composed from the file system
tree that was created via the given osbuild stages.
For the sake of backwards compatibility, legacy support was enabled
by default. Flip this around, so that leaving the parameter out
means disabling it.
This is more intuitive, and will pave the way for dropping support
for the value being a bool in the future.
`osbuild-composer` always passes the argumnet explicitly, though
still always as a boolean.
Signed-off-by: Tom Gundersen <teg@jklm.no>
The sources.json file contains repositories for x86_64 and s390,
this patch adds the same for aarch64 Fedora 30 so that the example
pipeline can be build easily.
The "legacy" option in stages/org.osbuild.grub2 defaults to True:
`legacy = options.get("legacy", True)`
If both legacy and uefi is enable, the stage will try to create a hybrid
image, but that fails for aarch64. This patch makes sure we use uefi
mode only which fixed this issue.
I tested the patch by hand together with the fix from this PR:
https://github.com/osbuild/osbuild/pull/308
We will mention this example in our man-page, so make sure it actually
works. This imports all sources into the pipeline definition and
adjusts the syntax to match what we expect.
Even on ubuntu we can build rpm-based pipelines without bootstrapping
via fedora 27. Drop the build env from the travis config and from our
samples directory.
Signed-off-by: Tom Gundersen <teg@jklm.no>
As long as this matches the build environment, this does not make
a differenece, but let us not depend on this.
This will be useful when automatically transforming dnf to rpm
pipelines, as the platform_module_id is needed as input to
osbuild-composer's dnf-json tool.
Performed using this script:
```
cat $1 | jq '(.stages[]? | select(.name == "org.osbuild.dnf") | .options.module_platform_id) |= . + "platform:f30"' | sponge $1
cat $1 | jq '(.build.pipeline.stages[]? | select(.name == "org.osbuild.dnf") | .options.module_platform_id) |= . + "platform:f30"' | sponge $1
```
Signed-off-by: Tom Gundersen <teg@jklm.no>
Add a new sample that supports hybrid boot, i.e. legacy boot and
UEFI, like f30-hybrid-qcow2, but with a separate boot partition.
The latter means that 'fix-bls' as well as the 'grub2' stage need
to be adapted for it: 'fix-bls' needs have the prefix set to "/"
and grub2 needs to have a 'boot_fs_uuid' option pointing to the
uuid of the filesystem for the boot partition.
Instead of directly encoding the repo configuration make use of the
new sources api. The corresponding entry in samples/sources.json
which is referenced here ("sha256:450d4c0…") was added with the
previous commit ("92cfc57d720…").
"it's the future! 🛸" - Lars Karlitski
Bare bones example for image creation on s390x with MBR partition
layout and zipl bootloader. The kernel command line is configured
via the new org.osbuild.kernel-cmdline stage which needs to be run
before the dnf stage.
Add a example demonstrating hybrid boot. The main ingredients are:
- grub2 stage has `uefi` and `legacy` options
- GPT partition layout
- BIOS boot partition so that core image for grub2 legacy can
be stored (small, 1MB, type 21686148-6449-6E6F-744E-656564454649)
- ESP partition so UEFI can load grub*.efi (via the shim)
With the introduction of the `bootloader` option, grub2 legacy
installation setting changed. Before, grub2 legacy installation
was dependent on the partition scheme, i.e. only when dos/mbr
layout was used grub2 got installed. After the change the default
is to install it unless `bootloader.type" is explicitly set, even
if the partition layout is GPT. But a legacy grub2 installation
on GPT requires a BIOS boot partition, so the new default is not
right for the case of pure (non-hyrid) UEFI images.
Therefore revert to the old behavior of only defaulting to grub2
legacy if the option is not explicitly set *and* the partition
layout is "dos"/"mbr".
Adapt the f30-qcow2-gpt sample, which is non-uefi grub2 legacy
but with GPT and a bios boot partition, to explicitly request
the grub2 bootloader.
Include a example of a ppc64le (open firmware) pipeline based on
f30-s390x. It includes changes to the grub2 stage (selecting the
platform), the partition table contains the PReP partition needed
to store the grub2 core image.
Pipelines encode which source content they need in the form of
repository metadata checksums (or rpm checksums). In addition, they
encode where they fetch that source content from in the form of URLs.
This is overly specific and doesn't have to be in the pipeline's hash:
the checksum is enough to specify an image.
In practice, this precluded using alternative ways of getting at source
packages, such as local mirrors, which could speed up development.
Introduce a new osbuild API: sources. With it, a stage can query for a
way to fetch source content based on checksums.
The first such source is `org.osbuild.dnf`, which returns repository
configuration for a metadata checksum. Note that the dnf stage continues
to verify that the content it received matches the checksum it expects.
Sources are implemented as programs, living in a `sources` directory.
They are run on the host (i.e., uncontained) right now. Each source gets
passed options, which are taken from a new command line argument to
osbuild, and an array of checksums for which to return content.
This API is only available to stages right now.
Based on the UEFI sample (f30-base-uefi.json). NB: the inclusion
of the dracut-config-generic is needed to disable "host-only" for
dracut so the initramfs will include the virtio_blk block device
driver that is needed to mount the root file system when running
the image in qemu.
We've been using a generic `osbuild-run`, which sets up the build
environment (and works around bugs) for all build roots. It is already
getting unwieldy, because it tries to detect the OS for some things it
configures. It's also about to cause problems for RHEL, which doesn't
currently support a python3 shebang without having /etc around.
This patch changes the `build` key in a pipeline to not be a pipeline
itself, but an object with `runner` and `pipeline` keys. `pipeline` is
the build pipeline, as before. `runner` is the name of the runner to
use. Runners are programs in the `runners` subdirectory.
Three runners are included in this patch. They're copies of osbuild-run
for now (except some additions for rhel82). The idea is that each of
them only contains the minimal setup code necessary for an OS, and that
we can review what's needed when updating a build root.
Also modify the `--build-pipeline` command line switch to accept such a
build object (instead of a pipeline) and rename it accordingly, to
`--build-env`.
Correspondingly, `OSBUILD_TEST_BUILD_PIPELINE` → `OSBUILD_TEST_BUILD_ENV`.
Downloading the gpg key is fragile and kept causing our tests to fail.
In general, we want to limit the network access, so let's just embed
the gpg keys directly in the pipeline.
Fixes#133.
Signed-off-by: Tom Gundersen <teg@jklm.no>
It is similar to the official Fedora cloud base image except for few
minor differences. The reason for this divergence is that we don't want
to include all hacks that are currently present in the official
kickstart file. You can see it here as a reference:
https://pagure.io/fedora-kickstarts/blob/master/f/fedora-cloud-base.ks#_149
build-from-yum.json is the one that's being used for testing on Ubuntu.
Remove base-from-yum.json, because it's confusing to have two similarly
named pipelines like this.
Otherwise, sfdik would pick one at random. We want our images to be
reproducible to the extent possible, so we must move all randomness
out of the assemblers when we can.
Signed-off-by: Tom Gundersen <teg@jklm.no>
This allows given packages to be excluded from the transaction. This
is useful if you want to install a group with certain exceptions.
A common thing to do in kicktstart files is:
```
rm -f /boot/*-rescue*
```
By instead excluding the dracut-rescue-config package we end up
with:
```
"deleted_files": [
"/etc/kernel/postinst.d",
"/usr/lib/dracut/dracut.conf.d/02-rescue.conf",
"/usr/lib/kernel/install.d/51-dracut-rescue.install",
"/boot/initramfs-0-rescue-ffffffffffffffffffffffffffffffff.img",
"/boot/vmlinuz-0-rescue-ffffffffffffffffffffffffffffffff"
],
```
Signed-off-by: Tom Gundersen <teg@jklm.no>
Opt in to supporting the most common ones, if we want to support more
we can add support as the need arises.
Signed-off-by: Tom Gundersen <teg@jklm.no>
