The `size` option was using `-l`, which in fact was `--extents`.
Fix that to use `--size` and add a new option `extents` that
will in fact call translate to `--extents` and this replace the
current use of the `size` option.
Adapt the `fedora-ostree-image` test manifest to use `extents`.
Add RHEL 7.9 example manifests. Add them to a `rhel` sub-directory in
the test/data/manifests directory since we cannot re-generate them
in the normal github actions, because they require access to RHEL
content.
Ability to set the bootloader backend that OSTree should use. NB:
normally this should be set to `none` since in modern distros and
bootloaders the BLS is used and the BLS snippets are generated on
`none` but none of the of the specific bootloader tools are run,
like `grub2-mkconfig` for grub.
Update the fedora image manifest to use that config setting.
Add an optional `remote` to the deployment of the ostree so that
the it is tied to the specified remote. This is needed later for
updating the commit from that remote.
Greenboot is the idea of automatically rolling back bad updates,
i.e. updates that do not boot successfully. The implementation
is split between the boot loader and a user space component.
The latter sets two variables `boot_counter`, which indicates
the maximum number of boot attempts and `boot_success` which
tells the boot laoder if a previous boot was successful. The
bootloader on the other hand will decrement the counter variable
and reset the success indicator one.
An implementation of the user space component for rpm-ostree is
called `greenboot`.
Instead of using the version specific, pre-depsolved f34 build manifest,
use the new version agnostic build manifest (fedora-build.mpp). NB: this
is included directly as mpp so that its variables get defined by the
including manifest. This should make it even easier to update manifests
to new fedora releases.
Include a build manifest that is itself not have tied to a specified
version and thus is meant to be included with the following vars
pre-defined as .mpp file:
- arch architecture (x86_64)
- releasever release version (f34)
- snapshot rpmrepo snapshot (20210326)
Properly label the build root for the f34 build root manifest v2.
Also label the cp and tar binaries with `install_exec_t` so they
can read and copy labels unknown to the host.
The order of entries in a dictionary is not specified by the JSON
standard and hard to control when marshalling dictionaries in Go.
Since the order of mounts is important and the wrong order leads
to wrong mount trees change the `mounts` field to an array. This
breaks existing manifests but after careful deliberation it was
concluded that the original schema with mounts as dictionaries
is not something we want to support. Apologies to everyone.
Adjust the schema of the copy and zipl stage accordingly.
Instead of having the OSTree repo in the anaconda root squashfs,
put the it to the root of the iso. This has several advantages:
first and foremost, we don't have to make a huge squahfs file-
system. We don't have to compress the repo. Additionally, the
repo is now easily accessible by mounting the iso.
Intead of overwriting the standard ananconda kickstart file, we
move it to the root of the iso. To have anaconda pick it up an
additional kernel command line parameter is required.
F34+ has the xorg-x11-server-utils package split up. Install the
individual new packages we need for anaconda.
See Anaconda commit 55371c996861a47da9504b31118858dd3257a8b0
Use variables for various often occuring contants, like the
release, ref and isolabel. Additionally, use variables for
the snapshot so it can be changed easily.
Allow the manifest variables, defined via mpp-vars, to be used from
within the mpp blocks. For this template strings are used, where
variables are marked via `$`. We cannot use the `mpp-format` logic
easily there, since that is processed after other mpp directives
have been processed.
As a result remove the built-in substitution from support from dnf
dep-solving, since we had to post-process the resulting urls with
variable substitution afterwards. Now that is covered with this
more generic mechanism.
The manifests set a "en_US" locale but this causes gnome-terminal to not
run due a non UTF-8 locale being used, which is an unsupported config:
gnome-terminal-server[1899]: Non UTF-8 locale (ISO-8859-1) is not supported!
Reported-by: Stephen Smoogen <ssmoogen@redhat.com>
Use the new partiton layout support in MPP. NB: start data as
well as the size of the last partition have been omitted now,
since `sfdisk` will figure those out for us.
Previous versions of mpp would already set the arch and basearch
substitution, which would work for dep-solving itself, but not
properly re-write the resulting URLs which means that the manifest
was broken. Fix this by properly replacing the substitutions in
the URL. Also support official 'releasever' substitution.
Instead of passing dictionaries around that are inconvenient to
use in code and even more in the `mpp-format-*` directives, use
a simple class to represent package information. Use that in
the `pkginfo` dict that can be accessed via `mpp-format-*`. Use
the `evra` property instead of string manipulation in the
`fedora-boot.mpp.json` and `-ostree-bootiso.mpp.json` manifest.
This uses size computations to simplify the partition size/offsets
and the depsolv results to get the kernel version.
This makes no changes to the resulting json files.
Add new `org.osbuild.cloud-init` stage, which currently allows to create
configuration files for cloud-init under `/etc/cloud/cloud.cfg.d`. The
stage supports only a very limited subset of cloud-init configuration
options, which is covering needs of RHEL AMI images.
The schema mandates that if the 'configuration_files' option is
specified, then at least one configuration file must be defined. In
addition each section of the configuration must contain at least one
property (section or configuration option).
Add `python3-pyyaml` package to the `F34-build` testing manifest,
because it is required for running and testing the new stage.
Regenerate all affected manifests.
Add test for the new stage.
Update the `osbuild-ci` container image used for testing to a new tag,
which includes python3-pyyaml, the dependency of the new stage.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The logic to sort urls was added globally in `mpp-import-pipeline`
but only the in the v1 code path was the `state.manifest_urls`
variable set and thus for v2 the actual sorting did not happen.
Fix this and set the `manifest_urls` to the `org.osbuild.curl`
items, which makes sense because we only know how to sort those.
For `setfiles` in the `org.osbuild.selinux` to work properly it
needs itself have the correct labels. This is true for setfiles
on the host, but also needs to be true for the `setfiles` that
is located in the build root. Therefore we need to label the
build root via `org.osbuild.selinux`. Additionally re-label
the `cp` and `tar` binary to have the `install_exec_t` which is
needed to read and write unknown labels (to the host). Since
`cp` and `tar` are used to read (and write) files inside stages
and assemblers they need to have the special label.
The commit that added the ostree tarball to manifest version 2
went in after the PR to sort the urls and thus the source urls
for that manifest were not sorted. This of course no breaks ci
which makes sure that the test data up to date. I blame the
ci model used by github but I am also sorry.
In both mpp-depsolve and mpp-import-pipeline, sort the packages to
url dictionary before writing the JSON. This makes it easier to
look for packages but more importantly ensures that the resulting
set of packages has the same ordering in the sources section
independently of how it was assembled.
Instead of operating directly on a file, which was previously specified
by `filename`, operate on a device. This is more flexible since a file
can be accessed via a loop back device; but the inverse is obviously
not true, like other devices can not be accessed via a plain file.
Therefore, re-factor the stage to use a device and adapt the existing
test (`fedora-ostree-image`).
Add a new manifest that creates an ostree commit, deploys that,
creates a raw image and copies the deployment into it. The
resulting artefact is a bootlabel qcow2 image.
Move from using 'zram' to 'zram-generator-defaults' in the ostree bootiso
testing manifest. More information is available in Fedora 33 Change
document [1].
Add org.osbuild.kernel-cmdline stage to fedora-boot.json manifest
because of change in how grub handles the kernel command line arguments
[2].
GRUB2 Stage 2 checksums in assemblers test are updated. The change have
been verified by building the fedora-boot.json manifest with each checked
filesystem and booting the image in QEMU with legacy mode.
[1] https://fedoraproject.org/wiki/Changes/SwapOnZRAM
[2] https://github.com/osbuild/osbuild-composer/pull/982#issuecomment-697356929
Signed-off-by: Tomas Hozza <thozza@redhat.com>
This new manifests creates an boot iso which contains an ostree
commit that is build in the same manifest. The boot iso can then
be installed via the ISO.
This is to test the functionality of the new `saved_entry` grub2
config file. Ideally an integration test would install a new non-
default kernel and check that it does not get selected. Something
for the future.
In addition to the required base layer, provided via the the input
of the same name, the oci-archive stage now accepts up to nine
additional layers that get added on top of each other, sorted in
ascending order, i.e. `layer.1` to `layer.9`.
Adapt the `fedora-ostree-container` example manifest so that the
ostree commit is now in a separate layer, which makes it possible
to share the base layer between different commits container.
Change the test manifests that use containers to not include
docs when installing. Also don't install docs in the build root
for those manifests. Since the fedora-ostree-container.mpp is
being built in CI, this also tests that the new option.
