Port the `org.osbuild.ostree.commit` assembler to a new assembler
like stage. The notable differences are: the fs tree preparation
now happens in the `ostree.preptree` stage and there is no option
to create a tarball, since that should be handled by a tar stage.
This new stage is a combination of the existing `rpm-ostree` stage
and the old `ostree.commit` assembler. It first does prepare a new
OSTree compliant root fs (previously done in `ostree.commit`) and
then uses `rpm-ostree tree postprocess` to perform various post-
processing on the tree to make it fully OSTree compliant.
Explicitly declare the schema for version 2. Notable differences
are the declaration of the `packages` input as well is the lack
of that property in `options`.
Convert the `org.osbuild.ostree` stage to use inputs instead of
sources. In the format (version 1) loading code, convert the
stage to use an input based on the existing stage options.
The stage was structured in a way so that it supports many different
inputs, like, `archive` and in the future maybe trees, git checkouts.
This does not fit very well into the new input model, where instead
seperate `copy.tree` and `copy.files` stages might make more sense.
Since the stage is not been used anywhere, and mainly got added
as a way to get the ignition dracut module via an archive of a git
repo, remove the stage for now. A replacement will be brought back
in the future.
The sysconfig directory contains a variety of system configuration files. The
values found in each of the configuration files may need to be specified
before first boot.
Currently, only the kernel and network configs can be modified and will
override existing files.
Add new org.osbuild.rhsm stage to configure to configure RHSM DNF
plugins. The stage currently supports only enabling / disabling the DNF
plugins. The stage's configuration schema allows extending it in the
future to configure other aspects of RHSM if needed.
The schema specifies each DNF plugin as an explicit object. The reason
is that although currently only setting of one common option (enabled)
is allowed, the 'subscription-manager' plugin's configuration actually
allows one additional plugin-specific option. The stage may support
setting it in the future, which will be easier with distinct objects for
each plugin.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Instead of reading the arguments from sys.stdin, which requires
that stdin is setup properly for that in the runner, use the new
api.arguments() method to directly fetch the arguments.
Also fix missing newlines between imports and methods to be more
PEP-8 complaint, where needed.
Generate and report metadata about all the packages that were
installed. This information will be needed by composer, especially
the 'sigmd5' bit, for integration with koji[1].
[1] https://docs.pagure.org/koji/content_generator_metadata/
The `script` and `test` stages should not be used in produciton, and
their use should be discouraged in general. They may make sense for
debugging, but should not be shipped.
The test stage is still used by the boot tests, so leave that for now,
and only drop the scripts stage.
Signed-off-by: Tom Gundersen <teg@jlkm.no>
The documentation states that the stage accepts any options and
dumps them but the schema was not allowing any. Fix that and also
change the sample to include a random option.
It should be possible to use the stage to only set the default
target or disable services. There is no need to always require
having `enabled_services`.
Add the ability to mask services, which is done e.g. when building
installers. See systemctl(1) for more information about masked
services.
Modify the existing stage test to include a test for it.
Add a new `labels` option that can contain `path`: `label` pairs
to overwrite the default labels for `path`.
NB: These manually set labels will not survive a relabeling and
are most useful to adjust policy in the buildroot, e.g. for `cp`
to be able to copy labels unknown to the host, by labeling it
as `system_u:object_r:install_exec_t:s0`.
This brings it in line with the JSON schema. Also fix a bug where the
wrong root was used for checking the signatures.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Add a new stage option `initramfs-args`, that will in turn be set
on the Treefile that is then passed to rpm-ostree. This will pass
the options on to dracut when building the initramfs module. NB:
the `ostree` dracut module will also always be automatically
included by rpm-ostree.
This stage will create a file '/boot/ignition.firstboot' that will,
with the help of support in grub, trigger ignition on the first
boot. The `network` option can be used to overwrite the default
network configuration set in grub2.
Add support for ignition[1] via a new `ignition` stage option. If
enabled, a new section is added to the main grub.cfg that will
create a 'ignition_firstboot' variable meant to be included in the
kernel command line configuration.
The grub.cfg snippet was taken from 'src/grub.cfg' of Fedora CoreOS
Assembler[2] at ec05cde20d3449fab8e4c76493ffa1ebd9b0b626 but with
PR #1373 applied to not hard-code the dhcp options.
[1] https://github.com/coreos/ignition
[2] https://github.com/coreos/coreos-assembler/
When adding a new user with a uid that already exists, the "-o",
option needs to be added. Always do so when the uid is specified.
A use case for this is e.g. an installer iso with a `install`
user that has 0 (like root) for its uid (and gid).
Support setting uids, gids with values of `0` as well as passwords
and descriptions with the empty string, by explicitly checking the
value of each against `None`, because simple `if` conditionals are
false for those.
The content hash of each RPM is already verified, so verifying
signatures again is not necessary if the manifest generation is trusted,
and verifying signatures does not help if the manifest generation is
not.
Let us follow what DNF does and default to not verify signatures, but in
order to preserve features already in use we still allow opting in to
verifying signatures as before on a per RPM basis.
This will make it possible to install unsigned RPMs, or a mixed of
signed and unsigned RPMs.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Extract the grub.cfg redirect config as GRUB_REDIRECT_TEMPLATE,
meant to be used via python's string.Template class. Document
its intended use and also the template options.
Extract the strings for grub.cfg into a GRUB_CFG_TEMPLATE multi-line
string and turn it into a template meant to be used via python's
string.Template class. Document it, especially the template options.
Instead of having two different places within conditionals where
the configuration is written, have only one common. In the case
of hybrid boot, in the end there will be two grub configuration
files: the canonical one in /boot/grub2/grub.cfg and a redirect
one, in the EFI directory that will redirect to the canonical.
In case of legacy only, only the canonical one in the default
location (/boot/grub2/grub.cfg) will be written.
For EFI only mode, only /boot/efi/EFI/<vendor>/grub.cfg will
be written with the main grub configuration data.
Thus the writing of the main grub configuration will now always
be written in exactly one place (now line 319).
Move the write_grub_cfg_redirect to the new GrubConfig object as
write_redirect. Add a `separate_boot` property to be used by the
new write_redirect. Remove the corresponding variable and also
the `grub_fs` variable since that is now all handled by the
GrubConfig object.
The reason behind this is to combine all the necessary state in the
object instead of passing it all to the write_grub_cfg function.
The idea is that as more things will get configurable, say the
timeout or ignition support, more things need to be passed to it
and thus it is better to an object where these config options can
be set and then combined when writing the config.
Fedora CoreOS[1] uses a dracut module[2] together with a systemd
generator[3] to mount the file system, including the root one.
Thus neither '/etc/fstab' nor a `root=` kernel command line
option is needed. Support that use case by making the 'rootfs'
option optional.
[1] https://github.com/coreos/fedora-coreos-config/tree/testing-devel/
[2] overlay.d/05core/usr/lib/dracut/modules.d/40ignition-ostree
[3] overlay.d/05core/usr/lib/systemd/system-generators/coreos-boot-mount-generator
The ignition-dracut module for Fedora CoreOS and anaconda both have
code to populate '/var' via systemd-tmpfiles. In images that where
said dracut module is not used, but '/var' needs to be populate, it
can no be done by setting the `populate_var` option.
Fix all occurrences of format-strings without any interpolation. pylint
warns about those (and for some reason did not do so for our modules).
A followup will fix the pylint tests, so make sure all the warnings are
resolved.
For all currently supported modules, i.e. stages and assemblers,
convert the STAGE_DESC and STAGE_INFO into a proper doc-string.
Rename the STAGE_OPTS into SCHEMA.
Refactor meta.ModuleInfo loading accordingly.
The script to be used for the conversion is:
--- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< ---
import os
import sys
import osbuild
import osbuild.meta
from osbuild.meta import ModuleInfo
def find_line(lines, start):
for i, l in enumerate(lines):
if l.startswith(start):
return i
return None
def del_block(lines, prefix):
start = find_line(lines, prefix)
end = find_line(lines[start:], '"""')
print(start, end)
del lines[start:start+end+1]
def main():
index = osbuild.meta.Index(os.curdir)
modules = []
for klass in ("Stage", "Assembler"):
mods = index.list_modules_for_class(klass)
modules += [(klass, module) for module in mods]
for m in modules:
print(m)
klass, name = m
info = ModuleInfo.load(os.curdir, klass, name)
module_path = ModuleInfo.module_class_to_directory(klass)
path = os.path.join(os.curdir, module_path, name)
with open(path, "r") as f:
data = list(f.readlines())
i = find_line(data, "STAGE_DESC")
print(i)
del data[i]
del_block(data, "STAGE_INFO")
i = find_line(data, "STAGE_OPTS")
data[i] = 'SCHEMA = """\n'
docstr = '"""\n' + info.desc + "\n" + info.info + '"""\n'
doclst = docstr.split("\n")
doclst = [l + "\n" for l in doclst]
data = [data[0]] + doclst + data[1:]
with open(path, "w") as f:
f.writelines(data)
if __name__ == "__main__":
main()
Copies files obtained via a `source` to the tree. Multiple files or
directories can be copied by specifying multiple entries in `paths`.
If no paths are specified the whole contents of `source` is copied.
The source and the target path for each individual path entry might
optionally be specified via `from` and `to`, respectively; if no
path is given for any of the two, the root `/` is assumed.
Currently only an 'archive' 'source' is supported that in turn uses
the existing 'org.osbuild.files' source to fetch an archive (tarball)
and extracts it to a temporary directory.
Change all the schemata to not allow additional properties. This
should help with misspelled properties as well as missing schema
information in the stage itself.
Done via a small python3 script:
--- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< ---
import os
import sys
def list_stages(base):
return [(base, f) for f in os.listdir(base) if f.startswith("org.osbuild")]
stages = list_stages("stages")
stages += list_stages("assemblers")
def find_line(lines, start):
for i, l in enumerate(lines):
if l.startswith(start):
return i
return None
NOADD = '"additionalProperties": false'
for stage in stages:
with open(f"{stage[0]}/{stage[1]}", "r") as f:
print(f"{stage[0]}/{stage[1]}", file=sys.stderr)
data = f.readlines()
i = find_line(data, 'STAGE_OPTS = """')
if i:
data.insert(i+1, NOADD + ",\n")
else:
i = find_line(data, 'STAGE_OPTS = ""')
if i:
data[i] = f'STAGE_OPTS = """\n'
data.insert(i+1, NOADD + "\n")
data.insert(i+2, '"""\n')
with open(f"{stage[0]}/{stage[1]}", "w") as f:
f.writelines(data)
