Commit graph

359 commits

Author SHA1 Message Date
Thomas Lavocat
bb30ffa062 Add the option of compressing using lz4
Using lz4 in squashfs reduces the time to complete the stage from 446s
to 4s on my laptop while making the produced iso going from 1.6G to
2.0G.
2022-01-11 13:20:11 +01:00
Christian Kellner
ee96b11faf stages/grub2: ability to not write kernel cmdline
Currently we always write the kernel command line to the `grubenv`
file, if only to include the root device. Starting with Fedora 33
and thus RHEL 9, the kernel command line included statically in
the BLS snippets and the grubenv `kernelopts` variable not used.
Instead one of the {/usr/lib,/etc}/kernel/cmdline files is read
and the parameters in them used during the creation of the BLS
snippets.
Therefore we add a new `write_cmdline` option that, if set to
FALSE, will prevent us from writing the kernel command line.
2022-01-06 15:09:33 +00:00
Christian Kellner
d4f275e024 stages/grub2: fix whitespace in docstring
For you, David.
2022-01-06 15:09:33 +00:00
Christian Kellner
7a2b8ac107 stages/grub2: extract uuid js schema definition
The uuid schema is duplicated in a few places. Extract it and use
references to it instead.
2022-01-06 15:09:33 +00:00
Jelle van der Waa
e3515caf01 stages: Remove excess whitespace between words 2022-01-06 16:01:26 +01:00
Jelle van der Waa
3d4be5f059 stages/org.osbuild.pacman: Make /dev/stdin available
Installing the mkinitcpio kernel preset's requires /dev/stdin to be
available for calling install.
2022-01-06 16:01:26 +01:00
Jelle van der Waa
97202e53b0 stages: Respect grub_home for grub legacy boot
Additionally add a new QEMU image output target for testing
grub2/mkinitcpio stages.
2022-01-06 16:01:26 +01:00
Ondřej Budai
0b8dbd4bc3 stages/kickstart: ensure a newline at the end of the file
To simplify extending of the kickstart file.

Fixes: rhbz#2036971
2022-01-04 18:36:37 +01:00
Ondřej Budai
fe379b631b stages/kickstart: remove dead code
post nor anaconda wasn't ever set to something truthy, let's just remove them
2022-01-04 18:36:37 +01:00
Tomas Hozza
a9e1070ce2 Fix typo in tar stage schema option enum
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-22 11:50:08 +01:00
Jelle van der Waa
0cbd7898c7 Add mkinitcpio stage
This stage generates the initrd for Arch Linux and derivates.
2021-12-21 10:44:55 +01:00
Tomas Hozza
9786d1f0d6 stages: allow using sysconfig stage multiple times.
The sysconfig stage currently does not produce expected results when
used multiple times within the same pipeline. Specifically, the stage
always truncates respective configuration files for properties `kernel`
and `network`, if if these are not set in the stage options. Due to this
reason, the outcome of the image builds may depend on the order of
multiple occurrences of the sysconfig stage.

The following two pipeline snippets would produce different
configuration files content:

Configuration files are truncated:
```
{
  "type": "org.osbuild.sysconfig",
  "options": {
    "kernel": {
      "update_default": true,
      "default_kernel": "kernel"
    },
    "network": {
      "networking": true,
      "no_zero_conf": true
    }
  }
},
{
  "type": "org.osbuild.sysconfig",
  "options": {
    "network-scripts": {
      "ifcfg": {
        "eth0": {
          "bootproto": "dhcp",
          "device": "eth0",
          "ipv6init": false,
          "onboot": true,
          "peerdns": true,
          "type": "Ethernet",
          "userctl": true
        }
      }
    }
  }
},
```

No configuration files are truncated:
```
{
  "type": "org.osbuild.sysconfig",
  "options": {
    "network-scripts": {
      "ifcfg": {
        "eth0": {
          "bootproto": "dhcp",
          "device": "eth0",
          "ipv6init": false,
          "onboot": true,
          "peerdns": true,
          "type": "Ethernet",
          "userctl": true
        }
      }
    }
  }
},
{
  "type": "org.osbuild.sysconfig",
  "options": {
    "kernel": {
      "update_default": true,
      "default_kernel": "kernel"
    },
    "network": {
      "networking": true,
      "no_zero_conf": true
    }
  }
},
```

Change the stage to not touch respective configuration files if the
`kernel` and `network` properties are not set in the stage options.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-17 08:44:54 +01:00
Achilleas Koutsou
646dd238fe stages: add new org.os.build.pacman and org.osbuild.pacman.conf stage
Pacman is the default package manager for Arch Linux and derivates, the
pacman.conf stage generate a valid pacman.conf configuration file.

Co-Authored-By: Jelle van der Waa <jvanderwaa@redhat.com>
2021-12-15 23:22:15 +01:00
Tomas Hozza
d7989a5c26 Add new stage for configuring DNF Automatic
Add a new stage `org.osbuild.dnf-automatic.config` for configuring DNF
Automatic.

The stage changes persistent DNF Automatic configuration. Currently, only
a subset of options can be set:
  - 'commands' section
    - apply_updates
    - upgrade_type

Fix #908

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-15 18:49:13 +01:00
Tomas Hozza
cd4ac1c75a Add new stage for creating YUM / DNF repo files
Add a new stage `org.osbuild.yum.repos` for creating YUM / DNF `.repo`
files in `/etc/yum.repos.d`. All repo-specific options are supported but
only a subset of options which can be set for a repo as well as in the
[main] section are supported.

Add unit test for the new stage.

Fix #907

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-09 18:51:51 +01:00
Jelle van der Waa
f965ca8510 stages/users: Explicitly create a home directory
On distributions such as Arch Linux the home directory is not created by
default.
2021-12-09 16:48:31 +01:00
Sanne Raymaekers
5b3ebd7912 stages/org.osbuild.dnf.config: Edit /etc/dnf/dnf.conf
Fixes #906
2021-12-09 15:53:57 +01:00
Christian Kellner
d8a4f9d063 stages: add new org.osbuild.crypttab stage
New stage to configure /etc/crypttab to setup encrypted block devices.
See the documentation of the stage for more information.
2021-12-09 00:44:21 +00:00
Christian Kellner
2801c17730 stages: add new luks2 stage
New stage to initialize LUKS2 container on a given device, usually a
loopback device bound to a partition. The passphrase and uuid of the
container need to be specified. Optionally the cipher, label, sector
size and sub-label can be specified. Requires the cryptsetup binary
to be install in the build root.
2021-12-09 00:44:21 +00:00
Tomas Hozza
0df902d6bb sshd.config stage: support PermitRootLogin option
Add support for `PermitRootLogin` option in the
`org.osbuild.sshd.config` stage.

I kept the "yes" and "no" values for consistency with other stage
options. While it will make the implementation in osbuild-composer
harder, it won't be impossible as we already have a precedence for doing
it this way (e.g. in the `org.osbuild.pam.limits.conf`).

Modify the stage unit tests to check the new option.

Remove the empty `org.osbuild.sshd.config` stage from `a.mpp.json`
since it does not add any value and it actually made the `tree-diff`
tool provide a weird tree diff results.

Fix #910

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-07 15:20:50 +00:00
Christian Kellner
bd5b60a857 stages: remove org.osbuild.test.timeout
Was only used in `test_buildroot` and got replaced with a direct
call to `sleep`.
2021-12-07 09:47:01 +00:00
AaronH88
99c739fd60 test: test buildroot read_with_timeout function
- Added a new stage that is stuck in an infinite loop
- Added two tests that use this stage and force a timeout
2021-12-03 14:29:36 +00:00
Jelle van der Waa
9d7d952ddf stages/grub2.inst: Make grub-mkimage binary configurable
On Arch Linux grub-mkimage is provided by the `grub` package which is
grub2 and grub (1) no longer exists.
2021-11-29 20:28:44 +01:00
Christian Kellner
0fec32e079 stages/ostree.fillvar: fill the correct var
The idea behind the stage is to provision `var` of the stateroot,
i.e. the `var` the is shared amongst all deployments for a given
os (indicated by `osname`, e.g. `fedora`, `centos`, ...).
For `systemd-tmpfiles` to infer the correct paths, it needs to be
run on the deployment. The `var` of the latter needs to be bind-
mounted to the `var` of the stateroot, because it is shared. This
was always the intention but not what the code did. Fix this by
getting the `var` of the stateroot and bind it to the `var` of
the deployment.
NB: In reality this never mattered since systemd-tmpfiles is also
run during system startup.
2021-11-26 17:20:11 +01:00
Alexander Larsson
079598fa4f osbuild: Add org.osbuild.gunzip stage
This stage the same args and formats as org.osbuild.untar (and as such
much code is just copied from that stage), except it runs gunzip
instead.  I need this to uncompress the aarch64 kernel when directly
uefi-booting it.
2021-11-19 00:16:04 +00:00
Tom Gundersen
f7bf23faba stages/authconfig: run authconfig
This applies the default authconfig settings to the tree.

Note that the `/backups` directory is removed. The tool creaset
this, and by default it should not exist, so this should be a
noop. However, if you run this on a tree with existing backups,
they would be lost.
2021-11-08 20:43:51 +01:00
Ondřej Budai
306fd3ed96 stages/yum.config: add an option to configure langpacks plugin
The new stage enables users to configure the langpacks plugin of YUM.
Currently, only locales option is supported.
2021-11-08 17:17:02 +01:00
Christian Kellner
52cb27631b stages/selinux: ability to force an auto-relabel
This should not be needed in any case but can be a sledgehammer
for situations where we cannot properly label a file; it turns
out such a scenario is if a label, lets call it `a1`, is is an
alias to another label, lets call it `l1`. Setting `a1` will
lead to `l1` being read back, and thus copying the label `a1`
will result on the label `l1` being copied instead. Now if the
target distribution does not have `l1` but only has `a1` we
cannot set it and thus will end up with an unlabeled file.
2021-11-05 16:06:13 +00:00
Tom Gundersen
9da89de8b5 stages/pwquality.conf: set pwquality configuration
This allows the password policy to be configured.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-11-05 16:53:54 +01:00
Christian Kellner
5301270bff stages/rhsm: add support to configure yum plugins
Adds support to configure `yum-plugins`, which currently is a full
alias for `dnf-plugins`, although this might change in the future,
in case dnf options diverge from yum. It allows for both yum and
dnf plugins to be configured at the same time since on RHEL 7 both
files will be present.
2021-11-05 15:56:54 +01:00
Christian Kellner
0e9fcad8fc stages/rhsm: extract plugins defintion
This is so that it can be reused by a new `yum-plugins` section.
2021-11-05 15:56:54 +01:00
Tomas Hozza
58ec1c9a83 Add new org.osbuild.yum.config stage
Add a new stage for modifying YUM global configuration.

Add a unit test case for the newly added stage.

Because we test stages on Fedora, where there is no YUM, and this stage
is mostly intended for being used with RHEL-7 images, the stage does not
produce error in case the `/etc/yum.conf` file does not exist. It rather
produces a warning and creates the file. Ideally the stage would produce
an error in case the configuration file does not exist, but that would
be impossible to test on recent Fedora.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-11-05 12:10:33 +01:00
Achilleas Koutsou
58708c29d2 stages/cloud-init: disable default_flow_style when writing configs 2021-11-05 12:08:38 +01:00
Achilleas Koutsou
9b5f13fc6d stages/cloud-init: add support for configuring output logging 2021-11-05 12:08:38 +01:00
Achilleas Koutsou
4ac004e178 stages/cloud-init: add support for configuring reporting handlers
It's possible the keys "logging" and "telemetry" can be arbitrary names.
If that's the case, we can change the schema without breaking backwards
compatibility, so defining known keys is safer.
2021-11-05 12:08:38 +01:00
Achilleas Koutsou
3ee19645a3 stages/cloud-init: add support for configuring Azure datasource
Datasources are defined in an array to preserve ordering and are then
converted to objects.
2021-11-05 12:08:38 +01:00
Christian Kellner
12eb14378a stages: add new org.osbuild.cron.script stage
New stage to schedule a script via a /etc/cron.{hourly, daily ...}.
Currently only a simple command that will be put into an generated
script with the given name. Later more options might be added.
2021-11-05 11:53:11 +01:00
Ondřej Budai
266c3d1953 stages/grub2: add support for terminal, serial and timeout config
This commit adds options to the org.osbuild.grub2 stage to configure
terminal input and output, serial console and timeout.

The functionality and configuration schema is the same as in the legacy
grub2 stage.
2021-11-04 17:35:24 +01:00
Tom Gundersen
36176ab377 stages/waagent.conf: set WALinuxAgent configuration
This is requried to comply with Azure marketplace best
practices. The WALinuxAgent should not handle formating or
swap, as that is done by cloud-init.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-11-03 18:32:40 +01:00
Tom Gundersen
304f1e3f9f stages/sshd.config: set sshd configuration
This is required for images to be importable to the AWS
marketplace. Both PasswordAuthentication and
ChallengeResponseAuthentication must be disabled.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-11-03 18:32:40 +01:00
Tomas Hozza
daacf002fe Support 'install' command in org.osbuild.modprobe stage
Add support for new 'install' command in the org.osbuild.modprobe stage.

Extend the unit test coverage to test the new command.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-11-03 16:11:54 +00:00
Christian Kellner
bd4ab1d36f stages/lvm2.create: fix 'size' and add 'extents'
The `size` option was using `-l`, which in fact was `--extents`.
Fix that to use `--size` and add a new option `extents` that
will in fact call translate to `--extents` and this replace the
current use of the `size` option.
Adapt the `fedora-ostree-image` test manifest to use `extents`.
2021-11-03 15:09:35 +00:00
Christian Kellner
f16b606716 stages/grub2.legacy: new stage for non-bls config
Configure grub2 but instead of using the Bootloader Specification (BLS)
it uses traditional menu entries for the individual boot entries. This
is needed since RHEL7 does not have grub2 with BLS support.
2021-10-30 00:54:09 +01:00
Christian Kellner
da5150084e stages/parted: new stage to partition a device
Like the existing sfdisk stage, the parted stage can be used to create
a partition table on the specified device. In contrast to sfdisk, it
does not support uuids, neither for specifying the partition types,
nor the actual uuid of the partition. The current implementation only
supports GPT.
This stage is meant to be used on older systems, like RHEL 7, where
sfdisk exists but does not support GPT (or --json).
2021-10-30 00:54:09 +01:00
Christian Kellner
d76db390c8 stages/grub2.inst: ensure /var/tmp exists
Ensure that `/var/tmp` exists in the build root before trying to
create a file there. It seems to not be there on RHEL 7 build
roots.
2021-10-13 17:10:09 +02:00
Christian Kellner
21ad9fa399 stages/qemu: fix 'compat' option
The option got renamed to `compat` (and moved into the `qemu`
object) when the stage was extracted from the `qemu` assembler;
but the code, taken from the assembler, still used the old
`qcow2_compat` name for the option. Fix this.
2021-09-23 11:00:47 +02:00
Christian Kellner
99dd623383 stages: pam_limits.conf → pam.limits.conf
Mostly my personal preference, but we do not have any stage
with an underscore in the name, so let not start now.
2021-09-14 15:31:12 +02:00
Tomas Hozza
46ff53d5f4 Add a new stage for configuring pam_limits module
Add a new stage `org.osbuild.pam_limits.conf`, which created
configuration files for `pam_limits` module in /etc/security/limits.d.

Add unit test for the new stage.

Fix #788

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-09 15:00:30 +02:00
Christian Kellner
7ec305a343 ostree.config: add bootloader config option
Ability to set the bootloader backend that OSTree should use. NB:
normally this should be set to `none` since in modern distros and
bootloaders the BLS is used and the BLS snippets are generated on
`none` but none of the of the specific bootloader tools are run,
like `grub2-mkconfig` for grub.

Update the fedora image manifest to use that config setting.
2021-09-09 12:29:54 +02:00
Tomas Hozza
187681f17d Add a new stage for setting kernel parameters via sysctl.d
Add a new stage `org.osbuild.sysctld` for setting kernel parameters at
boot by creating a configuration file in /usr/lib/sysctl.d. At least
one parameter must be specified for the stage.

Add unit test for the new stage.

Fix #790

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-09-09 11:16:11 +02:00