Commit graph

388 commits

Author SHA1 Message Date
Christian Kellner
e51d08ba9e stages/ostree.passwd: fix subid source path
The sub{g,u}id files in the commit are located in `/usr/etc` not
`/etc`. Read the files from there.
2022-05-16 17:51:56 +02:00
Christian Kellner
20daf92cf3 stages/udev.rules: use correct separator
The key-value-operator expressions should separated by comma
not space. Fix this.
2022-05-16 17:19:40 +02:00
Christian Kellner
d04ce2e1ef stages: add new org.osbuild.udev.rules stage
Add new stage to create udev rules files; add a simple stage test
to check for basic functionality.
2022-05-06 12:46:03 +02:00
Alexander Larsson
9bbe024ba2 org.osbuild.luks2.format: Support dm-integrity
I've been looking at:
  https://archive.fosdem.org/2018/schedule/event/cryptsetup/attachments/slides/2506/export/events/attachments/cryptsetup/slides/2506/fosdem18_cryptsetup_aead.pdf

And it seems if you want integrity checking, dm-crypt is not enough,
but should be combined with dm-integrity. This allows this by using
the --integrity option with cryptsetup.

It should be noted that the slides above mention that this is marked
experimental in the docs because the existing algorithms usable for
this is a bit slow. Something to be aware of if you want to use this.
2022-04-30 15:26:16 +02:00
Christian Kellner
061c2012ed stage/ostree.passwd: also merge /etc/sub{u,g}id
When merging user and group database from individual commits also merge
the corresponding /etc/sub{u,g}id database. These are created when the
users are added via `useradd` and thus also need to be imported with
the corresponding users.
2022-04-28 14:38:24 +01:00
Christian Kellner
b57376d160 pipeline: drop CAP_MAC_ADMIN by default
Drop `CAP_MAC_ADMIN` from the default capabilities which is needed
to write and read(!) unknown SELinux labels. Adjust the stages
that need to read or write SELinux labels accordingly.
2022-04-27 23:05:11 +01:00
Christian Kellner
0c8f5c7ef0 pipeline: drop CAP_{NET_ADMIN,SYS_PTRACE} caps
Drop CAP_{NET_ADMIN,SYS_PTRACE} from the default capabilities which
are only needed to run bwrap from inside a stage which is done by
the `ostree.commit` and `ostree.preptree` stages, so retain them
directly there.
2022-04-27 23:05:11 +01:00
Alexander Larsson
68481f48ae Support specifying multiple devices in all mkfs versions (like xfs)
mkfs.xfs already has `additionalProperties: true` for the devices
section, as this is necessary for example when creating lvm2 setups.
This should be possible for other filesystem types too.
2022-04-19 18:00:15 +02:00
Tomas Hozza
bbe6127ddb Stages: support vmdk subformat in the qemu stage
Support specifying the subformat for the `vmdk` image type in the
`org.osbuild.qemu` stage.
2022-04-05 15:01:34 +02:00
Enric Balletbo i Serra
31eed79760 stages/sfdisk: support changing GPT partition attribute bits
When is specified change the GPT partition attribute bits. The attributes
is a comma list of bits numbers or bit names. You can take a look at the
manual page of sfdisk to see the supported attribute bits.
2022-03-24 00:05:32 +01:00
Tomas Hozza
3dea4b934c stages/rpm: don't verify signatures during install
If instructed, the rpm stage checks all digests and signatures of a package
explicitly using `rpmkeys` tool. The default stage behavior is that no
package signatures are checked when installed by the stage (not even
explicitly).

For these reasons, the package signature checking is supposed to be
disabled when installing rpm packages. This was achieved by passing the
`--define "_pkgverify_level none"` option to rpm. However this option
specifies only requirements for a package to be installed and `none`
means that packages without any signature are accepted by rpm. If the
package signature is deemed BAD, the package installation fails even
though this option has been passed to rpm.

There are valid cases when even packages which signature marked as BAD
should be installed. It may happen, that the GPG key used to sign a
package uses an algorithm not allowed by the system crypto policy, e.g.
SHA1. If such GPG key is imported on the system and a package signed
using it is being installed, its installation would fail when the
package is read by rpm. This is because its signatures are by default
checked if they exist in the package.

The desired behavior to not check any package signatures when installing
a package is instead achieved by using `--nosignature` rpm option. It
turns off the whole signature checking mechanism.

Use the `--nosignature` rpm option instead of
`--define "_pkgverify_level none"`, when installing packages using rpm.

Fix https://github.com/osbuild/osbuild/issues/991
2022-03-22 18:41:12 +01:00
Christian Kellner
d38cdb6425 stages/selinux: directly call setfilecon
Instead of using `chcon`, directly call `selinux.setfilecon`.
On systems without SELinux support, i.e. coreutils was built
without `<selinux.h>` present, `chcon` will return `ENOTSUP`
for all calls to SElinux functions like `setfilecon` even if
the selinux libraries are later installed.
Therefore we directly call the new osbuild helper function,
which really is just a wrapper around setting extended file
attributes and thus will work even if SELinux support is not
compiled into coreutils.
The only other thing `chcon` is doing besides a cal to the
`setfilecon` method is to convert the context string to a
new `contex_t` and back to validate it. This should not be
needed since the kernel will do this for us. On system
without SELinux support `context_new` will also not validate
the context.
2022-03-18 20:36:10 +01:00
Christian Kellner
d8f36b55fa stages/grub2: option to configure GRUB_DEFAULT
Add a new attribute `config.default` that when set will be written to
`GRUB_DEFAULT`. This should be set to `saved` when a `saved_entry` is
specified so that the functionality will be preserved if the grub cfg
gets regenerated (which is really should not, but we can not prohibit
it).
2022-03-04 16:38:41 +01:00
Tomas Hozza
9054da7d25 stages/firewall: fix fail when setting only the default zone
When the firewall stage is provided with stage options, which set only
the default firewall zone, the `firewall-offline-cmd` command is
executed unconditionally without any parameters. This is because in this
case `ports`, `enabled_services` and `disabled_services` are all an
empty lists. This results in a failure with the following error message:

`Opening of '/etc/sysconfig/system-config-firewall' failed, exiting.`

Make sure that the second invocation of `firewall-offline-cmd` happens
conditionally, only when at least one of the `ports`, `enabled_services`
or `disabled_services` is a non-empty list.

Adjust the stage test to cover this scenario.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-03-04 10:02:27 +01:00
Christian Kellner
5a5da44c06 stages/rpm: option to import gpg keys from tree
Add a new option `gpgkeys.fromtree` that when specified will
import the specified gpg keys from files located in the tree,
such as `/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release`.
2022-03-04 08:37:38 +01:00
Tomas Hozza
e71a91f5d0 Extend firewall stage to set the default zone
Extend the firewall stage to allow setting the default firewall zone.
Modify the stage unit test accordingly.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-03-02 16:43:22 +01:00
Jelle van der Waa
b7fe5c724f stages: add org.osbuild.pacman.mirrorlist.conf
Stage for configuring the pacman mirrorlist file.
2022-03-01 16:07:27 +01:00
Jelle van der Waa
11f8eef5b5 stages: add pacman keyring stage
This stage initialises the pacman keyring which is required for pacman
packages to be installed.
2022-03-01 16:07:27 +01:00
Jelle van der Waa
25d43dd82e stages: add the ability to configure pacman repos
Without configured repositories the generated pacman.conf isn't super
useful. Pacman supports two different ways to configure a repository
either with a Server = line or sourced from pacman's mirrorlist.
2022-03-01 16:07:27 +01:00
Christian Kellner
8f75d1e7a1 stages: add new org.osbuild.fdo stage
This stage is needed to write down the FDO DIUN pub key root certs
needed to talk to the manufacturer server to grab the device credentials
for provisioning and later onboarding

Co-Authored-By: Antonio Murdaca <runcom@linux.com>
2022-02-23 15:32:02 +00:00
Christian Kellner
d8bfe13fdd stages/oci-archive: fix creation time format
According to OCI image specification the `created` property needs
to be in the format describe in RFC 3339, section 5.6 "Internet Date /
Time Format". The suffix is the "time-offset" which is either "Z" or
"time-numoffset".
Before commit 25b5679[2] we the datetime object, created via `utcnow`
did not have timezone information so "Z" was manually appended. This
was changed with commit 25b5679 and now the timezone information was
indeed included, which also meant that `isoformat` now included the
offset, i.e. `+00:00`. But the `Z` was still appended and the result
was `+00:00Z`, something that is not standard compliant. Fix this
by removing the extra `Z`.

[1] https://datatracker.ietf.org/doc/html/rfc3339#section-5.6
[2] 25b567990b
2022-02-22 10:04:08 +00:00
Antonio Murdaca
f768781589 stages: add luks remove-key stage
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-02-16 15:43:19 +01:00
Antonio Murdaca
25ecd12b3d stages: add clevis-luks-bind stage
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-02-16 15:43:19 +01:00
Alexander Larsson
e9d02a27bb skopeo stage: remove overlay/backingFsBlockDev file after install
Each time the overlay backend runs on an xfs fs it creates the
file "overlay/backingFsBlockDev" in the containers storage directory.

It is not needed in the image as skopeo recreates it each boot, and
since it is a block device its existance means we can't store the
resulting tree in ostree. Lets just get rid of it.
2022-02-15 19:13:00 +01:00
Roy Golan
ed82f33c8f Fix type orci-archive/oci-archive
Signed-off-by: Roy Golan <rgolan@redhat.com>
2022-02-14 14:15:19 +01:00
Alexander Larsson
46a228df38 Add support for installing containers in images
This adds a stage called org.osbuild.skopeo that installs docker and
oci archive files into the container storage of the tree being
constructed.

The source can either be a file from another pipeline, for example one
created with the existing org.osbuild.oci-archive stage, or it can
be using the new org.osbuild.skopeo source and org.osbuild.containers
input, which will download an image from a registry and install that.

There is an optional option in the install stage that lets you
configure a custom storage location, which allows the use of the
additionalimagestores option in the container storage.conf
to use a read-only image stores (instead of /var/lib/container).

Note: skopeo fails to start if /etc/containers/policy.json is
not available, so we bind mount it from the build tree to the
buildroot if available.
2022-02-10 14:43:17 +01:00
Alexander Larsson
1897eaf8bc org.osbuild.oci-archive: Try to create more reproducible images
This tries to make the various tar-balls produced by the stage more
likely to be identical in separate runs. We do this by sorting
the names and removing some unnecessary metadata for the files.

The most important thing to get right is the layer tarball, because
that is what defines the container id. We sort the names to avoid
random differences, and drop ctimes and atimes because these are
generally just set to the something near the current build time which
is not useful to encode in the container image. This is as opposed
to the mtime which generally comes from e.g. the rpms that where
installed in the pipeline.

For the actual archive tarball we can standardize metadata even more,
because none of the metadata are used when consuming the archive.
2022-02-09 09:58:49 +01:00
Alexander Larsson
25b567990b oci-archive stage: Use source-epoch as creation data
If the source-epoch is specified, this is used as the creation date
instead of the current time.
2022-02-09 09:58:49 +01:00
Thomas Lavocat
31fb124716 bootiso: add the option to compress using lz4 2022-02-08 16:02:36 +01:00
Thomas Lavocat
bb30ffa062 Add the option of compressing using lz4
Using lz4 in squashfs reduces the time to complete the stage from 446s
to 4s on my laptop while making the produced iso going from 1.6G to
2.0G.
2022-01-11 13:20:11 +01:00
Christian Kellner
ee96b11faf stages/grub2: ability to not write kernel cmdline
Currently we always write the kernel command line to the `grubenv`
file, if only to include the root device. Starting with Fedora 33
and thus RHEL 9, the kernel command line included statically in
the BLS snippets and the grubenv `kernelopts` variable not used.
Instead one of the {/usr/lib,/etc}/kernel/cmdline files is read
and the parameters in them used during the creation of the BLS
snippets.
Therefore we add a new `write_cmdline` option that, if set to
FALSE, will prevent us from writing the kernel command line.
2022-01-06 15:09:33 +00:00
Christian Kellner
d4f275e024 stages/grub2: fix whitespace in docstring
For you, David.
2022-01-06 15:09:33 +00:00
Christian Kellner
7a2b8ac107 stages/grub2: extract uuid js schema definition
The uuid schema is duplicated in a few places. Extract it and use
references to it instead.
2022-01-06 15:09:33 +00:00
Jelle van der Waa
e3515caf01 stages: Remove excess whitespace between words 2022-01-06 16:01:26 +01:00
Jelle van der Waa
3d4be5f059 stages/org.osbuild.pacman: Make /dev/stdin available
Installing the mkinitcpio kernel preset's requires /dev/stdin to be
available for calling install.
2022-01-06 16:01:26 +01:00
Jelle van der Waa
97202e53b0 stages: Respect grub_home for grub legacy boot
Additionally add a new QEMU image output target for testing
grub2/mkinitcpio stages.
2022-01-06 16:01:26 +01:00
Ondřej Budai
0b8dbd4bc3 stages/kickstart: ensure a newline at the end of the file
To simplify extending of the kickstart file.

Fixes: rhbz#2036971
2022-01-04 18:36:37 +01:00
Ondřej Budai
fe379b631b stages/kickstart: remove dead code
post nor anaconda wasn't ever set to something truthy, let's just remove them
2022-01-04 18:36:37 +01:00
Tomas Hozza
a9e1070ce2 Fix typo in tar stage schema option enum
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-22 11:50:08 +01:00
Jelle van der Waa
0cbd7898c7 Add mkinitcpio stage
This stage generates the initrd for Arch Linux and derivates.
2021-12-21 10:44:55 +01:00
Tomas Hozza
9786d1f0d6 stages: allow using sysconfig stage multiple times.
The sysconfig stage currently does not produce expected results when
used multiple times within the same pipeline. Specifically, the stage
always truncates respective configuration files for properties `kernel`
and `network`, if if these are not set in the stage options. Due to this
reason, the outcome of the image builds may depend on the order of
multiple occurrences of the sysconfig stage.

The following two pipeline snippets would produce different
configuration files content:

Configuration files are truncated:
```
{
  "type": "org.osbuild.sysconfig",
  "options": {
    "kernel": {
      "update_default": true,
      "default_kernel": "kernel"
    },
    "network": {
      "networking": true,
      "no_zero_conf": true
    }
  }
},
{
  "type": "org.osbuild.sysconfig",
  "options": {
    "network-scripts": {
      "ifcfg": {
        "eth0": {
          "bootproto": "dhcp",
          "device": "eth0",
          "ipv6init": false,
          "onboot": true,
          "peerdns": true,
          "type": "Ethernet",
          "userctl": true
        }
      }
    }
  }
},
```

No configuration files are truncated:
```
{
  "type": "org.osbuild.sysconfig",
  "options": {
    "network-scripts": {
      "ifcfg": {
        "eth0": {
          "bootproto": "dhcp",
          "device": "eth0",
          "ipv6init": false,
          "onboot": true,
          "peerdns": true,
          "type": "Ethernet",
          "userctl": true
        }
      }
    }
  }
},
{
  "type": "org.osbuild.sysconfig",
  "options": {
    "kernel": {
      "update_default": true,
      "default_kernel": "kernel"
    },
    "network": {
      "networking": true,
      "no_zero_conf": true
    }
  }
},
```

Change the stage to not touch respective configuration files if the
`kernel` and `network` properties are not set in the stage options.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-17 08:44:54 +01:00
Achilleas Koutsou
646dd238fe stages: add new org.os.build.pacman and org.osbuild.pacman.conf stage
Pacman is the default package manager for Arch Linux and derivates, the
pacman.conf stage generate a valid pacman.conf configuration file.

Co-Authored-By: Jelle van der Waa <jvanderwaa@redhat.com>
2021-12-15 23:22:15 +01:00
Tomas Hozza
d7989a5c26 Add new stage for configuring DNF Automatic
Add a new stage `org.osbuild.dnf-automatic.config` for configuring DNF
Automatic.

The stage changes persistent DNF Automatic configuration. Currently, only
a subset of options can be set:
  - 'commands' section
    - apply_updates
    - upgrade_type

Fix #908

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-15 18:49:13 +01:00
Tomas Hozza
cd4ac1c75a Add new stage for creating YUM / DNF repo files
Add a new stage `org.osbuild.yum.repos` for creating YUM / DNF `.repo`
files in `/etc/yum.repos.d`. All repo-specific options are supported but
only a subset of options which can be set for a repo as well as in the
[main] section are supported.

Add unit test for the new stage.

Fix #907

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-09 18:51:51 +01:00
Jelle van der Waa
f965ca8510 stages/users: Explicitly create a home directory
On distributions such as Arch Linux the home directory is not created by
default.
2021-12-09 16:48:31 +01:00
Sanne Raymaekers
5b3ebd7912 stages/org.osbuild.dnf.config: Edit /etc/dnf/dnf.conf
Fixes #906
2021-12-09 15:53:57 +01:00
Christian Kellner
d8a4f9d063 stages: add new org.osbuild.crypttab stage
New stage to configure /etc/crypttab to setup encrypted block devices.
See the documentation of the stage for more information.
2021-12-09 00:44:21 +00:00
Christian Kellner
2801c17730 stages: add new luks2 stage
New stage to initialize LUKS2 container on a given device, usually a
loopback device bound to a partition. The passphrase and uuid of the
container need to be specified. Optionally the cipher, label, sector
size and sub-label can be specified. Requires the cryptsetup binary
to be install in the build root.
2021-12-09 00:44:21 +00:00
Tomas Hozza
0df902d6bb sshd.config stage: support PermitRootLogin option
Add support for `PermitRootLogin` option in the
`org.osbuild.sshd.config` stage.

I kept the "yes" and "no" values for consistency with other stage
options. While it will make the implementation in osbuild-composer
harder, it won't be impossible as we already have a precedence for doing
it this way (e.g. in the `org.osbuild.pam.limits.conf`).

Modify the stage unit tests to check the new option.

Remove the empty `org.osbuild.sshd.config` stage from `a.mpp.json`
since it does not add any value and it actually made the `tree-diff`
tool provide a weird tree diff results.

Fix #910

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-12-07 15:20:50 +00:00
Christian Kellner
bd5b60a857 stages: remove org.osbuild.test.timeout
Was only used in `test_buildroot` and got replaced with a direct
call to `sleep`.
2021-12-07 09:47:01 +00:00