Commit graph

425 commits

Author SHA1 Message Date
Christian Kellner
16f1c560cc stages/rpm: support marking install as ostree
An ostree system can be identified as such by the presence of a
marker file: /run/ostree-booted. The `rpm-ostree` tool also
creates this marker during the _installation_ of the system[1,2].
Recently, starting with F36, the authselect package has taken
has become mandatory[3] and is now owning the nsswitch config.
An rpm-ostree system, which has parts of the user database in
/usr, release on the nss-altfiles to read these databases. The
necessary entries are added during the post-processing, which
is called in our `org.osbuild.ostree.preptree` stage[4]. This
installation is skipped though if the nsswitch is the file is
a symlink, indicating that it is owned by some other package,
like authselect. So the F36 authselect change first broke rpm-
ostree[6]. The fix was to check for `/run/ostree-booted` in
the authselect scriptlet and special case this situation[7,8].
Now, our `org.osbuild.rpm` stage does not yet have the ability
to pretend it is a running ostree system and thus we did not
get the special treatment resulting in nss-altfiles not being
enabled in our ostree commits. Therefore the passwd database in
/usr was not read and a lot of daemons and programs without a
valid user, like e.g. `sshd`.
This change introduces a new option, `ostree_booted` that if
set, will create the `/run/ostree-booted` marker and thus our
installation phase will get the same treatments from packages
as rpm-ostree. Hopefully.

[1] 730bec87b1/rust/src/builtins/compose/mod.rs (L24)
[2] 6211d1452e/src/app/rpmostree-compose-builtin-tree.cxx (L501)
[3] https://fedoraproject.org/wiki/Changes/Make_Authselect_Mandatory
[4] 7993c6f565/rust/src/composepost.rs (L635)
[5] d614caeca1/f/0010-spec-fix-detection-of-ostree-system.patch
[6] https://bugzilla.redhat.com/show_bug.cgi?id=2034360
[7] https://src.fedoraproject.org/rpms/authselect/c/d614caeca1a68f55542aefd0d76bda2691c85d24?branch=f36
[8] https://github.com/authselect/authselect/issues/48
2022-08-13 19:21:52 +01:00
Christian Kellner
776bab46ae stages/rpm: convert the mount code to python
Convert the code that sets up the bind mounts to /dev, /sys/ and
/proc from a bash script to python by using the new mount helper
that were moved from the `objectstore`.
2022-08-13 19:21:52 +01:00
Christian Kellner
8f95154a8c stages/rpm: extract machine-id code into function
Extract the code that creates the fake machine id, if it does not
exist, into its own function. Also convert the shell code into
Python code.
2022-08-13 19:21:52 +01:00
Christian Kellner
b49f3f91f9 util/mnt: extract MountGuard into new module
Extract the `MountGuard` class from all stages that defined it into
a new `mnt` utility module.
2022-08-13 19:21:52 +01:00
Christian Kellner
f05078f66e global: fix PEP-8 formatting
This patch was generated by running `autopep8 --diff` on the
source tree and then applying the diff.
2022-08-05 09:41:05 +02:00
Christian Kellner
a3bc5f7692 stages/gcp.guest-agent.conf: warning -> info
Inform when a new guest-agent configuration file is created,
but remove the warning prefix, since it is perfectly fine if
the file does not yet exist.
2022-08-04 08:08:02 +02:00
Christian Kellner
24ad788d3c stages/gcp.guest-agent.conf: specify encoding
When opening files, always specify the encoding as this will
become a linter warning in the near future.
2022-08-04 08:08:02 +02:00
Christian Kellner
cbffd9d104 stages/gcp.guest-agent.conf: add short header
Our format for the global module doc-string is usually:
"""
<short summary>

<longer description>
"""

Introduce the short summary and also reformat the rest of the
doc string a bit.
2022-08-04 08:08:02 +02:00
Christian Kellner
679688f66f stages/gcp.guest-agent.conf: fix some whitespaces
Be more PEP-8 (two lines between methods). Additionally, separate
the builtin imports from library imports and osbuild imports.
2022-08-04 08:08:02 +02:00
fkolwa
2bca7b07f3 stages: add instance_configs.cfg.distro
test/data: add instance_configs.cfg.distro test
2022-08-03 15:39:35 +02:00
Simon de Vlieger
f3e2c8d5e9 stage: add an rhsm.facts stage
This stage allows to manifests to define information to be put into a
built image. The information can then later be used by RHSM/Insights.

The facts are stored in `/usr/share/osbuild/rhsm.facts` and then
symlinked into `/etc/rhsm/facts/osbuild.facts`.
2022-08-03 08:54:51 +02:00
Christian Kellner
9956f54c97 stages/containers.storage.conf: support pytoml
RHEL 8 only has the old pytoml library, so we need to support that
as well. Try falling back if importing `toml` fails.
2022-07-23 11:12:15 +02:00
Christian Kellner
184ed2d71b stages/containers.storage.conf: fix filename prop
The schema said `filename` but we were accessing `location`. Ups.
2022-07-22 20:02:24 +02:00
Christian Kellner
6fee1051c7 stages/container.storage.conf: restrict locations
Only allow the two known working locations as values for `filename`.
2022-07-22 20:02:24 +02:00
Ondřej Budai
092f886904 stages/users: support a dot inside a username
From IEEE Std 1003.1-2017:

 3.437 User Name:
A string that is used to identify a user; see also User Database. To be
portable across systems conforming to POSIX.1-2017, the value is composed
of characters from the portable filename character set. The <hyphen-minus>
character should not be used as the first character of a portable user name.

 3.282 Portable Filename Character Set
The set of characters from which portable filenames are constructed.

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
a b c d e f g h i j k l m n o p q r s t u v w x y z
0 1 2 3 4 5 6 7 8 9 . _ -

The last three characters are the <period>, <underscore>, and <hyphen-minus>
characters, respectively.

---

The regex checking the username in org.osbuild.users stage omitted the
<period> character before. This commit adds it and modifies the stage
test in order to cover this change.

Fixes: rhbz#2104464
2022-07-21 14:52:57 +02:00
Simon de Vlieger
3fd864e5a9 osbuild: fix optional-types
Optional types were provided in places but were not always correct. Add
mypy checking and fix those that fail(ed).
2022-07-13 17:31:37 +02:00
Christian Kellner
6e66c69608 stages: add org.osbuild.containers.storage.conf
New stage to  create or modify `containers-storage.conf` config
files. The currently implement strategy is to merge the given
and existing configuration options.
2022-07-11 16:25:56 +02:00
Gianluca Zuccarelli
852fad9fcb oscap: image build remediation
Refactor the oscap remediation stage to
scan and run remediatoin on the image tree
rather than running the remediation at first
boot.
2022-07-06 09:59:00 +02:00
Christian Kellner
1235c62f8b stages/dnf.config: extend documentation
Add a few more words about the stage.
2022-06-30 09:37:27 +01:00
Christian Kellner
4c12967c0d stages/dnf.config: add support for tsflags
Add support for specifying rpm transaction flags via the `tsflags`
options. See `dnf.conf(5)` for more details.
2022-06-30 09:37:27 +01:00
Christian Kellner
c1c6ab6493 stages/dnf.config: fix schema for config.main
The properties for `main` were not properly defined and but also
not really checked since `additionalProperties` was not defined.
2022-06-30 09:37:27 +01:00
Christian Kellner
57b320f04f stages/dnf.config: specify encondig for open
This is a pyling warning `W1514` "using open without explicitly
specifying an encoding" in newer version, so fix this.
2022-06-30 09:37:27 +01:00
Christian Kellner
10b8c0dcdc stages/rpm.macros: new stage to set rpm macros
This is to persistently set specific RPM macros. Currently only
`_install_langs` is supported.
2022-06-30 00:17:23 +01:00
Christian Kellner
34ebe60bf0 stages/rpm: new option to set _install_langs
RPM supports installing only specific locales by setting the
`_install_langs` macro.
2022-06-30 00:17:23 +01:00
Christian Kellner
a6e7070b13 stages/oci-archive: make config optional in code
According to our schema `config` is optional but the code actually
accessed it unconditionally; fix that.
2022-06-30 00:03:49 +02:00
Christian Kellner
b761188b98 stages/oci-archive: fix comment to use single # 2022-06-30 00:03:49 +02:00
Christian Kellner
9b548d12c7 stages/oci-archive: specify encoding for open
This is a pyling warning `W1514` "using open without explicitly
specifying an encoding" in newer version, so fix this.
2022-06-30 00:03:49 +02:00
Christian Kellner
4d39f9ec2b stages: add new org.osbuild.rpmkeys.import stage
Add a new stage to import public keys into the RPM database via the
`rpmkeys` command. This is similar as to what the `org.osbuild.rpm`
stage already does but it uses inputs instead of inline data.
2022-06-25 13:19:11 +01:00
Jelle van der Waa
923cc60fdb stages: add vagrant stage for libvirt
This commit introduces a new vagrant stage and initially only supports
the libvirt backend. The vagrant stage takes a disk image as input
generates a Vagrantfile and metadata.json.
2022-06-23 11:06:12 +02:00
Christian Kellner
d602b5ff7f stages/mkfs.fat: pass -I command line option
The option will force `mkfs.fat` to ignore existing partitions on
the target device. The check is done via the corresponding device
node in sysfs, i.e. the contents of the `partition` attribute in
`/sys/dev/block/<major>:<minor>`. In certain situations this info
can be stale. Passing `-I` will work-around these situations.
2022-06-20 11:19:33 +02:00
Christian Kellner
86123da599 stages/qemu: expose vpc options
Currently we hard code the vpc options `subformat=fixed` and
`force_size`, which are needed to generate valid azure images
with newer versions of qemu. But for other use cases or other
versions of qemu these options might not be wanted or valid.
Expose all the options but with defaults corresponding to the
old behavior.
Add a unit test for the `force_size` option to check its
effect. Also add a check for the correct size to the existing, 
default value (i.e. `force_size` being `true`).
2022-06-15 13:41:11 +02:00
Gianluca Zuccarelli
66d2cf6df5 stages: add oscap-remediation stage
Add a new stage to handle openscap first boot
remediation. The openscap-remediation.service
looks for a `/system-update` symlink which
points to an openscap config file. This stage
creates both the necessary configuration and
the `/system-update` symlink.
2022-05-31 01:32:49 +02:00
Christian Kellner
e1c2d642dc stages/grub2.legacy: replace arch with platform
The `architecture` served two purposes: 1) the selection of the loader
and 2) the selection of the platform. Instead of inferring the latter
from `architecture`, it is now explicitly specified as a property of
the `bios` value, which in turn was transformed into an object.
The loader is still inferred but since `bios` is an object now there
is the option of adding an explicit `loader` option to it.
All this should make it more transparent what is happening and is
also more in line with the normal `grub2` stage.
2022-05-29 14:02:16 +02:00
Christian Kellner
dcc56d083f stages/grub2.legacy: make schema stricter
The code requires that `id`, `product` and `kernel` are defined for
each entry. Reflect that in the schema. Require at least one entry.
2022-05-29 14:02:16 +02:00
Christian Kellner
8137597ac4 stages/grub2.legacy: use x86 instead of x64
The usage `x64` is to specific since we could indeed boot 32 bit
`x86`, so just use that.
2022-05-28 17:13:17 +01:00
Christian Kellner
b96ee08ce1 stages/grub2.legacy: make the "bios" option a bool
We use an internal mapping from `architecture` to the needed
grub2 platform identifier, there is no need to pass it in.
Change it to a boolean.
2022-05-28 17:13:17 +01:00
Christian Kellner
e0fbbdaa5e stages: add new sgdisk stage
Can be used to create partition tables via GPT laypout via `sgdisk(8)`.
The schema of `partitions` is intentionally kept identical to the one
in `org.osbuild.sfdisk`.
Add corresponding tests.
2022-05-23 13:00:44 +02:00
Christian Kellner
e51d08ba9e stages/ostree.passwd: fix subid source path
The sub{g,u}id files in the commit are located in `/usr/etc` not
`/etc`. Read the files from there.
2022-05-16 17:51:56 +02:00
Christian Kellner
20daf92cf3 stages/udev.rules: use correct separator
The key-value-operator expressions should separated by comma
not space. Fix this.
2022-05-16 17:19:40 +02:00
Christian Kellner
d04ce2e1ef stages: add new org.osbuild.udev.rules stage
Add new stage to create udev rules files; add a simple stage test
to check for basic functionality.
2022-05-06 12:46:03 +02:00
Alexander Larsson
9bbe024ba2 org.osbuild.luks2.format: Support dm-integrity
I've been looking at:
  https://archive.fosdem.org/2018/schedule/event/cryptsetup/attachments/slides/2506/export/events/attachments/cryptsetup/slides/2506/fosdem18_cryptsetup_aead.pdf

And it seems if you want integrity checking, dm-crypt is not enough,
but should be combined with dm-integrity. This allows this by using
the --integrity option with cryptsetup.

It should be noted that the slides above mention that this is marked
experimental in the docs because the existing algorithms usable for
this is a bit slow. Something to be aware of if you want to use this.
2022-04-30 15:26:16 +02:00
Christian Kellner
061c2012ed stage/ostree.passwd: also merge /etc/sub{u,g}id
When merging user and group database from individual commits also merge
the corresponding /etc/sub{u,g}id database. These are created when the
users are added via `useradd` and thus also need to be imported with
the corresponding users.
2022-04-28 14:38:24 +01:00
Christian Kellner
b57376d160 pipeline: drop CAP_MAC_ADMIN by default
Drop `CAP_MAC_ADMIN` from the default capabilities which is needed
to write and read(!) unknown SELinux labels. Adjust the stages
that need to read or write SELinux labels accordingly.
2022-04-27 23:05:11 +01:00
Christian Kellner
0c8f5c7ef0 pipeline: drop CAP_{NET_ADMIN,SYS_PTRACE} caps
Drop CAP_{NET_ADMIN,SYS_PTRACE} from the default capabilities which
are only needed to run bwrap from inside a stage which is done by
the `ostree.commit` and `ostree.preptree` stages, so retain them
directly there.
2022-04-27 23:05:11 +01:00
Alexander Larsson
68481f48ae Support specifying multiple devices in all mkfs versions (like xfs)
mkfs.xfs already has `additionalProperties: true` for the devices
section, as this is necessary for example when creating lvm2 setups.
This should be possible for other filesystem types too.
2022-04-19 18:00:15 +02:00
Tomas Hozza
bbe6127ddb Stages: support vmdk subformat in the qemu stage
Support specifying the subformat for the `vmdk` image type in the
`org.osbuild.qemu` stage.
2022-04-05 15:01:34 +02:00
Enric Balletbo i Serra
31eed79760 stages/sfdisk: support changing GPT partition attribute bits
When is specified change the GPT partition attribute bits. The attributes
is a comma list of bits numbers or bit names. You can take a look at the
manual page of sfdisk to see the supported attribute bits.
2022-03-24 00:05:32 +01:00
Tomas Hozza
3dea4b934c stages/rpm: don't verify signatures during install
If instructed, the rpm stage checks all digests and signatures of a package
explicitly using `rpmkeys` tool. The default stage behavior is that no
package signatures are checked when installed by the stage (not even
explicitly).

For these reasons, the package signature checking is supposed to be
disabled when installing rpm packages. This was achieved by passing the
`--define "_pkgverify_level none"` option to rpm. However this option
specifies only requirements for a package to be installed and `none`
means that packages without any signature are accepted by rpm. If the
package signature is deemed BAD, the package installation fails even
though this option has been passed to rpm.

There are valid cases when even packages which signature marked as BAD
should be installed. It may happen, that the GPG key used to sign a
package uses an algorithm not allowed by the system crypto policy, e.g.
SHA1. If such GPG key is imported on the system and a package signed
using it is being installed, its installation would fail when the
package is read by rpm. This is because its signatures are by default
checked if they exist in the package.

The desired behavior to not check any package signatures when installing
a package is instead achieved by using `--nosignature` rpm option. It
turns off the whole signature checking mechanism.

Use the `--nosignature` rpm option instead of
`--define "_pkgverify_level none"`, when installing packages using rpm.

Fix https://github.com/osbuild/osbuild/issues/991
2022-03-22 18:41:12 +01:00
Christian Kellner
d38cdb6425 stages/selinux: directly call setfilecon
Instead of using `chcon`, directly call `selinux.setfilecon`.
On systems without SELinux support, i.e. coreutils was built
without `<selinux.h>` present, `chcon` will return `ENOTSUP`
for all calls to SElinux functions like `setfilecon` even if
the selinux libraries are later installed.
Therefore we directly call the new osbuild helper function,
which really is just a wrapper around setting extended file
attributes and thus will work even if SELinux support is not
compiled into coreutils.
The only other thing `chcon` is doing besides a cal to the
`setfilecon` method is to convert the context string to a
new `contex_t` and back to validate it. This should not be
needed since the kernel will do this for us. On system
without SELinux support `context_new` will also not validate
the context.
2022-03-18 20:36:10 +01:00
Christian Kellner
d8f36b55fa stages/grub2: option to configure GRUB_DEFAULT
Add a new attribute `config.default` that when set will be written to
`GRUB_DEFAULT`. This should be set to `saved` when a `saved_entry` is
specified so that the functionality will be preserved if the grub cfg
gets regenerated (which is really should not, but we can not prohibit
it).
2022-03-04 16:38:41 +01:00