Commit graph

469 commits

Author SHA1 Message Date
Sanne Raymaekers
bae4f77661 stages: add ovf stage
This stage generates an ovf descriptor and a manifest intended for
vSphere. The resulting artifacts can be tarred together with the vmdk
into an ova.
2023-03-28 23:07:38 +02:00
Antonio Murdaca
345b2a5997 stages/ignition: support multi kargs in network kcmdline
As of today the functionality is broken if somebody passes more than one
karg as the join would simply unify the strings but
ignition_network_kcmdline is unquoted and results in just the very first
karg to be consumed by the set command.

Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-02-27 17:39:14 +01:00
Tomáš Hozza
200c2b0129 stages/copy: add option to remove destination before copying
Extend the copy stage to optionally allow removing the destination
before copying. This allows one to not follow symlinks if the
destination is a symlink to a file. By default, `cp` would change
the file pointed to by the destination if it is symlink.

Extend the stage doc text to cover the behavior with regard to
destination being a symlink.

Add unit tests for the copy stage to also test the newly added option.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-20 10:31:06 +01:00
Achilleas Koutsou
d3229dc929 stages/shell.init: add top-level files property
Add a top level property "files" to the schema and move the rest of the
existing schema one level down.  This way we can support adding global
properties in the future if we ever need to expand the scope of the
stage.
2023-02-15 18:14:01 +01:00
Achilleas Koutsou
eb903a9596 stages/shell.init: restrict filename pattern
Old filename pattern was too wide and contained a bug in the form of
.-_, which defined a range instead of three characters.
2023-02-15 18:14:01 +01:00
Achilleas Koutsou
b848e5d9df stages/shell.init: add pattern for env var names
Pattern for valid environment variable names as defined in
The Open Group Base Specifications Issue 7, 2018 edition
IEEE Std 1003.1-2017 (Revision of IEEE Std 1003.1-2008)

https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html

Updated tests to match UPPERCASE ONLY var names.
2023-02-15 18:14:01 +01:00
Achilleas Koutsou
03b467da72 stages: add shell.init stage
New stage for writing shell init files in /etc/profile.d.
Currently only supports writing environment variables as key-value
pairs.
2023-02-08 13:30:39 +01:00
Tomáš Hozza
16063fff57 Add org.osbuild.chown stage
Add a new `org.osbuild.chown` stage for setting user and group ownershop
of files. The stage runs the `chown` from the image using `chroot` to
enable it to use users and groups that exist only in the (image) tree.

Add unit test testing the stage in various scenarios.

Co-authored-by: Janine Olear <pninak@web.de>
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-07 12:14:01 +01:00
Tomáš Hozza
db1fa5b3c3 stages/mkdir: enhance stage documentation
Explicitly mention the stage behavior with regard to setting mode on
newly created or existing directories.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-20 14:28:36 +01:00
Tomáš Hozza
d3a63cf056 Revert "stages/mkdir: explicitly set mode using os.chmod"
This reverts commit a988aacf99.

After some discussion, the original behavior was intentional. With the
added support for gracefully handling the existence of directories, the
stage would originally not set the mode of an existing directory, while
now it will. Additional issue is that `mkdir` applies the provided mode
- umask, which was intentional. Setting the same mode without taking
umask value into account is not desired.
2023-01-20 14:28:36 +01:00
Tomáš Hozza
b8e1450a35 stages/mkdir: add exist_ok option to not fail if directory exists
Add a new optional stage option to not fail if the specified directory
already exists. This will make it easier to support creation of custom
repositories via customizations in osbuild-composer. The reason is that
if a specified directory exists in an image, because it was created by
an RPM, then creating it would fail. However, the user may have
specified different mode for the directory, than it already has. Since
there is no way to know for sure if the directory already exists on the
image, without building the image itself, it is desired to handle this
case gracefully as valid in specific use cases.

The default behavior stays the same - specifying an existing directory
path will lead to an error.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-16 20:19:27 +01:00
Tomáš Hozza
a988aacf99 stages/mkdir: explicitly set mode using os.chmod
Documentation for os.mkdir() says that the mode is
ignored on some systems. Also umask value may affect
the final mode. So we set the mode explicitly.

Set the mode explicitly.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-16 20:19:27 +01:00
Achilleas Koutsou
2efdbe0277 stages/rhsm.facts: create facts file in /etc
Instead of creating the file in /usr/share and symlinking to /etc,
create it directly in /etc. This fixes an issue with SELinux labeling.
The file in /usr/share does not get labelled correctly because it
doesn't match the policy and causes issues with some tools (rhc).

See rhbz#2147450.
2023-01-03 09:27:13 +01:00
Eric Curtin
0007fc2065 ostree.config: add aboot (Android) bootloader config option
We want to add aboot to the list of possible bootloaders so we can
distinguish if we are using aboot or one of the other bootloaders.

Signed-off-by: Eric Curtin <ecurtin@redhat.com>
2022-12-15 11:27:02 +00:00
Christian Kellner
caad585907 stages: add new org.osbuild.ostree.encapsulate
Wrap OSTree commits into a oci container image.
2022-12-15 11:13:19 +01:00
Christian Kellner
e793cc0eb5 stages/users: create missing home directories
If a home directory is specified for an existing user that does
not have one, `usermod` does not create one. This case is now
detected and `mkhomedir_helper(8)` is run inside the chroot to
create the home dir. In Fedora this utility is provided by the
`pam` package so this is now installed in the corresponding
tests together with a new user that simulates the aforementioned
scenario.
Enahnce the stage description: drop an superflous line and add
a description for the home-dir scenario.
2022-12-07 17:25:30 +01:00
Christian Kellner
8ee740dff2 stages/users: don't create homedir when home is empty
Don't create a home dir if the `home` property is given, but empty.
Modify the users tests to create a user that has no home directory.
2022-12-07 17:25:30 +01:00
Christian Kellner
e5e70efc37 stages/users: accept identical uid for existing users
Accept a `uid` option for an existing user if it is the existing
one. This allows to have the same options for existing as well as
new users, which in turn allows for the same blueprint in Composer
for new and upgrade OSTree commits. In the latter we pre-fill the
password database from a previous commit, which is needed to make
sure that uids do not change. Since Composer can't know which of
the specified users in the blueprint are new ones or existing ones
it cannot easily omit the corresponding stage options. Thus the
stage options have to be the same for new and existing users.
2022-12-07 17:25:30 +01:00
Christian Kellner
a2e212bb26 stages/containers.storage: ability to specify a base file
In newer version of the container storage package the config file
moved from `/etc/containers` to `/usr/containers/`. The later is
not marked as config, so we don't want to change it.
The current containers code[1] will read _either_ a file in `usr`
or in `etc` depending on the existence of the latter. This we can
not just write the keys we want into a file in `/etc/containers`
without losing all other defaults set in the config file.
A new option `filebase` is therefore added, that when given will
be read and form the bases of the configuration data. Then data
from the target file (given via `filename`) will be merged into
and finally the actual configuration will be applied on top.

[1] 232bf398bd/types/options.go (L85)
2022-11-15 11:34:15 +01:00
Christian Kellner
b0ffe6c2b3 stages/containers.storage: allow setting options of empty sections
If the existing config does not have a section that is being set
by the stage it currently would crash since we assume that it
exists. Default to an empty dict in that case.
2022-11-15 11:34:15 +01:00
Christian Kellner
9b9c989d7b stages: add stage to build unified kernel images
Add new `org.osbuild.uki` stage to build unified kernel images.
2022-11-14 20:10:59 +01:00
Antonio Murdaca
f34bee944b grub2.iso: add timeout option
add the ability to configure the grub timeout. Today it defealts to 1
minute and we can't configure it from osbuild-composer either (the
simplified-installer is mainly using this afaict and 1 min is just too
much).

Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2022-11-11 17:22:14 +01:00
Antonio Murdaca
ea8261cad6 ignition: fix ignition_network_kcmdline
Signed-off-by: Antonio Murdaca <runcom@linux.com>
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2022-11-09 12:02:07 +01:00
Christian Kellner
7729e6225e stages: add new cpio.out stage
New stage that will assemble a tree into a CPIO archive with the
help of the `cpio` command.
2022-11-03 17:26:43 +01:00
David Rheinsberg
8464e80143 stages/ostree.preptree: fix broken link
Fix the broken "adapting-existing" link in the ostree.preptree
introdutory comment.
2022-10-26 15:04:53 +02:00
Colin Walters
6a2c78b85f stages/ostree.preptree: Link to rpm-ostree code
This code was clearly influenced/copied from rpm-ostree,
since it's now duplicated let's cross-link to help ensure that
if someone wants to change this they hopefully consider changing
rpm-ostree too.
2022-10-26 15:04:53 +02:00
Irene Diez
24b116213c stages/firewall: change zones/sources handling
Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-26 12:07:21 +02:00
Antonio Murdaca
833f2da3f9 Extend firewall stage to add sources
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-10-26 12:07:21 +02:00
Christian Kellner
976fbe178a stages/keymap: add font option
Allows the configuration of the console font. systemd-firstboot
does not allow to do it, so we need to manually tweak the file.
2022-10-25 11:20:11 +02:00
Christian Kellner
30e20050c6 stages/logind-systemd: add ReserveVT option
Add option to reserve a VT exclusively for autovt@.service activation.
See logind.conf(5) for more details.
2022-10-22 14:58:44 +02:00
Irene Diez
145a0cc273 stages: add org.osbuild.systemd-journald stage
This stage allows to configure a systemd-journald
config drop-in file located in
`/usr/lib/systemd/journald.conf.d`.

Currently it supports the following options for
the 'Journal' section: Storage, Compress, SplitMode,
MaxFileSec, MaxRetentionSec, SyncIntervalSec, Audit.

Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-21 09:04:51 +02:00
Ondřej Budai
000ac81777 stages/rpm: make the fake machine-id newline-terminated
From machine-id(5): machine ID is a single *newline-terminated*, hexadecimal,
32-character, lowercase ID.

This commit fixes that by adding a newline.

This broke us on Fedora 38:
This pull request added `set -e` to kernel-install:

https://github.com/systemd/systemd/pull/23881

This caused F38 pipeline failing because the script also runs
`read -r MACHINE_ID </etc/machine-id`. `read` returns 1 if the input doesn't
contain a newline, thus after the addition of `set -e`, the script started
crashing with our bad machine-id. This caused crashing of the whole image build
in the grub2 stage which requires kernel-install to work (because kernel-install
is responsible for putting vmlinuz into /boot).
2022-10-18 14:40:24 +02:00
Antonio Murdaca
8a7b6d382d stages: fix ostree config stage
Currently we're using `str(value)` on a boolean which yields `True` or
`False` - turns out ostree reads these values case sensitive and despite
setting `True|False`, it doesn't just work.

From jlebon on slack:

> the syntax is readonly=true . it's case sensitive

Fix the above and also just remove the loop as, while it's handy, we'll
have to differentiate between options' values anyway and it's just two
options we support today.

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-09-28 21:30:49 +02:00
David Rheinsberg
054bba5228 stages/greenboot: avoid new pylint suppressions
The `consider-using-f-string` suppression is new, thus old pylint will
complain about unknown directives. If we start ignoring unknown
directives for this reasons, we will no longer get warned about
misspelled directives. Hence, lets avoid this for now and just use an
f-string.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-14 12:26:37 +02:00
David Rheinsberg
cdd65d4fd9 stages/greenboot: ignore f-string recommendation
The explicit string-formatting is definitely easier to manage than an
f-string in the greenboot case. Make pylint suppress its recommendation.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-13 16:21:14 +02:00
Simon de Vlieger
ea6085fae6 osbuild: run isort on all files 2022-09-12 13:32:51 +02:00
Simon de Vlieger
b07aca5d86 modules: explicit encodings for open() 2022-09-09 15:33:29 +02:00
Alexander Larsson
f4bfce05e3 org.osbuild.oci-archive: Support setting Entrypoint
This allows setting Entrypoint (as well as Cmd) in the oci image,
as per the spec:

  https://github.com/opencontainers/image-spec/blob/main/config.md

Note: These two are not equivalent, the Cmd part is replaced by the
argument on the `docker run` commandline, whereas the entrypoint is
kept from the config, so it is important to expose both of these.
2022-09-05 13:10:47 +02:00
Sayan Paul
28854f452a Add greenboot configuration management via osbuild
The script will update /etc/greenboot/greenboot.conf if user passes
the parameter in the config. Right now this only tweaks one config but
it will/can be expanded if other use cases come.

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-08-23 16:53:55 +02:00
Achilleas Koutsou
b87eaf6032 stages/ostree.remotes: support contenturl
OSTree remotes can be configured with a contenturl that is separate from
the metadata url.  When specified, the contenturl is used for content
and the url is used only for metadata.
Updated the description of the url property for clarity.
2022-08-22 20:17:40 +02:00
Achilleas Koutsou
2bff83364b stages/ostree.remotes: support gpgkeypath
OSTree remotes can be configured to import keys from a file on disk or a
directory containing multiple keys.

When specified, explicitly enables gpg-verify=true.
2022-08-22 20:17:40 +02:00
Christian Kellner
a967c61d17 stages/rpm: ability to set database path
Add a new option `dbpath` that controls the database path of the
rpm database.
2022-08-15 18:22:59 +02:00
Christian Kellner
8c00907461 stages/rpm.macros: new-line terminate values
Needed if we want to specify multiple values and even for the
single value use case this is the right thing to do.
2022-08-15 18:22:59 +02:00
Christian Kellner
b33f178a11 stages/rpm.macros: allow setting of _dbpath
Allow setting the database path via the `_dbpath` variable.
2022-08-15 18:22:59 +02:00
Christian Kellner
16f1c560cc stages/rpm: support marking install as ostree
An ostree system can be identified as such by the presence of a
marker file: /run/ostree-booted. The `rpm-ostree` tool also
creates this marker during the _installation_ of the system[1,2].
Recently, starting with F36, the authselect package has taken
has become mandatory[3] and is now owning the nsswitch config.
An rpm-ostree system, which has parts of the user database in
/usr, release on the nss-altfiles to read these databases. The
necessary entries are added during the post-processing, which
is called in our `org.osbuild.ostree.preptree` stage[4]. This
installation is skipped though if the nsswitch is the file is
a symlink, indicating that it is owned by some other package,
like authselect. So the F36 authselect change first broke rpm-
ostree[6]. The fix was to check for `/run/ostree-booted` in
the authselect scriptlet and special case this situation[7,8].
Now, our `org.osbuild.rpm` stage does not yet have the ability
to pretend it is a running ostree system and thus we did not
get the special treatment resulting in nss-altfiles not being
enabled in our ostree commits. Therefore the passwd database in
/usr was not read and a lot of daemons and programs without a
valid user, like e.g. `sshd`.
This change introduces a new option, `ostree_booted` that if
set, will create the `/run/ostree-booted` marker and thus our
installation phase will get the same treatments from packages
as rpm-ostree. Hopefully.

[1] 730bec87b1/rust/src/builtins/compose/mod.rs (L24)
[2] 6211d1452e/src/app/rpmostree-compose-builtin-tree.cxx (L501)
[3] https://fedoraproject.org/wiki/Changes/Make_Authselect_Mandatory
[4] 7993c6f565/rust/src/composepost.rs (L635)
[5] d614caeca1/f/0010-spec-fix-detection-of-ostree-system.patch
[6] https://bugzilla.redhat.com/show_bug.cgi?id=2034360
[7] https://src.fedoraproject.org/rpms/authselect/c/d614caeca1a68f55542aefd0d76bda2691c85d24?branch=f36
[8] https://github.com/authselect/authselect/issues/48
2022-08-13 19:21:52 +01:00
Christian Kellner
776bab46ae stages/rpm: convert the mount code to python
Convert the code that sets up the bind mounts to /dev, /sys/ and
/proc from a bash script to python by using the new mount helper
that were moved from the `objectstore`.
2022-08-13 19:21:52 +01:00
Christian Kellner
8f95154a8c stages/rpm: extract machine-id code into function
Extract the code that creates the fake machine id, if it does not
exist, into its own function. Also convert the shell code into
Python code.
2022-08-13 19:21:52 +01:00
Christian Kellner
b49f3f91f9 util/mnt: extract MountGuard into new module
Extract the `MountGuard` class from all stages that defined it into
a new `mnt` utility module.
2022-08-13 19:21:52 +01:00
Christian Kellner
f05078f66e global: fix PEP-8 formatting
This patch was generated by running `autopep8 --diff` on the
source tree and then applying the diff.
2022-08-05 09:41:05 +02:00
Christian Kellner
a3bc5f7692 stages/gcp.guest-agent.conf: warning -> info
Inform when a new guest-agent configuration file is created,
but remove the warning prefix, since it is perfectly fine if
the file does not yet exist.
2022-08-04 08:08:02 +02:00