Commit graph

235 commits

Author SHA1 Message Date
Gianluca Zuccarelli
586d6bbe43 stages/yum.repo: add sslverify field
Add `sslverify` field to the yum.repo stage.
2023-05-03 20:53:52 +02:00
Sanne Raymaekers
33597e82a3 test/data/stages: add wsl.conf test 2023-04-11 10:11:08 +02:00
Simon de Vlieger
12e4e541c3 stage/systemd.preset: be able to write a preset file
The right way to enable services is to use a preset file instead of
writing directly into /etc. This adds a new stage called
`org.osbuild.systemd.preset` to do so.
2023-04-05 14:34:39 +02:00
Achilleas Koutsou
532a4c1166 test: add manifest-list test for skopeo stage
Added another skopeo stage to skopeo/a.mpp.json with a skopeo source for
a container hosted on the osbuild-composer gitlab registry.  The name
points to a manifest list, which refers to two containers (amd64 and
arm64) that contain a single text file (README.md).  The `index` field
is enabled to include the manifest-list as an extra input to the stage.

The diff is updated with the new expected file list.
The containers were created with buildah:

  amd=$(buildah from --arch=amd64 scratch)
  arm=$(buildah from --arch=arm64 scratch)
  buildah config --created-by "Achilleas Koutsou" "${amd}"
  buildah config --created-by "Achilleas Koutsou" "${arm}"
  buildah copy "${amd}" README.md
  buildah copy "${arm}" README.md
  amdid=$(buildah commit --format=docker --rm "${amd}")
  armid=$(buildah commit --format=docker --rm "${arm}")
  name="registry.gitlab.com/redhat/services/products/image-builder/ci/osbuild-composer/manifest-list-test"
  buildah manifest create "${name}" "${amdid}" "${armid}"

  podman manifest push --all "${name}" dir:container
2023-03-31 14:57:26 +02:00
Sanne Raymaekers
bae4f77661 stages: add ovf stage
This stage generates an ovf descriptor and a manifest intended for
vSphere. The resulting artifacts can be tarred together with the vmdk
into an ova.
2023-03-28 23:07:38 +02:00
Tomáš Hozza
200c2b0129 stages/copy: add option to remove destination before copying
Extend the copy stage to optionally allow removing the destination
before copying. This allows one to not follow symlinks if the
destination is a symlink to a file. By default, `cp` would change
the file pointed to by the destination if it is symlink.

Extend the stage doc text to cover the behavior with regard to
destination being a symlink.

Add unit tests for the copy stage to also test the newly added option.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-20 10:31:06 +01:00
Achilleas Koutsou
d3229dc929 stages/shell.init: add top-level files property
Add a top level property "files" to the schema and move the rest of the
existing schema one level down.  This way we can support adding global
properties in the future if we ever need to expand the scope of the
stage.
2023-02-15 18:14:01 +01:00
Achilleas Koutsou
b848e5d9df stages/shell.init: add pattern for env var names
Pattern for valid environment variable names as defined in
The Open Group Base Specifications Issue 7, 2018 edition
IEEE Std 1003.1-2017 (Revision of IEEE Std 1003.1-2008)

https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html

Updated tests to match UPPERCASE ONLY var names.
2023-02-15 18:14:01 +01:00
Achilleas Koutsou
d5701dac28 test: add unit test for org.osbuild.shell.init 2023-02-08 13:30:39 +01:00
Tomáš Hozza
16063fff57 Add org.osbuild.chown stage
Add a new `org.osbuild.chown` stage for setting user and group ownershop
of files. The stage runs the `chown` from the image using `chroot` to
enable it to use users and groups that exist only in the (image) tree.

Add unit test testing the stage in various scenarios.

Co-authored-by: Janine Olear <pninak@web.de>
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-07 12:14:01 +01:00
Tomáš Hozza
d3a63cf056 Revert "stages/mkdir: explicitly set mode using os.chmod"
This reverts commit a988aacf99.

After some discussion, the original behavior was intentional. With the
added support for gracefully handling the existence of directories, the
stage would originally not set the mode of an existing directory, while
now it will. Additional issue is that `mkdir` applies the provided mode
- umask, which was intentional. Setting the same mode without taking
umask value into account is not desired.
2023-01-20 14:28:36 +01:00
Tomáš Hozza
b8e1450a35 stages/mkdir: add exist_ok option to not fail if directory exists
Add a new optional stage option to not fail if the specified directory
already exists. This will make it easier to support creation of custom
repositories via customizations in osbuild-composer. The reason is that
if a specified directory exists in an image, because it was created by
an RPM, then creating it would fail. However, the user may have
specified different mode for the directory, than it already has. Since
there is no way to know for sure if the directory already exists on the
image, without building the image itself, it is desired to handle this
case gracefully as valid in specific use cases.

The default behavior stays the same - specifying an existing directory
path will lead to an error.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-16 20:19:27 +01:00
Achilleas Koutsou
fdf064b15a test: update rhsm.facts diff 2023-01-03 09:27:13 +01:00
Christian Kellner
1d193008d4 test/data: set source epoch for certain pipelines
For selected pipelines, set the source epoch. The date is set to
the current updates snapshot date (in epoch).
2022-12-15 13:10:35 +00:00
Christian Kellner
acc54c5202 test/ostree-image: +fedora-ostree-native-container
Test building the new fedora-ostree-native-container manifest.
2022-12-15 11:13:19 +01:00
Christian Kellner
f04ea2bab2 test/stages/users: ignore non-deterministic files
The `/etc/shadow` and `/etc/shadow-` files are changing (salting),
so we should not compare them in the test.
2022-12-08 14:44:39 +01:00
Christian Kellner
e793cc0eb5 stages/users: create missing home directories
If a home directory is specified for an existing user that does
not have one, `usermod` does not create one. This case is now
detected and `mkhomedir_helper(8)` is run inside the chroot to
create the home dir. In Fedora this utility is provided by the
`pam` package so this is now installed in the corresponding
tests together with a new user that simulates the aforementioned
scenario.
Enahnce the stage description: drop an superflous line and add
a description for the home-dir scenario.
2022-12-07 17:25:30 +01:00
Christian Kellner
8ee740dff2 stages/users: don't create homedir when home is empty
Don't create a home dir if the `home` property is given, but empty.
Modify the users tests to create a user that has no home directory.
2022-12-07 17:25:30 +01:00
Christian Kellner
48a4419705 test/data: persist the journal for ostree images
Include the new journald config stage to configure journald to
persist the journal. This is needed since we don't create the
`/var/log/journal` directory that journald uses to switch the
default to persistent storage. But instead of creating that
directory, we explicitly configure journald via the new stage.
This is also what Fedora CoreOS does.
2022-11-16 17:51:56 +01:00
Christian Kellner
638d5b834d test/data: add 'rw' kernel flag to ostree image
A recent commit (8a7b6d3) fixed the ostree config stage and thus
we actually properly set the readonly flag for the deployment.
This broke the image since we did not specify the `rw` kernel flag
and as a consequence the ostree mounting code in the initrd broke.
2022-11-16 17:51:56 +01:00
Christian Kellner
ed67839183 test/data: add rootfs to UKI
Create a squashfs image from the root file system and append it
to the existing initram fs. Passing `root=live:/rootfs.img`
is enough to use the existing dmsquash machiner in dracut.
2022-11-14 20:10:59 +01:00
Christian Kellner
053f027c67 test/data: add UKI test manifest
Example UKI manifest.
2022-11-14 20:10:59 +01:00
Christian Kellner
bca3fe62d5 test/data: add binutils to fedora-build-v2
Add binutils for `objcopy` needed to build UKIs.
2022-11-14 20:10:59 +01:00
Christian Kellner
7729e6225e stages: add new cpio.out stage
New stage that will assemble a tree into a CPIO archive with the
help of the `cpio` command.
2022-11-03 17:26:43 +01:00
Irene Diez
24b116213c stages/firewall: change zones/sources handling
Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-26 12:07:21 +02:00
Christian Kellner
976fbe178a stages/keymap: add font option
Allows the configuration of the console font. systemd-firstboot
does not allow to do it, so we need to manually tweak the file.
2022-10-25 11:20:11 +02:00
Christian Kellner
30e20050c6 stages/logind-systemd: add ReserveVT option
Add option to reserve a VT exclusively for autovt@.service activation.
See logind.conf(5) for more details.
2022-10-22 14:58:44 +02:00
Irene Diez
145a0cc273 stages: add org.osbuild.systemd-journald stage
This stage allows to configure a systemd-journald
config drop-in file located in
`/usr/lib/systemd/journald.conf.d`.

Currently it supports the following options for
the 'Journal' section: Storage, Compress, SplitMode,
MaxFileSec, MaxRetentionSec, SyncIntervalSec, Audit.

Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-21 09:04:51 +02:00
Irene Diez
b96f715c6e test: add README.md on how to make tests for stages
Adds a README.md which explains what do you need
to do to add a new test for an stage

Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-18 19:02:34 +02:00
Sanne Raymaekers
925ca9b41e sources/ostree: set contenturl when pulling from remote
If a contenturl is specified, the url is used only for metadata. This is
useful when the actual content is hosted separately.
2022-10-14 12:04:54 +02:00
David Rheinsberg
1bdf038d36 ci: update container and actions
Update the osbuild-ci container and privdocker action to the most recent
builds.

This changes the ostree-image-tests since the Fedora update pulled in
util-linux/fdisk changes that align partition sizes. Hence, the ostree
tests need to be changed to have aligned partition sizes as well. For
more information, see:

    commit 921c7da55ec78350e4067b3fd6b7de6f299106ee
    Author: Karel Zak <kzak@redhat.com>
    Date:   Thu Jan 27 10:50:45 2022 +0100

        libfdisk: (gpt) align size of partition by default

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-23 12:08:10 +02:00
Simon de Vlieger
ea6085fae6 osbuild: run isort on all files 2022-09-12 13:32:51 +02:00
Christian Kellner
07d3605385 manifests: set db path in ostree commit
rpm-ostree is setting the database path to /usr/share/rpm during
the build. Do so as well.
2022-08-15 18:22:59 +02:00
Christian Kellner
c28269f8d9 manifests: enable cloud-init services in commit
It is instabled, but not a enabled via presets so we have to
explicitly enable it.
2022-08-13 19:21:52 +01:00
Christian Kellner
5570e2a92e manifests: upgrade ostree commit, image to F36
Fedora 34 is end of life and we ought to be testing with newer
releases so that we catch issues like the authselect one, that
is detailed in "stages/rpm: support marking install as ostree".
2022-08-13 19:21:52 +01:00
Christian Kellner
899fdcd5d9 manifests: set rpm/ostree_booted for commit
See "stages/rpm: support marking install as ostree" for more
information why this is needed on Fedora 36 and newer.
2022-08-13 19:21:52 +01:00
Christian Kellner
10255dbdcc test/manifests: base ostree image on commit
Instead of re-defining the ostree-commit in the ostree-image.mpp
file, import the needed pipelines to build the commit and use
that.
2022-08-13 19:21:52 +01:00
Christian Kellner
d2bcaffbdc test/manifests: add update repos in ostree-commit
Include modular and updates repositories for ostree commit and the
new build manifest.
2022-08-13 19:21:52 +01:00
Christian Kellner
073359166d test/manifests: add generic fedora build manifest
Basically a copy of f34-build-v2 but uses variables. Use it form
fedora-ostree-commit.
2022-08-13 19:21:52 +01:00
Christian Kellner
056a33041b manifests: use variables in ostree-commit
Instead of hard-coding the release and the snapshot, use variables,
which should help updating to newer releases and snapshots.
2022-08-13 19:21:52 +01:00
Christian Kellner
4b11989ce3 manifests: convert ostree commit to format v2
Convert the manifest to use version 2 format. Version 1 is really
not used much anymore because composer was completely ported to
using v2. Welcome to the future, ostree commit.
2022-08-13 19:21:52 +01:00
Christian Kellner
c3b172e16b manifests/ostree-commit: update package list
Include cryptsetup and lvm2 so that commit can be deployed to
images that are encrypted and have lvm2 layout. Additionally,
include greenboot. Remove the explicit dependency on ostree
and systemd-udev, which are automatically pulled in.
2022-08-13 19:21:52 +01:00
fkolwa
2bca7b07f3 stages: add instance_configs.cfg.distro
test/data: add instance_configs.cfg.distro test
2022-08-03 15:39:35 +02:00
Simon de Vlieger
f3e2c8d5e9 stage: add an rhsm.facts stage
This stage allows to manifests to define information to be put into a
built image. The information can then later be used by RHSM/Insights.

The facts are stored in `/usr/share/osbuild/rhsm.facts` and then
symlinked into `/etc/rhsm/facts/osbuild.facts`.
2022-08-03 08:54:51 +02:00
Ondřej Budai
092f886904 stages/users: support a dot inside a username
From IEEE Std 1003.1-2017:

 3.437 User Name:
A string that is used to identify a user; see also User Database. To be
portable across systems conforming to POSIX.1-2017, the value is composed
of characters from the portable filename character set. The <hyphen-minus>
character should not be used as the first character of a portable user name.

 3.282 Portable Filename Character Set
The set of characters from which portable filenames are constructed.

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
a b c d e f g h i j k l m n o p q r s t u v w x y z
0 1 2 3 4 5 6 7 8 9 . _ -

The last three characters are the <period>, <underscore>, and <hyphen-minus>
characters, respectively.

---

The regex checking the username in org.osbuild.users stage omitted the
<period> character before. This commit adds it and modifies the stage
test in order to cover this change.

Fixes: rhbz#2104464
2022-07-21 14:52:57 +02:00
Christian Kellner
6e66c69608 stages: add org.osbuild.containers.storage.conf
New stage to  create or modify `containers-storage.conf` config
files. The currently implement strategy is to merge the given
and existing configuration options.
2022-07-11 16:25:56 +02:00
Christian Kellner
07a45ba34c test/data: include python3-toml in v2 build root
Prepares for testing stages that use python3-toml to edit config
files.
2022-07-11 16:25:56 +02:00
Gianluca Zuccarelli
c379dccdf1 test/data: update oscap.remediation tests
The oscap.remediation stage now runs the
remediation during build time rather than
creating the necessary files to run the
remediation on first-boot. The tests have
been updated to reflect these changes.
2022-07-06 09:59:00 +02:00
Christian Kellner
4c12967c0d stages/dnf.config: add support for tsflags
Add support for specifying rpm transaction flags via the `tsflags`
options. See `dnf.conf(5)` for more details.
2022-06-30 09:37:27 +01:00
Christian Kellner
f078420879 manifests/fedora-container: persist _install_langs
Persist the `_install_langs` via the new `org.osbuild.rpm.macros`
stage.
2022-06-30 00:17:23 +01:00