Commit graph

2160 commits

Author SHA1 Message Date
schutzbot
845f58631c Post release version bump
[skip ci]
2022-10-26 08:18:16 +00:00
Christian Kellner
976fbe178a stages/keymap: add font option
Allows the configuration of the console font. systemd-firstboot
does not allow to do it, so we need to manually tweak the file.
2022-10-25 11:20:11 +02:00
Christian Kellner
30e20050c6 stages/logind-systemd: add ReserveVT option
Add option to reserve a VT exclusively for autovt@.service activation.
See logind.conf(5) for more details.
2022-10-22 14:58:44 +02:00
Irene Diez
145a0cc273 stages: add org.osbuild.systemd-journald stage
This stage allows to configure a systemd-journald
config drop-in file located in
`/usr/lib/systemd/journald.conf.d`.

Currently it supports the following options for
the 'Journal' section: Storage, Compress, SplitMode,
MaxFileSec, MaxRetentionSec, SyncIntervalSec, Audit.

Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-21 09:04:51 +02:00
SchutzBot
dd830a2796 schutzfile: update manifest-db ref 2022-10-20 2022-10-20 11:07:01 +02:00
Irene Diez
b96f715c6e test: add README.md on how to make tests for stages
Adds a README.md which explains what do you need
to do to add a new test for an stage

Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-18 19:02:34 +02:00
Ondřej Budai
000ac81777 stages/rpm: make the fake machine-id newline-terminated
From machine-id(5): machine ID is a single *newline-terminated*, hexadecimal,
32-character, lowercase ID.

This commit fixes that by adding a newline.

This broke us on Fedora 38:
This pull request added `set -e` to kernel-install:

https://github.com/systemd/systemd/pull/23881

This caused F38 pipeline failing because the script also runs
`read -r MACHINE_ID </etc/machine-id`. `read` returns 1 if the input doesn't
contain a newline, thus after the addition of `set -e`, the script started
crashing with our bad machine-id. This caused crashing of the whole image build
in the grub2 stage which requires kernel-install to work (because kernel-install
is responsible for putting vmlinuz into /boot).
2022-10-18 14:40:24 +02:00
Jan Macku
9d26789b78 ci(packit): Use YAML Merge Key syntax to reduce duplicity 2022-10-18 13:04:03 +02:00
Jan Macku
f894d64f62 ci: Update .packit.yaml - update/remove deprecated options 2022-10-18 13:04:03 +02:00
Jan Macku
9e2add32a0 ci: Remove unused .travis.yml config 2022-10-18 13:04:03 +02:00
Sanne Raymaekers
fb3ac490d5 schutzbot/mockbuild: stop running mock as root
Running mock as root is broken, this commit can be reverted once
rhbz#2135203 is fixed.
2022-10-17 18:36:02 +02:00
Sanne Raymaekers
925ca9b41e sources/ostree: set contenturl when pulling from remote
If a contenturl is specified, the url is used only for metadata. This is
useful when the actual content is hosted separately.
2022-10-14 12:04:54 +02:00
Sanne Raymaekers
cc9d05c201 sources/ostree: fix mTLS certs remote option
These options take the format --set "KEY_VALUE". Also fix the string
formatting.
2022-10-14 12:04:54 +02:00
Jakub Rusz
c638cbcb83 rpmbuild: build rpms on RHEL 8.8 and 9.2 nightly 2022-10-14 10:24:33 +02:00
schutzbot
6523fd8d41 Post release version bump
[skip ci]
2022-10-12 08:19:37 +00:00
Sanne Raymaekers
fcaad0462a sources/ostree: pull from remote using rhsm mTLS certs
The consumer certs are used to uniquely identify a system against
candlepin. These consumer certs can be used to identify the system when
pulling from RH controlled ostree repositories.
2022-10-11 16:49:45 +02:00
Christian Kellner
1c81d1e966 monitor/log: show build root and runner
In the `LogMonitor`, print the build root and its runner for each
pipelie. That could help identifying issues with runners.
2022-10-11 12:49:16 +02:00
Christian Kellner
9480009189 runners: remove aliased runner
Now that we can automatically detect the best available runner for
a requested one, we don't need to maintain the link farm with the
explicit mapping anymore.
2022-10-11 12:49:16 +02:00
Christian Kellner
5bdc8d030c osbuild: auto-detect best available runner
Use the new `Index.detect_runner` method that will give us the best
available runner for a requested one. To do so a new `pipeline.Runner`
class is introduced that stores the `meta.RunnerInfo` class for the
specific runner and the original name that was requested.
In the manifest loading and describing functions of the formats, use
`Index.detect_runner` to get the `RunnerInfo` for a requested runner
and then wrap it in a `pipeline.Runner` object, which is then passed
to the `Manifest.add_pipeline` method.
See also commit "meta: ability to auto-detect runner".
Adjust all test.
2022-10-11 12:49:16 +02:00
Christian Kellner
0554ac652b test/fmt/v1: use existing runner in manifests
Instead of using a non-existing runner `org.osbuild.test` use an
existing one `org.osbuild.linux`. This prepares the switch to
using runner auto-detection, which will rely on existing runners.
2022-10-11 12:49:16 +02:00
Christian Kellner
77e717f4f4 buildroot: explicitly bind mount runner
Instead of relying on the assumption that the specific runner will
be in `/run/osbuild/lib/runners/` we now bind-mount the runner at a
specific well known path and execute it from there.
2022-10-11 12:49:16 +02:00
Christian Kellner
7a8b3d3c6f meta: ensure we have an absolute path for Index
Ensure that we always have an absolute path so that all paths
in `ModuleInfo` and `RunnerInfo` are absolute as well.
2022-10-11 12:49:16 +02:00
Christian Kellner
e923f559ab meta: cache host runner
The host runner does not change and we can thus avoid looking it up
over and over again.
2022-10-11 12:49:16 +02:00
Christian Kellner
ebf967ad1f pipeline: reformat line for readability
The line got too long.
2022-10-11 12:49:16 +02:00
Christian Kellner
7c399f15df meta: include path in RunnerInfo
This is so that once we have a `RunnerInfo` we can directly run
it without needed a back-reference to the index.
2022-10-11 12:49:16 +02:00
Christian Kellner
683a8cbfa7 meta: cache list of runners
Instead of enumerating all existing runners -- doing i/o -- we
cache the list at the `Index` level.
2022-10-11 12:49:16 +02:00
Christian Kellner
c755068bd2 meta: new host runner detection logic
This is a port of `pipeline.detect_host_runner` in combination
with the new runner detection logic.
2022-10-11 12:49:16 +02:00
Christian Kellner
ec1c5bb37c test: checks for runner detection
Add a test suite for the runner detection logic.
2022-10-11 12:49:16 +02:00
Christian Kellner
49dc76c434 test: add new test suite for 'meta' module
Move the checks for `meta.Schema` from `test_osbuild.py` into the new
test suite, converting it to use pytest in the process.
2022-10-11 12:49:16 +02:00
Christian Kellner
6695294a29 meta: ability to auto-detect runner
The way that runners were designed is the following: For each distro
we have a specific runner. In case a new version of the distro can
use the previous runner, we just create a symlink. In case a new
distro version needs adjustments, the runner is copied and adjusted.
This is a very clean and obvious design. There is one big drawback:
For each new distribution a symlink must be created before it can be
used. For Fedora that should ideally happen when it is branched; and
this will, ipso facto, always be a symlink since at the time of the
branching the new distro is the old distro. But at this very moment
osbuild will be broken since it does not contain the new runner; the
only way to prevent this is to create the corresponding new runner
before the distro is branched, where it then must be a symlink too.
This very much suggest that instead of the explicit symlink, which
does not /that/ much clarity, the existing "old" runner should just
work for the new distribution. This commit implements the logic to
do just that: all existing runners are parsed into a distro and
version tuple and then, given a specific requested distro, the best
matching one is return.
2022-10-11 12:49:16 +02:00
Antonio Murdaca
8a7b6d382d stages: fix ostree config stage
Currently we're using `str(value)` on a boolean which yields `True` or
`False` - turns out ostree reads these values case sensitive and despite
setting `True|False`, it doesn't just work.

From jlebon on slack:

> the syntax is readonly=true . it's case sensitive

Fix the above and also just remove the loop as, while it's handy, we'll
have to differentiate between options' values anyway and it's just two
options we support today.

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-09-28 21:30:49 +02:00
schutzbot
14e3cfc860 Post release version bump
[skip ci]
2022-09-28 08:21:46 +00:00
Thomas Lavocat
bc06a037ee manifest-db: propage the osbuild SHA on manifest-db
Add a bot to update the SHA dependency to OSBuild on manifest-db
automatically.

The OSBuild SHA will be changed only if the manifest-db SHA was updated
on the last commit landed on main. This way this ensure that CI ran
successfully for this exact version of OSBuild against manifest-db's
actual state. Therefore it is safe to assume that we can upgrade the
manifest dependency without running the whole CI again.
2022-09-23 14:18:38 +02:00
David Rheinsberg
c9ac6d5ee6 test/src: improve file enumeration
Improve the file-enumeration to first check for all files ending in
`*.py`, and for everything else run `/bin/file` on it, and check for
python mime-types.

Note that we expect the caller to open all those files anyway, and thus
this should not any significant overhead, unless we end up with lots of
non-python files in the repository (which is highly unlikely). But even
then, `/bin/file` only reads the first few bytes of a file, and will
default if those are not sufficient to detect the file type. Hence, not
much overhead is expected by this.

This change will now start adding scripts in ./tools/ to the linter,
including osbuild-mpp.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-23 12:08:10 +02:00
David Rheinsberg
1bdf038d36 ci: update container and actions
Update the osbuild-ci container and privdocker action to the most recent
builds.

This changes the ostree-image-tests since the Fedora update pulled in
util-linux/fdisk changes that align partition sizes. Hence, the ostree
tests need to be changed to have aligned partition sizes as well. For
more information, see:

    commit 921c7da55ec78350e4067b3fd6b7de6f299106ee
    Author: Karel Zak <kzak@redhat.com>
    Date:   Thu Jan 27 10:50:45 2022 +0100

        libfdisk: (gpt) align size of partition by default

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-23 12:08:10 +02:00
David Rheinsberg
4b09088661 test/isort: apply diff to full tree
Apply the isort modifications to the entire source tree, not just the
selected python files of test-src.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-23 12:08:10 +02:00
David Rheinsberg
5aaa4e602e test/isort: print diff
Make isort print the diff so we can actually fix the problems. This
might be overly verbose when introducing isort, but for new changes it
is very nice to see the actual code it complains about.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-23 12:08:10 +02:00
David Rheinsberg
2d6d902428 tree: pep8 + linter fixes
For some reasons I forgot to fix those in the previous runs. Fix a
linter and pep8 warning.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-23 12:08:10 +02:00
Simon Steinbeiss
f3033dd579 packit: Enable Bodhi updates for unstable Fedoras
As we don't only want to get Bodhi updates for the stable releases, but
also the ones still in development, we need to use 'fedora-branched'.

See https://packit.dev/docs/configuration/#aliases
2022-09-23 11:06:11 +02:00
Vit Mojzis
34c801e670 selinux: Update based on latest packaging guide
https://fedoraproject.org/wiki/SELinux/IndependentPolicy

- Add dependency on selinux-policy-targeted
- Move %selinux_relabel_pre to osbuild-selinux
- Start shipping osbuild interface file
- Exclude installed policy module file from RPM verification

Signed-off-by: Vit Mojzis <vmojzis@redhat.com>
2022-09-23 10:10:31 +02:00
Ondřej Budai
c824e18b45 packit: add epel-9 to copr_build
Currently, we build only in the CS9 chroot. It contains newer packages than
RHEL 9 has which causes the package built in CS9 chroot uninstallable
on RHEL 9 - selinux-policy is the usual suspect.

Let's enable builds in the EPEL 9 chroot (which is actually EPEL 9 on RHEL 9)
in order to have a repository for users that use RHEL 9.

This was requested in the Image Builder chat room.
2022-09-15 09:48:35 +02:00
David Rheinsberg
054bba5228 stages/greenboot: avoid new pylint suppressions
The `consider-using-f-string` suppression is new, thus old pylint will
complain about unknown directives. If we start ignoring unknown
directives for this reasons, we will no longer get warned about
misspelled directives. Hence, lets avoid this for now and just use an
f-string.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-14 12:26:37 +02:00
schutzbot
a298254dce Post release version bump
[skip ci]
2022-09-14 08:18:10 +00:00
Thomas Lavocat
77153698c5 ci: remove the composer image test
This test that compiles and compare image-info from manifests is
redundant with the tests from manifest-db.
2022-09-14 10:00:43 +02:00
David Rheinsberg
a65cf82b5e osbuild-mpp: fix pylint warnings
Fix a bunch of pylint warnings about coding-style. Nothing really major.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-13 17:41:27 +02:00
David Rheinsberg
73ad1a3eac osbuild-mpp: avoid symbol shadowing
Avoid using `os` or `input` as symbol names, since they shadow
previously defined symbols.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-13 17:41:27 +02:00
David Rheinsberg
0f49ed9f98 osbuild-mpp: implement virtual functions
Make sure to correctly implement virtual functions and raise
NotImplementedError, as suggested by the standard library. Pylint will
otherwise complain about unavailable functions being accessed.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-13 17:41:27 +02:00
David Rheinsberg
0471c646a8 osbuild-mpp: fix YAMLError.problem accessors
The `problem` member is only defined if a problem-mark is present. If
the mark is not set, we get no details on the error whatsoever and have
to just print a generic error message.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-13 17:41:27 +02:00
David Rheinsberg
8c6049f197 osbuild-mpp: fix PartitionTable.__getitem__() bail-out
The __getitem__() function incorrectly returns `None` if the
partition-type is not found. Make sure to always return IndexError if
the requested value is outside the supported range.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-13 17:41:27 +02:00
David Rheinsberg
8c367dfa6f tree: fix pylint warnings
This fixes pylint warnings on our modules that are currently not part of
CI-pylint. The fixes should all be straightforward.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-13 16:21:14 +02:00