Commit graph

2172 commits

Author SHA1 Message Date
Ygal Blum
6cdc27366b osbuild-mpp: recognize manifest without mediaType and with manifests fields as a list
According to the OCI Image Index specification the mediaType field is not mandatory
Assume that it is a list if mediaType is not set while manifests field is

Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
2022-11-08 11:25:25 +01:00
Christian Kellner
7729e6225e stages: add new cpio.out stage
New stage that will assemble a tree into a CPIO archive with the
help of the `cpio` command.
2022-11-03 17:26:43 +01:00
Christian Kellner
51b973f54d ci: update to containers/privdocker@552e30c
The privdocker action got update to node 16, since node 12 is
deprecated and support for it will be remove by summer 2023[1].

[1] https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/
2022-11-03 12:59:52 +01:00
Simon de Vlieger
3421826d2f osbuild-dev: a new tool to help with manifests 2022-11-02 17:55:13 +01:00
Christian Kellner
58bafaad98 ci: upgrade to actions/checkout@v3
This uses the new node 16 version (see [1]), which is the future proof
version, since node 12 will be deprecated by summer 2023 (see [2]).

[1] upstream commits:
8f9e05e482
a12a3943b4

[2] https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/
2022-11-02 00:14:57 +01:00
schutzbot
1ecc784386 schutzfile: Update snapshots to 20221028 2022-10-29 17:04:55 +02:00
Jan Macku
5a904f4c39 ci(lint): add shell linter - Differential ShellCheck
It performs differential ShellCheck scans and report results directly in pull request.

documentation: https://github.com/redhat-plumbers-in-action/differential-shellcheck

Co-authored-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-10-26 16:45:09 +02:00
schutzbot
9f92b81056 schutzfile: Update snapshots to 20221025 2022-10-26 15:19:17 +02:00
David Rheinsberg
8464e80143 stages/ostree.preptree: fix broken link
Fix the broken "adapting-existing" link in the ostree.preptree
introdutory comment.
2022-10-26 15:04:53 +02:00
Colin Walters
6a2c78b85f stages/ostree.preptree: Link to rpm-ostree code
This code was clearly influenced/copied from rpm-ostree,
since it's now duplicated let's cross-link to help ensure that
if someone wants to change this they hopefully consider changing
rpm-ostree too.
2022-10-26 15:04:53 +02:00
Irene Diez
24b116213c stages/firewall: change zones/sources handling
Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-26 12:07:21 +02:00
Antonio Murdaca
833f2da3f9 Extend firewall stage to add sources
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-10-26 12:07:21 +02:00
schutzbot
845f58631c Post release version bump
[skip ci]
2022-10-26 08:18:16 +00:00
Christian Kellner
976fbe178a stages/keymap: add font option
Allows the configuration of the console font. systemd-firstboot
does not allow to do it, so we need to manually tweak the file.
2022-10-25 11:20:11 +02:00
Christian Kellner
30e20050c6 stages/logind-systemd: add ReserveVT option
Add option to reserve a VT exclusively for autovt@.service activation.
See logind.conf(5) for more details.
2022-10-22 14:58:44 +02:00
Irene Diez
145a0cc273 stages: add org.osbuild.systemd-journald stage
This stage allows to configure a systemd-journald
config drop-in file located in
`/usr/lib/systemd/journald.conf.d`.

Currently it supports the following options for
the 'Journal' section: Storage, Compress, SplitMode,
MaxFileSec, MaxRetentionSec, SyncIntervalSec, Audit.

Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-21 09:04:51 +02:00
SchutzBot
dd830a2796 schutzfile: update manifest-db ref 2022-10-20 2022-10-20 11:07:01 +02:00
Irene Diez
b96f715c6e test: add README.md on how to make tests for stages
Adds a README.md which explains what do you need
to do to add a new test for an stage

Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-18 19:02:34 +02:00
Ondřej Budai
000ac81777 stages/rpm: make the fake machine-id newline-terminated
From machine-id(5): machine ID is a single *newline-terminated*, hexadecimal,
32-character, lowercase ID.

This commit fixes that by adding a newline.

This broke us on Fedora 38:
This pull request added `set -e` to kernel-install:

https://github.com/systemd/systemd/pull/23881

This caused F38 pipeline failing because the script also runs
`read -r MACHINE_ID </etc/machine-id`. `read` returns 1 if the input doesn't
contain a newline, thus after the addition of `set -e`, the script started
crashing with our bad machine-id. This caused crashing of the whole image build
in the grub2 stage which requires kernel-install to work (because kernel-install
is responsible for putting vmlinuz into /boot).
2022-10-18 14:40:24 +02:00
Jan Macku
9d26789b78 ci(packit): Use YAML Merge Key syntax to reduce duplicity 2022-10-18 13:04:03 +02:00
Jan Macku
f894d64f62 ci: Update .packit.yaml - update/remove deprecated options 2022-10-18 13:04:03 +02:00
Jan Macku
9e2add32a0 ci: Remove unused .travis.yml config 2022-10-18 13:04:03 +02:00
Sanne Raymaekers
fb3ac490d5 schutzbot/mockbuild: stop running mock as root
Running mock as root is broken, this commit can be reverted once
rhbz#2135203 is fixed.
2022-10-17 18:36:02 +02:00
Sanne Raymaekers
925ca9b41e sources/ostree: set contenturl when pulling from remote
If a contenturl is specified, the url is used only for metadata. This is
useful when the actual content is hosted separately.
2022-10-14 12:04:54 +02:00
Sanne Raymaekers
cc9d05c201 sources/ostree: fix mTLS certs remote option
These options take the format --set "KEY_VALUE". Also fix the string
formatting.
2022-10-14 12:04:54 +02:00
Jakub Rusz
c638cbcb83 rpmbuild: build rpms on RHEL 8.8 and 9.2 nightly 2022-10-14 10:24:33 +02:00
schutzbot
6523fd8d41 Post release version bump
[skip ci]
2022-10-12 08:19:37 +00:00
Sanne Raymaekers
fcaad0462a sources/ostree: pull from remote using rhsm mTLS certs
The consumer certs are used to uniquely identify a system against
candlepin. These consumer certs can be used to identify the system when
pulling from RH controlled ostree repositories.
2022-10-11 16:49:45 +02:00
Christian Kellner
1c81d1e966 monitor/log: show build root and runner
In the `LogMonitor`, print the build root and its runner for each
pipelie. That could help identifying issues with runners.
2022-10-11 12:49:16 +02:00
Christian Kellner
9480009189 runners: remove aliased runner
Now that we can automatically detect the best available runner for
a requested one, we don't need to maintain the link farm with the
explicit mapping anymore.
2022-10-11 12:49:16 +02:00
Christian Kellner
5bdc8d030c osbuild: auto-detect best available runner
Use the new `Index.detect_runner` method that will give us the best
available runner for a requested one. To do so a new `pipeline.Runner`
class is introduced that stores the `meta.RunnerInfo` class for the
specific runner and the original name that was requested.
In the manifest loading and describing functions of the formats, use
`Index.detect_runner` to get the `RunnerInfo` for a requested runner
and then wrap it in a `pipeline.Runner` object, which is then passed
to the `Manifest.add_pipeline` method.
See also commit "meta: ability to auto-detect runner".
Adjust all test.
2022-10-11 12:49:16 +02:00
Christian Kellner
0554ac652b test/fmt/v1: use existing runner in manifests
Instead of using a non-existing runner `org.osbuild.test` use an
existing one `org.osbuild.linux`. This prepares the switch to
using runner auto-detection, which will rely on existing runners.
2022-10-11 12:49:16 +02:00
Christian Kellner
77e717f4f4 buildroot: explicitly bind mount runner
Instead of relying on the assumption that the specific runner will
be in `/run/osbuild/lib/runners/` we now bind-mount the runner at a
specific well known path and execute it from there.
2022-10-11 12:49:16 +02:00
Christian Kellner
7a8b3d3c6f meta: ensure we have an absolute path for Index
Ensure that we always have an absolute path so that all paths
in `ModuleInfo` and `RunnerInfo` are absolute as well.
2022-10-11 12:49:16 +02:00
Christian Kellner
e923f559ab meta: cache host runner
The host runner does not change and we can thus avoid looking it up
over and over again.
2022-10-11 12:49:16 +02:00
Christian Kellner
ebf967ad1f pipeline: reformat line for readability
The line got too long.
2022-10-11 12:49:16 +02:00
Christian Kellner
7c399f15df meta: include path in RunnerInfo
This is so that once we have a `RunnerInfo` we can directly run
it without needed a back-reference to the index.
2022-10-11 12:49:16 +02:00
Christian Kellner
683a8cbfa7 meta: cache list of runners
Instead of enumerating all existing runners -- doing i/o -- we
cache the list at the `Index` level.
2022-10-11 12:49:16 +02:00
Christian Kellner
c755068bd2 meta: new host runner detection logic
This is a port of `pipeline.detect_host_runner` in combination
with the new runner detection logic.
2022-10-11 12:49:16 +02:00
Christian Kellner
ec1c5bb37c test: checks for runner detection
Add a test suite for the runner detection logic.
2022-10-11 12:49:16 +02:00
Christian Kellner
49dc76c434 test: add new test suite for 'meta' module
Move the checks for `meta.Schema` from `test_osbuild.py` into the new
test suite, converting it to use pytest in the process.
2022-10-11 12:49:16 +02:00
Christian Kellner
6695294a29 meta: ability to auto-detect runner
The way that runners were designed is the following: For each distro
we have a specific runner. In case a new version of the distro can
use the previous runner, we just create a symlink. In case a new
distro version needs adjustments, the runner is copied and adjusted.
This is a very clean and obvious design. There is one big drawback:
For each new distribution a symlink must be created before it can be
used. For Fedora that should ideally happen when it is branched; and
this will, ipso facto, always be a symlink since at the time of the
branching the new distro is the old distro. But at this very moment
osbuild will be broken since it does not contain the new runner; the
only way to prevent this is to create the corresponding new runner
before the distro is branched, where it then must be a symlink too.
This very much suggest that instead of the explicit symlink, which
does not /that/ much clarity, the existing "old" runner should just
work for the new distribution. This commit implements the logic to
do just that: all existing runners are parsed into a distro and
version tuple and then, given a specific requested distro, the best
matching one is return.
2022-10-11 12:49:16 +02:00
Antonio Murdaca
8a7b6d382d stages: fix ostree config stage
Currently we're using `str(value)` on a boolean which yields `True` or
`False` - turns out ostree reads these values case sensitive and despite
setting `True|False`, it doesn't just work.

From jlebon on slack:

> the syntax is readonly=true . it's case sensitive

Fix the above and also just remove the loop as, while it's handy, we'll
have to differentiate between options' values anyway and it's just two
options we support today.

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-09-28 21:30:49 +02:00
schutzbot
14e3cfc860 Post release version bump
[skip ci]
2022-09-28 08:21:46 +00:00
Thomas Lavocat
bc06a037ee manifest-db: propage the osbuild SHA on manifest-db
Add a bot to update the SHA dependency to OSBuild on manifest-db
automatically.

The OSBuild SHA will be changed only if the manifest-db SHA was updated
on the last commit landed on main. This way this ensure that CI ran
successfully for this exact version of OSBuild against manifest-db's
actual state. Therefore it is safe to assume that we can upgrade the
manifest dependency without running the whole CI again.
2022-09-23 14:18:38 +02:00
David Rheinsberg
c9ac6d5ee6 test/src: improve file enumeration
Improve the file-enumeration to first check for all files ending in
`*.py`, and for everything else run `/bin/file` on it, and check for
python mime-types.

Note that we expect the caller to open all those files anyway, and thus
this should not any significant overhead, unless we end up with lots of
non-python files in the repository (which is highly unlikely). But even
then, `/bin/file` only reads the first few bytes of a file, and will
default if those are not sufficient to detect the file type. Hence, not
much overhead is expected by this.

This change will now start adding scripts in ./tools/ to the linter,
including osbuild-mpp.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-23 12:08:10 +02:00
David Rheinsberg
1bdf038d36 ci: update container and actions
Update the osbuild-ci container and privdocker action to the most recent
builds.

This changes the ostree-image-tests since the Fedora update pulled in
util-linux/fdisk changes that align partition sizes. Hence, the ostree
tests need to be changed to have aligned partition sizes as well. For
more information, see:

    commit 921c7da55ec78350e4067b3fd6b7de6f299106ee
    Author: Karel Zak <kzak@redhat.com>
    Date:   Thu Jan 27 10:50:45 2022 +0100

        libfdisk: (gpt) align size of partition by default

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-23 12:08:10 +02:00
David Rheinsberg
4b09088661 test/isort: apply diff to full tree
Apply the isort modifications to the entire source tree, not just the
selected python files of test-src.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-23 12:08:10 +02:00
David Rheinsberg
5aaa4e602e test/isort: print diff
Make isort print the diff so we can actually fix the problems. This
might be overly verbose when introducing isort, but for new changes it
is very nice to see the actual code it complains about.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-23 12:08:10 +02:00
David Rheinsberg
2d6d902428 tree: pep8 + linter fixes
For some reasons I forgot to fix those in the previous runs. Fix a
linter and pep8 warning.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-09-23 12:08:10 +02:00