Commit graph

2177 commits

Author SHA1 Message Date
Christian Kellner
ecb24a8eb7 util: add module to parse PE32+ files
Add an new module with utility functions to inspect PE32+ files,
mainly listing the sections and their addresses and sizes.
Include a simple test to check that we can successfully parse the
EFI stub contained in systemd (systemd-udev package).
2022-11-14 20:10:59 +01:00
Antonio Murdaca
f34bee944b grub2.iso: add timeout option
add the ability to configure the grub timeout. Today it defealts to 1
minute and we can't configure it from osbuild-composer either (the
simplified-installer is mainly using this afaict and 1 min is just too
much).

Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2022-11-11 17:22:14 +01:00
Thomas Lavocat
9d1eb6cecc ci: add tags to AWS instances
The new terraform runners support an extended lists of tags to get a
greater precision on the stats we can extract from AWS cost center.
2022-11-10 20:09:26 +01:00
Antonio Murdaca
ea8261cad6 ignition: fix ignition_network_kcmdline
Signed-off-by: Antonio Murdaca <runcom@linux.com>
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2022-11-09 12:02:07 +01:00
schutzbot
70bed2e984 Post release version bump
[skip ci]
2022-11-09 08:19:42 +00:00
Ygal Blum
6cdc27366b osbuild-mpp: recognize manifest without mediaType and with manifests fields as a list
According to the OCI Image Index specification the mediaType field is not mandatory
Assume that it is a list if mediaType is not set while manifests field is

Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
2022-11-08 11:25:25 +01:00
Christian Kellner
7729e6225e stages: add new cpio.out stage
New stage that will assemble a tree into a CPIO archive with the
help of the `cpio` command.
2022-11-03 17:26:43 +01:00
Christian Kellner
51b973f54d ci: update to containers/privdocker@552e30c
The privdocker action got update to node 16, since node 12 is
deprecated and support for it will be remove by summer 2023[1].

[1] https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/
2022-11-03 12:59:52 +01:00
Simon de Vlieger
3421826d2f osbuild-dev: a new tool to help with manifests 2022-11-02 17:55:13 +01:00
Christian Kellner
58bafaad98 ci: upgrade to actions/checkout@v3
This uses the new node 16 version (see [1]), which is the future proof
version, since node 12 will be deprecated by summer 2023 (see [2]).

[1] upstream commits:
8f9e05e482
a12a3943b4

[2] https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/
2022-11-02 00:14:57 +01:00
schutzbot
1ecc784386 schutzfile: Update snapshots to 20221028 2022-10-29 17:04:55 +02:00
Jan Macku
5a904f4c39 ci(lint): add shell linter - Differential ShellCheck
It performs differential ShellCheck scans and report results directly in pull request.

documentation: https://github.com/redhat-plumbers-in-action/differential-shellcheck

Co-authored-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-10-26 16:45:09 +02:00
schutzbot
9f92b81056 schutzfile: Update snapshots to 20221025 2022-10-26 15:19:17 +02:00
David Rheinsberg
8464e80143 stages/ostree.preptree: fix broken link
Fix the broken "adapting-existing" link in the ostree.preptree
introdutory comment.
2022-10-26 15:04:53 +02:00
Colin Walters
6a2c78b85f stages/ostree.preptree: Link to rpm-ostree code
This code was clearly influenced/copied from rpm-ostree,
since it's now duplicated let's cross-link to help ensure that
if someone wants to change this they hopefully consider changing
rpm-ostree too.
2022-10-26 15:04:53 +02:00
Irene Diez
24b116213c stages/firewall: change zones/sources handling
Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-26 12:07:21 +02:00
Antonio Murdaca
833f2da3f9 Extend firewall stage to add sources
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-10-26 12:07:21 +02:00
schutzbot
845f58631c Post release version bump
[skip ci]
2022-10-26 08:18:16 +00:00
Christian Kellner
976fbe178a stages/keymap: add font option
Allows the configuration of the console font. systemd-firstboot
does not allow to do it, so we need to manually tweak the file.
2022-10-25 11:20:11 +02:00
Christian Kellner
30e20050c6 stages/logind-systemd: add ReserveVT option
Add option to reserve a VT exclusively for autovt@.service activation.
See logind.conf(5) for more details.
2022-10-22 14:58:44 +02:00
Irene Diez
145a0cc273 stages: add org.osbuild.systemd-journald stage
This stage allows to configure a systemd-journald
config drop-in file located in
`/usr/lib/systemd/journald.conf.d`.

Currently it supports the following options for
the 'Journal' section: Storage, Compress, SplitMode,
MaxFileSec, MaxRetentionSec, SyncIntervalSec, Audit.

Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-21 09:04:51 +02:00
SchutzBot
dd830a2796 schutzfile: update manifest-db ref 2022-10-20 2022-10-20 11:07:01 +02:00
Irene Diez
b96f715c6e test: add README.md on how to make tests for stages
Adds a README.md which explains what do you need
to do to add a new test for an stage

Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-18 19:02:34 +02:00
Ondřej Budai
000ac81777 stages/rpm: make the fake machine-id newline-terminated
From machine-id(5): machine ID is a single *newline-terminated*, hexadecimal,
32-character, lowercase ID.

This commit fixes that by adding a newline.

This broke us on Fedora 38:
This pull request added `set -e` to kernel-install:

https://github.com/systemd/systemd/pull/23881

This caused F38 pipeline failing because the script also runs
`read -r MACHINE_ID </etc/machine-id`. `read` returns 1 if the input doesn't
contain a newline, thus after the addition of `set -e`, the script started
crashing with our bad machine-id. This caused crashing of the whole image build
in the grub2 stage which requires kernel-install to work (because kernel-install
is responsible for putting vmlinuz into /boot).
2022-10-18 14:40:24 +02:00
Jan Macku
9d26789b78 ci(packit): Use YAML Merge Key syntax to reduce duplicity 2022-10-18 13:04:03 +02:00
Jan Macku
f894d64f62 ci: Update .packit.yaml - update/remove deprecated options 2022-10-18 13:04:03 +02:00
Jan Macku
9e2add32a0 ci: Remove unused .travis.yml config 2022-10-18 13:04:03 +02:00
Sanne Raymaekers
fb3ac490d5 schutzbot/mockbuild: stop running mock as root
Running mock as root is broken, this commit can be reverted once
rhbz#2135203 is fixed.
2022-10-17 18:36:02 +02:00
Sanne Raymaekers
925ca9b41e sources/ostree: set contenturl when pulling from remote
If a contenturl is specified, the url is used only for metadata. This is
useful when the actual content is hosted separately.
2022-10-14 12:04:54 +02:00
Sanne Raymaekers
cc9d05c201 sources/ostree: fix mTLS certs remote option
These options take the format --set "KEY_VALUE". Also fix the string
formatting.
2022-10-14 12:04:54 +02:00
Jakub Rusz
c638cbcb83 rpmbuild: build rpms on RHEL 8.8 and 9.2 nightly 2022-10-14 10:24:33 +02:00
schutzbot
6523fd8d41 Post release version bump
[skip ci]
2022-10-12 08:19:37 +00:00
Sanne Raymaekers
fcaad0462a sources/ostree: pull from remote using rhsm mTLS certs
The consumer certs are used to uniquely identify a system against
candlepin. These consumer certs can be used to identify the system when
pulling from RH controlled ostree repositories.
2022-10-11 16:49:45 +02:00
Christian Kellner
1c81d1e966 monitor/log: show build root and runner
In the `LogMonitor`, print the build root and its runner for each
pipelie. That could help identifying issues with runners.
2022-10-11 12:49:16 +02:00
Christian Kellner
9480009189 runners: remove aliased runner
Now that we can automatically detect the best available runner for
a requested one, we don't need to maintain the link farm with the
explicit mapping anymore.
2022-10-11 12:49:16 +02:00
Christian Kellner
5bdc8d030c osbuild: auto-detect best available runner
Use the new `Index.detect_runner` method that will give us the best
available runner for a requested one. To do so a new `pipeline.Runner`
class is introduced that stores the `meta.RunnerInfo` class for the
specific runner and the original name that was requested.
In the manifest loading and describing functions of the formats, use
`Index.detect_runner` to get the `RunnerInfo` for a requested runner
and then wrap it in a `pipeline.Runner` object, which is then passed
to the `Manifest.add_pipeline` method.
See also commit "meta: ability to auto-detect runner".
Adjust all test.
2022-10-11 12:49:16 +02:00
Christian Kellner
0554ac652b test/fmt/v1: use existing runner in manifests
Instead of using a non-existing runner `org.osbuild.test` use an
existing one `org.osbuild.linux`. This prepares the switch to
using runner auto-detection, which will rely on existing runners.
2022-10-11 12:49:16 +02:00
Christian Kellner
77e717f4f4 buildroot: explicitly bind mount runner
Instead of relying on the assumption that the specific runner will
be in `/run/osbuild/lib/runners/` we now bind-mount the runner at a
specific well known path and execute it from there.
2022-10-11 12:49:16 +02:00
Christian Kellner
7a8b3d3c6f meta: ensure we have an absolute path for Index
Ensure that we always have an absolute path so that all paths
in `ModuleInfo` and `RunnerInfo` are absolute as well.
2022-10-11 12:49:16 +02:00
Christian Kellner
e923f559ab meta: cache host runner
The host runner does not change and we can thus avoid looking it up
over and over again.
2022-10-11 12:49:16 +02:00
Christian Kellner
ebf967ad1f pipeline: reformat line for readability
The line got too long.
2022-10-11 12:49:16 +02:00
Christian Kellner
7c399f15df meta: include path in RunnerInfo
This is so that once we have a `RunnerInfo` we can directly run
it without needed a back-reference to the index.
2022-10-11 12:49:16 +02:00
Christian Kellner
683a8cbfa7 meta: cache list of runners
Instead of enumerating all existing runners -- doing i/o -- we
cache the list at the `Index` level.
2022-10-11 12:49:16 +02:00
Christian Kellner
c755068bd2 meta: new host runner detection logic
This is a port of `pipeline.detect_host_runner` in combination
with the new runner detection logic.
2022-10-11 12:49:16 +02:00
Christian Kellner
ec1c5bb37c test: checks for runner detection
Add a test suite for the runner detection logic.
2022-10-11 12:49:16 +02:00
Christian Kellner
49dc76c434 test: add new test suite for 'meta' module
Move the checks for `meta.Schema` from `test_osbuild.py` into the new
test suite, converting it to use pytest in the process.
2022-10-11 12:49:16 +02:00
Christian Kellner
6695294a29 meta: ability to auto-detect runner
The way that runners were designed is the following: For each distro
we have a specific runner. In case a new version of the distro can
use the previous runner, we just create a symlink. In case a new
distro version needs adjustments, the runner is copied and adjusted.
This is a very clean and obvious design. There is one big drawback:
For each new distribution a symlink must be created before it can be
used. For Fedora that should ideally happen when it is branched; and
this will, ipso facto, always be a symlink since at the time of the
branching the new distro is the old distro. But at this very moment
osbuild will be broken since it does not contain the new runner; the
only way to prevent this is to create the corresponding new runner
before the distro is branched, where it then must be a symlink too.
This very much suggest that instead of the explicit symlink, which
does not /that/ much clarity, the existing "old" runner should just
work for the new distribution. This commit implements the logic to
do just that: all existing runners are parsed into a distro and
version tuple and then, given a specific requested distro, the best
matching one is return.
2022-10-11 12:49:16 +02:00
Antonio Murdaca
8a7b6d382d stages: fix ostree config stage
Currently we're using `str(value)` on a boolean which yields `True` or
`False` - turns out ostree reads these values case sensitive and despite
setting `True|False`, it doesn't just work.

From jlebon on slack:

> the syntax is readonly=true . it's case sensitive

Fix the above and also just remove the loop as, while it's handy, we'll
have to differentiate between options' values anyway and it's just two
options we support today.

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-09-28 21:30:49 +02:00
schutzbot
14e3cfc860 Post release version bump
[skip ci]
2022-09-28 08:21:46 +00:00
Thomas Lavocat
bc06a037ee manifest-db: propage the osbuild SHA on manifest-db
Add a bot to update the SHA dependency to OSBuild on manifest-db
automatically.

The OSBuild SHA will be changed only if the manifest-db SHA was updated
on the last commit landed on main. This way this ensure that CI ran
successfully for this exact version of OSBuild against manifest-db's
actual state. Therefore it is safe to assume that we can upgrade the
manifest dependency without running the whole CI again.
2022-09-23 14:18:38 +02:00