Commit graph

2189 commits

Author SHA1 Message Date
Christian Kellner
76d6bfa4e8 test/objectstore: use helper to assert contents 2022-11-16 11:09:44 +01:00
Christian Kellner
3e8d2c21dc pipeline: opt out of copy-on-write for objects
When committing an object to the store, clone it if the current
stage is not the latests stage, i.e. `todo` has still entries.
This is the second step of the removal of copy-on-write support
in `Object`.
2022-11-16 11:09:44 +01:00
Christian Kellner
daa2e1c3bb objectstore: option to clone object on commit
Add a new `clone` parameter to the `commit` method on `ObjectStore`
that when used will clone the object to the store instead of using
the `store_tree` method which moves the object and resets it. This
is the first step of removing copy-on-write support from `Object`.
2022-11-16 11:09:44 +01:00
Christian Kellner
1762048c1f objectstore: add clone method for object
Right now this is basically a clone(!) of `export` but this will
change in the future when we change the layout of how objects
are stored.
2022-11-16 11:09:44 +01:00
Christian Kellner
c3c06a1ebd objectstore: small comment fix
Just fix a typo, and start the comment with a capital letter.
2022-11-16 11:09:44 +01:00
schutzbot
0b870d88f0 schutzfile: Update snapshots to 20221115 2022-11-15 16:26:56 +01:00
Christian Kellner
a2e212bb26 stages/containers.storage: ability to specify a base file
In newer version of the container storage package the config file
moved from `/etc/containers` to `/usr/containers/`. The later is
not marked as config, so we don't want to change it.
The current containers code[1] will read _either_ a file in `usr`
or in `etc` depending on the existence of the latter. This we can
not just write the keys we want into a file in `/etc/containers`
without losing all other defaults set in the config file.
A new option `filebase` is therefore added, that when given will
be read and form the bases of the configuration data. Then data
from the target file (given via `filename`) will be merged into
and finally the actual configuration will be applied on top.

[1] 232bf398bd/types/options.go (L85)
2022-11-15 11:34:15 +01:00
Christian Kellner
b0ffe6c2b3 stages/containers.storage: allow setting options of empty sections
If the existing config does not have a section that is being set
by the stage it currently would crash since we assume that it
exists. Default to an empty dict in that case.
2022-11-15 11:34:15 +01:00
Christian Kellner
ed67839183 test/data: add rootfs to UKI
Create a squashfs image from the root file system and append it
to the existing initram fs. Passing `root=live:/rootfs.img`
is enough to use the existing dmsquash machiner in dracut.
2022-11-14 20:10:59 +01:00
Christian Kellner
053f027c67 test/data: add UKI test manifest
Example UKI manifest.
2022-11-14 20:10:59 +01:00
Christian Kellner
bca3fe62d5 test/data: add binutils to fedora-build-v2
Add binutils for `objcopy` needed to build UKIs.
2022-11-14 20:10:59 +01:00
Christian Kellner
9b9c989d7b stages: add stage to build unified kernel images
Add new `org.osbuild.uki` stage to build unified kernel images.
2022-11-14 20:10:59 +01:00
Christian Kellner
ecb24a8eb7 util: add module to parse PE32+ files
Add an new module with utility functions to inspect PE32+ files,
mainly listing the sections and their addresses and sizes.
Include a simple test to check that we can successfully parse the
EFI stub contained in systemd (systemd-udev package).
2022-11-14 20:10:59 +01:00
Antonio Murdaca
f34bee944b grub2.iso: add timeout option
add the ability to configure the grub timeout. Today it defealts to 1
minute and we can't configure it from osbuild-composer either (the
simplified-installer is mainly using this afaict and 1 min is just too
much).

Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2022-11-11 17:22:14 +01:00
Thomas Lavocat
9d1eb6cecc ci: add tags to AWS instances
The new terraform runners support an extended lists of tags to get a
greater precision on the stats we can extract from AWS cost center.
2022-11-10 20:09:26 +01:00
Antonio Murdaca
ea8261cad6 ignition: fix ignition_network_kcmdline
Signed-off-by: Antonio Murdaca <runcom@linux.com>
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2022-11-09 12:02:07 +01:00
schutzbot
70bed2e984 Post release version bump
[skip ci]
2022-11-09 08:19:42 +00:00
Ygal Blum
6cdc27366b osbuild-mpp: recognize manifest without mediaType and with manifests fields as a list
According to the OCI Image Index specification the mediaType field is not mandatory
Assume that it is a list if mediaType is not set while manifests field is

Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
2022-11-08 11:25:25 +01:00
Christian Kellner
7729e6225e stages: add new cpio.out stage
New stage that will assemble a tree into a CPIO archive with the
help of the `cpio` command.
2022-11-03 17:26:43 +01:00
Christian Kellner
51b973f54d ci: update to containers/privdocker@552e30c
The privdocker action got update to node 16, since node 12 is
deprecated and support for it will be remove by summer 2023[1].

[1] https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/
2022-11-03 12:59:52 +01:00
Simon de Vlieger
3421826d2f osbuild-dev: a new tool to help with manifests 2022-11-02 17:55:13 +01:00
Christian Kellner
58bafaad98 ci: upgrade to actions/checkout@v3
This uses the new node 16 version (see [1]), which is the future proof
version, since node 12 will be deprecated by summer 2023 (see [2]).

[1] upstream commits:
8f9e05e482
a12a3943b4

[2] https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/
2022-11-02 00:14:57 +01:00
schutzbot
1ecc784386 schutzfile: Update snapshots to 20221028 2022-10-29 17:04:55 +02:00
Jan Macku
5a904f4c39 ci(lint): add shell linter - Differential ShellCheck
It performs differential ShellCheck scans and report results directly in pull request.

documentation: https://github.com/redhat-plumbers-in-action/differential-shellcheck

Co-authored-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-10-26 16:45:09 +02:00
schutzbot
9f92b81056 schutzfile: Update snapshots to 20221025 2022-10-26 15:19:17 +02:00
David Rheinsberg
8464e80143 stages/ostree.preptree: fix broken link
Fix the broken "adapting-existing" link in the ostree.preptree
introdutory comment.
2022-10-26 15:04:53 +02:00
Colin Walters
6a2c78b85f stages/ostree.preptree: Link to rpm-ostree code
This code was clearly influenced/copied from rpm-ostree,
since it's now duplicated let's cross-link to help ensure that
if someone wants to change this they hopefully consider changing
rpm-ostree too.
2022-10-26 15:04:53 +02:00
Irene Diez
24b116213c stages/firewall: change zones/sources handling
Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-26 12:07:21 +02:00
Antonio Murdaca
833f2da3f9 Extend firewall stage to add sources
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-10-26 12:07:21 +02:00
schutzbot
845f58631c Post release version bump
[skip ci]
2022-10-26 08:18:16 +00:00
Christian Kellner
976fbe178a stages/keymap: add font option
Allows the configuration of the console font. systemd-firstboot
does not allow to do it, so we need to manually tweak the file.
2022-10-25 11:20:11 +02:00
Christian Kellner
30e20050c6 stages/logind-systemd: add ReserveVT option
Add option to reserve a VT exclusively for autovt@.service activation.
See logind.conf(5) for more details.
2022-10-22 14:58:44 +02:00
Irene Diez
145a0cc273 stages: add org.osbuild.systemd-journald stage
This stage allows to configure a systemd-journald
config drop-in file located in
`/usr/lib/systemd/journald.conf.d`.

Currently it supports the following options for
the 'Journal' section: Storage, Compress, SplitMode,
MaxFileSec, MaxRetentionSec, SyncIntervalSec, Audit.

Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-21 09:04:51 +02:00
SchutzBot
dd830a2796 schutzfile: update manifest-db ref 2022-10-20 2022-10-20 11:07:01 +02:00
Irene Diez
b96f715c6e test: add README.md on how to make tests for stages
Adds a README.md which explains what do you need
to do to add a new test for an stage

Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-18 19:02:34 +02:00
Ondřej Budai
000ac81777 stages/rpm: make the fake machine-id newline-terminated
From machine-id(5): machine ID is a single *newline-terminated*, hexadecimal,
32-character, lowercase ID.

This commit fixes that by adding a newline.

This broke us on Fedora 38:
This pull request added `set -e` to kernel-install:

https://github.com/systemd/systemd/pull/23881

This caused F38 pipeline failing because the script also runs
`read -r MACHINE_ID </etc/machine-id`. `read` returns 1 if the input doesn't
contain a newline, thus after the addition of `set -e`, the script started
crashing with our bad machine-id. This caused crashing of the whole image build
in the grub2 stage which requires kernel-install to work (because kernel-install
is responsible for putting vmlinuz into /boot).
2022-10-18 14:40:24 +02:00
Jan Macku
9d26789b78 ci(packit): Use YAML Merge Key syntax to reduce duplicity 2022-10-18 13:04:03 +02:00
Jan Macku
f894d64f62 ci: Update .packit.yaml - update/remove deprecated options 2022-10-18 13:04:03 +02:00
Jan Macku
9e2add32a0 ci: Remove unused .travis.yml config 2022-10-18 13:04:03 +02:00
Sanne Raymaekers
fb3ac490d5 schutzbot/mockbuild: stop running mock as root
Running mock as root is broken, this commit can be reverted once
rhbz#2135203 is fixed.
2022-10-17 18:36:02 +02:00
Sanne Raymaekers
925ca9b41e sources/ostree: set contenturl when pulling from remote
If a contenturl is specified, the url is used only for metadata. This is
useful when the actual content is hosted separately.
2022-10-14 12:04:54 +02:00
Sanne Raymaekers
cc9d05c201 sources/ostree: fix mTLS certs remote option
These options take the format --set "KEY_VALUE". Also fix the string
formatting.
2022-10-14 12:04:54 +02:00
Jakub Rusz
c638cbcb83 rpmbuild: build rpms on RHEL 8.8 and 9.2 nightly 2022-10-14 10:24:33 +02:00
schutzbot
6523fd8d41 Post release version bump
[skip ci]
2022-10-12 08:19:37 +00:00
Sanne Raymaekers
fcaad0462a sources/ostree: pull from remote using rhsm mTLS certs
The consumer certs are used to uniquely identify a system against
candlepin. These consumer certs can be used to identify the system when
pulling from RH controlled ostree repositories.
2022-10-11 16:49:45 +02:00
Christian Kellner
1c81d1e966 monitor/log: show build root and runner
In the `LogMonitor`, print the build root and its runner for each
pipelie. That could help identifying issues with runners.
2022-10-11 12:49:16 +02:00
Christian Kellner
9480009189 runners: remove aliased runner
Now that we can automatically detect the best available runner for
a requested one, we don't need to maintain the link farm with the
explicit mapping anymore.
2022-10-11 12:49:16 +02:00
Christian Kellner
5bdc8d030c osbuild: auto-detect best available runner
Use the new `Index.detect_runner` method that will give us the best
available runner for a requested one. To do so a new `pipeline.Runner`
class is introduced that stores the `meta.RunnerInfo` class for the
specific runner and the original name that was requested.
In the manifest loading and describing functions of the formats, use
`Index.detect_runner` to get the `RunnerInfo` for a requested runner
and then wrap it in a `pipeline.Runner` object, which is then passed
to the `Manifest.add_pipeline` method.
See also commit "meta: ability to auto-detect runner".
Adjust all test.
2022-10-11 12:49:16 +02:00
Christian Kellner
0554ac652b test/fmt/v1: use existing runner in manifests
Instead of using a non-existing runner `org.osbuild.test` use an
existing one `org.osbuild.linux`. This prepares the switch to
using runner auto-detection, which will rely on existing runners.
2022-10-11 12:49:16 +02:00
Christian Kellner
77e717f4f4 buildroot: explicitly bind mount runner
Instead of relying on the assumption that the specific runner will
be in `/run/osbuild/lib/runners/` we now bind-mount the runner at a
specific well known path and execute it from there.
2022-10-11 12:49:16 +02:00