Commit graph

2198 commits

Author SHA1 Message Date
Christian Kellner
881b2bb5c9 mounts: convert paths to str during serialization
When creating the JSON data, call `os.fspath` on all paths, like
`root` and `devices.tree` to ensure they are strings; this allows
for tree to be an object that conforms to `os.PathLike`.
2022-11-21 17:26:53 +01:00
Christian Kellner
0a29694593 devices: convert tree to str during serialization
When creating the JSON data, call `os.fspath` on `tree` to ensure
it is a string; this allows for tree to be an object that conforms
to `os.PathLike`.
2022-11-21 17:26:53 +01:00
Christian Kellner
48a4419705 test/data: persist the journal for ostree images
Include the new journald config stage to configure journald to
persist the journal. This is needed since we don't create the
`/var/log/journal` directory that journald uses to switch the
default to persistent storage. But instead of creating that
directory, we explicitly configure journald via the new stage.
This is also what Fedora CoreOS does.
2022-11-16 17:51:56 +01:00
Christian Kellner
638d5b834d test/data: add 'rw' kernel flag to ostree image
A recent commit (8a7b6d3) fixed the ostree config stage and thus
we actually properly set the readonly flag for the deployment.
This broke the image since we did not specify the `rw` kernel flag
and as a consequence the ostree mounting code in the initrd broke.
2022-11-16 17:51:56 +01:00
Christian Kellner
74e1dea1f7 objectstore: remove context manager from Object
As `ObjectStore.object` is currently not used via a context
manager anywhere in the source, remove the code.
2022-11-16 11:09:44 +01:00
Christian Kellner
28b8252a04 objectstore: implicit clone based on object ids
If the object's id does not match with the one supplied for the
commit, we create a clone. Otherwise we store the tree.
The code path is arranged in a way that we always go through
`Object.store_tree` so we always call `Object.finalize` as a
prepration for the future, where we might actually do something
meaningful in the finalizer, like reset the *times or count the
tree size.
2022-11-16 11:09:44 +01:00
Christian Kellner
5346025031 objectstore: remove copy on write from object
Remove copy-on-write support from `objectstore.Object`. The main
reason for introducing copy-on-write was to save an additional
copy in the non DAG-pipeline model[1]. With the introduction of
the latter and the explicit `--export` option, we can achieve the
same result without the complexity of copy-on-write semantics.

[1] See commit 39213b7, part of 3b7c87d5..42a365d1 changeset.
2022-11-16 11:09:44 +01:00
Christian Kellner
afc82ee465 test/objectstore: always setup a fresh store
There is little use in sharing the store between test, quite to
opposite: all tests expect a clean store and some currently set
that up themselves. Create a fresh store for each test.
2022-11-16 11:09:44 +01:00
Christian Kellner
0a41742d27 test/objectstore: small check for clone on commit
Add a small test that checks we indeed copied the object by
verifying a file in the store has the same content after
committing but a different inode.
2022-11-16 11:09:44 +01:00
Christian Kellner
76d6bfa4e8 test/objectstore: use helper to assert contents 2022-11-16 11:09:44 +01:00
Christian Kellner
3e8d2c21dc pipeline: opt out of copy-on-write for objects
When committing an object to the store, clone it if the current
stage is not the latests stage, i.e. `todo` has still entries.
This is the second step of the removal of copy-on-write support
in `Object`.
2022-11-16 11:09:44 +01:00
Christian Kellner
daa2e1c3bb objectstore: option to clone object on commit
Add a new `clone` parameter to the `commit` method on `ObjectStore`
that when used will clone the object to the store instead of using
the `store_tree` method which moves the object and resets it. This
is the first step of removing copy-on-write support from `Object`.
2022-11-16 11:09:44 +01:00
Christian Kellner
1762048c1f objectstore: add clone method for object
Right now this is basically a clone(!) of `export` but this will
change in the future when we change the layout of how objects
are stored.
2022-11-16 11:09:44 +01:00
Christian Kellner
c3c06a1ebd objectstore: small comment fix
Just fix a typo, and start the comment with a capital letter.
2022-11-16 11:09:44 +01:00
schutzbot
0b870d88f0 schutzfile: Update snapshots to 20221115 2022-11-15 16:26:56 +01:00
Christian Kellner
a2e212bb26 stages/containers.storage: ability to specify a base file
In newer version of the container storage package the config file
moved from `/etc/containers` to `/usr/containers/`. The later is
not marked as config, so we don't want to change it.
The current containers code[1] will read _either_ a file in `usr`
or in `etc` depending on the existence of the latter. This we can
not just write the keys we want into a file in `/etc/containers`
without losing all other defaults set in the config file.
A new option `filebase` is therefore added, that when given will
be read and form the bases of the configuration data. Then data
from the target file (given via `filename`) will be merged into
and finally the actual configuration will be applied on top.

[1] 232bf398bd/types/options.go (L85)
2022-11-15 11:34:15 +01:00
Christian Kellner
b0ffe6c2b3 stages/containers.storage: allow setting options of empty sections
If the existing config does not have a section that is being set
by the stage it currently would crash since we assume that it
exists. Default to an empty dict in that case.
2022-11-15 11:34:15 +01:00
Christian Kellner
ed67839183 test/data: add rootfs to UKI
Create a squashfs image from the root file system and append it
to the existing initram fs. Passing `root=live:/rootfs.img`
is enough to use the existing dmsquash machiner in dracut.
2022-11-14 20:10:59 +01:00
Christian Kellner
053f027c67 test/data: add UKI test manifest
Example UKI manifest.
2022-11-14 20:10:59 +01:00
Christian Kellner
bca3fe62d5 test/data: add binutils to fedora-build-v2
Add binutils for `objcopy` needed to build UKIs.
2022-11-14 20:10:59 +01:00
Christian Kellner
9b9c989d7b stages: add stage to build unified kernel images
Add new `org.osbuild.uki` stage to build unified kernel images.
2022-11-14 20:10:59 +01:00
Christian Kellner
ecb24a8eb7 util: add module to parse PE32+ files
Add an new module with utility functions to inspect PE32+ files,
mainly listing the sections and their addresses and sizes.
Include a simple test to check that we can successfully parse the
EFI stub contained in systemd (systemd-udev package).
2022-11-14 20:10:59 +01:00
Antonio Murdaca
f34bee944b grub2.iso: add timeout option
add the ability to configure the grub timeout. Today it defealts to 1
minute and we can't configure it from osbuild-composer either (the
simplified-installer is mainly using this afaict and 1 min is just too
much).

Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2022-11-11 17:22:14 +01:00
Thomas Lavocat
9d1eb6cecc ci: add tags to AWS instances
The new terraform runners support an extended lists of tags to get a
greater precision on the stats we can extract from AWS cost center.
2022-11-10 20:09:26 +01:00
Antonio Murdaca
ea8261cad6 ignition: fix ignition_network_kcmdline
Signed-off-by: Antonio Murdaca <runcom@linux.com>
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2022-11-09 12:02:07 +01:00
schutzbot
70bed2e984 Post release version bump
[skip ci]
2022-11-09 08:19:42 +00:00
Ygal Blum
6cdc27366b osbuild-mpp: recognize manifest without mediaType and with manifests fields as a list
According to the OCI Image Index specification the mediaType field is not mandatory
Assume that it is a list if mediaType is not set while manifests field is

Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
2022-11-08 11:25:25 +01:00
Christian Kellner
7729e6225e stages: add new cpio.out stage
New stage that will assemble a tree into a CPIO archive with the
help of the `cpio` command.
2022-11-03 17:26:43 +01:00
Christian Kellner
51b973f54d ci: update to containers/privdocker@552e30c
The privdocker action got update to node 16, since node 12 is
deprecated and support for it will be remove by summer 2023[1].

[1] https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/
2022-11-03 12:59:52 +01:00
Simon de Vlieger
3421826d2f osbuild-dev: a new tool to help with manifests 2022-11-02 17:55:13 +01:00
Christian Kellner
58bafaad98 ci: upgrade to actions/checkout@v3
This uses the new node 16 version (see [1]), which is the future proof
version, since node 12 will be deprecated by summer 2023 (see [2]).

[1] upstream commits:
8f9e05e482
a12a3943b4

[2] https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/
2022-11-02 00:14:57 +01:00
schutzbot
1ecc784386 schutzfile: Update snapshots to 20221028 2022-10-29 17:04:55 +02:00
Jan Macku
5a904f4c39 ci(lint): add shell linter - Differential ShellCheck
It performs differential ShellCheck scans and report results directly in pull request.

documentation: https://github.com/redhat-plumbers-in-action/differential-shellcheck

Co-authored-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-10-26 16:45:09 +02:00
schutzbot
9f92b81056 schutzfile: Update snapshots to 20221025 2022-10-26 15:19:17 +02:00
David Rheinsberg
8464e80143 stages/ostree.preptree: fix broken link
Fix the broken "adapting-existing" link in the ostree.preptree
introdutory comment.
2022-10-26 15:04:53 +02:00
Colin Walters
6a2c78b85f stages/ostree.preptree: Link to rpm-ostree code
This code was clearly influenced/copied from rpm-ostree,
since it's now duplicated let's cross-link to help ensure that
if someone wants to change this they hopefully consider changing
rpm-ostree too.
2022-10-26 15:04:53 +02:00
Irene Diez
24b116213c stages/firewall: change zones/sources handling
Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-26 12:07:21 +02:00
Antonio Murdaca
833f2da3f9 Extend firewall stage to add sources
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-10-26 12:07:21 +02:00
schutzbot
845f58631c Post release version bump
[skip ci]
2022-10-26 08:18:16 +00:00
Christian Kellner
976fbe178a stages/keymap: add font option
Allows the configuration of the console font. systemd-firstboot
does not allow to do it, so we need to manually tweak the file.
2022-10-25 11:20:11 +02:00
Christian Kellner
30e20050c6 stages/logind-systemd: add ReserveVT option
Add option to reserve a VT exclusively for autovt@.service activation.
See logind.conf(5) for more details.
2022-10-22 14:58:44 +02:00
Irene Diez
145a0cc273 stages: add org.osbuild.systemd-journald stage
This stage allows to configure a systemd-journald
config drop-in file located in
`/usr/lib/systemd/journald.conf.d`.

Currently it supports the following options for
the 'Journal' section: Storage, Compress, SplitMode,
MaxFileSec, MaxRetentionSec, SyncIntervalSec, Audit.

Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-21 09:04:51 +02:00
SchutzBot
dd830a2796 schutzfile: update manifest-db ref 2022-10-20 2022-10-20 11:07:01 +02:00
Irene Diez
b96f715c6e test: add README.md on how to make tests for stages
Adds a README.md which explains what do you need
to do to add a new test for an stage

Signed-off-by: Irene Diez <idiez@redhat.com>
2022-10-18 19:02:34 +02:00
Ondřej Budai
000ac81777 stages/rpm: make the fake machine-id newline-terminated
From machine-id(5): machine ID is a single *newline-terminated*, hexadecimal,
32-character, lowercase ID.

This commit fixes that by adding a newline.

This broke us on Fedora 38:
This pull request added `set -e` to kernel-install:

https://github.com/systemd/systemd/pull/23881

This caused F38 pipeline failing because the script also runs
`read -r MACHINE_ID </etc/machine-id`. `read` returns 1 if the input doesn't
contain a newline, thus after the addition of `set -e`, the script started
crashing with our bad machine-id. This caused crashing of the whole image build
in the grub2 stage which requires kernel-install to work (because kernel-install
is responsible for putting vmlinuz into /boot).
2022-10-18 14:40:24 +02:00
Jan Macku
9d26789b78 ci(packit): Use YAML Merge Key syntax to reduce duplicity 2022-10-18 13:04:03 +02:00
Jan Macku
f894d64f62 ci: Update .packit.yaml - update/remove deprecated options 2022-10-18 13:04:03 +02:00
Jan Macku
9e2add32a0 ci: Remove unused .travis.yml config 2022-10-18 13:04:03 +02:00
Sanne Raymaekers
fb3ac490d5 schutzbot/mockbuild: stop running mock as root
Running mock as root is broken, this commit can be reverted once
rhbz#2135203 is fixed.
2022-10-17 18:36:02 +02:00
Sanne Raymaekers
925ca9b41e sources/ostree: set contenturl when pulling from remote
If a contenturl is specified, the url is used only for metadata. This is
useful when the actual content is hosted separately.
2022-10-14 12:04:54 +02:00