Commit graph

654 commits

Author SHA1 Message Date
Michael Vogt
0528ccc3f0 osbuild: add support to exclude_paths to setfiles()
This is needed because on a mounted `bootc` container `setfiles`
without excluding `/sysroot` will create many warnings like:
```
setfiles: conflicting specifications for /run/osbuild/tree/sysroot/ostree/repo/objects/00/0ef9ada2ee87792e8ba21afd65aa00d79a1253018832652b8694862fb80e84.file and /run/osbuild/tree/usr/lib/firmware/cirrus/cs35l41-dsp1-spk-prot-103c8b8f-r1.bin.xz, using system_u:object_r:lib_t:s0.
```
but simply excluding this dir fixes them.
2024-03-20 18:05:51 +01:00
Simon de Vlieger
a9f4024ded lint: fix up adjusted stages 2024-03-19 15:32:45 +01:00
Simon de Vlieger
bb58892571 stages: externalize schemas
Copy all schemas into the relevant `.meta.json` files instead of having
them contained inside the stages.
2024-03-19 15:32:45 +01:00
Simon de Vlieger
02b6d696ef stages: consistent docstrings
Make docstrings consistent across stages regarding whitespace and
summary line.
2024-03-19 15:32:45 +01:00
Michael Vogt
7431653882 test: add (trivial) test for ConditionPathIsDirectory 2024-03-19 10:19:26 +01:00
Sayan Paul
548d2a51ea stage/unit-create:new systemd unit config params
Added new systemd unit config : ConditionPathIsDirectory

Signed-off-by: Sayan Paul <paul.sayan@gmail.com>
2024-03-19 10:19:26 +01:00
Michael Vogt
155e24e4cb stage: user test that tests adduser is called the right way
The issue with mocking subprocess.call() is that we will not
catch if arguments are passed as eg. integers. So instead use
the `mock_command()` helper so that a real binary is called.
2024-03-18 20:37:21 +01:00
Michael Vogt
fd0167f130 test: return container_id in make_container
The current `make_container()` helper is a bit silly (which is
entirely my fault). It requires a container tag as input but all
tests end up creating a random number for this input. So instead
just remove the input and return the container_id from the podman
build in the contextmanager and use that.
2024-03-18 20:36:19 +01:00
Michael Vogt
df224fb32b stages: add support for kernel-options to bootc-install-to-fs
This commit adds support to use the `--kopt` arguments to
`bootc install to-filesystems`. It is not strictly needed right
now though.
2024-03-14 15:07:05 +01:00
Michael Vogt
626077ffc0 stages: add root ssh options to bootc.install-to-filesystem
To start using the `bootc.instal-to-filesystem` stage we need
to be able to test the generated images. This requires a login
on the disk. Traditionally we did that via the `users` stage.

But on a bootc system we do not want to modify the disk after
bootc did the install to avoid messing with things like the
selinux labels or (future) fsverity setups.

So for now we will use the `--root-ssh-authorized-keys` feature
to inject support for login.

So this commit adds a new option to the stage called
`root-ssh-authorized-keys`.
2024-03-14 15:07:05 +01:00
Michael Vogt
3e11b57815 meta: rename -meta.json to .meta.json
With `.meta.json` it's two "real" file extension. The `-meta.json`
feels less clear.
2024-03-14 13:37:57 +01:00
Michael Vogt
9b09ed9eb4 osbuild: allow json data to come from a {stage}-meta.json file
Instead of always parsing the python stage to load meta information
allow the user of a new `{stage}-meta.json` file. This is a first
step towards allowing modules to be written in a different language
than python. It also has some practical advantages:
- slightly faster as it avoids calling python to output the schemas
- easier to write schemas as this can be done in a real json editor
  now
- more extensible in a future where stages maybe binaries with
  shlib dependencies that are only satisfied in the buildroot
  but not on the host
2024-03-14 13:37:57 +01:00
Michael Vogt
439f7f08c7 test: add minimal test for expiredate for user stage 2024-03-12 07:38:31 +01:00
Andre Marianiello
f6ae58151e stages/org.osbuild.users: add expire date
Add the expiredate field to the users option in the org.osbuild.users
stage. This option maps to the --expiredate option of useradd/usermod,
which can be useful when creating users whose password must be changed
upon first login.
2024-03-12 07:38:31 +01:00
Simon de Vlieger
a895aa177c stages/kickstart: add noswap to autopart
Allow passing the `noswap` option to `autopart`.
2024-03-11 11:50:34 +01:00
Michael Vogt
d43194c0b3 test: add unit-{type,path} test coverage 2024-03-08 16:20:48 -08:00
Gianluca Zuccarelli
6c0973238d utils/mnt: fix mount permissions
This is a follow up to #1550 where we enabled a `rw` permissions mode,
which is not ideal since it would theoretically be possible to set both
`ro` and `rw` modes at the same time. This commit fixes the issue by only
allowing one option at a time.

Fixes #1588
2024-03-07 13:01:47 +00:00
Sayan Paul
2d2cdd8097 stage/systemd-unit-create:paremeterize file location
User can now customize the systemd unit load path.
User can select between etc or usr , defaults to 'usr'.
Also user can customize the scope of the service between global
or system, defaults to system.

Signed-off-by: Sayan Paul <paul.sayan@gmail.com>
2024-03-07 14:04:00 +02:00
Michael Vogt
9baca1fe90 stages: add small unit test for the gzip stage 2024-03-04 11:18:13 +01:00
Luke Yang
cfaabe618f stages/org.osbuild.gzip: add compression level option
Allow compression level to be specified instead of defaulting to 1. This is needed for CoreOS Assembler.
2024-03-04 11:18:13 +01:00
Michael Vogt
30f740ec9f stages: add test for coreos.platform generate_console_settings_file
And a tiny tweak to avoid an empty line at the start of the
`boot/grub2/console.cfg`
2024-02-28 10:37:01 +01:00
Michael Vogt
e9c31c035b stages: tweak process_platforms_json to be slightly shorter
With the test the helper can now be slightly simplified. Because
we only have two results it seems easier to just use them directly
than to store them in an intermediate result struct.
2024-02-28 10:37:01 +01:00
Michael Vogt
6cc7309890 stages: add unit test for coreos.platform process_platforms_json
Add unit test for `coreos.platform` process_platforms_json helper.
2024-02-28 10:37:01 +01:00
Luke Yang
7709b7caeb stages/org.osbuild.tar: add sparse option
Add a sparse option to the tar stage which make files in the archive sparse if they are sparse on disk. This is needed for CoreOS Assember.
2024-02-27 16:46:19 -05:00
Renata Ravanelli
0b7dcf3cf4 Add coreos.platforms stage
- Process all necessary operations related to CoreOS
platforms is crucial and specific to CoreOS. This step
is essential for CoreOS exclusively.

- Our approach to handling 'platforms.json' may change as we
advance with the OSBuild work. However, we don't have a clear
vision about how it will be in the future yet, particularly as
we also manage similar components within the osbuild composer
to configure cloud parameters. We probably will know better
when we start working with the cloud artifacts.

As a summary, let's add it know to unblock us, and if we find a
better approach in the future, we can always go back and remove it.

Signed-off-by: Renata Ravanelli <rravanel@redhat.com>
2024-02-26 16:02:28 +01:00
Renata Ravanelli
ddf4478ea9 util: Add bls module
- Add functions for appending kernel parameters to the
Boot Loader Specification (BLS) as needed.

Signed-off-by: Renata Ravanelli <rravanel@redhat.com>
2024-02-26 16:02:28 +01:00
Michael Vogt
322974695c stages(dracut): add small unittest for initoverlayfs
Small followup for https://github.com/osbuild/osbuild/pull/1586
that includes a basic check that the initoverlayfs option calls
the right binary.
2024-02-22 11:59:21 +01:00
Gianluca Zuccarelli
6b0d2d7a3c stages/ostree-container-deploy: mkdir /var/tmp
This stage crashes when using `containers-storage` if `/var/tmp`
doesn't exist, since the directory is required by `skopeo` for big
files. See:
4ad2f46269/storage/storage_src.go (L163)
2cb3c9d3c2/internal/tmpdir/tmpdir.go (L18)
2024-02-22 11:58:54 +01:00
Achilleas Koutsou
591593ea00 testutil: make_container context manager
Make make_container a context manager so we can reliably clean up
containers that were created in tests.
2024-02-21 17:55:37 +01:00
Michael Vogt
f7e4febb2c stage,test: extract make_container() test helper 2024-02-21 17:55:37 +01:00
Tomáš Hozza
92e75c375c Stages/grub2.legacy: make config options a subset of grub2 stage
While integrating the changes to grub2 config stage options in the
`osbuild/images` repository, I noticed that the code for grub2.legacy
is embedding the grub2 stage config options struct in its own config
options struct [1]. However, the grub2 stage options contain also the
`default` entry, while the grub2.legacy stage does not allow this option
in its stage options.

This change fixes that and ensures that the grub2 stage options config
section is a subset of the grub2.legacy stage options config section.

[1] d5644f3115/pkg/osbuild/grub2_legacy_stage.go (L85)

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-02-21 17:51:28 +01:00
Eric Curtin
134a4cca26 stages/dracut: Add functionality to build initoverlayfs with dracut
As an alternative to just initramfs. Upstream initoverlayfs project:

https://github.com/containers/initoverlayfs

Signed-off-by: Eric Curtin <ecurtin@redhat.com>
2024-02-21 11:53:48 +01:00
Tomáš Hozza
be1ba5be08 Stages/grub2: extend default config options
Extend the stage to allow explicitly configuring more grub2 default
config options. Preserve the defaults for options which were previously
hard-coded.

Extend the stage unit test to verify setting of the new grub2 default
config options.

Related to https://issues.redhat.com/browse/RHEL-19583

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-02-20 19:28:43 +01:00
Tomáš Hozza
84fc4c9903 Extend unit test for org.osbuild.grub2 stage
Test the content of the /etc/default/grub config file based on the
provided stage options.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-02-20 19:28:43 +01:00
Tomáš Hozza
7895424b78 Stages/grub2.legacy: extend default config options
Extend the stage to allow explicitly configuring more grub2 default
config options. Preserve the defaults for options which were previously
hard-coded.

Extend the stage unit test to verify setting of the new grub2 default
config options.

Related to https://issues.redhat.com/browse/RHEL-19583

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-02-20 19:28:43 +01:00
Tomáš Hozza
21ebed9305 Add unit test for org.osbuild.grub2.legacy
For now, test the content of /etc/default/grub configuration file.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-02-20 19:28:43 +01:00
Michael Vogt
43c83c01e6 stages(org.osbuild.systemd.unit.create): add small unit test
Small unit test for the new `org.systemd.unit.create` stage. It
will do basic schema validation checks and ensures that a simple
service definition is correctly translated to a systemd unit.
2024-02-19 15:12:55 +01:00
Evgeny Kolesnikov
46e1bc1e2b stages/oscap.remediation: Mount host's /proc and /dev earlier
We need /dev/urandom for the first pass as well.
2024-02-17 02:10:58 +01:00
Michael Vogt
90193d007f stages/bootc.install-to-fs: fix root mount handling
The bootc.install-to-filesystem code needs to run against the
root directory of a mounted tree, i.e. with /boot, /boot/efi
mounted. So change the code so that the target dir is the
the "mounts" paths. This is similar to how bootupd works and
the caller need to arrange the right mount setup there.
2024-02-16 18:05:50 +01:00
Sayan Paul
e858dc72c3 stage/systemd-unit:stage to create systemd unit file
Add systemd unit files in osbuild stage

This stage creates systemd unit file in `/usr/lib/systemd/system/`.
The stage accepts filename which must end with `.service`.Section
`Unit` , `Service` , `Install` accepts various parameters as per
the systemd documentaion.`systemd-analyze verify` is be performed
after the .service file is created to check for potential errors.

Signed-off-by: Sayan Paul <paul.sayan@gmail.com>
2024-02-16 11:55:11 +01:00
Michael Vogt
c3af3173eb stages(container-deploy): podman mount failure test/tweaks
This commit adds a test that ensures that the output of
podman mount is part of the error message. While writing
the test I also tweaked the code slightly so that we only
try to `podman umount` if we managed to successfully mount.
2024-02-12 17:41:21 -08:00
Michael Vogt
c0f670f035 test: add manifests for bootc.install-to-filesystem
This commit adds example manifests for a bootc.install-to-filesystem
system. It does not do more with them because running a full test
requires a working podman which is difficult to use inside our
GH runners that are already running inside docker.
2024-02-09 12:03:09 +01:00
Michael Vogt
af360b0d71 stages: add test for bootc.install-to-filesystem 2024-02-09 12:03:09 +01:00
Ondřej Budai
226b50eba5 stages: add new org.osbuild.bootc.install-to-filesystem
Support the `boot install to-filesystem` capability to install a
bootc image to a filesystem.
2024-02-09 12:03:09 +01:00
Luke Yang
619a64f0bd stages/org.osbuild.qemu: make qcow2 compression optional
Modify the stages/org.osbuild.qemu stage such that compression is
optional. This resolves the image size differences between an image
built with coreos assember vs osbuild, as discussed in:
https://github.com/coreos/fedora-coreos-tracker/issues/1653#issuecomment-1928342241
2024-02-07 20:31:10 -05:00
Dusty Mabe
e1cbf92673 ostree: add convenience function for using default OSTree deployment
This adds a `default: true` option for all cases where OSTree
information is specified in schemas and allows for the information
to be picked up from the filesystem.

This is a safe operation because when building disk images there is
no known case where having two deployments makes sense. In the case
there ever were a case then the osname, ref, and serial options still
exist and can be used.

Co-authored-by: Luke Yang <luyang@redhat.com>
Co-authored-by: Michael Vogt <michael.vogt@gmail.com>
2024-02-07 18:50:38 -05:00
Dusty Mabe
2021b915f1 stages(copy): allow copying from a mount or the tree
It seems like an artifical limitation to prevent copying from a mount
or from one location in the tree to another. It just so happens we need
this functionality when building CoreOS images because we want to take
a file embedded in the OSTree at a location and copy it to another
location in the tree. The particular example here is we want to copy
/usr/share/coreos-assembler/platforms.json -> /boot/coreos/platforms.json
See https://github.com/coreos/coreos-assembler/pull/3709

Allowing to copy from/to the tree we can now do something like:

```
- type: org.osbuild.copy
  options:
    paths:
      - from: tree:///usr/share/coreos-assembler/platforms.json
        to: tree:///boot/coreos/platforms.json
  mounts:
    - name: ostree.deployment
      type: org.osbuild.ostree.deployment
      options:
        deployment:
          ref: ostree/1/1/0
          osname:
            fedora-coreos
```
2024-02-07 18:38:03 -05:00
Michael Vogt
705397e7f6 test: add simple zipl_inst tests
Co-authored-by: Nikita Dubrovskii <nikita@linux.ibm.com>
2024-02-07 18:08:44 -05:00
Nikita Dubrovskii
e427289887 stages(zipl.inst): support appending kernel options
In some cases it could useful to add additional kernel options without
modifying BLS config, so any subsequent call to `zipl` would ignore them.
We are going to use this to append `ignition.firstboot` to kernel cmdline
for CoreOS on s390x.
2024-02-07 18:08:44 -05:00
Colin Walters
161c19601a container-deploy: Output stderr on failure
This stage was failing for me in bib, with this change I now
get more useful information from podman's stderr, e.g.:

```
RuntimeError: Failed to mount image (125): time="2024-02-06T14:23:06Z" level=error msg="Unmounting /var/lib/containers/storage/overlay/06456126e7c06cf1b21de024e08e64eddead2b8d03779be213e63aeeea9dec94/merged: invalid argument"
Error: creating overlay mount (...snip...)
fuse: device not found, try 'modprobe fuse' first
fuse-overlayfs: cannot mount: No such file or directory
```
2024-02-07 16:11:19 +01:00