Commit graph

681 commits

Author SHA1 Message Date
Tomáš Hozza
13c098cfdd Stages/SELinux: force auto-relabel of full contexts
Previously, the SELinux stage would not force full contexts reset when
forcing auto-relabel on first boot. As a result, all files remained
`unconfined_u` after the auto-relabeling on first boot and only the type
part was reset.

We really need to mimic the behavior of `fixfiles -F onboot` command,
which creates the `/.autorelabel` file with "-F" in it.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-04-22 16:33:59 +02:00
Michael Vogt
2f18beba17 stages: allow bootloader --append via kickstart
This commit allows use to append kernel commandline options via
the kickstart file. This is useful for e.g. the bootc installer
where we support customizing this via blueprints.
2024-04-19 13:14:14 +02:00
Achilleas Koutsou
f255fba09f stage/systemd.unit.create: move systemd-analyze verify to tests
Verifying the systemd unit also checks if any referred systemd units
(Wants, Requires, After) exist and if all commands in Exec exist and are
executable.  Without '--root', the systemd-analyze verify command is
testing this against files in the build root, which isn't valid.

Units and binaries might not exist in the build root when referenced in
the image root tree, making the unit fail when when it's valid.
Conversely, the verification can succeed by finding executables in the
build root that don't exist in the image root tree when it should be
failing.

When verifying user units, systemd expects runtime directories.

All of this makes it quite difficult to verify systemd units properly
when building an image.  The call is useful for making sure the unit is
structured properly, but the user unit verification setup is difficult
to accomplish in a general way while building.

Remove the systemd-analyze verify step from the stage.  Move it to the
unit test so that we have some assurance that our unit file structure is
correct and things work as expected.  Create referenced unit files and
commands to make the unit valid.
2024-04-18 17:20:57 +02:00
Achilleas Koutsou
86baf802d5 test/systemd_unit_create: Wants, Requires, After
Add test values for Wants, Requires, and After.
Adding multiple values to test that arrays work and made sure they're
all different.
The units need to be valid, real unit names otherwise the
'systemd-analyze verify' check will fail.
2024-04-18 17:20:57 +02:00
Achilleas Koutsou
d40948ce61 stage/systemd.unit.create: add After option
Support the After option in the Unit section of the unit file.
2024-04-18 17:20:57 +02:00
Michael Vogt
388e367392 stages: add support for --target-imgref to bootc install to-filesystem
We currently do not set the `--target-imgref` and do not allow to
override it. This means that on a fresh deploy it is set to an
incorrect value. This commit allows to set it via the
org.osbuild.bootc.install-to-filesystem stage.
2024-04-17 15:06:33 +02:00
Tomáš Hozza
5b75592fef Stages/cloud-init: support additional datasources
Add "Ec2" and "None" datasources, which is needed for RHEL-7.9 EC2
images.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-04-16 07:50:09 +02:00
Michael Vogt
7c04e1c596 stages(groups): port to v2 and allow devices/mounts
This will allow us to generate `bootc install to-filesystem` compatible
devices/mount setups for the groups stage.
2024-04-15 13:56:12 +02:00
Michael Vogt
2a0027557f stages(groups): add minimal parameter validation test
The schema will move to v2 so we need to ensure that at least some
minimal validation is done that the old and the new schema work
the same way.
2024-04-15 13:56:12 +02:00
Paweł Poławski
f3a5267e65 Tests: Add unit tests for org.osbuild.mkfs.xfs stage 2024-04-09 23:56:06 +02:00
Michael Vogt
38bcef9378 stages(mkdir): allow to pass in devices/mounts
This allows to combine `org.osbuild.mkdir` with the `osbuild.deployment`
mount and with the upcoming `org.osbuild.bind` mount. The use case is
that we need to create the dir `/var/home` so that `useradd` from inside
a ostree root works (there /home is a symlink and useradd will not
follow the symlink and create a dir in the target by itself).

This allows to write:
```json
        {
          "type": "org.osbuild.mkdir",
          "options": {
            "paths": [
	      {
		"path": "/var/home"
	      }
	    ]
          },
          "devices": {
            "disk": {
              "type": "org.osbuild.loopback",
              "options": {
                "filename": "disk.raw",
                "partscan": true
              }
            }
          },
          "mounts": [
            {
              "name": "part4",
              "type": "org.osbuild.ext4",
              "source": "disk",
              "target": "/",
              "partition": 4
            },
            {
              "name": "part3",
              "type": "org.osbuild.ext4",
              "source": "disk",
              "target": "/boot",
              "partition": 3
            },
            {
              "name": "part2",
              "type": "org.osbuild.fat",
              "source": "disk",
              "target": "/boot/efi",
              "partition": 2
            },
            {
              "name": "ostree.deployment",
              "type": "org.osbuild.ostree.deployment",
              "options": {
                "source": "mount",
                "deployment": {
                  "default": true
                }
              }
            },
            {
              "name": "bind",
              "type": "org.osbuild.bind",
	      "target": "tree://",
	      "options": {
		"source": "mount://"
	      }
            }
          ]
        },
```
to fix this.
2024-04-09 17:12:20 +02:00
Paweł Poławski
488eee7bc0 Tests: Add unit tests for org.osbuild.mkfs.btrfs stage 2024-04-09 00:39:11 -07:00
Michael Vogt
2f858d32e4 stages: add "devices/mounts" as allowed inputs for users/selinux
When moving to `bootc install to-filesystem` we will need support
for mounting the deployed disk and writing to the deployment root
this requires that we teach the users and selinux stages to
have them available. This is a first step towards this.

It also adds tests to ensure the options can be passed.
2024-04-05 16:13:42 +02:00
Michael Vogt
ba85d30cee stage(selinux): move to schema_2 to allow adding mounts/devices
This is a preparation to allow adding mounts/devices to the users
stage so that we can eventually support bootc install to-filesystem.
2024-04-05 16:13:42 +02:00
Michael Vogt
72a2334fbe states(users): move to schema_2 to allow adding mounts/devices
This is a preparation to allow adding mounts/devices to the users
stage so that we can eventually support bootc install to-filesystem.

It also adds some smoke tests for the schema to ensure it's still
valid.
2024-04-05 16:13:42 +02:00
Michael Vogt
91f47ddf03 stages(zip): add test for zip stage
Add a small integration test that runs zip with various options
to ensure it works correctly.
2024-04-04 13:54:34 +02:00
Luke Yang
ac8a2a4f30 stages: add org.osbuild.zip
In CoreOS Assembler, some hyperv artifact we `zip` for compression. This
new stage is modeled after the `org.osbuild.tar` stage with necessary
modifications.
2024-04-04 13:54:34 +02:00
Achilleas Koutsou
0dc816c2f9 stages/systemd.unit: multiple Environment options
Update the org.osbuild.systemd.unit stage to also support multiple
Environment options where each is an object with {key: value}.  Enable
the allow_no_value option in configparser so we can add the multiple
entries.
2024-04-04 10:36:04 +02:00
Achilleas Koutsou
e3fd572b94 test/systemd_unit_create: add new options to unit test
Add the new options to the unit test.
2024-04-04 10:36:04 +02:00
Achilleas Koutsou
2d3c48207b stages/systemd.unit.create: comment for allow_no_value
Add a comment explaining why we need to set allow_no_value=True for the
configparser.
2024-04-04 10:36:04 +02:00
Achilleas Koutsou
77e7c0538a stages/systemd.unit.create: Environment and EnvironmentFile
Support the Environment and EnvironmentFile options in the Service
section of the unit file.
The Environment option is set as an object with keys "key" and "value"
and the key is validated with a pattern.

Updated the stage to special-case the Environment option.
2024-04-04 10:36:04 +02:00
Paweł Poławski
ae8f3f4a1e Stage: Schema migration to separated JSON files
In the past input schema for stages has been moved to standalone
JSON files. Looks like org.osbuild.tar was missed during the migration.
2024-03-30 11:03:29 +01:00
Michael Vogt
ec496769c5 stages: allow bootc.install-to-filesystem work without selinux
By default "bootc" will refuse to work on a non-selinux system if
the bootc container requires selinux. This is a sensible approach
in general but for us it's tricky because we want to be able to
generate images when running on developer machines or CI machines
that may not necessarily have selinux. So make bootc more relaxed.
2024-03-26 12:55:11 +01:00
Achilleas Koutsou
fa0605b89f stages: reformat meta.json files
Reformat all meta.json files for consistency.

Formatted with `jq --indent 2 .`
2024-03-26 01:20:37 +01:00
Michael Vogt
0a72145b13 stages(chattr): fix rebase artifact 2024-03-25 18:26:53 +01:00
Michael Vogt
249107a028 stages,test: fix lint errors and add basic unit tests
Add very simple unit tests as a starting point for the new
parsing functions in `util/parsing.py`.
2024-03-25 18:26:53 +01:00
Renata Ravanelli
6d4d1962eb util: Consolidate parse functions into util
- Move functions to the 'util' to centralize common
functionality, reducing code duplication and improving
maintainability across the codebase.

Signed-off-by: Renata Ravanelli <rravanel@redhat.com>
2024-03-25 18:26:53 +01:00
Michael Vogt
0528ccc3f0 osbuild: add support to exclude_paths to setfiles()
This is needed because on a mounted `bootc` container `setfiles`
without excluding `/sysroot` will create many warnings like:
```
setfiles: conflicting specifications for /run/osbuild/tree/sysroot/ostree/repo/objects/00/0ef9ada2ee87792e8ba21afd65aa00d79a1253018832652b8694862fb80e84.file and /run/osbuild/tree/usr/lib/firmware/cirrus/cs35l41-dsp1-spk-prot-103c8b8f-r1.bin.xz, using system_u:object_r:lib_t:s0.
```
but simply excluding this dir fixes them.
2024-03-20 18:05:51 +01:00
Simon de Vlieger
a9f4024ded lint: fix up adjusted stages 2024-03-19 15:32:45 +01:00
Simon de Vlieger
bb58892571 stages: externalize schemas
Copy all schemas into the relevant `.meta.json` files instead of having
them contained inside the stages.
2024-03-19 15:32:45 +01:00
Simon de Vlieger
02b6d696ef stages: consistent docstrings
Make docstrings consistent across stages regarding whitespace and
summary line.
2024-03-19 15:32:45 +01:00
Michael Vogt
7431653882 test: add (trivial) test for ConditionPathIsDirectory 2024-03-19 10:19:26 +01:00
Sayan Paul
548d2a51ea stage/unit-create:new systemd unit config params
Added new systemd unit config : ConditionPathIsDirectory

Signed-off-by: Sayan Paul <paul.sayan@gmail.com>
2024-03-19 10:19:26 +01:00
Michael Vogt
155e24e4cb stage: user test that tests adduser is called the right way
The issue with mocking subprocess.call() is that we will not
catch if arguments are passed as eg. integers. So instead use
the `mock_command()` helper so that a real binary is called.
2024-03-18 20:37:21 +01:00
Michael Vogt
fd0167f130 test: return container_id in make_container
The current `make_container()` helper is a bit silly (which is
entirely my fault). It requires a container tag as input but all
tests end up creating a random number for this input. So instead
just remove the input and return the container_id from the podman
build in the contextmanager and use that.
2024-03-18 20:36:19 +01:00
Michael Vogt
df224fb32b stages: add support for kernel-options to bootc-install-to-fs
This commit adds support to use the `--kopt` arguments to
`bootc install to-filesystems`. It is not strictly needed right
now though.
2024-03-14 15:07:05 +01:00
Michael Vogt
626077ffc0 stages: add root ssh options to bootc.install-to-filesystem
To start using the `bootc.instal-to-filesystem` stage we need
to be able to test the generated images. This requires a login
on the disk. Traditionally we did that via the `users` stage.

But on a bootc system we do not want to modify the disk after
bootc did the install to avoid messing with things like the
selinux labels or (future) fsverity setups.

So for now we will use the `--root-ssh-authorized-keys` feature
to inject support for login.

So this commit adds a new option to the stage called
`root-ssh-authorized-keys`.
2024-03-14 15:07:05 +01:00
Michael Vogt
3e11b57815 meta: rename -meta.json to .meta.json
With `.meta.json` it's two "real" file extension. The `-meta.json`
feels less clear.
2024-03-14 13:37:57 +01:00
Michael Vogt
9b09ed9eb4 osbuild: allow json data to come from a {stage}-meta.json file
Instead of always parsing the python stage to load meta information
allow the user of a new `{stage}-meta.json` file. This is a first
step towards allowing modules to be written in a different language
than python. It also has some practical advantages:
- slightly faster as it avoids calling python to output the schemas
- easier to write schemas as this can be done in a real json editor
  now
- more extensible in a future where stages maybe binaries with
  shlib dependencies that are only satisfied in the buildroot
  but not on the host
2024-03-14 13:37:57 +01:00
Michael Vogt
439f7f08c7 test: add minimal test for expiredate for user stage 2024-03-12 07:38:31 +01:00
Andre Marianiello
f6ae58151e stages/org.osbuild.users: add expire date
Add the expiredate field to the users option in the org.osbuild.users
stage. This option maps to the --expiredate option of useradd/usermod,
which can be useful when creating users whose password must be changed
upon first login.
2024-03-12 07:38:31 +01:00
Simon de Vlieger
a895aa177c stages/kickstart: add noswap to autopart
Allow passing the `noswap` option to `autopart`.
2024-03-11 11:50:34 +01:00
Michael Vogt
d43194c0b3 test: add unit-{type,path} test coverage 2024-03-08 16:20:48 -08:00
Gianluca Zuccarelli
6c0973238d utils/mnt: fix mount permissions
This is a follow up to #1550 where we enabled a `rw` permissions mode,
which is not ideal since it would theoretically be possible to set both
`ro` and `rw` modes at the same time. This commit fixes the issue by only
allowing one option at a time.

Fixes #1588
2024-03-07 13:01:47 +00:00
Sayan Paul
2d2cdd8097 stage/systemd-unit-create:paremeterize file location
User can now customize the systemd unit load path.
User can select between etc or usr , defaults to 'usr'.
Also user can customize the scope of the service between global
or system, defaults to system.

Signed-off-by: Sayan Paul <paul.sayan@gmail.com>
2024-03-07 14:04:00 +02:00
Michael Vogt
9baca1fe90 stages: add small unit test for the gzip stage 2024-03-04 11:18:13 +01:00
Luke Yang
cfaabe618f stages/org.osbuild.gzip: add compression level option
Allow compression level to be specified instead of defaulting to 1. This is needed for CoreOS Assembler.
2024-03-04 11:18:13 +01:00
Michael Vogt
30f740ec9f stages: add test for coreos.platform generate_console_settings_file
And a tiny tweak to avoid an empty line at the start of the
`boot/grub2/console.cfg`
2024-02-28 10:37:01 +01:00
Michael Vogt
e9c31c035b stages: tweak process_platforms_json to be slightly shorter
With the test the helper can now be slightly simplified. Because
we only have two results it seems easier to just use them directly
than to store them in an intermediate result struct.
2024-02-28 10:37:01 +01:00
Michael Vogt
6cc7309890 stages: add unit test for coreos.platform process_platforms_json
Add unit test for `coreos.platform` process_platforms_json helper.
2024-02-28 10:37:01 +01:00