The items of an array are defined under the `items` key, not under `paths`.
Let's fix this.
Btw, this is possible because JSON Schema itself doesn't use
additionalProperties = false. This allows extending the schemas easily, but
is sadly a bit error-prone.
Sadly, since this issue effectively disabled validation of the stage options,
we also need to relax the schema a bit:
We found out that there are manifests in the wild, that use relative paths,
instead of absolute ones. Thus, this commit changes the validation regex to
allow relative paths. However, this now emits a warning and it's strongly
discouraged. The associated stage test was modified to accommodate for this.
Co-authored-by: Tomáš Hozza <thozza@redhat.com>
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This is a feature that was added in rpm-ostree 2023.10 and is needed
for the new transient /etc feature to work. What it does is change the
labeling of /usr/etc to match those of /etc, so that /usr/etc can be used
directly as a bind-mount or an overlay mount when mounted on /etc.
See https://github.com/coreos/rpm-ostree/pull/4640 for details.
This drains some of the logic out of `main()` into a
`ostree_commit_deploy()` function. Doing this will make it easier
to diff this stage with the recently added `ostree.deploy.container`
stage.
This commit also changes the `ref` in the schema to be optional,
which is a fixup for 3cc733d. We need to make the ref optional because
the ref could come from the user in the toplevel schema or it could
come from input commit in the schema.
This stage is similar to ostree.deploy, but deploys from a container
image rather than from an OSTree commit by using the `ostree container
image deploy` command. An example stage definition could look like:
```
- type: org.osbuild.ostree.deploy.container
options:
osname: fedora-coreos
target_imgref: ostree-remote-registry:fedora:quay.io/fedora/fedora-coreos:stable
mounts:
- /boot
- /boot/efi
kernel_opts:
- rw
- console=tty0
- console=ttyS0
- ignition.platform.id=qemu
- '$ignition_firstboot'
inputs:
images:
type: org.osbuild.containers
origin: org.osbuild.source
mpp-resolve-images:
images:
- source: quay.io/fedora/fedora-coreos
tag: stable
```
Co-authored-by: Dusty Mabe <dusty@dustymabe.com>
Add a new attribute, "keys", to allow specifying multiple public SSH
keys to install to a users authorized_keys file.
This maintains backwards compatibility with the existing "key" attribute
that can only specify a single SSH key to install into the file (without
using some newline hacks).
Signed-off-by: Michael Ho <michael.ho@ieee.org>
This will allow a user to specify an input directly to the deploy
stage rather than requiring a ostree.pull stage to be called first.
Adding inputs will also be useful when we soon add support for
deploying from a container rather than just deploying from an existing
ostree commit in a repo.
I've been informed that this stage is old and shouldn't be used for
newly created manifests. Let's add a comment to the top for future
explorers of this code base.
Some platforms like the TI AM62 require a particular FAT geometry for
their CPU to read the file system (and thus the bootloader). Failing
that the CPU will simply not boot and keep looking for a bootloader.
Let's add some options to enforce a particular filesystem geometry
through the -g option of mkfs.fat.
Signed-off-by: Maxime Ripard <mripard@kernel.org>
The current options building code for mkfs.fat prevents to have multiple
options at a time since it will assign the opts list to either the label
or fatsize options.
If both are set, only the last one, fatsize, will be set.
Use the occasion to rewrite the code a bit and limit the number of
lists concatenation.
Signed-off-by: Maxime Ripard <mripard@kernel.org>
This will hoist even more code into util out of the skopeo stage.
Now a caller can call:
with containers.container_source(image) as (image_name, image_source):
print(f"{image_name}, {image_source}")
to process containers inputs.
This hoists container handling code from the skopeo stage into
util/containers. It is prep for adding another stage that accepts
containers as an input. The code is common so we should share it
amongst all stages that use containers as input.
For Fedora CoreOS we don't actually have any root= kernel command
line option in our baked images. We have services that rely on this
and set up sysroot on first boot. The code in this stage doesn't
require for this option to have been provided and actually gracefully
handles when it's not provided. Let's just change the schema to also
not require it.
Osbuild doesn't support creating btrfs over multiple devices, thus it should
be fine to have only single and dup in the schema (other options are for raid)
systemd unit stage will now be able to add user unit dropin,
this is done by adding an additional field:unit-type
which is set to system by default. It also adds ability
to update the unit config with ConditionPathExists.
Relevant testing updated for the new workflow.
Signed-off-by: Sayan Paul <paul.sayan@gmail.com>
This adjustment allows the definition of the mark with the RPMs and runs
DNF after installing the RPMs to put the proper markings in the DNF
state database. See #455.
This ensures that packages don't get removed during `autoremove` leading
to broken systems.
rpm-ostree 2023.2 dropped the `rpm-ostree container-encapsulate` entrypoint.
Instead, we have to use `rpm-ostree compose container-encapsulate`.
Adjust the code that it selects the correct entrypoint based on the rpm-ostree
version.
Since the new stage now requires python-yaml, add it to the buildroot and
regenerate all manifests.
We have a lot of copypasta of `def ostree()` but in this
case it's actually rpm-ostree, and we should be clear about that.
(Also in the future this code will need to change
to use the non-deprecated entrypoint, see
https://github.com/coreos/rpm-ostree/pull/4527 )
`tox` is a standard testing tool for Python projects, this allows you to
test locally with all your installed Python version with the following
command:
`tox -m test -p all`
To run the tests in parallel for all supported Python versions.
To run linters or type analysis:
```
tox -m lint -p all
tox -m type -p all
```
This commit *also* disables the `import-error` warning from `pylint`,
not all Python versions have the system-installed Python libraries
available and they can't be fetched from PyPI.
Some linters have been added and the general order linters run in has
been changed. This allows for quicker test failure when running
`tox -m lint`. As a consequence the `test_pylint` test has been removed
as it's role can now be fulfilled by `tox`.
Other assorted linter fixes due to newer versions:
- use a str.join method (`consider-using-join`)
- fix various (newer) mypy and pylint issues
- comments starting with `#` and no space due to `autopep8`
This also changes our CI to use the new `tox` setup and on top of that
pins the versions of linters used. This might move into separate
requirements.txt files later on to allow for easier updating of those
dependencies.
Add a new stage to generate an OpenSCAP tailoring file. The stage
overrides a base OpenSCAP profile by enabling and disabling user
selected rules and creates a new profile name which can be used for
OpenSCAP scanning and remediation.
This will hopefully help debug problems when the signature check fails.
You need to manually take the hash and look it up in the manifest
sources list to figure out what package failed since this stage doesn't
have access to sources.
Anaconda has deprecated the `kickstart_modules` directive and instead
now has `activatable_`, `forbidden_`, and `optional_modules`. This is
available starting in Fedora 35.
This change allows frontend code (`osbuild-composer`) to put version
checks in place to write the correct keys.
Fedora and RHEL 9 kernels support it and since it's an interesting alternative
to other compression methods, we should support it in osbuild.
I also took the liberty of sorting the compression methods alphabetically.
The partition name in the sgdisk stage was previously quoted when
passed to sgdisk as an argument. I think that this was done because
the sgdisk man page states that:
```
If you want to set a name that includes a space, enclose it in
quotation marks, as in sgdisk -c 1:"Sample Name" /dev/sdb.
```
However, this should apply only when sgdisk is run in a shell, so that
the argument is not split by shell into multiple arguments and passes
as a single string.
The stage is executing sgdisk using Python `subprocess` module, which
does not need strings with spaces to be quoted, because they are passed
to the command as separate items which are not split in any way.
The previous behavior of the stage was that these quotes became part of
the actual partition name in the partition table.
After a discussion within the team, we determined that this is a bug.
However, fixing it would result in osbuild producing a different
artifact for the same manifest, compared to osbuild version without such
fix. This is undesired.
For backward compatibility, a new `quote_partition_name` property is
added to the stage options, which can be used to make the stage not
quote the partition name when passed to `sgdisk`. As a result, the
partition name won't be quoted in the partition table.
The default stage behavior us kept.
Modify unit tests to use this option by default.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
util-linux 2.38.1, at least, does not accept raw bit indexes for the
reserved bits (Bit0-2). The undefined ones are out of reach as well and
will have sfdisk throw an error. Only the GUID specific ones can be
passed as raw indexes. This can be verified with the --part-attrs
option. It replicates the format of the --dump output:
$ dd if=/dev/zero of=disk.img bs=$((4<<10)) count=$((32<<10)
$ sgdisk disk.img -n 0:0:+64M -t 0:0FC63DAF-8483-4772-8E79-3D69D8477DE4 -c 0:root
$ sgdisk disk.img -A1:set:{0,1,2,3,48}
$ sfdisk --dump disk.img
label: gpt
label-id: 7484F730-3429-47BF-8A72-3A7AE1F2D86C
device: disk.img
unit: sectors
first-lba: 34
last-lba: 262110
sector-size: 512
disk.img1 : start= 2048, size= 131072, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4, uuid=404694AC-247D-43B3-9907-A468E5C038A3, name="root", attrs="RequiredPartition NoBlockIOProtocol LegacyBIOSBootable GUID:48"
$ sfdisk --part-attrs disk.img 1 0
unsupported GPT attribute bit '0'
sfdisk: disk.img: partition 1: failed to set partition attributes
While the --dump output prefixes the GUID specific bits with "GUID:",
that is not necessary for setting them, which is consistent with the
man-page.
Signed-off-by: Eric Chanudet <echanude@redhat.com>
This allows enabling the ext4 "verity" feature (which is currently
default to off). This will be needed in the automotive work we're
doing.
Signed-off-by: Alexander Larsson <alexl@redhat.com>
The code uses x86_64 as a default, see:
basearch = options.get("basearch", "x86_64")
Let's declare that explicitly in the schema to prevent any confusion.
The right way to enable services is to use a preset file instead of
writing directly into /etc. This adds a new stage called
`org.osbuild.systemd.preset` to do so.
When a manifest list is matched with a container image, the skopeo
stage will merge the specified manifest into the container image dir
before copying it to the registry in the OS tree.
If there is no manifest to merge, we maintain the old behaviour of
symlinking the source to work around the ":" in filename issue.
Otherwise, we copy the container directory so that we can merge the
manifest in the new location.
