Osbuild doesn't support creating btrfs over multiple devices, thus it should
be fine to have only single and dup in the schema (other options are for raid)
systemd unit stage will now be able to add user unit dropin,
this is done by adding an additional field:unit-type
which is set to system by default. It also adds ability
to update the unit config with ConditionPathExists.
Relevant testing updated for the new workflow.
Signed-off-by: Sayan Paul <paul.sayan@gmail.com>
This adjustment allows the definition of the mark with the RPMs and runs
DNF after installing the RPMs to put the proper markings in the DNF
state database. See #455.
This ensures that packages don't get removed during `autoremove` leading
to broken systems.
rpm-ostree 2023.2 dropped the `rpm-ostree container-encapsulate` entrypoint.
Instead, we have to use `rpm-ostree compose container-encapsulate`.
Adjust the code that it selects the correct entrypoint based on the rpm-ostree
version.
Since the new stage now requires python-yaml, add it to the buildroot and
regenerate all manifests.
We have a lot of copypasta of `def ostree()` but in this
case it's actually rpm-ostree, and we should be clear about that.
(Also in the future this code will need to change
to use the non-deprecated entrypoint, see
https://github.com/coreos/rpm-ostree/pull/4527 )
`tox` is a standard testing tool for Python projects, this allows you to
test locally with all your installed Python version with the following
command:
`tox -m test -p all`
To run the tests in parallel for all supported Python versions.
To run linters or type analysis:
```
tox -m lint -p all
tox -m type -p all
```
This commit *also* disables the `import-error` warning from `pylint`,
not all Python versions have the system-installed Python libraries
available and they can't be fetched from PyPI.
Some linters have been added and the general order linters run in has
been changed. This allows for quicker test failure when running
`tox -m lint`. As a consequence the `test_pylint` test has been removed
as it's role can now be fulfilled by `tox`.
Other assorted linter fixes due to newer versions:
- use a str.join method (`consider-using-join`)
- fix various (newer) mypy and pylint issues
- comments starting with `#` and no space due to `autopep8`
This also changes our CI to use the new `tox` setup and on top of that
pins the versions of linters used. This might move into separate
requirements.txt files later on to allow for easier updating of those
dependencies.
Add a new stage to generate an OpenSCAP tailoring file. The stage
overrides a base OpenSCAP profile by enabling and disabling user
selected rules and creates a new profile name which can be used for
OpenSCAP scanning and remediation.
This will hopefully help debug problems when the signature check fails.
You need to manually take the hash and look it up in the manifest
sources list to figure out what package failed since this stage doesn't
have access to sources.
Anaconda has deprecated the `kickstart_modules` directive and instead
now has `activatable_`, `forbidden_`, and `optional_modules`. This is
available starting in Fedora 35.
This change allows frontend code (`osbuild-composer`) to put version
checks in place to write the correct keys.
Fedora and RHEL 9 kernels support it and since it's an interesting alternative
to other compression methods, we should support it in osbuild.
I also took the liberty of sorting the compression methods alphabetically.
The partition name in the sgdisk stage was previously quoted when
passed to sgdisk as an argument. I think that this was done because
the sgdisk man page states that:
```
If you want to set a name that includes a space, enclose it in
quotation marks, as in sgdisk -c 1:"Sample Name" /dev/sdb.
```
However, this should apply only when sgdisk is run in a shell, so that
the argument is not split by shell into multiple arguments and passes
as a single string.
The stage is executing sgdisk using Python `subprocess` module, which
does not need strings with spaces to be quoted, because they are passed
to the command as separate items which are not split in any way.
The previous behavior of the stage was that these quotes became part of
the actual partition name in the partition table.
After a discussion within the team, we determined that this is a bug.
However, fixing it would result in osbuild producing a different
artifact for the same manifest, compared to osbuild version without such
fix. This is undesired.
For backward compatibility, a new `quote_partition_name` property is
added to the stage options, which can be used to make the stage not
quote the partition name when passed to `sgdisk`. As a result, the
partition name won't be quoted in the partition table.
The default stage behavior us kept.
Modify unit tests to use this option by default.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
util-linux 2.38.1, at least, does not accept raw bit indexes for the
reserved bits (Bit0-2). The undefined ones are out of reach as well and
will have sfdisk throw an error. Only the GUID specific ones can be
passed as raw indexes. This can be verified with the --part-attrs
option. It replicates the format of the --dump output:
$ dd if=/dev/zero of=disk.img bs=$((4<<10)) count=$((32<<10)
$ sgdisk disk.img -n 0:0:+64M -t 0:0FC63DAF-8483-4772-8E79-3D69D8477DE4 -c 0:root
$ sgdisk disk.img -A1:set:{0,1,2,3,48}
$ sfdisk --dump disk.img
label: gpt
label-id: 7484F730-3429-47BF-8A72-3A7AE1F2D86C
device: disk.img
unit: sectors
first-lba: 34
last-lba: 262110
sector-size: 512
disk.img1 : start= 2048, size= 131072, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4, uuid=404694AC-247D-43B3-9907-A468E5C038A3, name="root", attrs="RequiredPartition NoBlockIOProtocol LegacyBIOSBootable GUID:48"
$ sfdisk --part-attrs disk.img 1 0
unsupported GPT attribute bit '0'
sfdisk: disk.img: partition 1: failed to set partition attributes
While the --dump output prefixes the GUID specific bits with "GUID:",
that is not necessary for setting them, which is consistent with the
man-page.
Signed-off-by: Eric Chanudet <echanude@redhat.com>
This allows enabling the ext4 "verity" feature (which is currently
default to off). This will be needed in the automotive work we're
doing.
Signed-off-by: Alexander Larsson <alexl@redhat.com>
The code uses x86_64 as a default, see:
basearch = options.get("basearch", "x86_64")
Let's declare that explicitly in the schema to prevent any confusion.
The right way to enable services is to use a preset file instead of
writing directly into /etc. This adds a new stage called
`org.osbuild.systemd.preset` to do so.
When a manifest list is matched with a container image, the skopeo
stage will merge the specified manifest into the container image dir
before copying it to the registry in the OS tree.
If there is no manifest to merge, we maintain the old behaviour of
symlinking the source to work around the ":" in filename issue.
Otherwise, we copy the container directory so that we can merge the
manifest in the new location.
Add an extra optional input type to the skopeo stage called
`manifest-lists`. This is a list of file-type inputs that must be a
list of manifest lists, downloaded by the skopeo-index source.
The manifests are parsed and automatically associated with an image from
the required `images` inputs. If any manifest list is specified and not
used, this is an error.
Adding manifest-lists currently has no effect.
Change the local storage format for containers to the `dir` format.
The `dir` format will be used to retain signatures and manifests.
The remove-signatures option is removed since the storage format now
supports them.
The final move (os.rename()) at the end of the fetch_one() method now
creates the checksum directory if it doesn't exist and moves the child
archive into it, adding to any existing archives that might exist in
other formats (from a previous version downloading a `docker-archive`).
Dropped the .tar suffix from the symlink in the skopeo stage since it's
not necessary and the target of the link might be a directory now.
The parent class exists() method checks if there is a *file* in the
sources cache that matches the checksum. For containers, this used to
be a file called container-image.tar under a directory that matches the
checksum, so for containers it always returned False. Added an override
for the skopeo source that checks for the new directory archive.
The format so far was assumed to be `docker-archive` if the container
was coming from a source and `oci-archive` if it was coming from a
pipeline. The source format will now be changed to `dir` instead of
`docker-archive`. The pipeline format remains `oci-archive`.
With the new archive format being `dir`, the source can't be linked into
the build root and is bind mounted instead with the use of a MountGuard
created with the instance of the service, and torn down when the service
is stopped.
The _data field is removed from the map functions. It was unused and
these functions aren't part of the abstract class so they don't need to
have consistent signatures.
Update the skopeo stage with support for the newly supported `dir`
format.
As of today the functionality is broken if somebody passes more than one
karg as the join would simply unify the strings but
ignition_network_kcmdline is unquoted and results in just the very first
karg to be consumed by the set command.
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
Extend the copy stage to optionally allow removing the destination
before copying. This allows one to not follow symlinks if the
destination is a symlink to a file. By default, `cp` would change
the file pointed to by the destination if it is symlink.
Extend the stage doc text to cover the behavior with regard to
destination being a symlink.
Add unit tests for the copy stage to also test the newly added option.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add a top level property "files" to the schema and move the rest of the
existing schema one level down. This way we can support adding global
properties in the future if we ever need to expand the scope of the
stage.
Pattern for valid environment variable names as defined in
The Open Group Base Specifications Issue 7, 2018 edition
IEEE Std 1003.1-2017 (Revision of IEEE Std 1003.1-2008)
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html
Updated tests to match UPPERCASE ONLY var names.
Add a new `org.osbuild.chown` stage for setting user and group ownershop
of files. The stage runs the `chown` from the image using `chroot` to
enable it to use users and groups that exist only in the (image) tree.
Add unit test testing the stage in various scenarios.
Co-authored-by: Janine Olear <pninak@web.de>
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Explicitly mention the stage behavior with regard to setting mode on
newly created or existing directories.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This reverts commit a988aacf99.
After some discussion, the original behavior was intentional. With the
added support for gracefully handling the existence of directories, the
stage would originally not set the mode of an existing directory, while
now it will. Additional issue is that `mkdir` applies the provided mode
- umask, which was intentional. Setting the same mode without taking
umask value into account is not desired.
Add a new optional stage option to not fail if the specified directory
already exists. This will make it easier to support creation of custom
repositories via customizations in osbuild-composer. The reason is that
if a specified directory exists in an image, because it was created by
an RPM, then creating it would fail. However, the user may have
specified different mode for the directory, than it already has. Since
there is no way to know for sure if the directory already exists on the
image, without building the image itself, it is desired to handle this
case gracefully as valid in specific use cases.
The default behavior stays the same - specifying an existing directory
path will lead to an error.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Documentation for os.mkdir() says that the mode is
ignored on some systems. Also umask value may affect
the final mode. So we set the mode explicitly.
Set the mode explicitly.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Instead of creating the file in /usr/share and symlinking to /etc,
create it directly in /etc. This fixes an issue with SELinux labeling.
The file in /usr/share does not get labelled correctly because it
doesn't match the policy and causes issues with some tools (rhc).
See rhbz#2147450.
We want to add aboot to the list of possible bootloaders so we can
distinguish if we are using aboot or one of the other bootloaders.
Signed-off-by: Eric Curtin <ecurtin@redhat.com>
If a home directory is specified for an existing user that does
not have one, `usermod` does not create one. This case is now
detected and `mkhomedir_helper(8)` is run inside the chroot to
create the home dir. In Fedora this utility is provided by the
`pam` package so this is now installed in the corresponding
tests together with a new user that simulates the aforementioned
scenario.
Enahnce the stage description: drop an superflous line and add
a description for the home-dir scenario.
