The code uses x86_64 as a default, see:
basearch = options.get("basearch", "x86_64")
Let's declare that explicitly in the schema to prevent any confusion.
The test case was testing a subset of the functionality, which is
already tested by test case in `tests.yml`. Delete it, since it does not
add any value.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Fix the following errors:
```
osbuild/util/lvm2.py:117: error: Only instance methods can be decorated with @property
osbuild/api.py:50: error: Only instance methods can be decorated with @property
osbuild/sources.py:85: error: Only instance methods can be decorated with @property
```
Chaining of `@classmethod` and `@property` has been deprecated since
Python 3.11 with a note that chaining didn't work correctly in some
cases.
Relevant links:
https://github.com/python/mypy/issues/13746https://docs.python.org/3.11/whatsnew/3.11.html#language-builtins
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Previously, the unit test depended on osbuild modules being installed on
the system. As a result, this made the test not work in CI where we do
not install osbuild when running unit tests. In addition, the stage
executed by the unit test would use different version of osbuild
internals than the version that is being tests, which could result in
issues or not testing the intended code.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The directory does not exist when the unit test is run in CI. Handle
this case by ensuring that parent directories are created as needed.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
In CI, we list specific tests from `test/run` to run them in parallel.
This is different than what we do with tests in `test/mod` and
`test/src`.
It seems that as a result, we did not run tests from the following
files in CI:
- `test_devices.py`
- `test_mount.py`
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The right way to enable services is to use a preset file instead of
writing directly into /etc. This adds a new stage called
`org.osbuild.systemd.preset` to do so.
Added another skopeo stage to skopeo/a.mpp.json with a skopeo source for
a container hosted on the osbuild-composer gitlab registry. The name
points to a manifest list, which refers to two containers (amd64 and
arm64) that contain a single text file (README.md). The `index` field
is enabled to include the manifest-list as an extra input to the stage.
The diff is updated with the new expected file list.
The containers were created with buildah:
amd=$(buildah from --arch=amd64 scratch)
arm=$(buildah from --arch=arm64 scratch)
buildah config --created-by "Achilleas Koutsou" "${amd}"
buildah config --created-by "Achilleas Koutsou" "${arm}"
buildah copy "${amd}" README.md
buildah copy "${arm}" README.md
amdid=$(buildah commit --format=docker --rm "${amd}")
armid=$(buildah commit --format=docker --rm "${arm}")
name="registry.gitlab.com/redhat/services/products/image-builder/ci/osbuild-composer/manifest-list-test"
buildah manifest create "${name}" "${amdid}" "${armid}"
podman manifest push --all "${name}" dir:container
Add support for resolving manifest lists in osbuild-mpp.
Adds an `index` boolean field to the container image struct for
mpp-resolve-images. When enabled, the preprocessor will also store the
manifest-list digest as a separate skopeo-index source and add it to the
skopeo stage under the `manifest-lists` input.
When a manifest list is matched with a container image, the skopeo
stage will merge the specified manifest into the container image dir
before copying it to the registry in the OS tree.
If there is no manifest to merge, we maintain the old behaviour of
symlinking the source to work around the ":" in filename issue.
Otherwise, we copy the container directory so that we can merge the
manifest in the new location.
Add an extra optional input type to the skopeo stage called
`manifest-lists`. This is a list of file-type inputs that must be a
list of manifest lists, downloaded by the skopeo-index source.
The manifests are parsed and automatically associated with an image from
the required `images` inputs. If any manifest list is specified and not
used, this is an error.
Adding manifest-lists currently has no effect.
A new source module that can download a multi-image manifest list from a
container registry. This module is very similar to the skopeo source,
but instead downloads a manifest list with `--multi-arch=index-only`.
The checksum of the source object must be the digest of the manifest
list that will be stored and the manifest that is downloaded must be a
manifest-list.
Change the local storage format for containers to the `dir` format.
The `dir` format will be used to retain signatures and manifests.
The remove-signatures option is removed since the storage format now
supports them.
The final move (os.rename()) at the end of the fetch_one() method now
creates the checksum directory if it doesn't exist and moves the child
archive into it, adding to any existing archives that might exist in
other formats (from a previous version downloading a `docker-archive`).
Dropped the .tar suffix from the symlink in the skopeo stage since it's
not necessary and the target of the link might be a directory now.
The parent class exists() method checks if there is a *file* in the
sources cache that matches the checksum. For containers, this used to
be a file called container-image.tar under a directory that matches the
checksum, so for containers it always returned False. Added an override
for the skopeo source that checks for the new directory archive.
The format so far was assumed to be `docker-archive` if the container
was coming from a source and `oci-archive` if it was coming from a
pipeline. The source format will now be changed to `dir` instead of
`docker-archive`. The pipeline format remains `oci-archive`.
With the new archive format being `dir`, the source can't be linked into
the build root and is bind mounted instead with the use of a MountGuard
created with the instance of the service, and torn down when the service
is stopped.
The _data field is removed from the map functions. It was unused and
these functions aren't part of the abstract class so they don't need to
have consistent signatures.
Update the skopeo stage with support for the newly supported `dir`
format.
Manifest-db is finally unblocked and we can update the reference commit.
Done manually this time to speed up the process.
I'm also updating the set of distributions we are testing on, upgrading
it to what's is generated on manifest db.
As of today the functionality is broken if somebody passes more than one
karg as the join would simply unify the strings but
ignition_network_kcmdline is unquoted and results in just the very first
karg to be consumed by the set command.
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
Extend the copy stage to optionally allow removing the destination
before copying. This allows one to not follow symlinks if the
destination is a symlink to a file. By default, `cp` would change
the file pointed to by the destination if it is symlink.
Extend the stage doc text to cover the behavior with regard to
destination being a symlink.
Add unit tests for the copy stage to also test the newly added option.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add a top level property "files" to the schema and move the rest of the
existing schema one level down. This way we can support adding global
properties in the future if we ever need to expand the scope of the
stage.
Pattern for valid environment variable names as defined in
The Open Group Base Specifications Issue 7, 2018 edition
IEEE Std 1003.1-2017 (Revision of IEEE Std 1003.1-2008)
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html
Updated tests to match UPPERCASE ONLY var names.
To avoid kernel panics if the kernel attempts to recover the filesystem
when it's mounted as readonly. Offer the possiblity to use the
norecovery option for journaling file systems (Xfs, Ext4, Btrfs).
Add a new `org.osbuild.chown` stage for setting user and group ownershop
of files. The stage runs the `chown` from the image using `chroot` to
enable it to use users and groups that exist only in the (image) tree.
Add unit test testing the stage in various scenarios.
Co-authored-by: Janine Olear <pninak@web.de>
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
