This pulls the list of snapshots from the rpmrepo API, greps the
codebase for all uses of rpmrepo.osbuild.org that look like a snapshot
name, and then checks to make sure they are still valid.
This is the same script as osbuild-composer, except that it also skips
checking ./test/data/stages/ which has rpm urls that look like snapshot
urls.
There hasn't ever been an official tool for this and everyone was just using
the output from the test to generate the diff. This commit introduces a proper
tool for generating these files.
The schemas allowed for `inputs` changed a while back; most new
manifests use a different format than what `osbuild-dev` initially
understood.
This adds all schemas.
Add the translation logic to handle the attrs field of sfdisk as
supported by org.osbuild.sfdisk and documented in its schema. With the
schema taking an int array, some translation is required to populate the
sfdisk command appropriately.
Amend the example schema to reflect the change.
Signed-off-by: Eric Chanudet <echanude@redhat.com>
`os.scandir()` can accept file descriptors only since Python 3.7. The
tool would previously fail with exception when run using Python 3.6.
The solution is to provide a path, which is done using a symlink in
procfs (this is already used within the tool).
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Add support for resolving manifest lists in osbuild-mpp.
Adds an `index` boolean field to the container image struct for
mpp-resolve-images. When enabled, the preprocessor will also store the
manifest-list digest as a separate skopeo-index source and add it to the
skopeo stage under the `manifest-lists` input.
According to the OCI Image Index specification the mediaType field is not mandatory
Assume that it is a list if mediaType is not set while manifests field is
Signed-off-by: Ygal Blum <ygal.blum@gmail.com>
For some reasons I forgot to fix those in the previous runs. Fix a
linter and pep8 warning.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
Avoid using `os` or `input` as symbol names, since they shadow
previously defined symbols.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
Make sure to correctly implement virtual functions and raise
NotImplementedError, as suggested by the standard library. Pylint will
otherwise complain about unavailable functions being accessed.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
The `problem` member is only defined if a problem-mark is present. If
the mark is not set, we get no details on the error whatsoever and have
to just print a generic error message.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
The __getitem__() function incorrectly returns `None` if the
partition-type is not found. Make sure to always return IndexError if
the requested value is outside the supported range.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
This mergest the handling of process_stages() and process_format() into
just one process_format(), which incrementally tracks the call stack
of the formating, which allows it to detect when it is hitting a stage
and can call _process_stage().
This means it is possible to mix things like mpp-if and mpp-join with
stages.
For example, you can do complex combinations like:
pipelines:
- name: rootfs
stages:
mpp-join:
- - type: org.ostree.foo
- mpp-if: use_bar
then:
type: org.osbuild.bar
- mpp-if: use_extra_stages
then:
mpp-eval: extra_stages
This is particularly useful if you included something and you want
to mpp-join something that was set in a variable.
Both file embedding and depsolves start by iterating over all stages,
and we want to add another similar one for container installs, so
break out the iteration over the containers so that it is done in
one place only.
This introduces a new dependency resolver to osbuild-mpp for Arch Linux
which uses the pacman package manager. The used solver is determined by
the `solver` field in the `mpp-depsolve` object inside the manifest
file, if it does not exists it falls back to the DepSolver for dnf/rpm.
Co-Authored-By: Jelle van der Waa <jvanderwaa@redhat.com>
This moves the handling of includes to the manifest loader, thus
supporting nested includes. search_dirs is moved to a property of the
Manifest so that it can be tracked during loads.
In addition we need to fix Manifest.path to the actual path that was
loaded instead of whatever the parent include said, so that relative
includes are handled from the proper location of the loaded manifest.
In process_embed_files(), it assumed that a stage had a "type" field,
which breaks if a stage is e.g. a `mpp-if` node, so use .get() instead
of raw dict lookups.
yaml files are essentially compatible with json files, although
they have some advantages, like allowing comments and being easier
for humans to read/write.
This changes the reading of the file to use a yaml parser instead of
a json parser, but still produces json at the end. I tried manually
converting a json file to yaml and running osbuild-mpp, and it produced
an identical file.
By default the yaml parser doesn't respect order so i had to tweak
the loader a bit to use OrderedDict.
Extract the code that finds a file and opens it from the existing
method that find manifests and opens them. This is so that the
former code can be re-used.
We now allow there to be default variables (none so far) which gets
installed as the initial value of manifest.vars.
Additionally, when including a file the default comes from the current
state of the including manifest, allowing the included manifest to use
variables from the base file.
