This mergest the handling of process_stages() and process_format() into
just one process_format(), which incrementally tracks the call stack
of the formating, which allows it to detect when it is hitting a stage
and can call _process_stage().
This means it is possible to mix things like mpp-if and mpp-join with
stages.
For example, you can do complex combinations like:
pipelines:
- name: rootfs
stages:
mpp-join:
- - type: org.ostree.foo
- mpp-if: use_bar
then:
type: org.osbuild.bar
- mpp-if: use_extra_stages
then:
mpp-eval: extra_stages
This is particularly useful if you included something and you want
to mpp-join something that was set in a variable.
Add to new autopep8 check to the checks suit. Ideally we would
move the `test.src` test over from tests.yml, but running that
in an updated container leads to a whole lot of new warnings,
most of them related to "unspecified encoding" in `open` calls.
Instead of using `subprocess` with `check=True`, which will
echo the command used, including all the files it checked,
check the error code and if non-zero, fail the test with a
nice small error message.
Our format for the global module doc-string is usually:
"""
<short summary>
<longer description>
"""
Introduce the short summary and also reformat the rest of the
doc string a bit.
This stage allows to manifests to define information to be put into a
built image. The information can then later be used by RHSM/Insights.
The facts are stored in `/usr/share/osbuild/rhsm.facts` and then
symlinked into `/etc/rhsm/facts/osbuild.facts`.
The v2 manifest requires the `version` key to be present to be
distinguishable from v1. While technically the manifest can be used
standalong without a `version` key, it does prevent us implementing the
manifest correctly in osbuild (i.e., we are unable to process a v2
manifest without the `version` key, because we are unable to autodetect
it then).
Mark the key as required. It does kind of break backwards compatibility
of the schema, but at the same time we always treated it this way,
anyway. So this should be fine.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
We need to initialize `schema` to `None`, otherwise it will be an access
to an uninitialized variable when looking up invalid schemata:
[...]
File "[...]/osbuild/meta.py", line 583, in get_schema
schema = Schema(schema, name or klass)
UnboundLocalError: local variable 'schema' referenced before assignment
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
The schema input of Schema.__init__ is a python-native representation
of a JSON object, so it can be any kind of dictionary. Furthermore, it
is optional.
Fix the type to be Optional[Dict].
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
Fix the wrong symbol prefixes for the Fat-Mounter. Looks like a
copy-paste from the Xfs-mounter.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
The noop mount does not use FileSystemMountService, and as such never
calls into `mount(1)`. Drop this information from the documentation.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
From IEEE Std 1003.1-2017:
3.437 User Name:
A string that is used to identify a user; see also User Database. To be
portable across systems conforming to POSIX.1-2017, the value is composed
of characters from the portable filename character set. The <hyphen-minus>
character should not be used as the first character of a portable user name.
3.282 Portable Filename Character Set
The set of characters from which portable filenames are constructed.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
a b c d e f g h i j k l m n o p q r s t u v w x y z
0 1 2 3 4 5 6 7 8 9 . _ -
The last three characters are the <period>, <underscore>, and <hyphen-minus>
characters, respectively.
---
The regex checking the username in org.osbuild.users stage omitted the
<period> character before. This commit adds it and modifies the stage
test in order to cover this change.
Fixes: rhbz#2104464
Instead of downloading the image directly to the temporary directory
and then moving that temporary directory into the cache use one more
intermediate directory and move that into the cache. The reason is
that on Python 3.6 removing the temporary directory itself will make
Python crash like this:
Python 3.6.8 (default, Sep 9 2021, 07:49:02)
[GCC 8.5.0 20210514 (Red Hat 8.5.0-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tempfile
>>> with tempfile.TemporaryDirectory(prefix="tmp-download-") as tmpdir:
... import os
... os.rename(tmpdir, "/tmp/foo")
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
File "/usr/lib64/python3.6/tempfile.py", line 809, in __exit__
self.cleanup()
File "/usr/lib64/python3.6/tempfile.py", line 813, in cleanup
_shutil.rmtree(self.name)
File "/usr/lib64/python3.6/shutil.py", line 477, in rmtree
onerror(os.lstat, path, sys.exc_info())
File "/usr/lib64/python3.6/shutil.py", line 475, in rmtree
orig_st = os.lstat(path)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp-download-adl86mwa'
Use `subprocess.check_output` instead of `run(..., capture_output=True)`
since the latter only got added in Python 3.7 and our codebase needs to
be compatible with 3.6 due to RHEL 8.x.
New stage to create or modify `containers-storage.conf` config
files. The currently implement strategy is to merge the given
and existing configuration options.
Update osbuild(1) to reflect changes of the last year. In particular,
we want to document behavior of the v2-manifest and only silently
continue to support v1-manifests.
Lets also introduce new chapters on output control, caching support,
and monitoring.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
Make sure all --help output is consistent. In this particular case,
each line should consistently start with a lower-case character and
avoid a leading `the`.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
The oscap.remediation stage now runs the
remediation during build time rather than
creating the necessary files to run the
remediation on first-boot. The tests have
been updated to reflect these changes.
This adds a `osbuild --version` command that prints the current osbuild
version in use. Allows users to confirm their osbuild is up to date
enough to use newer features.
