The v2 manifest requires the `version` key to be present to be
distinguishable from v1. While technically the manifest can be used
standalong without a `version` key, it does prevent us implementing the
manifest correctly in osbuild (i.e., we are unable to process a v2
manifest without the `version` key, because we are unable to autodetect
it then).
Mark the key as required. It does kind of break backwards compatibility
of the schema, but at the same time we always treated it this way,
anyway. So this should be fine.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
We need to initialize `schema` to `None`, otherwise it will be an access
to an uninitialized variable when looking up invalid schemata:
[...]
File "[...]/osbuild/meta.py", line 583, in get_schema
schema = Schema(schema, name or klass)
UnboundLocalError: local variable 'schema' referenced before assignment
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
The schema input of Schema.__init__ is a python-native representation
of a JSON object, so it can be any kind of dictionary. Furthermore, it
is optional.
Fix the type to be Optional[Dict].
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
Fix the wrong symbol prefixes for the Fat-Mounter. Looks like a
copy-paste from the Xfs-mounter.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
The noop mount does not use FileSystemMountService, and as such never
calls into `mount(1)`. Drop this information from the documentation.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
From IEEE Std 1003.1-2017:
3.437 User Name:
A string that is used to identify a user; see also User Database. To be
portable across systems conforming to POSIX.1-2017, the value is composed
of characters from the portable filename character set. The <hyphen-minus>
character should not be used as the first character of a portable user name.
3.282 Portable Filename Character Set
The set of characters from which portable filenames are constructed.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
a b c d e f g h i j k l m n o p q r s t u v w x y z
0 1 2 3 4 5 6 7 8 9 . _ -
The last three characters are the <period>, <underscore>, and <hyphen-minus>
characters, respectively.
---
The regex checking the username in org.osbuild.users stage omitted the
<period> character before. This commit adds it and modifies the stage
test in order to cover this change.
Fixes: rhbz#2104464
Instead of downloading the image directly to the temporary directory
and then moving that temporary directory into the cache use one more
intermediate directory and move that into the cache. The reason is
that on Python 3.6 removing the temporary directory itself will make
Python crash like this:
Python 3.6.8 (default, Sep 9 2021, 07:49:02)
[GCC 8.5.0 20210514 (Red Hat 8.5.0-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tempfile
>>> with tempfile.TemporaryDirectory(prefix="tmp-download-") as tmpdir:
... import os
... os.rename(tmpdir, "/tmp/foo")
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
File "/usr/lib64/python3.6/tempfile.py", line 809, in __exit__
self.cleanup()
File "/usr/lib64/python3.6/tempfile.py", line 813, in cleanup
_shutil.rmtree(self.name)
File "/usr/lib64/python3.6/shutil.py", line 477, in rmtree
onerror(os.lstat, path, sys.exc_info())
File "/usr/lib64/python3.6/shutil.py", line 475, in rmtree
orig_st = os.lstat(path)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp-download-adl86mwa'
Use `subprocess.check_output` instead of `run(..., capture_output=True)`
since the latter only got added in Python 3.7 and our codebase needs to
be compatible with 3.6 due to RHEL 8.x.
New stage to create or modify `containers-storage.conf` config
files. The currently implement strategy is to merge the given
and existing configuration options.
Update osbuild(1) to reflect changes of the last year. In particular,
we want to document behavior of the v2-manifest and only silently
continue to support v1-manifests.
Lets also introduce new chapters on output control, caching support,
and monitoring.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
Make sure all --help output is consistent. In this particular case,
each line should consistently start with a lower-case character and
avoid a leading `the`.
Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
The oscap.remediation stage now runs the
remediation during build time rather than
creating the necessary files to run the
remediation on first-boot. The tests have
been updated to reflect these changes.
This adds a `osbuild --version` command that prints the current osbuild
version in use. Allows users to confirm their osbuild is up to date
enough to use newer features.
Add a new stage to import public keys into the RPM database via the
`rpmkeys` command. This is similar as to what the `org.osbuild.rpm`
stage already does but it uses inputs instead of inline data.
Introduce a new class to manage inputs, `InputManger` and move the
code to map inputs from the `Input` here. The main insight of why
the logic should be place here is that certain information is needed
to map inputs, independently of specific type: the path to the input
directory, `root`, the store API, `storeapi` and the service manager
instance to start the actual service. Instead of passing all this
information again and again to the `Input` class, we now have a
specialized (service) manager class for inputs that has all the
needed information all the time.
This commit introduces a new vagrant stage and initially only supports
the libvirt backend. The vagrant stage takes a disk image as input
generates a Vagrantfile and metadata.json.
The idea of this test case was to check that two identical trees are
only stored once, via their treesum in the object store; but this
functionality was removed in commit e97f6ef34 and instead of treesums
random uuids are now used. As a result there is no de-duplication
anymore -- the subject of the test. So remove the test.
Check for existing checkpoint in `Pipeline.build_stages` by trying to
get the object, instead of just checking for its existence. Later, if
no checkpoints were found, i.e. `tree` is `None`, create a new object.
This avoids mixing of new object creation and object access.
Instead of iterating over the stages via indices, iterate over the
stages directly. To be able to do so, collect the stages that need
to be built in a deque and then drain it from the other end.
Also invoke `monitor.finish` when the pipeline failed to built.
There is no need to not invoke it in that case. This also will
allow us to print some information in the monitor in tha case.
