Commit graph

2057 commits

Author SHA1 Message Date
Christian Kellner
f05078f66e global: fix PEP-8 formatting
This patch was generated by running `autopep8 --diff` on the
source tree and then applying the diff.
2022-08-05 09:41:05 +02:00
Christian Kellner
bb644cdd07 test: add have_autopep8 method
Add a helper to check if the `autopep8` command is available.
2022-08-05 09:41:05 +02:00
Christian Kellner
620f7c0f3e test/pylint: extract source files into fixture
Extract the code that discovers the source files into a fixture;
this is done so that we can share that other future tests.
2022-08-05 09:41:05 +02:00
Christian Kellner
bf2789fd91 test/pylint: convert to pytest
Convert the test to using pytest instead of unittest.
2022-08-05 09:41:05 +02:00
Christian Kellner
a3bc5f7692 stages/gcp.guest-agent.conf: warning -> info
Inform when a new guest-agent configuration file is created,
but remove the warning prefix, since it is perfectly fine if
the file does not yet exist.
2022-08-04 08:08:02 +02:00
Christian Kellner
24ad788d3c stages/gcp.guest-agent.conf: specify encoding
When opening files, always specify the encoding as this will
become a linter warning in the near future.
2022-08-04 08:08:02 +02:00
Christian Kellner
cbffd9d104 stages/gcp.guest-agent.conf: add short header
Our format for the global module doc-string is usually:
"""
<short summary>

<longer description>
"""

Introduce the short summary and also reformat the rest of the
doc string a bit.
2022-08-04 08:08:02 +02:00
Christian Kellner
679688f66f stages/gcp.guest-agent.conf: fix some whitespaces
Be more PEP-8 (two lines between methods). Additionally, separate
the builtin imports from library imports and osbuild imports.
2022-08-04 08:08:02 +02:00
fkolwa
2bca7b07f3 stages: add instance_configs.cfg.distro
test/data: add instance_configs.cfg.distro test
2022-08-03 15:39:35 +02:00
schutzbot
f91bb0a24f Post release version bump
[skip ci]
2022-08-03 08:18:04 +00:00
Simon de Vlieger
f3e2c8d5e9 stage: add an rhsm.facts stage
This stage allows to manifests to define information to be put into a
built image. The information can then later be used by RHSM/Insights.

The facts are stored in `/usr/share/osbuild/rhsm.facts` and then
symlinked into `/etc/rhsm/facts/osbuild.facts`.
2022-08-03 08:54:51 +02:00
schutzbot
5fb7acc53e Post release version bump
[skip ci]
2022-07-27 09:06:45 +00:00
David Rheinsberg
376cbffd13 schemas/osbuild2: mark version as required
The v2 manifest requires the `version` key to be present to be
distinguishable from v1. While technically the manifest can be used
standalong without a `version` key, it does prevent us implementing the
manifest correctly in osbuild (i.e., we are unable to process a v2
manifest without the `version` key, because we are unable to autodetect
it then).

Mark the key as required. It does kind of break backwards compatibility
of the schema, but at the same time we always treated it this way,
anyway. So this should be fine.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-07-25 16:01:48 +02:00
David Rheinsberg
10f076f34d schemas/osbuild2: drop trailing spaces
Drop trailing spaces from the schema.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-07-25 16:01:48 +02:00
David Rheinsberg
38662b22a6 meta: fix lookup of invalid schemata
We need to initialize `schema` to `None`, otherwise it will be an access
to an uninitialized variable when looking up invalid schemata:

    [...]
      File "[...]/osbuild/meta.py", line 583, in get_schema
        schema = Schema(schema, name or klass)
    UnboundLocalError: local variable 'schema' referenced before assignment

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-07-25 16:01:48 +02:00
David Rheinsberg
4a40b732ea meta: fix type annotation of schema input
The schema input of Schema.__init__ is a python-native representation
of a JSON object, so it can be any kind of dictionary. Furthermore, it
is optional.

Fix the type to be Optional[Dict].

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-07-25 16:01:48 +02:00
David Rheinsberg
b41e591b02 docs/osbuild-manifest.5: fix typo
Fix a missing space in:

    `whichproduce` -> `which produce`

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-07-25 16:01:48 +02:00
Alexander Todorov
4911ca5b65 tests: Update OSBUILD_COMPOSER_COMMIT for testing on Fedora 36 2022-07-25 15:58:43 +02:00
Alexander Todorov
40b76a6bb2 COMPOSER-1622: Enable testing on Fedora 36 2022-07-25 15:58:43 +02:00
Christian Kellner
9956f54c97 stages/containers.storage.conf: support pytoml
RHEL 8 only has the old pytoml library, so we need to support that
as well. Try falling back if importing `toml` fails.
2022-07-23 11:12:15 +02:00
Christian Kellner
184ed2d71b stages/containers.storage.conf: fix filename prop
The schema said `filename` but we were accessing `location`. Ups.
2022-07-22 20:02:24 +02:00
Christian Kellner
6fee1051c7 stages/container.storage.conf: restrict locations
Only allow the two known working locations as values for `filename`.
2022-07-22 20:02:24 +02:00
David Rheinsberg
d77e87f3c1 mounts/fat: Xfs -> Fat
Fix the wrong symbol prefixes for the Fat-Mounter. Looks like a
copy-paste from the Xfs-mounter.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-07-22 18:06:24 +02:00
David Rheinsberg
2544486332 mounts/noop: drop wrong host-command reference
The noop mount does not use FileSystemMountService, and as such never
calls into `mount(1)`. Drop this information from the documentation.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-07-22 18:06:24 +02:00
Ondřej Budai
092f886904 stages/users: support a dot inside a username
From IEEE Std 1003.1-2017:

 3.437 User Name:
A string that is used to identify a user; see also User Database. To be
portable across systems conforming to POSIX.1-2017, the value is composed
of characters from the portable filename character set. The <hyphen-minus>
character should not be used as the first character of a portable user name.

 3.282 Portable Filename Character Set
The set of characters from which portable filenames are constructed.

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
a b c d e f g h i j k l m n o p q r s t u v w x y z
0 1 2 3 4 5 6 7 8 9 . _ -

The last three characters are the <period>, <underscore>, and <hyphen-minus>
characters, respectively.

---

The regex checking the username in org.osbuild.users stage omitted the
<period> character before. This commit adds it and modifies the stage
test in order to cover this change.

Fixes: rhbz#2104464
2022-07-21 14:52:57 +02:00
schutzbot
7dcc7d4499 Post release version bump
[skip ci]
2022-07-20 08:17:54 +00:00
Christian Kellner
51315a985a stages/skopeo: use extra intermediate download dir
Instead of downloading the image directly to the temporary directory
and then moving that temporary directory into the cache use one more
intermediate directory and move that into the cache. The reason is
that on Python 3.6 removing the temporary directory itself will make
Python crash like this:

Python 3.6.8 (default, Sep  9 2021, 07:49:02)
[GCC 8.5.0 20210514 (Red Hat 8.5.0-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tempfile
>>> with tempfile.TemporaryDirectory(prefix="tmp-download-") as tmpdir:
...     import os
...     os.rename(tmpdir, "/tmp/foo")

Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
  File "/usr/lib64/python3.6/tempfile.py", line 809, in __exit__
    self.cleanup()
  File "/usr/lib64/python3.6/tempfile.py", line 813, in cleanup
    _shutil.rmtree(self.name)
  File "/usr/lib64/python3.6/shutil.py", line 477, in rmtree
    onerror(os.lstat, path, sys.exc_info())
  File "/usr/lib64/python3.6/shutil.py", line 475, in rmtree
    orig_st = os.lstat(path)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp-download-adl86mwa'
2022-07-19 19:52:25 +02:00
Jakub Rusz
7cd4b4ea66 tests: run tests on RHEL-9.1 and 8.7 nightly 2022-07-16 18:59:06 +02:00
Christian Kellner
4647140808 source/skopeo: use subprocess.check_output
Use `subprocess.check_output` instead of `run(..., capture_output=True)`
since the latter only got added in Python 3.7 and our codebase needs to
be compatible with 3.6 due to RHEL 8.x.
2022-07-13 20:06:42 +02:00
Simon de Vlieger
3fd864e5a9 osbuild: fix optional-types
Optional types were provided in places but were not always correct. Add
mypy checking and fix those that fail(ed).
2022-07-13 17:31:37 +02:00
Christian Kellner
6e66c69608 stages: add org.osbuild.containers.storage.conf
New stage to  create or modify `containers-storage.conf` config
files. The currently implement strategy is to merge the given
and existing configuration options.
2022-07-11 16:25:56 +02:00
Christian Kellner
07a45ba34c test/data: include python3-toml in v2 build root
Prepares for testing stages that use python3-toml to edit config
files.
2022-07-11 16:25:56 +02:00
David Rheinsberg
2c1f61f9f2 docs: update osbuild(1)
Update osbuild(1) to reflect changes of the last year. In particular,
we want to document behavior of the v2-manifest and only silently
continue to support v1-manifests.

Lets also introduce new chapters on output control, caching support,
and monitoring.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-07-08 16:42:51 +02:00
David Rheinsberg
112bf3da51 osbuild: align --help output
Make sure all --help output is consistent. In this particular case,
each line should consistently start with a lower-case character and
avoid a leading `the`.

Signed-off-by: David Rheinsberg <david.rheinsberg@gmail.com>
2022-07-08 16:42:51 +02:00
schutzbot
b06c2311fd Post release version bump
[skip ci]
2022-07-06 08:26:42 +00:00
Gianluca Zuccarelli
c379dccdf1 test/data: update oscap.remediation tests
The oscap.remediation stage now runs the
remediation during build time rather than
creating the necessary files to run the
remediation on first-boot. The tests have
been updated to reflect these changes.
2022-07-06 09:59:00 +02:00
Gianluca Zuccarelli
852fad9fcb oscap: image build remediation
Refactor the oscap remediation stage to
scan and run remediatoin on the image tree
rather than running the remediation at first
boot.
2022-07-06 09:59:00 +02:00
Simon de Vlieger
5c25f17ab7 main: add a --version argument
This adds a `osbuild --version` command that prints the current osbuild
version in use. Allows users to confirm their osbuild is up to date
enough to use newer features.
2022-07-06 09:57:24 +02:00
Simon de Vlieger
ace6c3524b ci: push tags to gitlab
Tags are missing from GitLab (I pushed them once manually) while they
should really be there to be a full mirror.
2022-06-30 10:50:10 +02:00
Christian Kellner
1235c62f8b stages/dnf.config: extend documentation
Add a few more words about the stage.
2022-06-30 09:37:27 +01:00
Christian Kellner
4c12967c0d stages/dnf.config: add support for tsflags
Add support for specifying rpm transaction flags via the `tsflags`
options. See `dnf.conf(5)` for more details.
2022-06-30 09:37:27 +01:00
Christian Kellner
c1c6ab6493 stages/dnf.config: fix schema for config.main
The properties for `main` were not properly defined and but also
not really checked since `additionalProperties` was not defined.
2022-06-30 09:37:27 +01:00
Christian Kellner
57b320f04f stages/dnf.config: specify encondig for open
This is a pyling warning `W1514` "using open without explicitly
specifying an encoding" in newer version, so fix this.
2022-06-30 09:37:27 +01:00
Simon de Vlieger
dd2d4c8708 git: ignore common virtual env locations 2022-06-30 10:34:27 +02:00
Christian Kellner
f078420879 manifests/fedora-container: persist _install_langs
Persist the `_install_langs` via the new `org.osbuild.rpm.macros`
stage.
2022-06-30 00:17:23 +01:00
Christian Kellner
10b8c0dcdc stages/rpm.macros: new stage to set rpm macros
This is to persistently set specific RPM macros. Currently only
`_install_langs` is supported.
2022-06-30 00:17:23 +01:00
Christian Kellner
5fbfd710e4 manifests/fedora-container: only install en locale
This will shrink the container by around 13MB.
2022-06-30 00:17:23 +01:00
Christian Kellner
34ebe60bf0 stages/rpm: new option to set _install_langs
RPM supports installing only specific locales by setting the
`_install_langs` macro.
2022-06-30 00:17:23 +01:00
Christian Kellner
a6e7070b13 stages/oci-archive: make config optional in code
According to our schema `config` is optional but the code actually
accessed it unconditionally; fix that.
2022-06-30 00:03:49 +02:00
Christian Kellner
b761188b98 stages/oci-archive: fix comment to use single # 2022-06-30 00:03:49 +02:00