Commit graph

572 commits

Author SHA1 Message Date
Simon de Vlieger
ea6085fae6 osbuild: run isort on all files 2022-09-12 13:32:51 +02:00
Simon de Vlieger
98daf1a87d osbuild: setup isort autochecking
The `test-src` makefile target now also runs isort against all source
files and reports errors.
2022-09-12 13:32:51 +02:00
Simon de Vlieger
a5be1cc4d2 linting: use-implicit-booleaness-not-comparison
Newer warning from pylint, also consistent with how we do things
elsewhere. Note that this only applies to one file in the tests but
disabling it would be very weird for such a small fix.
2022-09-12 10:52:09 +02:00
Simon de Vlieger
5cb2da55f1 test: run mypy in test-src not in GH actions 2022-09-09 18:51:57 +02:00
Simon de Vlieger
e330e82cf0 lint: appease autopep8 2022-09-09 15:33:29 +02:00
Simon de Vlieger
38d2ab685c test: explicit encodings for open() 2022-09-09 15:33:29 +02:00
Christian Kellner
07d3605385 manifests: set db path in ostree commit
rpm-ostree is setting the database path to /usr/share/rpm during
the build. Do so as well.
2022-08-15 18:22:59 +02:00
Christian Kellner
c28269f8d9 manifests: enable cloud-init services in commit
It is instabled, but not a enabled via presets so we have to
explicitly enable it.
2022-08-13 19:21:52 +01:00
Christian Kellner
5570e2a92e manifests: upgrade ostree commit, image to F36
Fedora 34 is end of life and we ought to be testing with newer
releases so that we catch issues like the authselect one, that
is detailed in "stages/rpm: support marking install as ostree".
2022-08-13 19:21:52 +01:00
Christian Kellner
899fdcd5d9 manifests: set rpm/ostree_booted for commit
See "stages/rpm: support marking install as ostree" for more
information why this is needed on Fedora 36 and newer.
2022-08-13 19:21:52 +01:00
Christian Kellner
10255dbdcc test/manifests: base ostree image on commit
Instead of re-defining the ostree-commit in the ostree-image.mpp
file, import the needed pipelines to build the commit and use
that.
2022-08-13 19:21:52 +01:00
Christian Kellner
d2bcaffbdc test/manifests: add update repos in ostree-commit
Include modular and updates repositories for ostree commit and the
new build manifest.
2022-08-13 19:21:52 +01:00
Christian Kellner
073359166d test/manifests: add generic fedora build manifest
Basically a copy of f34-build-v2 but uses variables. Use it form
fedora-ostree-commit.
2022-08-13 19:21:52 +01:00
Christian Kellner
056a33041b manifests: use variables in ostree-commit
Instead of hard-coding the release and the snapshot, use variables,
which should help updating to newer releases and snapshots.
2022-08-13 19:21:52 +01:00
Christian Kellner
4b11989ce3 manifests: convert ostree commit to format v2
Convert the manifest to use version 2 format. Version 1 is really
not used much anymore because composer was completely ported to
using v2. Welcome to the future, ostree commit.
2022-08-13 19:21:52 +01:00
Christian Kellner
c3b172e16b manifests/ostree-commit: update package list
Include cryptsetup and lvm2 so that commit can be deployed to
images that are encrypted and have lvm2 layout. Additionally,
include greenboot. Remove the explicit dependency on ostree
and systemd-udev, which are automatically pulled in.
2022-08-13 19:21:52 +01:00
Christian Kellner
5c8d11ebe1 test/pylint: check error code instead of exception
Instead of using `subprocess` with `check=True`, which will
echo the command used, including all the files it checked,
check the error code and if non-zero, fail the test with a
nice small error message.
2022-08-05 09:41:05 +02:00
Christian Kellner
cae23de605 test/pylint: add autopep8 test
Add a test to ensure our source code is PEP-8 compliant.
2022-08-05 09:41:05 +02:00
Christian Kellner
f05078f66e global: fix PEP-8 formatting
This patch was generated by running `autopep8 --diff` on the
source tree and then applying the diff.
2022-08-05 09:41:05 +02:00
Christian Kellner
bb644cdd07 test: add have_autopep8 method
Add a helper to check if the `autopep8` command is available.
2022-08-05 09:41:05 +02:00
Christian Kellner
620f7c0f3e test/pylint: extract source files into fixture
Extract the code that discovers the source files into a fixture;
this is done so that we can share that other future tests.
2022-08-05 09:41:05 +02:00
Christian Kellner
bf2789fd91 test/pylint: convert to pytest
Convert the test to using pytest instead of unittest.
2022-08-05 09:41:05 +02:00
fkolwa
2bca7b07f3 stages: add instance_configs.cfg.distro
test/data: add instance_configs.cfg.distro test
2022-08-03 15:39:35 +02:00
Simon de Vlieger
f3e2c8d5e9 stage: add an rhsm.facts stage
This stage allows to manifests to define information to be put into a
built image. The information can then later be used by RHSM/Insights.

The facts are stored in `/usr/share/osbuild/rhsm.facts` and then
symlinked into `/etc/rhsm/facts/osbuild.facts`.
2022-08-03 08:54:51 +02:00
Ondřej Budai
092f886904 stages/users: support a dot inside a username
From IEEE Std 1003.1-2017:

 3.437 User Name:
A string that is used to identify a user; see also User Database. To be
portable across systems conforming to POSIX.1-2017, the value is composed
of characters from the portable filename character set. The <hyphen-minus>
character should not be used as the first character of a portable user name.

 3.282 Portable Filename Character Set
The set of characters from which portable filenames are constructed.

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
a b c d e f g h i j k l m n o p q r s t u v w x y z
0 1 2 3 4 5 6 7 8 9 . _ -

The last three characters are the <period>, <underscore>, and <hyphen-minus>
characters, respectively.

---

The regex checking the username in org.osbuild.users stage omitted the
<period> character before. This commit adds it and modifies the stage
test in order to cover this change.

Fixes: rhbz#2104464
2022-07-21 14:52:57 +02:00
Christian Kellner
6e66c69608 stages: add org.osbuild.containers.storage.conf
New stage to  create or modify `containers-storage.conf` config
files. The currently implement strategy is to merge the given
and existing configuration options.
2022-07-11 16:25:56 +02:00
Christian Kellner
07a45ba34c test/data: include python3-toml in v2 build root
Prepares for testing stages that use python3-toml to edit config
files.
2022-07-11 16:25:56 +02:00
Gianluca Zuccarelli
c379dccdf1 test/data: update oscap.remediation tests
The oscap.remediation stage now runs the
remediation during build time rather than
creating the necessary files to run the
remediation on first-boot. The tests have
been updated to reflect these changes.
2022-07-06 09:59:00 +02:00
Christian Kellner
4c12967c0d stages/dnf.config: add support for tsflags
Add support for specifying rpm transaction flags via the `tsflags`
options. See `dnf.conf(5)` for more details.
2022-06-30 09:37:27 +01:00
Christian Kellner
f078420879 manifests/fedora-container: persist _install_langs
Persist the `_install_langs` via the new `org.osbuild.rpm.macros`
stage.
2022-06-30 00:17:23 +01:00
Christian Kellner
10b8c0dcdc stages/rpm.macros: new stage to set rpm macros
This is to persistently set specific RPM macros. Currently only
`_install_langs` is supported.
2022-06-30 00:17:23 +01:00
Christian Kellner
5fbfd710e4 manifests/fedora-container: only install en locale
This will shrink the container by around 13MB.
2022-06-30 00:17:23 +01:00
Christian Kellner
4d39f9ec2b stages: add new org.osbuild.rpmkeys.import stage
Add a new stage to import public keys into the RPM database via the
`rpmkeys` command. This is similar as to what the `org.osbuild.rpm`
stage already does but it uses inputs instead of inline data.
2022-06-25 13:19:11 +01:00
Jelle van der Waa
923cc60fdb stages: add vagrant stage for libvirt
This commit introduces a new vagrant stage and initially only supports
the libvirt backend. The vagrant stage takes a disk image as input
generates a Vagrantfile and metadata.json.
2022-06-23 11:06:12 +02:00
Christian Kellner
e0db89284d tests/objectstore: remove "duplicate" test case
The idea of this test case was to check that two identical trees are
only stored once, via their treesum in the object store; but this
functionality was removed in commit e97f6ef34 and instead of treesums
random uuids are now used. As a result there is no de-duplication
anymore -- the subject of the test. So remove the test.
2022-06-21 15:08:32 +02:00
Christian Kellner
86123da599 stages/qemu: expose vpc options
Currently we hard code the vpc options `subformat=fixed` and
`force_size`, which are needed to generate valid azure images
with newer versions of qemu. But for other use cases or other
versions of qemu these options might not be wanted or valid.
Expose all the options but with defaults corresponding to the
old behavior.
Add a unit test for the `force_size` option to check its
effect. Also add a check for the correct size to the existing, 
default value (i.e. `force_size` being `true`).
2022-06-15 13:41:11 +02:00
Gianluca Zuccarelli
4ede85f176 test/data: add oscap.remediation tests
Add new tests for the `oscap.remediation`
stage.
2022-05-31 01:32:49 +02:00
Simon de Vlieger
ce9253788e tests: handle - in the sfdisk version test
When a `-` is in the version (meaning a version such as: `2.38-rc1`),
take only the part before the dash.

This closes #1036.
2022-05-30 16:04:03 +02:00
Christian Kellner
16b187f5d9 test: add grub2.legacy test
Add a basic check for the org.osbuild.grub2.legacy stage.
2022-05-29 14:02:16 +02:00
Christian Kellner
e0fbbdaa5e stages: add new sgdisk stage
Can be used to create partition tables via GPT laypout via `sgdisk(8)`.
The schema of `partitions` is intentionally kept identical to the one
in `org.osbuild.sfdisk`.
Add corresponding tests.
2022-05-23 13:00:44 +02:00
Christian Kellner
5edb54bc2d test/data: add gdisk to fedora 34 v2 build root
Will be use in later test. Separated as its own commit for clarity.
2022-05-23 13:00:44 +02:00
Thomas Lavocat
441e67a6f6 ostree: show commit metadata
This new API call allows one to check (among other things) if a commit
exists in a repo. It'll throw a RuntimeException if the commit is
missing.
2022-05-11 04:32:42 -05:00
Christian Kellner
f2aa688d3e test/monitor: properly initialize output
It was not initialized in `__init__`, do so.
2022-05-06 17:33:23 +02:00
Christian Kellner
d04ce2e1ef stages: add new org.osbuild.udev.rules stage
Add new stage to create udev rules files; add a simple stage test
to check for basic functionality.
2022-05-06 12:46:03 +02:00
Christian Kellner
1e4507c3d6 util/ostree: new class to store subordinate ids
Add a new class `SubIdsDB` as a database of subordinate Ids, like the
ones in `/etc/subuid` and `/etc/subgid`. Methods to read and write
data from these two files are provided.
Add corresponding unit tests.
2022-04-28 14:38:24 +01:00
Christian Kellner
4ac62abbc3 buildroot: ability to drop capabilities
Add a new member variable `caps` that if not `None` indicates the
capabilities to retain, i.e. all other capabilities not specified
will be dropped via `bubblewrap` (`--cap-drop`).
Add corresponding tests.
2022-04-27 23:05:11 +01:00
Christian Kellner
1874c71920 util/linux: add capability utilities 2022-04-27 23:05:11 +01:00
Christian Kellner
46fd8958bb test/util: convert util_linux to pytest
Convert the test from `unittest` to `pytest`. No semantic change.
2022-04-27 23:05:11 +01:00
Christian Kellner
4055774c7b test: remove old pipelines directory
Left over from the conversion of pipeline tests to test/data/manifests
and MPP via commits 7dcc946 and 020e78d.
2022-04-27 23:01:26 +01:00
Christian Kellner
99abc1373d inputs: support array of objects references
This extends the possible ways of passing references to inputs. The
current ways possible are:
 1) "plain references", an array of strings:
    ["ref1", "ref2", ...]
 2) "object references", a mapping of keys to objects:
    {"ref1": { <options> }, "ref2": { <options> }, ...}

This patch adds a new way:
  3) "array of object references":
    [{"id": "ref1", "options": { ... }}, {"id": ... }, ]

While osbuild promises to preserves the order for "object references"
not all JSON serialization libraries preserve the order since the
JSON specification does leave this up to the implementation.

The new "array of object references" thus allows for specifying the
references together with reference specific options and this in a
specific order.

Additionally this paves the way for specifying the same input twice,
e.g. in the case of the `org.osbuild.files` input where a pipeline
could then be specified twice with different files. This needs core
rework though, since internally we use dictionaries right now.
2022-04-21 16:39:58 +02:00