Commit graph

1766 commits

Author SHA1 Message Date
Jakub Rusz
8d67396ff3 tests: enable testing on aarch64 RHEL
Also update osbuild-composer version.

debug: skip openstack boot on aarch64
2021-11-29 14:46:09 +01:00
Jelle van der Waa
1b797dfcff gitignore: Ignore generated man pages 2021-11-26 19:39:42 +00:00
Christian Kellner
0fec32e079 stages/ostree.fillvar: fill the correct var
The idea behind the stage is to provision `var` of the stateroot,
i.e. the `var` the is shared amongst all deployments for a given
os (indicated by `osname`, e.g. `fedora`, `centos`, ...).
For `systemd-tmpfiles` to infer the correct paths, it needs to be
run on the deployment. The `var` of the latter needs to be bind-
mounted to the `var` of the stateroot, because it is shared. This
was always the intention but not what the code did. Fix this by
getting the `var` of the stateroot and bind it to the `var` of
the deployment.
NB: In reality this never mattered since systemd-tmpfiles is also
run during system startup.
2021-11-26 17:20:11 +01:00
Alexander Larsson
eefc52333e osbuild-mpp: Allow mpp-define-image nodes to use formating
We just process the formats after removing the mpp node, just like
we do in e.g. mpp-depsolv.
2021-11-26 11:51:42 +00:00
Alexander Larsson
4901e97236 osbuild-mpp: Allow included files to define partitions too 2021-11-26 11:51:42 +00:00
Alexander Larsson
ba8bc3a60a osbuild-mpp: Support nested includes
This moves the handling of includes to the manifest loader, thus
supporting nested includes. search_dirs is moved to a property of the
Manifest so that it can be tracked during loads.

In addition we need to fix Manifest.path to the actual path that was
loaded instead of whatever the parent include said, so that relative
includes are handled from the proper location of the loaded manifest.
2021-11-26 11:51:42 +00:00
Alexander Larsson
12335f9e5f osbuild-mpp: Be more careful about keys existing
In process_embed_files(), it assumed that a stage had a "type" field,
which breaks if a stage is e.g. a `mpp-if` node, so use .get() instead
of raw dict lookups.
2021-11-26 11:51:42 +00:00
Alexander Larsson
bfa619f1a2 osbuild-mpp: Fix default value for ManifestV2.pipelines
This should be [], not {}. This  was probably from copying the V1
value which is {} (for `pipeline`).
2021-11-26 11:51:42 +00:00
Alexander Larsson
29ecb1df67 osbuild-mpp: Support mpp-pipelines to include all or a subset of the pipelines
This makes a lot of sense, as some include could be built in a way the
includer doesn't know about that requires sub-pipelines.
2021-11-26 11:51:42 +00:00
Jakub Rusz
180e853c23 mockbuild: remove beta from rhel-9 template
Beta is out already, switch template to RHEL-9.
2021-11-24 13:26:54 +01:00
Jakub Rusz
de94251a32 ci: switch to rhel-9 runners
RHEL-9-Beta is no longer being developed. Switching to RHEL-9 instead.
Currently testing is not run there because of issues linked in the
comment.
2021-11-24 13:26:54 +01:00
Christian Kellner
d2d3d6d59e spec: require python3-pyyaml for osbuild-tools
osbuild-mpp has learned to read yaml files but with it gained a
hard dependency on python3-pyyaml. Specify so in the spec file.
2021-11-19 21:14:06 +01:00
Alexander Larsson
52c3cb12ba osbuild-mpp: Support loading yaml as well as json files
yaml files are essentially compatible with json files, although
they have some advantages, like allowing comments and being easier
for humans to read/write.

This changes the reading of the file to use a yaml parser instead of
a json parser, but still produces json at the end. I tried manually
converting a json file to yaml and running osbuild-mpp, and it produced
an identical file.

By default the yaml parser doesn't respect order so i had to tweak
the loader a bit to use OrderedDict.
2021-11-19 13:35:12 +01:00
Christian Kellner
04423e8a6a ci: update to latest osbuild ci image
Needed to get `cryptsetup` package needed for testing the luks2
related stages and devices.
2021-11-19 00:19:05 +00:00
Christian Kellner
bf655b9464 setup: disable new pylint warnings
For now disable some new warnings from pylint:
    - `consider-using-from-import`
    - `consider-using-with`
It probably makes sense to re-enable these in the future but
for now lets keep the code as is.
2021-11-19 00:19:05 +00:00
Christian Kellner
03232f1899 test/stages: basic gunzip stage test
Add a simple check for the new `org.osbuild.gunzip` stage.
2021-11-19 00:16:04 +00:00
Alexander Larsson
079598fa4f osbuild: Add org.osbuild.gunzip stage
This stage the same args and formats as org.osbuild.untar (and as such
much code is just copied from that stage), except it runs gunzip
instead.  I need this to uncompress the aarch64 kernel when directly
uefi-booting it.
2021-11-19 00:16:04 +00:00
Alexander Larsson
10333fa8ba osbuild-mpp: Report the path of the json file when parsing fails
This makes it a lot easier to understand failures when files are
including other files.
2021-11-18 21:00:13 +01:00
Ondřej Budai
d1ad772284 packit: enable copr builds for CS9 on main
I forgot to enable CS9 builds on main previously, let's fix that.
2021-11-18 19:46:46 +01:00
Christian Kellner
ac529563c3 test: use new mpp-embed in tests
This is to showcase it as much as to test its functionality. For this
the tar and xz stage tests have been converted. NB: only the mpp file
for each test is changed but the corresponding manifest is not.
The `sources/org.osbuild.inline` section has been kept otherwise the
ordering in the result manifest would change.
2021-11-18 13:41:06 +01:00
Christian Kellner
15759dd205 tools/osbuild-mpp: add ability to embed data
Add a new pre-processor directive to MPP that allows to define
`org.osbuild.inline` sources on the fly by embedding data or
files into the manifest.
2021-11-18 13:41:06 +01:00
Christian Kellner
22e670050f osbuild-mpp: extract finding files and opening
Extract the code that finds a file and opens it from the existing
method that find manifests and opens them. This is so that the
former code can be re-used.
2021-11-18 13:41:06 +01:00
Alexander Larsson
ffdbdf6235 osbuild-mpp: Support mpp-if
This allows optional manifest json values like "if blah, add stage foo",
or selecting between possible nodes, "if blah, stage is foo, else bar".
2021-11-18 11:33:57 +01:00
Alexander Larsson
04d880852a osbuild-mpp: Break out helper function
This breaks out the fakeroot handling used to rewrite a single node.
We'll use this in some new cases later.
2021-11-18 11:33:57 +01:00
kingsleyzissou
7fe8b922b4 Post release version bump
[skip ci]
2021-11-17 10:53:44 +00:00
Ondřej Budai
332e83b5cc packit: enable copr build for CS9 2021-11-15 18:40:00 +00:00
Christian Kellner
280aca8f07 test/lvm2: use LoopControl.loop_for_fd
Instead of having custom code that basically duplicates the
functionality of `LoopControl.loop_for_fd` use that instead.
Additionally, the version used in the test had a bug where
it did not re-create the Loop device in the main loop when
it was close due to an error, leading errors in subsequent
usages of the device that would often manifest in CI runs:
  fcntl.ioctl(self.fd, self.LOOP_SET_FD, fd)
  ValueError: file descriptor cannot be a negative integer (-1)
2021-11-12 17:40:00 +01:00
Simon Steinbeiss
2e9ffaf517 packit: Use upstream github release description
Setting this option enables packit to use the Github upstream release
description for the specfile instead of just using all commit messages.
2021-11-11 00:54:33 +00:00
ochosi
40bc216127 Post release version bump 2021-11-08 20:40:13 +00:00
Tom Gundersen
f7bf23faba stages/authconfig: run authconfig
This applies the default authconfig settings to the tree.

Note that the `/backups` directory is removed. The tool creaset
this, and by default it should not exist, so this should be a
noop. However, if you run this on a tree with existing backups,
they would be lost.
2021-11-08 20:43:51 +01:00
Ondřej Budai
306fd3ed96 stages/yum.config: add an option to configure langpacks plugin
The new stage enables users to configure the langpacks plugin of YUM.
Currently, only locales option is supported.
2021-11-08 17:17:02 +01:00
Christian Kellner
e63fa48504 formats/v2: fix describe for mount without source
Commit 5b1cd2b made `source` and `target` for mounts optional, but
the corresponding code in `describe` still assumes that the device
will always be present. Fix this so that source will only be used
if it is set.
2021-11-08 11:26:37 +00:00
Christian Kellner
52cb27631b stages/selinux: ability to force an auto-relabel
This should not be needed in any case but can be a sledgehammer
for situations where we cannot properly label a file; it turns
out such a scenario is if a label, lets call it `a1`, is is an
alias to another label, lets call it `l1`. Setting `a1` will
lead to `l1` being read back, and thus copying the label `a1`
will result on the label `l1` being copied instead. Now if the
target distribution does not have `l1` but only has `a1` we
cannot set it and thus will end up with an unlabeled file.
2021-11-05 16:06:13 +00:00
Tom Gundersen
9da89de8b5 stages/pwquality.conf: set pwquality configuration
This allows the password policy to be configured.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-11-05 16:53:54 +01:00
Christian Kellner
5301270bff stages/rhsm: add support to configure yum plugins
Adds support to configure `yum-plugins`, which currently is a full
alias for `dnf-plugins`, although this might change in the future,
in case dnf options diverge from yum. It allows for both yum and
dnf plugins to be configured at the same time since on RHEL 7 both
files will be present.
2021-11-05 15:56:54 +01:00
Christian Kellner
0e9fcad8fc stages/rhsm: extract plugins defintion
This is so that it can be reused by a new `yum-plugins` section.
2021-11-05 15:56:54 +01:00
Tomas Hozza
58ec1c9a83 Add new org.osbuild.yum.config stage
Add a new stage for modifying YUM global configuration.

Add a unit test case for the newly added stage.

Because we test stages on Fedora, where there is no YUM, and this stage
is mostly intended for being used with RHEL-7 images, the stage does not
produce error in case the `/etc/yum.conf` file does not exist. It rather
produces a warning and creates the file. Ideally the stage would produce
an error in case the configuration file does not exist, but that would
be impossible to test on recent Fedora.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-11-05 12:10:33 +01:00
Achilleas Koutsou
66a1fbad9d test/cloud-init: add new options to stage test 2021-11-05 12:08:38 +01:00
Achilleas Koutsou
58708c29d2 stages/cloud-init: disable default_flow_style when writing configs 2021-11-05 12:08:38 +01:00
Achilleas Koutsou
9b5f13fc6d stages/cloud-init: add support for configuring output logging 2021-11-05 12:08:38 +01:00
Achilleas Koutsou
4ac004e178 stages/cloud-init: add support for configuring reporting handlers
It's possible the keys "logging" and "telemetry" can be arbitrary names.
If that's the case, we can change the schema without breaking backwards
compatibility, so defining known keys is safer.
2021-11-05 12:08:38 +01:00
Achilleas Koutsou
3ee19645a3 stages/cloud-init: add support for configuring Azure datasource
Datasources are defined in an array to preserve ordering and are then
converted to objects.
2021-11-05 12:08:38 +01:00
Christian Kellner
12eb14378a stages: add new org.osbuild.cron.script stage
New stage to schedule a script via a /etc/cron.{hourly, daily ...}.
Currently only a simple command that will be put into an generated
script with the given name. Later more options might be added.
2021-11-05 11:53:11 +01:00
Ondřej Budai
266c3d1953 stages/grub2: add support for terminal, serial and timeout config
This commit adds options to the org.osbuild.grub2 stage to configure
terminal input and output, serial console and timeout.

The functionality and configuration schema is the same as in the legacy
grub2 stage.
2021-11-04 17:35:24 +01:00
Tom Gundersen
36176ab377 stages/waagent.conf: set WALinuxAgent configuration
This is requried to comply with Azure marketplace best
practices. The WALinuxAgent should not handle formating or
swap, as that is done by cloud-init.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-11-03 18:32:40 +01:00
Tom Gundersen
304f1e3f9f stages/sshd.config: set sshd configuration
This is required for images to be importable to the AWS
marketplace. Both PasswordAuthentication and
ChallengeResponseAuthentication must be disabled.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-11-03 18:32:40 +01:00
Tomas Hozza
daacf002fe Support 'install' command in org.osbuild.modprobe stage
Add support for new 'install' command in the org.osbuild.modprobe stage.

Extend the unit test coverage to test the new command.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-11-03 16:11:54 +00:00
msehnout
5df59a2251 Post release version bump 2021-11-03 15:19:47 +00:00
Christian Kellner
bd4ab1d36f stages/lvm2.create: fix 'size' and add 'extents'
The `size` option was using `-l`, which in fact was `--extents`.
Fix that to use `--size` and add a new option `extents` that
will in fact call translate to `--extents` and this replace the
current use of the `size` option.
Adapt the `fedora-ostree-image` test manifest to use `extents`.
2021-11-03 15:09:35 +00:00
Simon Steinbeiss
493b1f9a1b Let schutzbot do the post-release version bump
We need a privileged / admin user doing the post-release version bump as
this is a direct commit to main (i.e. without a PR) so switch to using
schutzbot with a scoped personal access token (only public_repo).
2021-11-01 21:58:35 +01:00