Commit graph

1743 commits

Author SHA1 Message Date
Alexander Larsson
04d880852a osbuild-mpp: Break out helper function
This breaks out the fakeroot handling used to rewrite a single node.
We'll use this in some new cases later.
2021-11-18 11:33:57 +01:00
kingsleyzissou
7fe8b922b4 Post release version bump
[skip ci]
2021-11-17 10:53:44 +00:00
Ondřej Budai
332e83b5cc packit: enable copr build for CS9 2021-11-15 18:40:00 +00:00
Christian Kellner
280aca8f07 test/lvm2: use LoopControl.loop_for_fd
Instead of having custom code that basically duplicates the
functionality of `LoopControl.loop_for_fd` use that instead.
Additionally, the version used in the test had a bug where
it did not re-create the Loop device in the main loop when
it was close due to an error, leading errors in subsequent
usages of the device that would often manifest in CI runs:
  fcntl.ioctl(self.fd, self.LOOP_SET_FD, fd)
  ValueError: file descriptor cannot be a negative integer (-1)
2021-11-12 17:40:00 +01:00
Simon Steinbeiss
2e9ffaf517 packit: Use upstream github release description
Setting this option enables packit to use the Github upstream release
description for the specfile instead of just using all commit messages.
2021-11-11 00:54:33 +00:00
ochosi
40bc216127 Post release version bump 2021-11-08 20:40:13 +00:00
Tom Gundersen
f7bf23faba stages/authconfig: run authconfig
This applies the default authconfig settings to the tree.

Note that the `/backups` directory is removed. The tool creaset
this, and by default it should not exist, so this should be a
noop. However, if you run this on a tree with existing backups,
they would be lost.
2021-11-08 20:43:51 +01:00
Ondřej Budai
306fd3ed96 stages/yum.config: add an option to configure langpacks plugin
The new stage enables users to configure the langpacks plugin of YUM.
Currently, only locales option is supported.
2021-11-08 17:17:02 +01:00
Christian Kellner
e63fa48504 formats/v2: fix describe for mount without source
Commit 5b1cd2b made `source` and `target` for mounts optional, but
the corresponding code in `describe` still assumes that the device
will always be present. Fix this so that source will only be used
if it is set.
2021-11-08 11:26:37 +00:00
Christian Kellner
52cb27631b stages/selinux: ability to force an auto-relabel
This should not be needed in any case but can be a sledgehammer
for situations where we cannot properly label a file; it turns
out such a scenario is if a label, lets call it `a1`, is is an
alias to another label, lets call it `l1`. Setting `a1` will
lead to `l1` being read back, and thus copying the label `a1`
will result on the label `l1` being copied instead. Now if the
target distribution does not have `l1` but only has `a1` we
cannot set it and thus will end up with an unlabeled file.
2021-11-05 16:06:13 +00:00
Tom Gundersen
9da89de8b5 stages/pwquality.conf: set pwquality configuration
This allows the password policy to be configured.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-11-05 16:53:54 +01:00
Christian Kellner
5301270bff stages/rhsm: add support to configure yum plugins
Adds support to configure `yum-plugins`, which currently is a full
alias for `dnf-plugins`, although this might change in the future,
in case dnf options diverge from yum. It allows for both yum and
dnf plugins to be configured at the same time since on RHEL 7 both
files will be present.
2021-11-05 15:56:54 +01:00
Christian Kellner
0e9fcad8fc stages/rhsm: extract plugins defintion
This is so that it can be reused by a new `yum-plugins` section.
2021-11-05 15:56:54 +01:00
Tomas Hozza
58ec1c9a83 Add new org.osbuild.yum.config stage
Add a new stage for modifying YUM global configuration.

Add a unit test case for the newly added stage.

Because we test stages on Fedora, where there is no YUM, and this stage
is mostly intended for being used with RHEL-7 images, the stage does not
produce error in case the `/etc/yum.conf` file does not exist. It rather
produces a warning and creates the file. Ideally the stage would produce
an error in case the configuration file does not exist, but that would
be impossible to test on recent Fedora.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-11-05 12:10:33 +01:00
Achilleas Koutsou
66a1fbad9d test/cloud-init: add new options to stage test 2021-11-05 12:08:38 +01:00
Achilleas Koutsou
58708c29d2 stages/cloud-init: disable default_flow_style when writing configs 2021-11-05 12:08:38 +01:00
Achilleas Koutsou
9b5f13fc6d stages/cloud-init: add support for configuring output logging 2021-11-05 12:08:38 +01:00
Achilleas Koutsou
4ac004e178 stages/cloud-init: add support for configuring reporting handlers
It's possible the keys "logging" and "telemetry" can be arbitrary names.
If that's the case, we can change the schema without breaking backwards
compatibility, so defining known keys is safer.
2021-11-05 12:08:38 +01:00
Achilleas Koutsou
3ee19645a3 stages/cloud-init: add support for configuring Azure datasource
Datasources are defined in an array to preserve ordering and are then
converted to objects.
2021-11-05 12:08:38 +01:00
Christian Kellner
12eb14378a stages: add new org.osbuild.cron.script stage
New stage to schedule a script via a /etc/cron.{hourly, daily ...}.
Currently only a simple command that will be put into an generated
script with the given name. Later more options might be added.
2021-11-05 11:53:11 +01:00
Ondřej Budai
266c3d1953 stages/grub2: add support for terminal, serial and timeout config
This commit adds options to the org.osbuild.grub2 stage to configure
terminal input and output, serial console and timeout.

The functionality and configuration schema is the same as in the legacy
grub2 stage.
2021-11-04 17:35:24 +01:00
Tom Gundersen
36176ab377 stages/waagent.conf: set WALinuxAgent configuration
This is requried to comply with Azure marketplace best
practices. The WALinuxAgent should not handle formating or
swap, as that is done by cloud-init.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-11-03 18:32:40 +01:00
Tom Gundersen
304f1e3f9f stages/sshd.config: set sshd configuration
This is required for images to be importable to the AWS
marketplace. Both PasswordAuthentication and
ChallengeResponseAuthentication must be disabled.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2021-11-03 18:32:40 +01:00
Tomas Hozza
daacf002fe Support 'install' command in org.osbuild.modprobe stage
Add support for new 'install' command in the org.osbuild.modprobe stage.

Extend the unit test coverage to test the new command.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-11-03 16:11:54 +00:00
msehnout
5df59a2251 Post release version bump 2021-11-03 15:19:47 +00:00
Christian Kellner
bd4ab1d36f stages/lvm2.create: fix 'size' and add 'extents'
The `size` option was using `-l`, which in fact was `--extents`.
Fix that to use `--size` and add a new option `extents` that
will in fact call translate to `--extents` and this replace the
current use of the `size` option.
Adapt the `fedora-ostree-image` test manifest to use `extents`.
2021-11-03 15:09:35 +00:00
Simon Steinbeiss
493b1f9a1b Let schutzbot do the post-release version bump
We need a privileged / admin user doing the post-release version bump as
this is a direct commit to main (i.e. without a PR) so switch to using
schutzbot with a scoped personal access token (only public_repo).
2021-11-01 21:58:35 +01:00
Christian Kellner
5828da8d62 test/data: use ostree.deployment in fedora image 2021-10-30 15:32:44 +01:00
Christian Kellner
8b5fe600c1 mounts: add new ostree.deployment service
Create a new OSTree deployment mount service that will set up bind
mounts inside the tree very much as it is done by OSTree in early
boot. This allows any stage to transparently work with OSTree
deployments.
2021-10-30 15:32:44 +01:00
Christian Kellner
a27b3f78bc mounts: include tree directory in arguments
Include the path to the tree for all mount services, so that future
services can e.g. bind mount something inside the tree.
2021-10-30 15:32:44 +01:00
Christian Kellner
f50146d3c5 mounts: allow empty returns from service
Allow mount services to return None, which means they have not 
actually mounted anything within the mount root. This might be
because they have bind mounted directories within the tree.
These mounts do not need any path translation.
2021-10-30 15:32:44 +01:00
Christian Kellner
08c1fbad4b mounts: separate file system mount service
Separate the current `MountService` into the more generic base mount
service and a specialized one for file systems.
2021-10-30 15:32:44 +01:00
Christian Kellner
7e776a0763 meta: allow mounts for all stages
If a stage has not itself defined the `mounts` property, allow any
mounts. This is in preparation to support specialized mounts, such
as bind mounts or ostree deployment mounts to transparently work
with any stage.
NB: devices are not allowed so this will not be applicable for the
current filesystem mounts.
2021-10-30 15:32:44 +01:00
Christian Kellner
5b1cd2b1c5 schema/v2: make mount source and target optional
The previous commit gave the individual mounts more control over the
source and target properties. Do not require them at the global
schema but hand the control if they are optional over to the modules.
2021-10-30 15:32:44 +01:00
Christian Kellner
02404ced94 mounts: change schema meta information
Define the mount schema in the actual mounts at a higher level. This
is in preparation to give the modules more control over the `source`
and `target` properties.
2021-10-30 15:32:44 +01:00
Christian Kellner
5694743ca6 mounts: introduce new mount manager class
Introduce a new specialized service manager class `MountManager` to
manage mounts. It uses the newly introduced `DeviceManager` to look
up devices and stores the reference to the mount point root path.
See the commit that introduced the `DeviceManager` for more info.
2021-10-30 15:32:44 +01:00
Christian Kellner
2447172125 devices: add device path helper functions
Add new helper functions that can translate from a managed device
to its path. One is relative and one is the absolute path on the
host, i.e. to the device node on the host.
2021-10-30 15:32:44 +01:00
Christian Kellner
a5e07cf506 devices: introduce new device manager class
Introduce a new class to manage devices, `DeviceManger` and move the
code to open devices from the `Device` here. The main insight of why
the logic should be place here is that certain information is needed
to open the devices, independently of specific type: the path to the
device node directory, `devpath`, the actual `tree` and the service
manager instance to start the actual service. Instead of passing all
this information again and again to the `Device` class, we now have
a specialized (service) manager class for devices that has all the
needed information all the time. Additionally, the special handling
of parent devices is moved from the pipeline to the service manager,
which is where it belongs.
This will make even more sense for mounts, where the `DeviceManger`
can then be passed to access the individual devices.
Port the test to use the `DeviceManager`.
2021-10-30 15:32:44 +01:00
Christian Kellner
94d8fb1ff7 test/data: add RHEL 7 manifests
Add RHEL 7.9 example manifests. Add them to a `rhel` sub-directory in
the test/data/manifests directory since we cannot re-generate them
in the normal github actions, because they require access to RHEL
content.
2021-10-30 00:54:09 +01:00
Christian Kellner
074741cc27 test/stages: add check for parted stage
Include a new test that writes a partition table to a disk and
then reads it back via `sfdisk` compares it against an layout
that was generated.
NB: This test needs `sfdisk` with `--json` support on the use host.
2021-10-30 00:54:09 +01:00
Christian Kellner
f951a4931e runners: add rhel7 runner
Create a runner for RHEL7. The one thing to note is that RHEL 7
makes use of ld.so.confd snippets and one important for us is
to include `/usr/lib64/iscsi` needed by qemu-img. Otherwise this
is a fairly simple and straight forward runner.
2021-10-30 00:54:09 +01:00
Christian Kellner
f16b606716 stages/grub2.legacy: new stage for non-bls config
Configure grub2 but instead of using the Bootloader Specification (BLS)
it uses traditional menu entries for the individual boot entries. This
is needed since RHEL7 does not have grub2 with BLS support.
2021-10-30 00:54:09 +01:00
Christian Kellner
da5150084e stages/parted: new stage to partition a device
Like the existing sfdisk stage, the parted stage can be used to create
a partition table on the specified device. In contrast to sfdisk, it
does not support uuids, neither for specifying the partition types,
nor the actual uuid of the partition. The current implementation only
supports GPT.
This stage is meant to be used on older systems, like RHEL 7, where
sfdisk exists but does not support GPT (or --json).
2021-10-30 00:54:09 +01:00
Christian Kellner
4a7e49c20b pipeline: don't bind-mount /boot from the host
When setting up the build root, only bind mount the `/boot` dir
from the supplied build tree, if the build tree is not the host
itself, since we never want to leak any host specific data and
the `/boot` directory should never be needed when building the
build root. The only reason `/boot` is mounted at all is for
the grub2 stage to copy efi binaries to the tree since they
directly installed to `/boot` by the respective bootloader
packages.
2021-10-30 00:16:03 +01:00
Christian Kellner
ccb26806fc buildroot: make mounting /boot optional
Currently, we take to paths from the root file system supplied
to the `BuildRoot` class: `/boot` and `/usr`. The reason for
mounting `/boot` is that grub2 and shim install efi binaries
there and for certain images we want to copy the binaries from
the build root and not install the respective packages.
However, if we build to build root itself, we probably don't
want the mount the hosts' `/boot` since we don't want to copy
anything from there. This change should give us the ability to
do exactly that.
2021-10-30 00:16:03 +01:00
Christian Kellner
d8a8dc84d6 setup.cfg: increase max-statements to 75
The default is 50, which we hit in a few places already. Let's do
75, equally arbitrary but gives us a bit more wiggle room.
2021-10-30 00:16:03 +01:00
Christian Kellner
de7fd7b402 runners: add new centos9 runner
Actually, rename the rhel90 runner to the centos9 runner, and
make the former a link to the latter, since in RHEL 9, CentOS
is the upstream and RHEL the downstream.
2021-10-30 00:14:16 +01:00
Ondřej Budai
a39d5af339 ci: remove 8.5 nightly testing
It no longer makes sense because:

- we don't make any changes to 8.5
- we don't regenerate test manifests for 8.5
- osbuild-composer for 8.5 is in the rhel-8.5.0 branch

Also, the latest-8.5.0 symlink was removed, which broke the CI.
2021-10-26 12:39:35 +02:00
Christian Kellner
c42c19356c mpp: fix long options
The correct way to specify long options is as separate arguments,
not as one argument separated by comma.
2021-10-25 18:01:25 +02:00
Alexander Larsson
611b6846e8 osbuild-mpp: Set the "arch" variable to the current rpm arch 2021-10-25 15:17:04 +02:00