Commit graph

1552 commits

Author SHA1 Message Date
Christian Kellner
9a786ce4fc stages/logind: write only one drop-in file
Instead of supporting writing an arbitrary number of configuration
files, just write one. This makes the stage and its schema clearer
and simpler. If more than one config file is needed, the stage can
be repeated multiple times. It is also more flexible since we can
in the future specify additional options at the top level which
will then be per-file, like a top level comment.
2021-07-22 01:05:29 +02:00
Christian Kellner
79c6e65976 stages/logind: configuration_files → config
Rename the `configuration_files` key to `config` key.
2021-07-22 01:05:29 +02:00
Christian Kellner
9f1d97a8b7 stages/modprobe: write only one config file
Instead of supporting writing an arbitrary number of configuration
files, just write one. This makes the stage and its schema clearer
and simpler. If more than one config file is needed, the stage can
be repeated multiple times. It is also more flexible since we can
in the future specify additional options at the top level which
will then be per-file, like a top level comment.
2021-07-22 01:05:29 +02:00
Christian Kellner
0190c991ae stages/dracut: write only one config file
Instead of supporting writing an arbitrary number of configuration
files, just write one. This makes the stage and its schema clearer
and simpler. If more than one config file is needed, the stage can
be repeated multiple times. It is also more flexible since we can
in the future specify additional options at the top level which
will then be per-file, like a top-level comment.
2021-07-22 01:05:29 +02:00
Christian Kellner
a1703dc298 stages/cloud-init: write only one config file
Instead of supporting writing an arbitrary number of configuration
files, just write one. This makes the stage and its schema clearer
and simpler. If more than one config file is needed, the stage can
be repeated multiple times. It is also more flexible since we can
in the future specify the directory, `/etc` vs `/usr` via a new
top level `directory` key.
2021-07-22 01:05:29 +02:00
Christian Kellner
0becf66454 stages/dracut.conf: configuration_files → config
Rename the `configuration_files` key to `config` key.
2021-07-22 01:05:29 +02:00
Christian Kellner
f259b4e5ca stages/modprobe: configuration_files → config
Rename the `configuration_files` key to `config` key.
2021-07-22 01:05:29 +02:00
Christian Kellner
9e259ae0bc stages/cloud-init: configuration_files → config
Rename the `configuration_files` key to `config` key.
2021-07-22 01:05:29 +02:00
Diaa Sami
8be74157ae test: fix wrongly named class
The test class for the v2 tests was wrongly named `TestFormatV1`.
2021-07-21 16:46:09 +02:00
Martin Sehnoutka
54d7128d65 util/rhsm: Implement a fallback to the previous behaviour
The problem is that some deployments might not have the redhat.repo
file, yet they might have the key and certificate to access Red Hat CDN.
If that was the case, the new approach would cause a regression compared
to the previous behavior.

This patch uses the previous method if the redhat.repo file is not
found or does not contain any matching URL.
2021-07-21 14:38:18 +02:00
Christian Kellner
0cab954d84 test/fmt_v2: more tests for mounts
Add tests for the duplicate mount, missing device detection.
2021-07-21 13:28:22 +02:00
Christian Kellner
3965291895 test/fmt_v2: include devices and mount
Include devices and mounts in the basic pipeline to check that
parsing and describing them works as expected.
2021-07-21 13:28:22 +02:00
Christian Kellner
a1df2d5d9d mounts: add org.osbuild.noop
Add a noop mount that is usefull for testing.
2021-07-21 13:28:22 +02:00
Christian Kellner
4d73760dcd formats/v2: fix mount description for source
The mount has a `source` not `device`. It is just called device
internally because for now it is always a device.
2021-07-21 13:28:22 +02:00
Christian Kellner
ae1296e33a formats/v2: mounts are arrays
The order of entries in a dictionary is not specified by the JSON
standard and hard to control when marshalling dictionaries in Go.
Since the order of mounts is important and the wrong order leads
to wrong mount trees change the `mounts` field to an array. This
breaks existing manifests but after careful deliberation it was
concluded that the original schema with mounts as dictionaries
is not something we want to support. Apologies to everyone.

Adjust the schema of the copy and zipl stage accordingly.
2021-07-21 13:28:22 +02:00
Christian Kellner
50627b713c schema/v2: make options for devices optional
Some devices might not need any options.
2021-07-21 13:28:22 +02:00
Christian Kellner
31943072c7 stages/noop: allow mounts and devices
Allow the noop stage to have mounts and devices, which is useful
for testing.
2021-07-21 13:28:22 +02:00
Diaa Sami
02ceb02d2a osbuild: Add validation for source references
Validate source references while loading manifests so that a bad
reference would result in a meaningful error message instead of a
hard-to-understand Python exception.
2021-07-20 20:22:27 +02:00
Christian Kellner
9e599fca17 tools/mpp: expand basurl after dep-solving
We create the urls for the packages after dep-solving. During the
big refactoring (802f4010) the code that would expand the base url
during that step got lost. Re-introduce that so that local repos
work correctly again.
Also make sure the path is absolute before calling `as_uri` since
it is not guranteed that the result will be a `file:///` uri.

Reported-by: Stephen Smoogen <smooge@smoogespace.com>
2021-07-20 13:23:58 +02:00
Christian Kellner
bd8cd7443a test/data: split up the bootiso stage
Use the new `squashfs`, `grub2.iso` and `isolinux` stage to re-
plicate what the `bootiso.mono` stage was doing, which is now
considered deprecated.
2021-07-16 15:19:58 +01:00
Christian Kellner
0910320e62 stages: add org.osbuild.isolinux
New stage to configure the isolinux boot loader.
2021-07-16 15:19:58 +01:00
Christian Kellner
228323f0dc stages: add org.osbuild.squashfs
New stage to create a squahfs image.
2021-07-16 15:19:58 +01:00
Christian Kellner
fb2786e4dd stages/grub2.iso: small refactoring
Small changes to the schema so it better aligns with the need of the
stage:
  move the efi properties to the top-level
  kernel information is under `kernel` object
  kernel command line options is an array now
Change `linuxefi` to `linux` which should work on RHEL 8 and is the
only thing that works on arm64.
Small PEP-8 fixes.

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2021-07-16 13:20:45 +02:00
Antonio Murdaca
dd13ec24fb stages: add org.osbuild.grub2.iso stage
Add a new stage to build a boot filesystem tree

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2021-07-16 13:20:45 +02:00
Ondřej Budai
f8c249cd8f ci: build rpms for Fedora 34 2021-07-16 13:02:38 +02:00
Christian Kellner
9a33fabd51 assembler/ostree.commit: fix copying of links
Commit 92cc269 fixed a bug where `/var` was copied into `/var`
resulting in `/var/var`. Sadly the fix broke copying links,
like `bin -> usr/bin`, where now the content of the link would
be copied but not the link itself. Use the `-t` command line
flag for `cp` which should ensure that we copy links as links
but also copy the contents for `/var` should the target dir,
i.e. `/var` already exist.
2021-07-16 10:32:12 +02:00
Christian Kellner
e8bf211e0c test/data: move ostree repo to the iso root
Instead of having the OSTree repo in the anaconda root squashfs,
put the it to the root of the iso. This has several advantages:
first and foremost, we don't have to make a huge squahfs file-
system. We don't have to compress the repo. Additionally, the
repo is now easily accessible by mounting the iso.
2021-07-15 23:08:51 +01:00
Christian Kellner
a19e261dbe test/data: move kickstart to the iso root
Intead of overwriting the standard ananconda kickstart file, we
move it to the root of the iso. To have anaconda pick it up an
additional kernel command line parameter is required.
2021-07-15 23:08:51 +01:00
Christian Kellner
c77170a924 test/data: create core user via kickstart
Create the users via the new users support that was added to the
kickstart stage, instead of having them in the OSTree commit.
2021-07-15 23:08:51 +01:00
Christian Kellner
9d3c420a38 test/data: update x11 pkgs for ostree-bootiso
F34+ has the xorg-x11-server-utils package split up. Install the
individual new packages we need for anaconda.

See Anaconda commit 55371c996861a47da9504b31118858dd3257a8b0
2021-07-15 23:08:51 +01:00
Christian Kellner
887e1bd8aa stages/kickstart: support for users and groups
Add support for creating groups and users during the installation
via the corresponding kickstart directives.
2021-07-15 23:08:51 +01:00
Christian Kellner
403dd5e2ec tools/lorax-templates-pkg: set executable bit 2021-07-15 23:08:51 +01:00
Ondřej Budai
19c4b43952 spec: make format of file modes consistent 2021-07-15 16:23:01 +01:00
Ondřej Budai
e9da5ccdc2 spec: make json schemas non-executable
rpmbuild actually complains about it:

*** WARNING: ./usr/share/osbuild/schemas/osbuild1.json is executable but has no shebang, removing executable bit
*** WARNING: ./usr/share/osbuild/schemas/osbuild2.json is executable but has no shebang, removing executable bit
2021-07-15 16:23:01 +01:00
Tomas Hozza
7617cb9a44 stages/authselect: rename profile_id option to profile
Rename the `profile_id` option to `profile` in the stage schema. This is
a follow-up to PR#696 comment after is has been merged [1].

[1] https://github.com/osbuild/osbuild/pull/696#discussion_r661253293

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-07-15 17:17:50 +02:00
Christian Kellner
cc39d5a8b4 test/data: use more variables in ostree-bootiso
Use variables for various often occuring contants, like the
release, ref and isolabel. Additionally, use variables for
the snapshot so it can be changed easily.
2021-07-14 23:51:34 +02:00
Christian Kellner
a9b3fbd92b test/data: auto-format ostree-bootiso
Re-indent and format the bootiso mpp. No actual content changes.
2021-07-14 23:51:34 +02:00
Christian Kellner
c4db24c481 tools/mpp: substitute vars in mpp blocks
Allow the manifest variables, defined via mpp-vars, to be used from
within the mpp blocks. For this template strings are used, where
variables are marked via `$`. We cannot use the `mpp-format` logic
easily there, since that is processed after other mpp directives
have been processed.
As a result remove the built-in substitution from support from dnf
dep-solving, since we had to post-process the resulting urls with
variable substitution afterwards. Now that is covered with this
more generic mechanism.
2021-07-14 23:51:34 +02:00
Christian Kellner
4a00895ba5 tools/mpp: sort import
Makes David happy, which makes us happy.
2021-07-14 23:51:34 +02:00
Christian Kellner
fd1b688f78 tools/mpp: single mpp node retrieval method
Introduce a helper method that will return the mpp node of a given
name or `None` if it does not exist. Delete it from the parent
dictionary if it did.

Reorder the typing imports as well to be sorted below the non-
`from` imports.
2021-07-14 23:51:34 +02:00
Christian Kellner
c774e76079 test/data: remove duplicated baseurl entries
We don't need to define them twice, remove them everwhere where
that was the case.
2021-07-14 23:51:34 +02:00
Christian Kellner
fdb0faca86 tools/mpp: extract vars initialization
Move the initialization for manifest vars out of `__init__`.
2021-07-14 23:51:34 +02:00
Ondřej Budai
4c3c7d7851 ci: enable tests on Fedora aarch64
because why not (:

RHEL might be more difficult, let's revisit it later.
2021-07-14 23:51:05 +02:00
Ondřej Budai
ea7eee8db6 ci: update composer to make tests more robust
f8fd3d04 switched F33 image tests to use rpmrepo rather than kernel mirrors
that are sometimes desynced. This should make the testsuite more stable.
2021-07-14 23:51:05 +02:00
Javier Martinez Canillas
6901c4dfc5 test/data: use UTF-8 locale on all manifests
The manifests set a "en_US" locale but this causes gnome-terminal to not
run due a non UTF-8 locale being used, which is an unsupported config:

gnome-terminal-server[1899]: Non UTF-8 locale (ISO-8859-1) is not supported!

Reported-by: Stephen Smoogen <ssmoogen@redhat.com>
2021-07-14 23:50:24 +02:00
Javier Martinez Canillas
2bc0088767 README: improve Build section
The current Build section is misleading, since running the commands that
are mentioned there will lead to only the osbuild module and osbuild-mpp
tool to be installed.

But none of the other required artifacts (sources, stages, schemas, etc)
will, causing the installed osbuild to not work at all. Instead, have an
Install section that explains how osbuild can be installed from RPMs.
2021-07-14 14:37:03 +02:00
Javier Martinez Canillas
897d702d2e docs: add --export option to the osbuild man page
This is used to export an image but isn't present in the osbuild man page.
2021-07-14 14:35:30 +02:00
Ondřej Budai
8396e73372 ci: upload rpms built in RHEL 8 CDN buildroot into rhel-8-cdn directory
Let's explain how RPMs for RHEL are built:

We use a subscribed RHEL 8.x machine and mock build these on it. Mock
initializes its own buildroot based on the latest RHEL 8 CDN content, see[1].
This means that the minor version of the buildroot is independent of the minor
version of the host.

However, we currently upload RPMs to a directory whose name consists also of
the minor version of the host. Our hosts are currently running RHEL 8.3 so
the RPMs are uploaded into rhel-8.3 directory despite them being built in the
RHEL 8.4 buildroot (RHEL 8 CDN buildroot specifically). This means that
we cannot guarantee that they are installable on RHEL 8.3 which is weird.

This commit adds a special case for hosts that run on subscribed RHEL and
thus build RPMs in a buildroot constructed from RHEL CDN. These RPMs are
now uploaded into rhel-8-cdn directory. This change more accurately reflects
the way we build our RPMs and removes some confusion.

[1]: https://github.com/rpm-software-management/mock/blob/main/mock-core-configs/etc/mock/templates/rhel-8.tpl#L37
2021-07-12 19:02:21 +02:00
Ondřej Budai
41e01326f0 mockbuild: register the host as soon as possible
So we can check if we are subscribed in the variable definitions.

I believe that registering the instance will go away soon nevertheless.
2021-07-12 19:02:21 +02:00
Ondřej Budai
54f233ac96 mockbuild: use REPO_BUCKET when defining the repo's base URL
Just deduplication of code.
2021-07-12 19:02:21 +02:00