Commit graph

1016 commits

Author SHA1 Message Date
Irene Diez
20a5b25320 distro/fedora: handle generated warnings during Manifest initialization
Signed-off-by: Irene Diez <idiez@redhat.com>
2023-03-31 14:14:57 +02:00
Irene Diez
2771af2530 distro/rhel7: handle generated warnings during Manifest initialization
Signed-off-by: Irene Diez <idiez@redhat.com>
2023-03-31 14:14:57 +02:00
Irene Diez
64a5946dc3 distro/rhel8: handle generated warnings during Manifest initialization
Signed-off-by: Irene Diez <idiez@redhat.com>
2023-03-31 14:14:57 +02:00
Irene Diez
d6b9b3a5bf distro/rhel9: handle generated warnings during Manifest initialization
Signed-off-by: Irene Diez <idiez@redhat.com>
2023-03-31 14:14:57 +02:00
Irene Diez
8022c227ba distro: allow to return warnings in the Manifest function
This changes the `Manifest` function of the `ImageType`
interface so that any warnings detected during the
`checkOptions` step of the manifest initialization can
be propagated back to the Weldr-API (see next commit).

Signed-off-by: Irene Diez <idiez@redhat.com>
2023-03-31 14:14:57 +02:00
Achilleas Koutsou
90cc2b18a8 distro/rhel8: assert error instead of skipping
We sometimes skip testing customizations for image types that don't
support them, other times, we check for a specific error.  If we don't
care about the specific error message that's received, running the test
and checking for error is better than skipping, because it ensures that
we're skipping the test for the right reasons and not accidentally
skipping a test for an image type that should support the
customizations.
2023-03-30 18:40:12 +02:00
Achilleas Koutsou
203dd5275e distro/rhel8: exclude azure-eap7-rhui from customization tests
Skip customizations tests for the EAP image.

Create a map of unsupported image types for each test to make adding new
ones easier.
Use a `nil` struct pointer for the pipeline names test instead of an
empty struct.  Non-nil customizations can't be compared.
2023-03-30 18:40:12 +02:00
Achilleas Koutsou
21d13a2ae3 distro/rhel8: specify ports for azure EAP
See COMPOSER-1859
2023-03-30 18:40:12 +02:00
Achilleas Koutsou
fb5accc227 distro/rhel8: add env vars to azure EAP
Add the required environment variables to Azure EAP.
See COMPOSER-1860
2023-03-30 18:40:12 +02:00
Achilleas Koutsou
3f299c7d43 distro: add ShellVars field to ImageConfig 2023-03-30 18:40:12 +02:00
Achilleas Koutsou
636f3ff237 distro/rhel8: add azure EAP 7 image type
Initialise the image type as an Azure RHUI type with the name
"azure-eap7-rhui" and a workload attached that provides the extra
packages required by the EAP workload.
2023-03-30 18:40:12 +02:00
Achilleas Koutsou
5b4e77797c distro/rhel8: remove outdated comment 2023-03-30 18:40:12 +02:00
Achilleas Koutsou
18ef5ae32d distro/rhel8: distro workload implementation and EAP
Add a local, private implementation of the Workload interface for RHEL
8.  The type should be used for concrete workloads that provide package
package lists for specific image types.

The eapWorkload() function creates a workload that provides the packages
required for EAP image type variants.
2023-03-30 18:40:12 +02:00
Achilleas Koutsou
77b8328999 distro: add workload to image type
Support specifying workload on an image type.
This isn't used yet, but when set, the workload will define packages and
services and not allow any blueprint customizations.

In the future, there will be multiple workloads with each image type
specifying one (but multiple image types can have the same default
workload).  The custom workload will allow all customizations that are
currently supported.
2023-03-30 18:40:12 +02:00
Tomáš Hozza
58e3716b89 distro: add support for RHEL-8.9 and RHEL-9.3
Add a basic support for building RHEL-8.9 and RHEL-9.3 images with
composer.

Add 8.9 and 9.3 repositories to the multitude of places where we have
them.

Generate image test manifests for 8.9 and 9.3. No functional testing is
added for 8.9 or 9.3 at this moment.

This change unblocks the RHEL Gating (since unit tests are currently
failing on 8.9 and 9.3 as on unknown distribution).

Related to https://issues.redhat.com/browse/COMPOSER-1924

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-03-27 11:39:24 +02:00
Sanne Raymaekers
03973acc1f distro/rhel: add payload repos to os package set
Before instantiating the manifest, any repositories that contain a
package set key contained in `PayloadPackageSets()` should be added to
the os package set in order to correctly generate the `org.osbuild.rpm`
stage for the os pipeline.

Otherwise options like GPG keys are not set correctly.

Fixes #3326
2023-03-23 13:39:04 +01:00
Brian C. Lane
3481e1d3ba Change the rpmmd cache directory structure to include the distro name
This causes dnf-json to use separate caches, allowing them to run in
parallel, with one lock per distribution. Multiple depsolves with the
same distribution in the blueprint will continue to be serial.
2023-03-23 11:26:40 +01:00
Simon de Vlieger
6704dfd609 distro/fedora: clarify requiredPartitionSizes 2023-03-20 11:04:14 +01:00
Simon de Vlieger
7e93df725c distro: move RequiredPartitionSizes into ImageType
It used to be defined on ImageOptions, but ImageType is a better fit for
this.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
92cdb0f082 iot-raw-image: change partition table to DOS
For parity with ImageFactory built images there should be only a DOS
partition table in the raw images, with a non-standard ID set for the
boot partition for Raspberry Pi compatibility.

This only applies to the aarch64 images.

As noted in #3220.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
39879a9f60 disk: add RequiredSizes to ImageOptions
These RequiredSizes are a map that is passed on to the partition table
logic which had hardcoded defaults. This makes it possible to define
either no RequiredSizes (`nil`) or empty RequiredSizes which means no
further constraint checks or partition resizes will be done.
2023-03-20 11:04:14 +01:00
Simon de Vlieger
053bbfcb98 iot-raw-image: set size to 4 GiB
The current Fedora IoT raw images are 4 GiB but we were using 10 GiB raw
images. Make this consistent to current.
2023-03-20 11:04:14 +01:00
Antonio Murdaca
414424cd30 simplified-installer: enable isolinux
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2023-03-15 12:35:50 +01:00
Simon de Vlieger
23627e9ce0 distro/fedora: add support for Fedora 39 2023-03-15 09:31:23 +01:00
Simon de Vlieger
a28d535577 image-installer: switch payload to minimal-rpm
With the addition of the minimal rpm package set and image type in #3181
the previously added bare metal package set is a duplicate. It's not
used elsewhere within fedora and the minimal rpm package set is
officially defined.
2023-03-08 12:15:23 +01:00
Simon de Vlieger
5f0d273248 distro: remove duplicate version checks for fonts
The new names of the packages are being added twice in two different
checks, remove the redundant code.
2023-03-08 12:12:18 +01:00
Brian C. Lane
7a4bb863dd Update deprecated io/ioutil functions
ioutil has been deprecated since go 1.16, this fixes all of the
deprecated functions we are using:

ioutil.ReadFile -> os.ReadFile
ioutil.ReadAll -> io.ReadAll
ioutil.WriteFile -> os.WriteFile
ioutil.TempFile -> os.CreateTemp
ioutil.TempDir -> os.MkdirTemp

All of the above are a simple name change, the function arguments and
results are exactly the same as before.

ioutil.ReadDir -> os.ReadDir

now returns a os.DirEntry but the IsDir and Name functions work the
same. The difference is that the FileInfo must be retrieved with the
Info() function which can also return an error.

These were identified by running:
golangci-lint run --build-tags=integration ./...
2023-03-07 09:22:23 -08:00
Sanne Raymaekers
b06863e468 distro/fedora: add open-vm-tools to vmdk package set 2023-03-01 14:35:56 +01:00
Achilleas Koutsou
f11589367e internal: FIXME notes for container package set
Tracing the package set and repository journey, I found that the
container package defined on the image type is never really used.

Added notes to fix later.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
b0a6b486a1 distro/test: set dnf-json path for solver
The value comes from the command line args of the test binary
(osbuild-composer-manifest-tests).  It works in our tests because we use
the default value, but if it was set differently it would have been
ignored.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
114ed20fab distro/test: include package sets field in manifest test repos
The test previously ignored the package-sets field in the repos in the
compose request.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
2a4cd1966d distro: pass all repos to initializeManifest
Pass through all repos to the initalizeManifest() function.  Each
pipeline will then select which repositories it needs based on the
PackageSets field of each repository.

Before, we only passed global repos down to the manifest generators and
pipeline-specific repositories would only be used if they were attached
to package sets and were handled explicitly by a pipeline generator.

The repositories of the "blueprint" package set are explicitly added to
the workload and returned by the "os" pipeline.
The repositories of the "installer" package set are explicitly added to
the "anaconda-tree" pipeline.

If a repository was specified for any other pipeline, for example
"build", the repositories for the that package set were never added to
the pipeline.

Fixes #3290
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
50f578924c distro: rename osPkgsKey from "packages" to "os"
Match the key for the OS packages to the name of the OS pipeline ("os").
We will use this key to identify package sets that are returned from the
OS pipeline since the manifest returns package sets indexed by each
pipeline's name.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
b918dc8a8c distro/test: add new test for pipeline-specific repos
New test that runs through a few different scenarios of assigning
repositories to specific package sets.  Each scenario defines a set of
global repositories (or none) and a set of pipeline-specific
repositories (or none) and an expected result.  It then calls the
distro's PackageSets() method.  The test checks that the package sets
that are returned are assigned the correct repositories based on the
test case's expected result.

A basic string-set struct is implemented to help with merging and
comparing string slices.

This test should currently fail due to #3290.
2023-02-23 16:22:42 +01:00
Achilleas Koutsou
777d27efbf distro/test: remove RHEL 7 exception from PipelineNames test
RHEL 7 rewrite makes this exception obsolete.
2023-02-23 16:22:42 +01:00
Irene Diez
38b6b9cdcf distro/rhel8+9: allow User/Group customizations on edge-si
Signed-off-by: Irene Diez <idiez@redhat.com>
2023-02-22 12:18:12 +01:00
Irene Diez
dbadd42a01 distro/rhel8+9: deprecate User/Group customi. on edge-commit/container
Signed-off-by: Irene Diez <idiez@redhat.com>
2023-02-22 12:18:12 +01:00
Tomáš Hozza
ffd0bdb7ad distro: add dir / files customizations policy check
Add a default policy for custom directories and files to constrain what
users can do. The intention is to ensure that directories and files can
be created only in `/etc` and also that none of the important
configuration files can be overwritten by this customization.

Add the policy validation to all distro implementation.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
b98f5dad70 distro: set dir / file BP customizations in OS customizations
Hook up the custom BP directories and files implementation with OS
pipeline implementation. The user-provided values are now set in the OS
customizations structure and will be used by the OS pipeline generator
when adding stages to the pipeline.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
0e2f221514 distro: validate directory / file BP customizations
Validate the user-provided directory and file BP customizations in
`checkOptions()`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
26e6983320 disk: move remaining mountpoint policy code to different packages
Move the `CheckMountpoints()` implementation to `blueprint` package,
since it does not operate on any data structures from the `disk`.

Move the default mountpoint allow list policy definition to the
`pathpolicy` package.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Tomáš Hozza
0e4dabd265 blueprint: delete HasIgnition() function
Unify how are allowed options checked in distro implementation in
relation to Ignition customization. Specifically, delete `HasIgnition()`
function and replace its use by `GetIgnition()` call and checking if it is
`nil`. This approach is consistent with how this is checked for other
customizations.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-02-22 12:17:36 +01:00
Antonio Murdaca
6afab3dff2 rhel9: fix conditionals for sysroot.readonly enablement
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-02-14 12:07:23 +01:00
Antonio Murdaca
1b9e008c29 rhel8: remove sysroot.readonly functionality
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-02-14 12:07:23 +01:00
Brian C. Lane
b0e388d115 cloudapi: Add subscription option for rhc
When rhc is selected it will install the required packages, register
using rhc and always enable insights.

When rhc is not selected it will use subscription manager for
registration, and optionally enable insights. Also installing required
packages.
2023-02-09 09:41:15 +01:00
Antonio Murdaca
b8ddbd3031 internal/distro/rhel9: enable ignition on 9+
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-02-07 13:30:22 +01:00
Antonio Murdaca
b3f1b0edc8 ignition: drop embedded provisioning url
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-02-07 13:30:22 +01:00
Ondřej Budai
2b1facb44d distro/rhel8: don't install missing MSFT key into azure-sap-rhui
The rhui-azure-rhel8-sap-ha package is currently missing
the /etc/pki/rpm-gpg/RPM-GPG-KEY-microsoft-azure-release key.
This makes the image type unbuildable, which causes some of our tests
to fail.

Overlay the generic RHUI config, so the missing key isn't imported.
See CLOUDX-336 for more information.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-02-07 13:16:12 +01:00
Ondřej Budai
bd5bffc8cf distro/rhel8: make sap image config methods private
They are not used in a different package, so they can be private.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-02-07 13:16:12 +01:00
Brian C. Lane
d12447408b Change RepoConfig.GPGKey to an array of key strings
DNF supports more than one GPG key. It is possible that one may be used for
signing packages, and another to sign the repository metadata. This
renamed GPGKey to GPGKeys internally. It does not change the on-disk
repository json format.
2023-02-01 10:27:58 +01:00