Commit graph

922 commits

Author SHA1 Message Date
Tom Gundersen
1bccf72b6b manifest/os: detect if the lvm2 package should be included
If the partition table includes logical volumes, the lvm2 package should be installed on
the target system.

Drop the corresponding logic from fedora/distro.go.
2022-07-07 12:00:56 +01:00
Tom Gundersen
ec8cc01f95 platform: introduce hardware platform abstraction
These objects describes the hardware an image runs on. Including
 - architecture
 - bootloader
 - required firmware

Use the platform abstraction to move firmware packages out of the package set
definitions.
2022-07-07 12:00:56 +01:00
Gianluca Zuccarelli
682481d4d7 distro: move isMountpointAllowed
Move the `isMountpointAllowed` function to the common
distro package since this function is shared used in
multiple distro packages.
2022-07-05 23:13:43 +01:00
Tom Gundersen
4e2cea61bb manifest/os: move packages needed for early boot into the pipeline definition
This simplifies the package set definitions in the fedora distro, and unifies the handling
of IoT and non-IoT images.
2022-07-05 22:39:40 +01:00
Tom Gundersen
03fc2d5bfd distro/fedora: install kernel and bootloader in ostree commits
This is a partial revert of 006c5b26, where kernel settings and bootloaders were only
installed on bootable systems.

However, ostree-based systems need the ability to pick up kernels and bootloaders from
the commit to install on the instance, so make this conditional on being bootable or an
ostree commit. This is probably an indication that we need a different abstraction.
2022-07-05 22:39:40 +01:00
Tom Gundersen
a45caf5756 distro/fedora: image type should be more specific than arch
Currently an image type could override the boot loader in the architecture, but we should
not allow an image type to select a bootloader type the architecture does not support. So
only in case the architecture supports hybrid boot do we allow the image type to select
one of the two types.

This enables UEFI on AMIs for aarch64.
2022-07-05 22:39:40 +01:00
Tom Gundersen
907b4a8686 distro/fedora: pipelines.go -> manifests.go
Simplify the manifest generation by doing it in terms of manifests rather than slices
of pipelines.
2022-07-05 12:15:50 +02:00
Tom Gundersen
ba31887ff2 fedora/pipelines/os: minor refactoring
Collect partition tables / boot loaders / kernels together and
make them conditional on the system being bootable.

As a side-effect, we no longer install the grub2 modules in ostree commits.
2022-07-05 12:15:50 +02:00
Tom Gundersen
de6c628069 manifest/os: drop kernelName and bootLoader from New()
The kernel name is optional and can be set later.

The bootloader we skip entirely. Instead, set the architecture, which now becomes
mandatory. Use it to deduce the bootloader, and in the future other pipelines can read
this property from the OS Pipeline, rather than having it passed in.
2022-07-05 12:15:50 +02:00
Tom Gundersen
fd5180d52d manifest/os: make grubLegacy and partitionTable optional
These should both default to being disabled, so move them away from the constructor.
Rename grubLegacy to BIOSPlatform and document that setting it enables BIOS support.
2022-07-05 12:15:50 +02:00
Tom Gundersen
64b4ad7c31 manifest/os: make ostree parameters optional
The OSTree parameters can be set after initialisation. We should only require parameters
to be set at initialisation time if we have no good defaults. In the case of OSTree the
default is to not enable OSTree support.
2022-07-05 12:15:50 +02:00
Tom Gundersen
b0b5a48c5f manifest/os: don't change the name based on ostree or not
Let the name reflect the pipeline type, and keep it the same whether or not
the contents is meant to be an OSTree commit or not.
2022-07-05 12:15:50 +02:00
Tom Gundersen
168aa40b2d manifest/build: don't expose extra build packages
The build packages should always be computed from its
dependents and there should be no need to override it. Drop the
ability, and all the unused code in fedora/*.

NOTE: due to a bug in a previous patch in this PR the extra
package set was never set on the build pipelines.
2022-07-04 23:04:29 +01:00
Tom Gundersen
83e2060976 distro/fedora: drop package set name translations
No longer name the packageSetChains after the package set, but
keep them named after the pipelines. This should be a
non-functional change as dnf-json does not care about what the
chains are called, only that the names are unique.
2022-07-04 23:04:29 +01:00
Tom Gundersen
fbad41ac5e manifest: boot packages - move over anaconda pkgs
The same set is used for building and booting, move over the boot
logic to the pipeline as well.
2022-07-04 23:04:29 +01:00
Tom Gundersen
41f6e428af manifest: build packages - move over anaconda pkgs
squashfs-tools is used by the mono stage, and lorax by lorax.

For now copy over the anaconda boot package set as-is, we can
refactor further from the pipelines.
2022-07-04 23:04:29 +01:00
Tom Gundersen
7c51aba903 manifest: build packages - move over installer pkgs
The `installerBuildPackageSet() are required by the ISOPipeline,
move it there.
2022-07-04 23:04:29 +01:00
Tom Gundersen
70d7a25a8e manifest: build packages - move over 'rpm-ostree'
Move `iotBuildPackages()` from the distro to the pipelines.
2022-07-04 23:04:29 +01:00
Tom Gundersen
e51e4e458f manifest: build packages - move over distro-wide package set
Move `distroBuildPackages()` from the distro to the pipelines.
2022-07-04 23:04:29 +01:00
Tom Gundersen
82356c419c distros/fedora/pipelines: pass the right packageChains
This fixes a bug introduced in a previous commit and simplifies the
passing of packageChains.
2022-07-04 23:04:29 +01:00
Tom Gundersen
4961a17ba8 manifest: implicitly track pipelines in manifest
Pipelines are now added to their manifest on creation, and we
ensure that dependants are associated with the same manifest.
2022-07-04 23:04:29 +01:00
Tom Gundersen
28b8a790b5 manifest: provide depsolved packages when serializing
Rather than providing packageSpecs when constructing the
Manifest, do so at Serialize() time.

This allows the overall flow to be:

```
m := Manifest.New()
m.AddPipeline()
...
c := m.GetPackageSetChains()
p := Depsolve(c)
m.Serialize(p)
```
2022-07-04 23:04:29 +01:00
Tom Gundersen
c042af7d9c distro/fedora/PackageSets: get from Manifest object
Before you read this patch, be warned: this is crazy. But rest
assured, this is as bad as it gets, and by the end of this PR it
will all be infinitely simpler than it ever was.

We want Manifests to be self-describing, and in particular, we want
manifest.GetPackageSetChains() to return what
ImageType.GetPackages does today. This should be achieved by
pipelines knowing their required package sets, and the caller
optionally amending additional package sets at Pipeline
initialisation time.

As a first step, while pipelines don't yet know anything about their
required package sets, simply pass in the precomputed
PackageSetChains and set them as optional on each pipeline. Then
we can read the chains back out of t he manifest.

This is not a functional change, but allows us to gradually move
package set definitions from the distros to the pipelines in follow
ups.
2022-07-04 23:04:29 +01:00
Tom Gundersen
be5ea6a9b8 distro/ImageType: let PackageSets depend on ImageOptions
The package sets for an image can depend on the blueprint, and
by the same logic there is no reason it should not be able to
depend on the image options.

This is so far a non-functional change, but makes a follow-up
commit simpler (though still without actually depending on
the image options to compute the package sets).
2022-07-04 23:04:29 +01:00
Tom Gundersen
0743eb2f81 manifest: use call-by-reference in pipeline interface
This allows interface methods to modify the state of the pipeline
objects. We will use this in subsequent patches.
2022-07-04 23:04:29 +01:00
Tom Gundersen
f791bde0e4 pipeline: fully encapsulate manifest generation
Make it the responsibility of each pipeline to track its required
sources, and use this to generate a manifest from its pipelines.

This removes the dependency on osbuild2 from distro.go, and the
only other uses of osbuild2 are to specify parameters in
ImageConfig, which will eventually go away too.

As a consequence, this only attaches the required sources for a
given manifest, rather than all known packages in the package set
map.
2022-07-04 23:04:29 +01:00
Tom Gundersen
d8f540c5cf distro/fedora/iotInstaller: drop unused package set
The IoT installer does not use the iotCommit package set. Drop it.
2022-07-04 23:04:29 +01:00
Achilleas Koutsou
cc5d32169d distro/rhel8: add rhel-8 alias to rhel-86
Add a plain `rhel-8` alias as the default distribution name and version
for the `rhel8` package.  The `rhel-86` distro is still available via
the NewRHEL86() constructor.  These two distributions are identical.

Repositories
------------

The rhel-8 repositories (repositories/rhel-8.json) are now set to the
CDN repositories with no minor version:
https://cdn.redhat.com/content/dist/rhel8/8/...

The rhel-8 test repositories (test/data/repositories/rhel-8.json) were
already set to the plain `8` repositories.  The Google repos have been
added.

The test case generator repositories used for `rhel-8` are the rpmrepo
snapshots as for rhel-86.
2022-07-01 23:52:13 +01:00
Achilleas Koutsou
57df5c2eaa Rename rhel86 package to rhel8
With 8.3 support dropped, the rhel86 package defines all the supported
RHEL 8 versions except 8.4.

RHEL 8.4 will be merged into the rhel8 package soon.
2022-07-01 23:52:13 +01:00
Achilleas Koutsou
723c34984c distro: remove rhel8
The rhel8 package represented RHEL 8.3, which is EOL.
The current rhel86 package will be renamed to rhel8 and be responsible
for building all RHEL 8 minor versions.
2022-07-01 23:52:13 +01:00
Achilleas Koutsou
444b8e33c9 distro: don't enable fdo-client service on RHEL 8.5
RHEL 8.5 doesn't have FDO and it's not installed, but we were enabling
the service for it in edge.
2022-06-30 18:34:45 +02:00
Christian Kellner
54ae069b94 distro: remove rhel 9 beta
This was a stop-gap until the actual rhel 9 distro was created. It
is in a sad state, quite broken and shout not be used by anybody.
Put it out of its misery.
2022-06-30 14:15:33 +02:00
Achilleas Koutsou
fc1d754999 distro: remove rhel85
Remove RHEL 8.5 distro source files.
RHEL 8.5 is now defined in the rhel86 package.
2022-06-30 11:14:21 +02:00
Achilleas Koutsou
436d8f9b43 distro/rhel86: add RHEL 8.5 variant
Define the distribution strings for RHEL 8.5 in distro/rhel86 and add
constructors.  Remove the old 8.5 from the distro registry and use the
new constructors.

Composer can now build RHEL 8.5 image-installer on aarch64, which wasn't
supported before.

RHEL 8.5 manifests have changed to minimise the differences from 8.6.
Some changes are fixes made in 8.6 but never backported to 8.5 because
of our (older) policy of not changing definitions after the release of a
distro.
Other changes are non-functional (e.g., stage or package order).

See the list below for the source of each change.

Manifest changes:
- Stage order changed for org.osbuild.systemd-logind and
  org.osbuild.rhsm.
- org.osbuild.grub2 options: config.default = "saved"
  Reverted 111cd8871f
- Partition sizes: RHEL 8.5 had extra arbitrarily sized padding for the
  header.  Now all partitions are sized to fit headers exactly.
  Original change at b7abef54e8.
- SELinux set to permissive in Anaconda.  This was changed in RHEL 8.6
  and 9.0 but never backported to 8.5.
  See a7fbe916b7.
- Installer isolevel set to 3.  Like above, this was changed in
  8.6 and 9.0.
  Original change at d8d161480e.
- Specify a remote for edge deployments.
  Original change at b18b4e80a0.
2022-06-30 11:14:21 +02:00
Achilleas Koutsou
56e6208fcf distro/rhel86: imagetypes and packages per RHEL version
Added utility function for comparing RHEL version strings.

Conditions added:
- greenboot subpackages were changed between RHEL 8.5 and RHEL 8.6.
- fido client packages aren't available in RHEL prior to 8.6.
- the ec2 SAP image type is not supported in RHEL prior to 8.6.
- the edge-simplified-installer and edge-raw-image image types are not
  supported in RHEL prior to 8.6.
    - They were previously supported in 8.5 without FDO support, but now
      it's dropped from 8.5 completely.
2022-06-30 11:14:21 +02:00
Achilleas Koutsou
e2e77a1190 distro/rhel86: reformat package sets
- One package per line for easier reading and diffing.
- Sorted alphabetically.  Exceptions made for sub-grouping.
2022-06-30 11:14:21 +02:00
Achilleas Koutsou
0a82fe4561 distro/rhel86: reorganise addImageTypes() call
One image type per line.
Sorted alphabetically.
2022-06-30 11:14:21 +02:00
Christian Kellner
c5a5da5f56 distro/fedora: add container image type
Add a container image type that is based on the existing fedora
container image. There is a delta in terms of the configuration
because osbuild does not yet provide all the neccessary means,
but the package set is already very close.
2022-06-29 10:02:46 +02:00
Christian Kellner
59edcc4a46 distro/fedora: only pass kernelName to NewOSPipeline for bootable images
If the image is not bootable it means that we do not need the kernel
version and thus we should not pass it along to `NewOSPipeline`,
because we might not even have a kernel in the package set.
2022-06-29 10:02:46 +02:00
Christian Kellner
776dfd2da6 distro: option to exclude docs during installation
We don't ship documentation in container images, add an option to
ImageConfig to disable the installation of docs during installation.
2022-06-29 10:02:46 +02:00
Christian Kellner
f37a791bd2 distro: new image config to disable selinux
We don't need to label container artifacts, so include an option
in the image config to disable it.
NB: This is a boolean that with the default false in order to
not have to change every existing image configuration.
2022-06-29 10:02:46 +02:00
Christian Kellner
516f30de17 distro: new blueprint helper where kernel package is optional
Add a new extended helper `GetPackagesEx` that includes a flag
to specify if the target is bootlable or not and only include
the kernel package in case it is set to true. Delegate to this
new helper from the existing `GetPackages`.
2022-06-29 10:02:46 +02:00
Tom Gundersen
4556312d22 pipeline: split qcow2 into format specific pipelines
Stages are procedural and named after the tool they wrap, but pipelines are declarative and should
be named after the kind of artefact they produce.

This splits the qemu (the tool) pipeline into qcow2, vmdk, and vpc (the formats) pipelines. In theory
we may have wanted to implemented through some shared helpers, but for now it seems trivial
enough that it is not worth it.
2022-06-27 19:11:26 +01:00
Tom Gundersen
1fa92f9091 pipeline: distinguish between optional and mandatory properties
The ideal is that the constructor takes mandatory properties as arguments, and fields in the struct
are all optional.

This clarifies that across the pipelines (or leaves TODOs where work remains), and where possible
makes fields optional by providing a valid default value.
2022-06-27 19:11:26 +01:00
Tom Gundersen
c001af63ec pipeline: further cleanups
This adds more documentation and makes more properties implicitly inherited rather than
repeated. This makes for less boilerplate, and gives us fewer things to keep in sync.
2022-06-27 19:11:26 +01:00
Tom Gundersen
f60092033b pipelines: make OSPipeline own the PartitionTable
The OSPipeline might need to know what disk layout it will be put onto, enforce this by making
the PartitionTable a property of the OSPipeline, and require child pipelines to query it when needed.
2022-06-27 19:11:26 +01:00
Tom Gundersen
ae34513d18 pipeline: move package to top level
The pipeline package is exists conceptually between the distro and the osbuild packages, so move
it to the top level rather than as a child of distro.

No functional change.
2022-06-27 19:11:26 +01:00
Tom Gundersen
e024ceb138 pipelines: cleanups
This is a collection of minor cleanups:
 - Start documenting the API
 - Enforce dependent pipelines have the correct type where necessary
 - Use data from dependent pipelines where possible
 - Start enforcing required fields
 - Move logic into the pipeline implementation where we can
2022-06-27 19:11:26 +01:00
Tom Gundersen
040a4ef6a1 pipelines: add Fedora IoT installer pipelines 2022-06-27 19:11:26 +01:00
Tom Gundersen
284eb9af59 distro/fedora: drop dead code 2022-06-27 19:11:26 +01:00