Commit graph

2499 commits

Author SHA1 Message Date
Achilleas Koutsou
cb0280c8c9 image: always enable users module in anaconda
Unconditionally enable the users module in Anaconda.
The module serves two purposes:
- It will create a user that's defined in the blueprint by adding the
  user creation stage in the kickstart file.
- It allows users to create user accounts interactively during
  installation.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
11b08da7a5 distro/rhel9: cleanup: delete unused functions 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
42274fd6a7 distro_test_common: refactor the kernel count test
- Add a list of image type names to skip with explanations.
- Run the test in two configurations: empty blueprint and blueprint with
  named kernel (kernel, kernel-debug).
- Count only unique kernel names.  Specifying the same kernel twice
  isn't an issue.  We might change this in the future, but for now the
  kernel selection logic is a bit messy.
- Temporarily skip the image-installer until we settle on a common rule
  for both Fedora and RHEL.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
d66bc11bc7 distro/rhel9: update tar image type to new definition
Add the image function and remove the pipeline function.
Remove the build package set.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
e100c57e58 image: new ImageKind: archive 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
108ac3931b distro/rhel9: update image-installer to new definition
Add the image function and remove the pipeline function.
Remove the build package set.

The image function (imageInstallerImage) was adapted from the same one
in Fedora.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
e031b45377 distro: set ostree ref during package collection
Similar to 194351e681 and
9ae47b6843cace3c9bef55d40a33fa11d8de391a.

Manifest initialisation during package collection uses options and
customizations to determine build root packages.  Just like adding a
fake ostree checksum and an unresolved container list, we also need to
add the ostree ref, which is used to determine if a pipeline is being
generated for an ostree-based image type.

This fixes manifest generation when an ostree commit or container is
being built with an embedded container through the cloud API.  Without
this change, the python toml module is not installed in the build root
and the container storage stage fails.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
60d2a88d4f distro/test: add containers to manifest test
Read the containers from the test manifests when regenerating for the
test.
Also move the ostree option handling higher and make it cleaner.  Only
apply the values if they're specified and needed.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
8612131351 store: add containers to test 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
09903bd0db manifest: support for container embedding
Adding support for container embedding.
The containers need to be specified in the image function (imageFunc)
arguments and when specified, propagate down to the OS pipeline
generator to add the necessary stages.

Support is added for RHEL 9.x and Fedora.

Requires a temporary container spec array with the info from the
blueprint for the first initialization of the manifest that's needed
when collecting required packages.
This should be simplified in the future.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
c4b5c229fb distro/rhel9: delete unused pipeline functions 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
2369869db7 distro/rhel9: update gce images to new definitions
add the image function and remove the pipeline function.
Remove the build package set.

Parameterise image config creation functions so that RHSM is added
conditionally based on distro name, like we did for AMI/EC2.

image: set the raw filename for the GCE image type

GCP requires that the raw image file inside the archive be named
'disk.raw'.  We set it on the imagePipeline while instantiating the
manifest as a workaround for now.

This should be changed to be configurable on the image type when
necessary, the same way the final filename is defined.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
a1332c5b82 manifest: add more customizations to live image
Extend the OSCustomizations to include options for dnf-automatic,
yum.repos, and gcp.guest-agent.
Propagate these options from the image configs in rhel9 down to the
OSCustomizations and add the stages to the os pipeline when they're
specified.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
581dde4215 manifest: use osbuild firewall options on os customization
OSCustomizations defined firewall options as blueprint firewall
customizations.  Changed it to instead be osbuild firewall stage
options.
This is swapping out one messy thing for another, but at least now we're
consistent in our mess.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
f0e29ab697 image: handle the GCE format in the live image
Add the options required by GCE to a tar pipeline at the end of the
manifest for images defined on the GCE format.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
8c361e510d platform: add special GCE platform format
GCE requires an archive with specific (tar) options, so we should have a
platform format defined separately for it.
Add the GCE image types to the x86_64 arch using the new platform
format.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
3fc8fd2cb9 manifest: support all tar options in tar pipeline
Make the tar pipeline take any implementation of the Pipeline interface
as an input argument.
Add support for all the tar stage options.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
1d1523828a distro/rhel9: update azure images to new definitions
Add the new image function and remove the pipeline function.
Remove the build package set.
Rename the last pipeline to xz for the RHUI image type.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
ae37ebb286 image: support compression on all live image types
Apply "xz" compression to any artifact pipeline if it's specified in the
image type.  The image filename should be applied to the pipeline only
if it's the last one, so we need to skip this assignment if we're going
to add a compression pipeline at the end.

This is a bit dirty but the plan is to remove artifact compression from
manifests and perform it during the export in osbuild-composer, so this
should go away soon.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
458f51128a manifest: set default VPC filename to image.vhd 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
4f60aa3c1d distro/rhel9: move azure image definitions to the top of the file 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
db88c734c1 manifest: add support for waagent and udevrules to liveImage
Extend the OSCustomizations to include the WAAgentConfig and UdevRules
options.
Propagate those options from the image's config down to the
OSCustomizations and add the stages to the os pipeline when they're
specified.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
370a0d1b29 manifest: add packages required by osbuild stages
Add python3-pyyaml to the build root when the OS pipeline includes the
cloud-init stage.
Add python3-iniparse to the build root when the OS pipeline includes
the dnf.config or rhsm stage.
The addition of the stages is detected using the corresponding
customizations.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
c35c4784a4 distro/rhel9: update ec2 and ec2-ha to new definitions
Add the image function and remove the pipeline function.
Remove the build package set.
Add "xz" compression and rename the last pipeline.
Propagate the compression option down to the liveImage() image creation
function.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
6fed422972 image: xz compress a pipeline if the image specifies it
If an image specifies xz compression, add an xz pipeline to the end of
the manifest.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
631b133352 manifest: make NewXZ() take any kind of pipeline for input
The XZ pipeline was initially only for the ostree raw image but now it
can be applied to the output of any pipeline.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
bc219c825e distro/rhel9: add compression option to imageType
The compression field defines the compression type for the final image,
if any.
This is a temporary change.  We should design a better way to specify
"transport compression" for images, compression that is not required by
the image type or platform, but simply done for transport convenience or
storage.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
bd4359ffd9 distro/rhel9: update ami to new definitions
Add the image function and remove the pipeline function.
Remove the build package set: build packages are added to the pipeline
on-demand.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
e2f7e1aed4 manifest: support X11KeymapLayouts in OS pipeline
In the OSCustomizations, we only apply the X11KeymapLayouts from the
image config if the keyboard customization is not specified.
Although we don't support specifying X11KeymapLayouts in the
customizations, it's related to the base Keymap and we should override
both settings in the image config if the base Keymap is specified.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
2541a2219f distro/rhel9: parameterise ec2/ami image config creation functions
Parameterise image config creation functions so that RHSM is added
conditionally based on distro name and version, instead of adding
conditions to the pipelines.
The pipeline generation function function should simply apply the
configurations without needing to know about the image type, distro
name, or version.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
5d760c48ad osbuild: remove Timeservers slice from chrony stage
The org.osbuild.chrony stage was extended to allow additional directives
alongside time servers [1].  The old Timeservers string slice was kept
for backwards compatibility.  Removing support for it in
osbuild-composer makes working with the stage's options simpler.  Using
the new struct slice Servers field and only specifying a Hostname for
each element is equivalent to the old behaviour, so no functionality is
lost.

This simplifies the chrony stage since no validation is required
anymore.
It also simplifies the propagation of configuration options through the
pipeline generation code which doesn't need to check for both types of
stage options.

[1] https://github.com/osbuild/osbuild/pull/692
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
a027b373a5 distro/rhel9: update vmdk image type to new definitions 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
e8285fb2bf manifest: set selinux labels for cp and tar conditionally
Only label cp and tar if they're found in the list of packages for the
build root.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
ce037e24be distro/rhel9: add image-installer to same platform as edge 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
10544e8016 distro/rhel9: update edge types to new definitions
Updated all edge image types except the simplified installer to the new
image definitions.
Added the image functions, based on the Fedora counterparts, and removed
the pipeline functions.
Removed the build package from the image definition.
Removed firmware packages from platform: not available in repos.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
12cbce58cb distro/rhel9: conditionally add RHSM to config
Only add the RHSM options to RHEL and not CS9.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
85a2029b4a manifest: support RHSM stage in OS pipeline 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
89202bdda8 manifest: don't add hostname stage if not specified
Skip the hostname stage if the hostname option is an empty string.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
0c6bbc6dff distro/rhel9: update qcow2 and openstack to new definitions
Add the image function and remove the pipeline function.
Remove the build package set: build packages are added to the pipeline
on-demand based on platform, environment, etc.
Copied the liveImage ImageKind generator from Fedora.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
51ac3e973e distro/rhel9: add support for imageFunc on image types
Add image and environment to the imageType struct.
Also define two new methods:
PackageSetsNew() and ManifestNew()

These get called by the corresponding base methods PackageSets() and
Manifest() when imageType.image is defined.  They will serve to
transition images to the new framework while also supporting the old
until all image types are updated.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
adc7a1cafa distro/rhel9: separate image type definitions
Split image type definitions into separate files by logical groups
(mostly by footprint and cloud platform) for easier navigation.
The image types are now all defined as package globals, like in Fedora.
Image specific package sets are defined in the file for the image type
grouping instead of the package_sets file.

A notable change is in the AWS/AMI types, which have different configs
based on OS version.  To get around this, we create each image type with
a function parameterised by the OS version string.  This is a bit messy
and inconsistent with other image types, but it's a temporary workaround
which will be cleaned up as the definitions get closer to the new
framework used in the fedora package.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
94eb5ca96d distro/rhel9: configure platforms for each image type
Create platform configurations for each image type based on the ones in
Fedora.
Currently this has no effect on the image definition / manifest.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
a0e05ea192 distro/rhel9: add platform to image type
Add platform attribute to imageType like we do in Fedora.
Currently this has no effect.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
1584770307 platform: add configs for ppc64le and s390x 2022-11-28 17:20:49 +01:00
Achilleas Koutsou
fe5835fece distro/rhel9: move imageType implementation to separate file
Move the imageType type and its methods to a separate file to make
them easier to manage.
This might get rewritten to have a default implementation for a lot of
the common functions that have simple, straightforward functionality
(e.g., return the value of an internal field).
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
4ec14ca5da distro/rhel9: move architecture implementation to separate file
Move the architecture type and its methods to a separate file to make
them easier to manage.
This might get merged with the other implementations to make a default
one because they're all (basically) the same.
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
7bcbab0ba5 distro/rhel9: update distribution implementation
Update the implementation of the distro.Distro interface to match the
one in Fedora.  The main change is that the runner is a runner.Runner
and not a string.

The distroMap is replaced by two functions that initialise a
distribution struct from a template (one for CentOS and one for RHEL).
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
310578757a runner: add python or platform-python to all build roots
Required for running osbuild modules.
In Fedora, so far, python3 was always pulled by some other build-root
dependency.  We should add it explicitly since it's required by all our
runners.
For RHEL and CentOS, the requirement is platform-python.

For RHEL 8 and CentOS 8, we also need to explicitly add python36 and for
RHEL 9 and CentOS 9 python3 because it's used to run the stages in
osbuild [1].

[1] ea8261cad6/runners/org.osbuild.rhel82 (L61)
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
5592e6a51f runner: add CentOS runner type
New runner type for CentOS 8 and 9.  Copied from RHEL
2022-11-28 17:20:49 +01:00
Achilleas Koutsou
3fa012b73a runner: fix RHEL runner (was fedora) 2022-11-28 17:20:49 +01:00