Adding support for container embedding.
The containers need to be specified in the image function (imageFunc)
arguments and when specified, propagate down to the OS pipeline
generator to add the necessary stages.
Support is added for RHEL 9.x and Fedora.
Requires a temporary container spec array with the info from the
blueprint for the first initialization of the manifest that's needed
when collecting required packages.
This should be simplified in the future.
Regenerate manifests for the gce and gce-rhui image types with the
rewritten definitions:
- remove unnecessary RPMs in the build root
artifact
- some stages changed order, but the pipelines be functionally
equivalent
add the image function and remove the pipeline function.
Remove the build package set.
Parameterise image config creation functions so that RHSM is added
conditionally based on distro name, like we did for AMI/EC2.
image: set the raw filename for the GCE image type
GCP requires that the raw image file inside the archive be named
'disk.raw'. We set it on the imagePipeline while instantiating the
manifest as a workaround for now.
This should be changed to be configurable on the image type when
necessary, the same way the final filename is defined.
Extend the OSCustomizations to include options for dnf-automatic,
yum.repos, and gcp.guest-agent.
Propagate these options from the image configs in rhel9 down to the
OSCustomizations and add the stages to the os pipeline when they're
specified.
OSCustomizations defined firewall options as blueprint firewall
customizations. Changed it to instead be osbuild firewall stage
options.
This is swapping out one messy thing for another, but at least now we're
consistent in our mess.
GCE requires an archive with specific (tar) options, so we should have a
platform format defined separately for it.
Add the GCE image types to the x86_64 arch using the new platform
format.
Regenerate manifests for the Azure and VHD image types with the
rewritten definitions:
- remove unnecessary RPMs in the build root
- changed order of rhsm stage: this should have no effect functionally
Apply "xz" compression to any artifact pipeline if it's specified in the
image type. The image filename should be applied to the pipeline only
if it's the last one, so we need to skip this assignment if we're going
to add a compression pipeline at the end.
This is a bit dirty but the plan is to remove artifact compression from
manifests and perform it during the export in osbuild-composer, so this
should go away soon.
Extend the OSCustomizations to include the WAAgentConfig and UdevRules
options.
Propagate those options from the image's config down to the
OSCustomizations and add the stages to the os pipeline when they're
specified.
Adding python3-pyyaml to the build root for manifests that contain
cloud-init in the OS pipeline.
Adding python3-iniparse to the build root for manifests that contain
rhsm or dnf.config in the OS pipeline.
Add python3-pyyaml to the build root when the OS pipeline includes the
cloud-init stage.
Add python3-iniparse to the build root when the OS pipeline includes
the dnf.config or rhsm stage.
The addition of the stages is detected using the corresponding
customizations.
Regenerate manifests for the EC2 and EC2-HA image types with the
rewritten definitions:
- remove unnecessary RPMs in the build root
- added dofstools to the image
- internal raw image filename changed: this has no effect on the final
artifact, since it's compressed and renamed before exporting.
- rhsm stage order changed.
Add the image function and remove the pipeline function.
Remove the build package set.
Add "xz" compression and rename the last pipeline.
Propagate the compression option down to the liveImage() image creation
function.
The compression field defines the compression type for the final image,
if any.
This is a temporary change. We should design a better way to specify
"transport compression" for images, compression that is not required by
the image type or platform, but simply done for transport convenience or
storage.
Regenerated manifests for the AMI image type with the rewritten
definitions:
- removed RPMs in the build root: packages are added to the build-root
on-demand by the payload pipelines.
- added dosfstools to the image: our new pipelines add fs tools for
every filesystem type that appears in the partition table.
- rhsm stage order changed.
In the OSCustomizations, we only apply the X11KeymapLayouts from the
image config if the keyboard customization is not specified.
Although we don't support specifying X11KeymapLayouts in the
customizations, it's related to the base Keymap and we should override
both settings in the image config if the base Keymap is specified.
Parameterise image config creation functions so that RHSM is added
conditionally based on distro name and version, instead of adding
conditions to the pipelines.
The pipeline generation function function should simply apply the
configurations without needing to know about the image type, distro
name, or version.
Regenerate all manifests that used the old chrony stage options with the
'timeservers' field. This is now replaced by the 'servers' field, which
is a slice of objects with a 'hostname' field instead of a slice of
strings.
The org.osbuild.chrony stage was extended to allow additional directives
alongside time servers [1]. The old Timeservers string slice was kept
for backwards compatibility. Removing support for it in
osbuild-composer makes working with the stage's options simpler. Using
the new struct slice Servers field and only specifying a Hostname for
each element is equivalent to the old behaviour, so no functionality is
lost.
This simplifies the chrony stage since no validation is required
anymore.
It also simplifies the propagation of configuration options through the
pipeline generation code which doesn't need to check for both types of
stage options.
[1] https://github.com/osbuild/osbuild/pull/692
Regenerated manifests for the vmdk image type with the rewritten
definitions:
- removed RPMs in the build root: packages are added to the build-root
on-demand by the payload pipelines. All removed package were
unnecessary for building the image.
Regenerated manifests for the edge image types with the rewritten
definitions:
- removed RPMs in the build root: packages are added to the build-root
on-demand by the payload pipelines. All removed package were
unnecessary for building the image.
- pipeline names changed.
- rpm stage for os pipeline now sets dbpath and ostree-booted option
(see [1] for original change in Fedora).
- installer pipelines changed to no longer use bootiso.mono stage, but
instead break it down to component stages (see [2] for original change
in Fedora).
Fedora IoT manifests updated to include selinux labelling changes.
The skopeo and containers storage stage have been temporarily dropped
since the new pipeline generator functions don't support it yet.
[1] https://github.com/osbuild/osbuild-composer/pull/2914
[2] https://github.com/osbuild/osbuild-composer/pull/3020
Updated all edge image types except the simplified installer to the new
image definitions.
Added the image functions, based on the Fedora counterparts, and removed
the pipeline functions.
Removed the build package from the image definition.
Removed firmware packages from platform: not available in repos.
Regenerated manifests for the QCOW2, OCI, and openstack image types with
the rewritten definitions.
The skopeo stage is not yet added by the live image pipeline.
The following changes in the manifests should affect the final image:
- Removed RPMs in the build root: packages are added to the build-root
on-demand by the payload pipelines. All removed packages were
unnecessary for building the image.
- /usr/bin/tar removed form selinux stage: tar should be added on-demand
if it's needed by any of the payload pipelines.
Add the image function and remove the pipeline function.
Remove the build package set: build packages are added to the pipeline
on-demand based on platform, environment, etc.
Copied the liveImage ImageKind generator from Fedora.
Add image and environment to the imageType struct.
Also define two new methods:
PackageSetsNew() and ManifestNew()
These get called by the corresponding base methods PackageSets() and
Manifest() when imageType.image is defined. They will serve to
transition images to the new framework while also supporting the old
until all image types are updated.
Split image type definitions into separate files by logical groups
(mostly by footprint and cloud platform) for easier navigation.
The image types are now all defined as package globals, like in Fedora.
Image specific package sets are defined in the file for the image type
grouping instead of the package_sets file.
A notable change is in the AWS/AMI types, which have different configs
based on OS version. To get around this, we create each image type with
a function parameterised by the OS version string. This is a bit messy
and inconsistent with other image types, but it's a temporary workaround
which will be cleaned up as the definitions get closer to the new
framework used in the fedora package.
Move the imageType type and its methods to a separate file to make
them easier to manage.
This might get rewritten to have a default implementation for a lot of
the common functions that have simple, straightforward functionality
(e.g., return the value of an internal field).
Move the architecture type and its methods to a separate file to make
them easier to manage.
This might get merged with the other implementations to make a default
one because they're all (basically) the same.
Update the implementation of the distro.Distro interface to match the
one in Fedora. The main change is that the runner is a runner.Runner
and not a string.
The distroMap is replaced by two functions that initialise a
distribution struct from a template (one for CentOS and one for RHEL).
Required for running osbuild modules.
In Fedora, so far, python3 was always pulled by some other build-root
dependency. We should add it explicitly since it's required by all our
runners.
For RHEL and CentOS, the requirement is platform-python.
For RHEL 8 and CentOS 8, we also need to explicitly add python36 and for
RHEL 9 and CentOS 9 python3 because it's used to run the stages in
osbuild [1].
[1] ea8261cad6/runners/org.osbuild.rhel82 (L61)
