Add a Distro enum to the Manifest struct for selecting package
selection.
Packages are sometimes renamed between distribution versions and since
we do the package selection in the Manifest, we need a way to select
distro-version-specific package names inside the manifest initialiser.
This may change in the future.
Do not expose the content of the manifest statically and instead rely on
the public methods to retrieve source specifications dynamically.
Since the methods require iterating through the pipelines to collect
source specifications, we should avoid calling the function multiple
times when we can reuse the returned values.
Define a public pipeline implementation that allows initialising with
content, serialising with resolved content, but produces no stages.
This is useful for testing.
Add an ostree commit source (instead of a resolved commit spec) to
pipelines that support ostree commits. Source specs are used when
initialising a manifest for package selection. The resolved commit spec
is added after manifest initialisation through the serialization
function for stage creation.
Pipelines that require or support an ostree commit (either as payload or
a parent) must return the source specs using getOSTreeCommitSources()
after initialisation and the commit specs using getOSTreeCommits()
during serialization.
Pipelines that don't require packages didn't need to implement the
serializeStart() method, but now we need to set the resolved ostree
commit spec when a pipelines requires it.
When creating a Manifest object, collect container SourceSpecs instead
of resolved Specs.
This is the same way we handle packages: The blueprint option is
converted to source specs and attached to the Manifest object during
creation. Later, the SourceSpecs will be resolved to full container
Specs and used during serialization.
Much like the GetPackageSetChains() manifest method, these two new
methods collect the container and ostree source specifications from the
pipelines that support them. Currently, only one pipeline per manifest
contains references to containers or ostree commits, but we collect them
in a map, keyed by the pipeline name, both for consistency with the
package sets and for any potential future changes that may require
differentiating which pipeline a content source belongs to.
In getPackageSetChain(), the workload repositories did not include the
ExtraBaseRepos.
In serialize(), when creating the rpm stage options (which collects
repository GPG keys), only the base repos were used, which is why we
previously had to merge repositories. Instead of merging repositories
in the calling function in distro, we should keep them separated so that
we can easily distinguish which repositories are only meant for the
blueprint or workload when we need to.
Add Content to the Manifest struct to represent unresolved packages,
container, and ostree commit specs that the caller should resolve before
serialising the manifest.
Move the FactsImageOptions from distro to the new rhsm/facts package.
At the same time define the values we use as an enum, including the
"test-manifest" value.
Though the values don't really matter, the test value is defined first
so it takes the 0 value, which feels nicer conceptually.
The field in the distro.ImageOptions is changed to be a pointer to allow
for nil values.
Add two new methods to the interface for returning container source
specifications and ostree commit source specifications respectively.
These are analogous to the package set chains; they are unresolved
source specifications that can be used with the appropriate resolver to
create a full specification for a manifest.
Only the Base pipeline implements the base empty functions for these for
now.
Copy the Marshal and Unmarshal functions from distro.Manifest to
manifest.OSBuildManifest to keep the same behaviour.
The Version() function isn't used, so let's drop it.
Removing the dependence of the manifest package on the distro package to
import manifest into distro.
Wherever arch names are needed, we use the enums from the platform
package instead.
Move the subscription options from distro to its own package.
Now we can import the manifest package into the distro package (instead
of the other way around) so we can work with the manifest.Manifest type
in distro.
The post install script customises the selinux policy, but is gated on
`selinuxenabled`, which will fail inside the buildroot container.
As a result it's never executed.
One of the IOT use cases for custom files and directories is to be able
to enable a custom systemd unit. This was previously not possible to do
via BP Customizations.
In addition, the test case for custom files and directories in
ostree-based images (currently only build-time images) takes advantage
of enabling a custom systemd unit file, this this will enable using the
same test scenario also for deploy-time ostree image types.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Extend the `OSTreeDeployment` with a list of custom files and
directories, that can be created in the deployed ostree commit. This
functionality is equivalent to the one that was added for the OS
pipeline.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This platform copies more files into `/boot` which are necessary to be
able to boot IoT on some single board computers.
We also immediately set this on the `Aarch64_IoT` platform which needs
u-boot to be placed in the `/boot`.
This closes#3312.
Add a helper function that collects all the manifest list digests from a
list of container specs and returns a FilesInput to be used with the
stage.
Use the function in the OS pipeline when adding containers. The
manifests input to the stage constructor will be empty if there are no
manifest lists in the container specs.
The skopeo stage in osbuild supports an second optional set of inputs
called `manifest-lists`. This is an array of files, i.e.,
`org.osbuild.files` type input.
To support this we need a new type for the skopeo stage inputs that can
encompass both input types, images and manifest-lists.
It turned out that when we stopped setting the kernel options in grubenv
file, we also stopped setting them in /etc/default/grub under
`GRUB_CMDLINE_LINUX`. This file is used by grub2-mkconfig when
generating grub configuration.
10_linux script executed by grub2-mkconfig recently started to overwrite
the /etc/kernel/cmdline, if its timestamp is older than the timestamp of
/etc/default/grub [1]. As a result, all kernel options were wiped out from
/etc/kernel/cmdline.
Make sure that we always set the `KernelOptions` in the grub2 stage
options, even if the `WriteCmdLine` is set to `false`.
In addition, unify the way we concatenate kernel options set in the
grub2 stage options. Some pipeline implementations were previously using
space, other were using comma. Space is now used everywhere.
Regenerate all affected image manifests.
[1] https://src.fedoraproject.org/rpms/grub2/c/fc76aed5333f56dd05400521a35b944a5df52ebc
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Fix the following error reported by Coverity:
```
Defect type: SUPPRESSED_ERROR
1. osbuild-composer-75/_build/src/github.com/osbuild/osbuild-composer/internal/manifest/os.go:546: suppressed_error: The error returned by the function "GetVerStrFromPackageSpecList(p.packageSpecs, "dracut-config-rescue")" is not explicitly checked.
2. osbuild-composer-75/_build/src/github.com/osbuild/osbuild-composer/internal/manifest/os.go:546: remediation: Explicitly check the return error.
544| }
545|
546|-> rescueVer, _ := rpmmd.GetVerStrFromPackageSpecList(p.packageSpecs, "dracut-config-rescue")
547| hasRescue := rescueVer != ""
548| bootloader = osbuild.NewGrub2LegacyStage(
```
Change the code to check if the called function returned an error or
not. If not error is returned, then the function returns a package
version string.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This is a workaround to make the systemd believe it's firstboot
when ignition runs on real firstboot.
Right now, since we ship /etc/machine-id, systemd thinks it's not firstboot
and ignition depends on it to run on the real firstboot to enable services from presets.
Since this only applies to artifacts with ignition and changing machineid-compat at
commit creation time may have undesiderable effect, we're doing it here as a stopgap.
We may revisit this in the future.
This patch also pins the version of osbuild because it depends on a fix
for the ignition stage.
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
Tracing the package set and repository journey, I found that the
container package defined on the image type is never really used.
Added notes to fix later.
Function that filters a list of repositories to return only the global
repos and any that define a given pipeline or package set name in their
list of PackageSets.
Extend the `OSCustomizations` with a list of custom files and
directories, that should be created in the image. If any `Files` or
`Directories` are specified in the `OSCustomizations`, the appropriate
osbuild stages will be added to the `os` pipeline. In addition to that,
any custom files data will be returned by the `getInline()` method of
the `os` pipeline.
This customization can't be yet used by users, because the translation
from BP customization to the `OSCustomizations` is missing.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Move the handling of the list of enabled and disabled systemd services
more to the end of the `os` pipeline, just before the SELinux stage.
This has no functional effect on produced images, but it will make it
nicer once the handling of the custom files and directories will be
added to the pipeline. Specifically it should be added right before the
services stage to allow enabling custom service files, but after all
other configurations that are applied to the image.
Regenerate all manifests.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
There are 3 different ways for subscriptions to be setup, test the
pipeline to make sure it includes the correct commands, and check the
package set chain to make sure it includes the packages needed to
support the selected subscription mode.
When rhc is selected it will install the required packages, register
using rhc and always enable insights.
When rhc is not selected it will use subscription manager for
registration, and optionally enable insights. Also installing required
packages.
When the image definitions were updated to the new framework, I failed
to update the dracut modules for Anaconda installers to match the
existing ones.
The changes in the manifest are at commit
c4af0a1886.
The nvdimm module and the additional drivers were removed.
The nvdimm module in particular is required for http boot but should
only be specified for RHEL 9. In RHEL 8 it is part of the default set
of modules.
See 02bb7a0b4f and
dc95382ba3 for the original commits that
introduced these changes.
The `FilesInputs` was since the beginning an XZ-specific implementation
of the input, but it was implemented in the `files_input.go` in a false
hope that it could be used as a generic stage inputs by any stages. It
turned out that various stages require different implementation of
its input. Specifically there is usually a stage-specific key, which has
assigned a common input type. For XZ stage, the key is `file`.
Remove `FilesInputs` and instead implement `XzStageInputs` which is now
accepted by the XZ stage.
Fix all affected pipeline implementations that use XZ stage.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Rework files input implementation to support all reference types
supported by the input schema. Also implement helper functions to
generate supported reference types. In some cases, the reference
supports e.g. referencing multiple pipelines in the stage inputs, but
this is currently not implemented, since no pipeline in composer uses
it.
Rework the files input unit tests to cover the new functionality.
Adjust all code affected by the changes made to files input.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
