As it turned out, people make mistakes and forget to write some parts of
code, unless a unit test screams at them. This is true for the
`InheritFrom()` method, which is not handling all members of the
`ImageConfig` structure.
Use reflection, instead of inheriting from each specific hard-coded
structure member. This will make the implementation future-proof in case
the `ImageConfig` structure is extended with additional members.
Using basic types as values in the `ImageConfig` structure makes it
impossible to distinguish if the empty value for the type was set
intentionally or if it is just the value the variable was initialized
to. This is very bad especially for `bool` type.
While working on unifying `vhd` and `azure-rhui` image types I found
out, that some newly added variables in the `ImageConfig` structure
were forgotten in the `InheritFrom()` method. This makes it impossible
to inherit their values from a parent configuration. This is however
required for the unification of `vhd` and `azure-rhui` image types. As
described above, it would be impossible to decide whether a `bool` value
should be inherited from the parent configuration or not. The only
solution is to use a pointer to the type. For consistency, use pointer
for all basic types.
Adjust distro implementations accordingly.
In podman v4.0.0 the default network backend was switched from cni to
netavark. However, podman will choose cni if there are already
containers, images, or cni networks preset on a system [1].
Starting with podman v4.2.0, containernetworking-plugins is no longer a
hard requirement for podman. So when an edge commit is built with an
embedded container, podman v4.2.0+ will choose the cni network and fail
with an error because the plugin isn't installed.
Adding the package explicitly alongside podman to avoid this issue with
future RHEL 9.1 edge builds when they include containers.
This change does not affect test manifests. The package is already
included in manifests as a dependency of podman < v4.2.0.
See rhbz#2123210
[1] a083f790ab/pkg/config/containers.conf (L275-L278)
For each of the supported distros start a goroutine to depsolve
'filesystem' which will preload the metadata making subsequent responses
faster.
This is safe to do without limits because we only supposed a limited
number of distros, and without additional locking because this is the
the same as hitting the API with multiple depsolve requests at the same
time.
Support for creating multiple amis from a single compose. It uses the
AWSEC2* jobs to push images to new regions, and share them with new
accounts.
The compose it depends upon has to have succeeded.
See https://github.com/BurntSushi/toml/issues/360
A recent change in BurntSushi/toml made encoding fail (later changed to
error) if a struct is marked as omitempty and is comparable. Go docs about
equality: https://go.dev/doc/go1#equality. Basically: A struct is comparable
if all of its fields are comparable. Slices are not comparable.
Customizations are marked as omitempty but they contain a lot of slices,
thus they are not comparable. The new version of BurntSushi/toml therefore
panics when we encode them.
The solution is to remove the omitempty tag from Customizations.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
When running an osbuild job, we read `/etc/redhat-release` to get the
host OS name to attach as metadata to the job result.
Only Fedora and RHEL ship this file, which makes the osbuild job always
fail on other distributions.
The main reason to report host OS back to the worker server is due to
Koji composes and the koji-finalize job, which pushes it to Koji. The
motivation is to have enough information to potentially re-instantiate
/ identify the original builder host OS. There are no specific
requirements on the string.
Modify the code to use `/etc/os-release` to determine the host OS. Fall
back to using `linux` as the host OS, in case reading `os-release`
fails, log the error and continue with the job. The `linux` fallback is
suggested by the `os-release` spec [1]
[1] https://www.freedesktop.org/software/systemd/man/os-release.html#ID=
Co-authored-by: Achilleas Koutsou <achilleas@koutsou.net>
The search command is more complicated than depsolve and dump. It needs
to return results based on the requested package names and globs.
Add a number of mock responses for the new search command, including
search results, all packages, and error responses that are triggered by
using special package names: nonexistingpkg, badpackage1, baddepsolve.
Instead of fetching all available packages from dnf-json and then
searching the results this uses SearchMetadata when a package name or
glob is passed to the API. It only uses FetchMetadata when fetching
the full list of packages.
This also fixes a bug where the error response to a projects/info
request used the id of 'ModulesError'. It now uses 'ProjectsError'.
Since the oscap remediation stage in osbuild runs
the oscap package in `chroot`, it is necessary to
install the `openscap-scanner` package to the image
itself rather than the build root.
The module is not present in official RHEL-9.1 ISO image and it is
causing boot issues when used with newer content. HTTP boot is
not affected by this change and works as expected.
Having the GPG check enabled for Google repos in `gce*` images will make
DNF try to import the relevant keys when upgrading, downgrading or
installing any packages from the repo. However due to Google still using
SHA-1 for GPG keys used to sign their RPMs, importing it will make any
transaction that includes such RPM to fail.
Disabling the GPG check will ensure that DNF won't attempt to import
Google GPG keys.
Related to https://issuetracker.google.com/issues/223626963
The repo is not needed any more, because the Google Cloud SDK is not
installed in the images by default. If anyone wants to install the SDK,
they can add the appropriate repo definition.
The repo is not needed any more, because the Google Cloud SDK is not
installed in the images by default. If anyone wants to install the SDK,
they can add the appropriate repo definition.
The Google SDK ships pre-compiled binaries. It is undesirable to install
it by default in `gce` and `gce-rhui` in its current shape. Also not
installing it does not anyhow affect the RHEL integration as the guest
OS in GCP.
The Google SDK ships pre-compiled binaries. It is undesirable to install
it by default in `gce` and `gce-rhui` in its current shape. Also not
installing it does not anyhow affect the RHEL integration as the guest
OS in GCP.
Extract the application into a utility method on `PartitionTable`.
In order for it to be usable for the first and second pass it does
take a `create` argument that controlls whether new partitons will
be created or return.
Since the LVM support was added to all distros, our disk
related code is adaptive, i.e. we will set the correct BLS
and grub2 prefix if there a `boot` partiton is present in
the layout after all customizations happen, which includes
LVMification.
One thing that was not yet fully working was layouts that
do not yet have a `/boot` partition but allow LVMification.
In that case `NewPartitionTable` and if `/boot` was the
first (or only) customization, would LVMify the partition
which in turn would create the `/boot` partition; but after
`newPT.ensureLVM()` the call to `newPT.createFilesystem`
with `/boot` would try to create another `/boot` mountpoint.
In order to deal with this situation correctly we are now
using a two phase approach: 1) enlarge existing mountpoints
and collect new ones. 2) if there are new ones and LMVify
was allowed, switch to LVM layout. Do a second pass and now
create or enlarge existing partitions, handling `/boot` in
the process.
Replace the simple allow list of paths with the more sophisticated
path policies. It enables us to e.g. allow one path but not any
sub-path. This will be useful for `/boot` where we want to allow
its customization but not any sub-path because that might actually
break booting.
Build a new path policy struct, ased on the new path trie struct.
It is designed to be able to store policies for paths. A Check
method can then be used to look up the policy for a given path
based on the defined policies.
Add a simple implementation of a path trie structure that can be
used to look up assoicated data for any given path. The constructor
will build the trie from a dict of paths to associated data. Later
modification is currently not support. Add tests for it creation
and lookup.
Add basic validation to ensure that the oscap
customizations are valid and required fields
have been provided. The validation also ensures
that the manifest generation errors out if
oscap customization has been enabled for older
or unsupported distros.
Add a package with the constants of the
valid oscap profiles. Add a function to
validate the available profiles against
an allow map of supported profiles. The
allowed function checks for both exact
matches and shorthand versions of the
oscap profiles.
Add support for embedding container images via the cloud API. For
this the container resolve job was plumbed into the cloud api's
handler and the API specification updated with a new `containers`
section that mimics the blueprint section with the same name.
