The host distro object was identical to the regular distro objects for a
while now. The constructors in the registry have been aliases to the
base constructors for a long time.
- Deleted all HostDistro constructors from the distributions.
- Changed the supported distro list to only contain base constructor
functions.
- The host distro in the distro registry is a copy of the base distro
that matches the host and does not call a separate constructor.
After introducing Go 1.18 to a project, it's required by law to convert at
least one method to a generic one.
Everyone hates IntToPtr, StringToPtr, BoolToPtr and Uint64ToPtr, so let's
convert them to the ultimate generic ToPtr one.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Fedora 35 is going EOL on Tue 2022-12-13. At the time of writing this commit
message, that's the next day. As we do releases on Wednesdays, the next
release will never find its way to F35 and thus, there's no point in keeping
support for it.
Let's delete everything that relates to Fedora 35. If there's something that
cannot be deleted (e.g. CI containers based on F35), let's upgrade it to F37.
TestCrossArchDepsolve now uses CentOS Stream 8 because RHEL 8.4 cannot read
F37 repository metadata. This is a similar issue to
https://bugzilla.redhat.com/show_bug.cgi?id=2004853 . Basically, newer
repositories can be only read by libmodulemd >= 2.11.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
- Add a list of image type names to skip with explanations.
- Run the test in two configurations: empty blueprint and blueprint with
named kernel (kernel, kernel-debug).
- Count only unique kernel names. Specifying the same kernel twice
isn't an issue. We might change this in the future, but for now the
kernel selection logic is a bit messy.
- Temporarily skip the image-installer until we settle on a common rule
for both Fedora and RHEL.
Similar to 194351e681 and
9ae47b6843cace3c9bef55d40a33fa11d8de391a.
Manifest initialisation during package collection uses options and
customizations to determine build root packages. Just like adding a
fake ostree checksum and an unresolved container list, we also need to
add the ostree ref, which is used to determine if a pipeline is being
generated for an ostree-based image type.
This fixes manifest generation when an ostree commit or container is
being built with an embedded container through the cloud API. Without
this change, the python toml module is not installed in the build root
and the container storage stage fails.
Adding support for container embedding.
The containers need to be specified in the image function (imageFunc)
arguments and when specified, propagate down to the OS pipeline
generator to add the necessary stages.
Support is added for RHEL 9.x and Fedora.
Requires a temporary container spec array with the info from the
blueprint for the first initialization of the manifest that's needed
when collecting required packages.
This should be simplified in the future.
This adds the `fedora-image-installer` and
`fedora-image-installer-preview` images.
The image installer type installs anaconda-webui on Fedora >= 38 to use
the new UI. It also writes its setting to
`/usr/share/anaconda/interactive-defaults.ks` as the current
anaconda-webui has not yet been tested in kickstart mode.
To do so manifest.Anaconda was expanded to take a (subset) of options
for a KickstartStage which is will write into interactive-defaults.ks.
And to take a list of additional modules to enable, so we can set up
Anaconda with all default modules.
Previously, this just happened silently and let to extremely odd errors. Let's
just print the error to simplify debugging the next time.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
8fdd158799 modified the Cloud API to resolve
ostree commits using a separate job. This change caused the API handler
to call PackageSets without any ostree options (because they are not resolved
yet).
Unfortunately, the new implementation of PackageSets initializes the manifest.
The initialization checks the options and if the type is iot-installer and
it doesn't have the fetch checksum for IoT, it just returns an error.
To work around this (we need an initialized manifest to create the chains),
this commit just gives the initialization method a dummy checksum. The ostree
options currently don't have any effect on the package sets, so this should
be fine.
In order to make this workaround at least slightly sane, a warning is printed,
there's a new test just for this behaviour and a long comment to remember to
delete these lines.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Instead of using the ostree.RequestParams in the OSTReeImageOptions,
define a new struct specific to ImageOptions for the ostree parameters.
This is almost identical to the new ostree.CommitSpec but the meaning of
the parameters changes based on image type and it would not be clear if
the CommitSpec was used in all cases. For example, the parameters of
the new OSTreeImageOptions do not always refer to the same commit. The
URL and Checksum may point to a parent commit to be pulled in to base
the new commit on, while the Ref refers to the new commit that will be
built (which may have a different ref from the parent).
The ostree.ResolveParams() function now returns two strings, the
resolved ref, which is replaced by the defaultRef if it's not specified
in the request, and the resolved parent checksum if a URL is specified.
The URL does not need to be returned since it's always the same as the
one specified in the request.
The function has been rewritten to make the logic more clear.
The docstring for the function has been rewritten to cover all use cases
and error conditions.
Remove the 'fedora' prefix from the canonical name for fedora-iot image
types. Make the previous names aliases.
This has little functional change since we're simply swapping the
canonical name with an existing alias.
Don't redefine the storage unit multiples in each distro, but use the
constants defined in the `common` package. This will make it easier to
split related image type definitions into separate files.
Include the platform packages when getting the build packages for the
RawOSTreeImage.
rpm-ostree is explicitly added for this image type.
dracut-config-generic and efibootmgr are temporarily added here, but we
should define a platform that includes them instead (some cleanup
required in general).
Add the new image type to the list in each architecture and update
tests.
Ignore ostree raw images in Kernel count test in distro_test_common:
Edge and IoT raw images don't need a kernel specified in their OS
pipeline. The kernel (and the OS in general, including all packages)
come from the commit that is pulled and deployed in the image.
This test passes on RHEL (for edge-raw-image types) because the
blueprint defaults to returning the main kernel, but this isn't
necessary and is likely to change in the near future.
Co-Authored-By: Ondřej Budai <ondrej@budai.cz>
The most interesting change is the removal of smc-meera-fonts in 37. As
suggested, rit-meera-new-fonts is used instead.
Existing F35 and F36 manifests updated with package changes.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
These are just super-simple to construct using a small helper.
It would be great if we can make `distro.go` totally version-agnostic but
that's something for the future.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Using basic types as values in the `ImageConfig` structure makes it
impossible to distinguish if the empty value for the type was set
intentionally or if it is just the value the variable was initialized
to. This is very bad especially for `bool` type.
While working on unifying `vhd` and `azure-rhui` image types I found
out, that some newly added variables in the `ImageConfig` structure
were forgotten in the `InheritFrom()` method. This makes it impossible
to inherit their values from a parent configuration. This is however
required for the unification of `vhd` and `azure-rhui` image types. As
described above, it would be impossible to decide whether a `bool` value
should be inherited from the parent configuration or not. The only
solution is to use a pointer to the type. For consistency, use pointer
for all basic types.
Adjust distro implementations accordingly.
Add basic validation to ensure that the oscap
customizations are valid and required fields
have been provided. The validation also ensures
that the manifest generation errors out if
oscap customization has been enabled for older
or unsupported distros.
Implement all of Fedora in terms of this new abstraction. What used to be the
manifest functions (and before that the pipeline functions) are now the image
functions, whose purpose is to instantiate the right image kind structs from the
image type definitions we currently have in the distro definition.
This is the first step to support embedding container images. Here
we add the `containers []container.Spec` argument to supply images
with resolved container specifications. For now all distros will
return an error in case a container is actually supplied since none
of them currently support embedding containers. NB: also no apis or
tools will actually resolve containers.
Fedora 34 is EOL, let's remove all traces of it, including:
- distro definition
- repositories (and test one)
- test manifests
- special package set rules
- hacks from the spec file
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
For now all it does is represent the name of the runner and what requirements
it has of the build pipeline.
Move some package definitions from the runner package set to where it belongs.
This represents how our systems should integrate into their environment, typically using
some sort of agent, or commonly cloud-init.
In the future we could imagine this representing network configuration or any other kind
of configuration necessary to reach the environment as well.
For now EC2 and Azure is supported, and stub environments are
added to show the idea, but these are not implemented/used
yet.
The workload encapsulates what the user wants to run on top of the image. Everything
else we do abstracts away the OS, the hardware, the environment, and what is left is what
matters: the workload.
For now only the `Custom` payload is implemented which requires the user to name the
packages they want installed, the repositories to pull them from and what systemd
services to enable.
A few other stub workloads are added to show the idea, but these are not used.
The ideal is for the workload to have only the minimal number of configuration options.
Pass PackageSets when initialising the Manifest, and read the chains back out.
This also fixes a bug where all repos were always used, rather than filtering per
package set.
Finally, this moves the 'chrony' inclusion from distro.go to the OSPipeline where
it belongs. In doing so the logic is changed slightly, where chrony is now
installed if NTP servers are configured (regardless of source), whereas in the
past it was included if the timezone was set in the blueprint (which made no sense).
If the kernel name is set, then the packaegSpecs must include the kernel package, ensure
this by including the kernel in the package list.
We currently include the kernel both in the userPackages and the base packages.
Including it only in the user packages does not work as the base could end up pulling it
in as well. However, it would be semantically more correct to include it only in the base
set, so if possible we should do that and drop it from the user packages (in a follow-up).
If the partition table includes logical volumes, the lvm2 package should be installed on
the target system.
Drop the corresponding logic from fedora/distro.go.
These objects describes the hardware an image runs on. Including
- architecture
- bootloader
- required firmware
Use the platform abstraction to move firmware packages out of the package set
definitions.
Currently an image type could override the boot loader in the architecture, but we should
not allow an image type to select a bootloader type the architecture does not support. So
only in case the architecture supports hybrid boot do we allow the image type to select
one of the two types.
This enables UEFI on AMIs for aarch64.
The build packages should always be computed from its
dependents and there should be no need to override it. Drop the
ability, and all the unused code in fedora/*.
NOTE: due to a bug in a previous patch in this PR the extra
package set was never set on the build pipelines.
No longer name the packageSetChains after the package set, but
keep them named after the pipelines. This should be a
non-functional change as dnf-json does not care about what the
chains are called, only that the names are unique.
Rather than providing packageSpecs when constructing the
Manifest, do so at Serialize() time.
This allows the overall flow to be:
```
m := Manifest.New()
m.AddPipeline()
...
c := m.GetPackageSetChains()
p := Depsolve(c)
m.Serialize(p)
```
Before you read this patch, be warned: this is crazy. But rest
assured, this is as bad as it gets, and by the end of this PR it
will all be infinitely simpler than it ever was.
We want Manifests to be self-describing, and in particular, we want
manifest.GetPackageSetChains() to return what
ImageType.GetPackages does today. This should be achieved by
pipelines knowing their required package sets, and the caller
optionally amending additional package sets at Pipeline
initialisation time.
As a first step, while pipelines don't yet know anything about their
required package sets, simply pass in the precomputed
PackageSetChains and set them as optional on each pipeline. Then
we can read the chains back out of t he manifest.
This is not a functional change, but allows us to gradually move
package set definitions from the distros to the pipelines in follow
ups.
The package sets for an image can depend on the blueprint, and
by the same logic there is no reason it should not be able to
depend on the image options.
This is so far a non-functional change, but makes a follow-up
commit simpler (though still without actually depending on
the image options to compute the package sets).
Make it the responsibility of each pipeline to track its required
sources, and use this to generate a manifest from its pipelines.
This removes the dependency on osbuild2 from distro.go, and the
only other uses of osbuild2 are to specify parameters in
ImageConfig, which will eventually go away too.
As a consequence, this only attaches the required sources for a
given manifest, rather than all known packages in the package set
map.
Add a container image type that is based on the existing fedora
container image. There is a delta in terms of the configuration
because osbuild does not yet provide all the neccessary means,
but the package set is already very close.
