Our package set was quite outdated. Let's sync it with the Fedora Cloud
kickstart:
https://pagure.io/fedora-kickstarts/blob/c3b160775a3b159f949ba931dcb68e520a460e12/f/fedora-cloud-base.ks
I manually compared built images and our image contain kernel, kernel-modules
and linux-firmware above the official package set:
- removing kernel and kernel-modules is tricky, because we always assume that
the default kernel is called kernel.
- removing linux-firmware is a big hack, since it breaks the RPM dependencies
inside the image.
Thus, let's ignore these for now, it's definitely better than before.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Fedora Cloud SIG considers the qcow2 (cloud base) image as a good fit also for
OpenStack. Let's make our openstack image just an alias of qcow2. Once again,
this removes some duplicated code.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Fedora doesn't build a separate AMI image. The ordinary Cloud Base image
is what's used for AWS.
Here's the code that's responsible for uploading ec2 images - it searches
for Cloud raw.xz images:
fcbface137/fedimg/consumers.py (L114)
And here's the pungi configuration showing that qcow2 and raw.xz are actually
built from the same kickstart:
https://pagure.io/pungi-fedora/blob/e080c0702f38c033025889e5fcc8d9fee5bb2026/f/fedora-cloud.conf#_142
Thus, we can just do the same thing: Let's base the AMI on the qcow2 to save
some bytes.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Demonstrate the new workflow for resolving containers.
1. First call Manifest().
2. Get container SourceSpecs from manifest struct.
3. Resolve them.
4. Serialize() with resolved container specs.
The changes in the test manifests are just the information about the
container sources (was a slice but is now a map) and the actual manifest
object isn't affected.
The TestDistro_Manifest test in distro_test_common is adapted
accordingly as well.
When creating a Manifest object, collect container SourceSpecs instead
of resolved Specs.
This is the same way we handle packages: The blueprint option is
converted to source specs and attached to the Manifest object during
creation. Later, the SourceSpecs will be resolved to full container
Specs and used during serialization.
Much like the GetPackageSetChains() manifest method, these two new
methods collect the container and ostree source specifications from the
pipelines that support them. Currently, only one pipeline per manifest
contains references to containers or ostree commits, but we collect them
in a map, keyed by the pipeline name, both for consistency with the
package sets and for any potential future changes that may require
differentiating which pipeline a content source belongs to.
The ImageType.PackageSets() function is going away and instead we will
rely on the ImageType.Manifest() function to both prepare the manifest
and return the package sets. The Manifest() function should never be
called without an ostree ref for ostree type images.
Use the new manifest generation procedure in the distro tests.
Use assert instead of require in TestImageTypePipelineNames to continue
running the rest of the subtests after a failure.
Some initialisations (image options and blueprint customizations) had to
be adjusted to work with the ImageType.Manifest() function.
Some helper functions in distro_test_common are no longer necessary and
have been removed.
The TestPipelineRepositories and TestImageTypePipelineNames tests must
be (partially) skipped for image types which specify a workload
(currently only azure-eap7-rhui), because they ignore payload
repositories.
The merging of payload repositories into the os pipeline had the
unwanted side effect of using the payload repos for the first depsolve
in the os chain when instead they should only be used for the second
(the depsolve for the blueprint or workload packages). This wasn't an
issue before because we didn't do the merging in the PackageSets()
function, but now we rely on the Manifest() function for that
functionality instead.
Before the merging of the two functions, the PackageSets() function did
not merge repositories and the repository-to-package-set mapping was
maintained correctly, but the merging was needed in the Manifest()
function so that rpm stage options were generated for all repositories.
With this change, we are removing the merging so that the mapping is
maintained, and will fix the rpm stage option generation in the pipeline
generator.
Use the new manifest generation procedure in the Weldr API.
Updated test distro to include the same packages from the PackageSets()
method in the Manifest.Content.PackageSets.
Copy the functionality of the ImageType.PackageSets() methods into
ImageType.Manifest() for each distro.
The Manifest() method now collects all package sets and repositories
from the blueprint and image type and after generating the Manifest
instance, calls the GetPackageSetChains() method to attach the computed
package sets to the Manifest before returning it.
The package sets in the call are now renamed to staticPackageSets to
differentiate from the dynamic (computed) package sets that are affected
by the manifest generation.
Pass the entire Blueprint to Manifest() instead of just the
Customizations. The goal is to combine the functionality of the
ImageType.PackageSets() and ImageType.Manifest() methods into one call.
Return manifest.Manifest from the Manifest() function without
serializing. The caller then has to call the manifest.Serialize()
function using the depsolved packages.
This moves towards changing the order of actions required to generate a
manifest. With this change, the manifest creation and depsolving can be
done independently, but this still requires instantiating the manifest
object twice (InstantiateManifest() is called in PackageSets() and
Manifest()), which we don't want to have to do.
Move the FactsImageOptions from distro to the new rhsm/facts package.
At the same time define the values we use as an enum, including the
"test-manifest" value.
Though the values don't really matter, the test value is defined first
so it takes the 0 value, which feels nicer conceptually.
The field in the distro.ImageOptions is changed to be a pointer to allow
for nil values.
Same as with the container SourceSpec, the struct specifies the required
information to resolve an ostree commit from a source (URL, ref, and
optional parent).
Renaming for consistency.
Copy the Marshal and Unmarshal functions from distro.Manifest to
manifest.OSBuildManifest to keep the same behaviour.
The Version() function isn't used, so let's drop it.
Removing the dependence of the manifest package on the distro package to
import manifest into distro.
Wherever arch names are needed, we use the enums from the platform
package instead.
Move the subscription options from distro to its own package.
Now we can import the manifest package into the distro package (instead
of the other way around) so we can work with the manifest.Manifest type
in distro.
Make checkOptions() take the whole blueprint and options. There is no
need to pass in the resolved containers separately since we only care
whether there are any containers defined for image types that don't
support them.
For backward compatibility, revert changes related to hybrid boot mode
for RHEL (RHUI) EC2 images before 9.3 release.
This change does not affect CentOS Stream 9 AMI images nor the RHEL AMI
build by the service or on-premise.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
For backward compatibility, revert changes related to hybrid boot mode
for RHEL (RHUI) EC2 images before 8.9 release.
This change does not affect CentOS Stream 8 AMI images nor the RHEL AMI
build by the service or on-premise.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Set the user provided BP customizations related to custom files and
directories to the iot raw-image type, to ensure that these get
created while deploying a commit.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
On RHEL-8, the x86_64 AMI / EC2 images used a BIOS-only partition table
layout, because the base partition table unification happened in the
past only on RHEL-9 and Fedora (inherited from RHEL-9).
To make things consistent and uniform across RHEL-8 and RHEL-9, I copied
the base partition table used by RHEL-9 AMI / EC2 images to RHEL-8. This
has a side-effect for aarch64 AMI / EC2, where the `/boot` partition
size changed from 512 MiB to 500 MiB, together with the partition GUID
to "Extended Boot Loader Partition GUID".
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The image already used base partition table with necessary layout to
support hybrid boot mode, so the change was just a matter of modifying
the associated platform.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
The image already used base partition table with necessary layout to
support hybrid boot mode, so the change was just a matter of modifying
the associated platform.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Fedora distro definition contained an empty `s390x` architecture with no
image types added to it. Let's remove it from the distro definition,
since it's adding no value in its current form.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
As a preparation to be able to determine the image boot mode when
importing it to the target environment (e.g. AWS), expose the
information on the `ImageType` level.
The image boot mode is determined based on the platform associated with
it.
The new method is not yet used by any code, but will be eventually used
by osbuild-composer server to set the proper value in the upload target
options for the worker. The worker will be then able to import the image
in the proper way to the cloud environment.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
This was the intention since the beginning (based on images built by
Google. Clean up code and mark the platform associated with GCE image
types as UEFI-only.
The only missing part is the default partition table used by the GCE
image, which is shared with other image types and still contains the
BIOS boot partition. I added a TODO comment to preserve this
information, but kept things as they are for now to not have to
introduce a new set of GCE-specific base partition tables.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Do not extend the image base package set with list of packages needed
for booting the OS, returned by `bootPackageSet()` based on the specific
image type, architecture and its boot type. This duplicated
functionality that is already handled by the platform associated image
and all the necessary packages are provided by the platform's
`GetPackages()` method and added to the base package list.
This reflects changes which were done in Fedora when it was ported to
the "new" image definitions, but were not ported to RHEL.
RHEL-8 GCE image type note:
After a previous change, the image boot type is now determined by the
associated platform and as a result, the GCE image type is marked as
supporting hybrid boot type, although it was meant to be UEFI only. As a
result, the package list returned by `bootPackageSet()` and previously
appended would contain grub2 BIOS-related packages. This is still the
case after this change, because the platform's `GetPackages()` method
will return the same list of packages in this case. However, the
platform used by RHEL-8 GCE image type has its `GetPackages()`
overridden by a different implementation not containing grub2 BIOS
related packages. For some reason, this change is not present in RHEL-9.
As a result, the grub2 BIOS related packages disappeared from the RHEL-8
GCE image package set, while there was no change in RHEL-9.
Keep the GCE image as is for now and make it an UEFI-only in a follow
up.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Remove the `legacy` from `architecture` struct, since this information
is already contained in the platform associated with the image.
This reflects changes which were done in Fedora when it was ported to
the "new" image definitions, but were not ported to RHEL.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
Remove the bootType from imageType and architecture structures and
determine the image boot type based on its associated platform.
This reflects changes which were done in Fedora when it was ported to
the "new" image definitions, but were not ported to RHEL.
GCE image type note:
This change has a side-effect on the GCE image type. It was meant to be
UEFI only, but the previous mixture of bootType set in the imageType and
the platform used for it made it a weird combination of almost hybrid
boot type, but not completely. For now, the grub2 BIOS-related packages
are added to the image content as a result. Eventually, the platform
used for the image should be changed to not support BIOS and the image
should also not have BIOS partition at all.
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
