Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The image type is only ever known (externally) as image-installer.
Renaming the internal variables and functions to reflect the name makes
the code easier to navigate.
Followup from, f34380d5b5 and
3a1765a5a8, copied to the rest of the RHEL
distro definitions.
For now, these customizations have no effect on the manifest.
The new `with-users` variants of the edge-installer test cases include
the user customizations in the blueprint, but the manifests are
(currently) the same as the corresponding base cases.
Add a new parameter `lvmify` to `NewPartitionTable` that, if set to
`true`, will cause the root partition to be wrapped in LVM in case
it is not in a LVM volume group. Set this to `false` for now so no
actual change should happen anywhere. Layouts where the root is
directly on a LUKS container are not yet supported.
Add tests for this.
This commit fixes#2347 by ensuring that a minimum
size of 1GB is set for all file systems. The only
exception to this is the `/usr` which is set to 2GB,
since this was the only mountpoint that was previously
being checked.
Replace the old CreateParittionTable() function with the new one called
NewPartitionTable() which works with the new interface types and
supports container-type setups (LUKS, LVM ,and Btrfs).
Changed usage in distro packages to take and carry around a pointer to
the new PartitionTable rather than a concrete type. The
NewPartitionTable() function returns a deep clone of the base
PartitionTable so the new pointer type can be moved and (if necessary)
modified freely without affecting the distro base PT.
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
This is needed so that we can do different things depending on the
given layout; this will be used in tests for now only. Only GPT
allows for arbitrary number of partitions and once we assert this
in code we will need to adjust the tests accordingly.
NB: This method might be removed again in the future, once generic
LVM support is added everywhere and the ability to differentiate
between MBR and GPT layouts is not needed anymore.
Do not rely on `distro.imageOptions` having any size information,
i.e. `Size` being `0`. Instead use `imageType.Size()` and the
information in the blueprint customization to calculate the size.
This makes the individual distro definitions idenpendent of the
API entry points that currently calculate the size, e.g.:
internal/cloudapi/v1/v1.go:PostCompose line 184
internal/cloudapi/v2/v2.go:PostCompose line 197
internal/kojiapi/server.go:PostCompose line 135
internal/weldr/api.go:composeHandler line 2289
Modify the signature of `CreatePartitionTable` so that it is
possible to return errors from the function. This is not yet
used, but will be in the near future. Change all call sites
accordingly: in most cases we can just bubble up the error.
Pass the `basePartitionTable` argument of `CreatePartitionTable`.
Now that we clone the partition table at the beginning of the
method there is no need to pass a copy of the partition table.
Each image type now implements BuildPipelines(), which returns a list of
pipeline names that set up the build environment, and
PayloadPipelines(), which returns a list of pipeline names that create
the OS image (all non-build pipeline names).
Older distros that produce v1 manifests should call the distro Fallback
functions to return the common defaults.
A Fallback function for the Exports() method is also added and called by
older distros.
All image types that produce v2 manifests (distros after RHEL 8.4)
should include the information in the image type definition and should
not rely on fallbacks for default values.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Previously it was only possible to configure separate partitions
for mountpoints in the allow list and their immediate subdirectories
only i.e. /var & /var/log
This fix allows for an arbitrary level of mountpoints, i.e. /var/log/audit,
/var/a/b/c/d/e and so on
OSBuild Composer can now build the RHEL 8.5 Raw Images. This images are
compressed raw images, i.e. a file that has a partition layout with an
deployed OSTree commit in it. It can be used to flash onto a hard drive
or booted in a virtual machine. An existing OSTree commit needs to
be provided.
The following image new types are supported: edge-raw-image.
Instead of using package sets at the distro, arch and image type
level and then merging them in `PackageSets`, store the function
that generates the package set in the image type and have them
return all the package set. In order to do so, they now take an
imageType parameter so that they can also return architecture
dependent packages.
Instead of having a common build package set defined at distro
struct level and merging them together with build packages in
the image type (and arches), we do the "inheritance" at the
package set level and append more specific packages to base
sets there. We also now ensure that each image type does have
a build package set defined.
The actual package set should not change for anything due to
this commit.
Split the common installer build packages from the one specific to
anaconda and edge.
NB: The "inheritance" is now done in the package sets rather than
outside, via package set merging.
The edge specific build packages, `edgeBuildPkgsKey` where defined
on the distro level but also always included in all actual edge
image types; there were thus duplicated.
This adds a new installer called the "Simplified Installer" for Edge.
In contrast to the existing insaller, which is based on Anaconda, this
new installer based on the CoreOS installer project[1], a small rust
based binary that is executed in the initramfs and will flash a raw
image to a specified installation device. For this a new blueprint
option is introduced. The raw image is created from an existing OSTree
commit and embedded into the resulting bootable iso. When booting the
iso the installation will automatically start witout any interaction
from the user.
NB: As with the existing edge installer, support is currently limited
to x86. The new installer also does not support non-uefi boot.
[1] https://github.com/coreos/coreos-installer
Co-Developed-by: Achilleas Koutsou <achilleas@koutsou.net>
Co-Developed-by: Antonio Murdaca <runcom@linux.com>
When building RHEL for Edge commits and a parent together with an
URL was specified, add a `org.osbuild.ostree.passwd` stage which
then will pre-load the uid/gid database with the data from the
parent commit. This ensures that uids and gids do not change for
the "child" commit.
Running the container on Openshift requires that the process inside the
container run without special permissions.
Switching to nginx and setting the following options that don't require
root privileges:
- Port 8080 (> 1024)
- pid file in '/tmp' instead of the default '/run' path
Also, the log file is chmod-ed to be world writable. Nginx always writes
to the default log file on startup, even if a different log file path is
specified in the configuration.
See rhbz#1945238
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Renamed tar-installer to image-installer.
This is a more appropriate name:
- It disassociates the image type from the "tar" image type. The two
should not be perceived to be connected.
- It's more descriptive. The format of the payload (tar) isn't relevant
to the purpose of the image type.
The problem: osbuild-composer used to have a rather uncomplete logic for
selecting client certificates and keys while fetching data from
repositories that use the "subscription model". In this scenario, every
repo requires the user to use a client-side TLS certificate. The problem
is that every repo can use its own CA and require a different pair of
a certificate and a key. This case wasn't handled at all in composer.
Furthermore, osbuild-composer can use remote workers which complicates
things even more.
Assumptions: The problem outlined above is hard to solve in the general
case, but Red Hat Subscription Manager places certain limitations on how
subscriptions might be used. For example, a subscription must be tight to
a host system, so there is no way to use such a repository in osbuild-composer
without it being available on the host system as well.
Also, if a user wishes to use a certain repository in osbuild-composer it
must be available on both hosts: the composer and the worker. It will come
with different pair of a client certificate and a key but otherwise, its
configuration remains the same.
The solution: Expect all the subscriptions to be registered in the
/etc/yum.repos.d/redhat.repo file. Read the mapping of URLs to certificates
and keys from there and use it. Don't change the manifest format and let
osbuild guess the appropriate subscription to use.
Add new image type definitions `ec2` and `ec2-ha` representing the
official RHEL ec2 image types.
Add a `xzArchivePipeline()`, which returns a pipeline producing a XZ
archive from a file produced by a different pipeline.
Add rpmrepo snapshots for `rhui` and `ha` repositories used to generate
image test cases. `rhui` is used by the `ec2` image and it is available
on x86_64 and aarch64 architectures. `ha` is used by the `ec2-ha` image
and it is available only for x86_64.
The new image type definitions are currently not used by any
API test case.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Previously, the support of UEFI has been captured only on the level or
architecture definition as a binary boolean value. In reality some of
the architectures are able to support legacy, UEFI or hybrid boot.
Introduce a new BootType value, defined on the architecture level, which
can be set to one of the three boot types mentioned above. The value set
on the architecture level can be overridden on the image type level in
the image type definition.
Add two unexported helper methods to the `imageType`, specifically
`getBootType()` which returns the boot type that should be used for the
image type and architecture combination. The values set explicitly in
the image type or architecture definition should not be used directly.
Second added method is `supportsUEFI()`, which returns boolean value
representing the fact if the image type supports UEFI boot.
Split and define the boot package sets separately for the legacy and
UEFI boot. The `PackageSets()` method of the imageType structure is
modified to take the boot type into consideration and append appropriate
package sets to the "os" package set.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Redefine the `ami` image type in RHEL-8.5 to be based on RHEL
ec2 images. The pipeline has different default settings, therefore the
common "os" pipeline is not used. The RHEL ec2 images have a different
default size than the original `ami` image definition. The RHEL ec2
images use a different default partitioning scheme. Their configuration
is slightly different for each architecture and the x86_64 version
of the image does not support UEFI.
Update rpmrepo snapshots used to generate RHEL-8.5 x86_64 and aarch64
image test cases.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
If there's no kernel in the main package set, the standard/default
kernel will be added while depsolving. This causes issues when an
alternative kernel is selected in the blueprint. Both kernels will be
installed (one from the blueprint and one from the main OS set) which
causes issues with ostree image types.
EDGE image types are defined under a different name for RHEL-8.5,
specifically they don't contain the "rhel-" prefix any more. To ensure
backward compatibility, add image type aliases for all EDGE image types
with the "rhel-" prefix.
Image type aliases are used only when getting a specific imageType
instance by its name. When listing all available image types for an
architecture, only the current image type names are returned, without
any aliases. This prevents the image types from being exposed multiple
times under different names via Weldr API.
Extend the distro unit tests to test image type aliases.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Originally, a copy of an architecture instance was always created when it
was added to a distro definition using the `addArches()` method.
However in reality, only a subset of structure members were copied,
which could create unexpected behavior and issues. This behavior is
identical to the behavior when image types are added to an architecture.
However the situation with image types differs in one aspect,
specifically that a single image type definition is usually reused
by multiple architecture definitions, while an architecture definition
is always used only by a single distribution definition.
Due to the fact that the image type contains a reference to the
architecture to which it has been added, the creation of a copy can not
be reasonably avoided. On the other hand, adding a copy of an architecture
to a distribution definition is not necessary.
Downside of creating copies of the architecture is that the image types
associated with it referred always to the original architecture
definition instance and not to the copy. So while references in the
direction of Distro -> Arch -> Image Type were correct and working, the
other direction was broken. Image Type -> (original) Arch -> (nil)
Distro.
Modify `distribution.AddArches()` method to directly add the passed
architecture instances to the distribution definition, instead of adding
their copies.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
To avoid packages specified in a blueprint from conflicting with exclude
lists, we depsolve blueprint packages separately and pass them into the
Manifest generator under the new "blueprint" package set key.
This approach has the added benefit that dependencies of packages
specified in the blueprint are not subject to exclusion in addition to
the explicitly named packages.
The OS pipeline which installs the packages for the base system merges
the two package sets before running the RPM stage. The signature of the
function is changed to explicitly require blueprint packages be
specified (though `nil` or empty slice is valid).
The kernel selection test is adapted to merge the package sets before
counting kernel package.
Adaptation of changes in
https://github.com/osbuild/osbuild-composer/pull/1349
