This is the first step to support embedding container images. Here
we add the `containers []container.Spec` argument to supply images
with resolved container specifications. For now all distros will
return an error in case a container is actually supplied since none
of them currently support embedding containers. NB: also no apis or
tools will actually resolve containers.
The test_distro Manifest, which is used in tests across multiple
packages, was using the old structure. Updated to the v2 structure and
adapted all tests.
The package sets for an image can depend on the blueprint, and
by the same logic there is no reason it should not be able to
depend on the image options.
This is so far a non-functional change, but makes a follow-up
commit simpler (though still without actually depending on
the image options to compute the package sets).
Move package set chain collation to the distro package and add
repositories to the package sets while returning the package sets from
their source, i.e., the ImageType.PackageSets() method.
This also removes the concept of "base repositories". There are no
longer repositories that are added implicitly to all package sets but
instead each package set needs to specify *all* the repositories it will
be depsolved against.
This paves the way for the requirement we have for building RHEL 7
images with a RHEL 8 build root. The build root package set has to be
depsolved against RHEL 8 repositories without any "base repos" included.
This is now possible since package sets and repositories are explicitly
associated from the start and there is no implicit global repository
set.
The change requires adding a list of PackageSet names to the core
rpmmd.RepoConfig. In the cloud API, repositories that are limited to
specific package sets already contain the correct package set names and
these are now copied to the internal RepoConfig when converting types in
genRepoConfig().
The user-specified repositories are only associated with the payload
package sets like before.
Return dummy values from the following methods:
- PackageSets
- PayloadPackageSets
- PackageSetsChains
Use package set names commonly used by recent distro definitions.
Package sets are based on values used by rpmmd mock implementation.
Adjust two Weldr API unit test check for the dummy values. Without
this fix, these unit tests would start failing after the move to
`rpmmd.DepsolvePackageSets()` in Weldr API compose handler.
Introduce a new method `PackageSetsChains()` to the `ImageType`
interface, which returns a named lists of package sets, which should be
depolved together in a chain.
Extend all distro implementations with the new method.
Add a unit test ensuring that if an image type defines some package set
name chains, that all of the listed package set names are present in the
package set map returned by the image type.
The method is currently not used anywhere. This is a preparation for
switching from current way of depsolving to the chain depsolving.
Adding three more combinations that weren't covered by previous tests:
- Supplying ref, parent, and URL: should result in an error
- Supplying ref and parent: OK
- Supplying parent, and URL: same as first case (ref gets default value
from image type)
Added default OSTreeRef() to test image type to cover the cases where
the ref isn't specified but affects the validation.
Separated and commented the test cases.
This is needed so that we can do different things depending on the
given layout; this will be used in tests for now only. Only GPT
allows for arbitrary number of partitions and once we assert this
in code we will need to adjust the tests accordingly.
NB: This method might be removed again in the future, once generic
LVM support is added everywhere and the ability to differentiate
between MBR and GPT layouts is not needed anymore.
Each image type now implements BuildPipelines(), which returns a list of
pipeline names that set up the build environment, and
PayloadPipelines(), which returns a list of pipeline names that create
the OS image (all non-build pipeline names).
Older distros that produce v1 manifests should call the distro Fallback
functions to return the common defaults.
A Fallback function for the Exports() method is also added and called by
older distros.
All image types that produce v2 manifests (distros after RHEL 8.4)
should include the information in the image type definition and should
not rely on fallbacks for default values.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Because there's only a few combinations of upload types and image types
that make sense, enforce correct combinations by eliminating upload
types.
Fixes#1775
The problem: osbuild-composer used to have a rather uncomplete logic for
selecting client certificates and keys while fetching data from
repositories that use the "subscription model". In this scenario, every
repo requires the user to use a client-side TLS certificate. The problem
is that every repo can use its own CA and require a different pair of
a certificate and a key. This case wasn't handled at all in composer.
Furthermore, osbuild-composer can use remote workers which complicates
things even more.
Assumptions: The problem outlined above is hard to solve in the general
case, but Red Hat Subscription Manager places certain limitations on how
subscriptions might be used. For example, a subscription must be tight to
a host system, so there is no way to use such a repository in osbuild-composer
without it being available on the host system as well.
Also, if a user wishes to use a certain repository in osbuild-composer it
must be available on both hosts: the composer and the worker. It will come
with different pair of a client certificate and a key but otherwise, its
configuration remains the same.
The solution: Expect all the subscriptions to be registered in the
/etc/yum.repos.d/redhat.repo file. Read the mapping of URLs to certificates
and keys from there and use it. Don't change the manifest format and let
osbuild guess the appropriate subscription to use.
composer doesn't support f34 or f35, but it should be possible to build
these even with the f33 distro definition. Introduce f34 and f35 repos
and aliases for f33.
Extend the "Test Distro" implementation and definition to contain two
architectures and make the second architecture contain two image types.
Add New2() function returning another "Test Distro".
Modify the `internal/store` unit tests to reflect changes done to the
"Test Distro".
Signed-off-by: Tomas Hozza <thozza@redhat.com>
imageTypeS2 implements the distro.ImageType interface buts generates a
Manifest matching the new osbuild v2 schema.
Two new image types are added to the rhel84 distro (x84_64 and aarch64)
for generating OCI containers contain an Edge (ostree) commit and, when
run, start a web serer to serve the commit.
The image type uses the new PackageSets map to define packages (and
excludes) for the image. The old methods (Packages() and
BuildPackages()) are implemented for compatibility with the old
workflow.
The image also defines an extra package set for the container that will
serve the package: "httpd" (and its dependencies).
The distro.ImageType interface has a new method: Exports()
It should return a list of names or IDs of artefacts that should be
exported from osbuild when the job is complete.
For the old image types, this is simply set to "assembler".
This replaces Packages() and BuildPackages() by returning a map of
package sets, the semantics of which is up to the distro to define.
They are meant to be depsolved and the result returned back as a
map to Manifest(), with the same keys.
No functional change.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Rather than setting this automagically, expose it to the caller. For
now the only caller we have simply passes it back in, so this is a
noop.
In follow-up commits this will be used to resolve the parent commit.
This is tested by verifying that the generated manifests do not
change.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Imagine this situation: You have a RHEL system booted from an image produced
by osbuild-composer. On this system, you want to use osbuild-composer to
create another image of RHEL.
However, there's currently something funny with partitions:
All RHEL images built by osbuild-composer contain a root xfs partition. The
interesting bit is that they all share the same xfs partition UUID. This might
sound like a good thing for reproducibility but it has a quirk.
The issue appears when osbuild runs the qemu assembler: it needs to mount all
partitions of the future image to copy the OS tree into it.
Imagine that osbuild-composer is running on a system booted from an imaged
produced by osbuild-composer. This means that its root xfs partition has this
uuid:
efe8afea-c0a8-45dc-8e6e-499279f6fa5d
When osbuild-composer builds an image on this system, it runs osbuild that
runs the qemu assembler at some point. As I said previously, it will mount
all partitions of the future image. That means that it will also try to
mount the root xfs partition with this uuid:
efe8afea-c0a8-45dc-8e6e-499279f6fa5d
Do you remember this one? Yeah, it's the same one as before. However, the xfs
kernel driver doesn't like that. It contains a global table[1] of all xfs
partitions that forbids to mount 2 xfs partitions with the same uuid.
I mean... uuids are meant to be unique, right?
This commit changes the way we build RHEL 8.4 images: Each one now has a
unique uuid. It's now literally a unique universally unique identifier. haha
[1]: a349e4c659/fs/xfs/xfs_mount.c (L51)
Rather than getting a set of base packages from the ImageType, and then
appending the requested packages from the blueprint, pass the blueprint
into the new Packages() function, and return the full set of packages to
be depsolved.
This allows us to also append packages based on other customizations
too, and use that to append chrony when the timezone is set. This
matches the behavior anaconda had, and there was a TODO item to do this,
which had been overlooked.
Fixes#787.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Rather than Manifest() returning an osbuild.Manifest object, introduce a
new distro.Manifest object which represents it as an opaque, JSON
serializable object. This new type has the following properties:
1) its serialization is compatible with the input to osbuild,
2) any valid osbuild input can be deserialized into it, and
3) marshalling and unmarshaling to and from JSON is lossless.
This means that even as we change the subset of valid osbulid manifests
that we support, we can still load any previous state from disk, and it
will continue to work just as before, even though we can no longer
deserialize it into our internal notion of osbuild.Manifest.
This fixes the underlying problem of which #685 was a symptom.
Signed-off-by: Tom Gundersen <teg@jklm.no>
When generating an osbuild manifest for an image type, we take a
customizations struct, which specifies the image-type-independent
customizations to apply. We also take the size argument, which is
specific to the image build and not part of the blueprint.
Introduce a new argument ImageOptions, which for now just wraps the size
argument. These options are specific to the image build/type, and
therefore does not belong with the other customizations.
For now this is a non-functional change, but follow-up commits will
introduce more types of image options.
Signed-off-by: Tom Gundersen <teg@jklm.no>
