Commit graph

75 commits

Author SHA1 Message Date
Tom Gundersen
919c484c79 distro: add Manifest method to interface
For now, this simply wraps Pipeline and Sources, and retruns the
resulting manifest object. In the future, Pipeline and Sources
may be dropped from the interface.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-03-03 22:25:49 +01:00
Lars Karlitski
17ca73ab14 osbuild-pipeline: validate distro when fetching it
We were verifying two things: if the passed distroArg exists in the
distribution mapping in common/types.go and if the it is an actually
registered distro. Since you cannot have distros registered that don't
correspond to a type, the first test is unnecessary.

Merge the two tests by moving the (much better) error message down into
the second test. This makes DistributionExists redundant, because
Registry.GetDistro() checks this implicitly.

Also, move ListDistributions() to the Registry object, because we want
to show distributions that are actually registered.

Add a test which checks that Registry.List() works and that all included
distributions register correctly.
2020-03-03 16:21:13 +01:00
Lars Karlitski
87e9c39532 distro: panic less often
Return errors from all distro's New() functions instead of logging and
returning nil. Also, return errors instead of panicking from
NewRegistry() and NewDefaultRegistry().
2020-03-03 11:53:06 +01:00
Lars Karlitski
d1965d6268 distro/*: don't go via Registry object in tests
These packages (and their tests) shouldn't access the distro package,
because that's cyclic.

Also, these packages should only test the objects they expose.
2020-03-03 11:53:06 +01:00
Lars Karlitski
ee3ef81714 distro: implement NewDefaultRegistry in terms of NewRegistry()
This makes it more apparent that NewDefaultRegistry() is a convenience
function and gets rid of a helper.
2020-03-03 11:53:06 +01:00
Lars Karlitski
e5eb673be6 distro: rename awkwardly named function
WithSingleDistro() doesn't follow go's naming convention for creating
objects (New*). Rename it to NewRegistry() and rename the old
NewRegistry() to NewDefaultRegistry().

The idea is that NewRegistry() can be used to create full Registry
objects from outside the package. NewDefaultRegistry() is a convenience
function that creates a Registry with all known distros.
2020-03-03 11:53:06 +01:00
Jacob Kozol
2a3a5b318b distro: add rhel-8.1 support
Images can be built for rhel 8.1. The pipeline generation and distro
tests are based off of the rhel 8.2 ones. Repository information as been
added for rhel 8.1. The repo urls are internal ones and will only work
if the user is on the Red Hat vpn.
2020-02-29 01:17:50 +01:00
Ondřej Budai
9e505b6659 distro: add public method for getting the host distro name
It will be useful in integration testing.
2020-02-26 16:58:39 +01:00
Brian C. Lane
cace702e9f Register Fedora 32 distro 2020-02-19 23:37:45 +01:00
Martin Sehnoutka
8e77e03284 distro: make it impossible to initialize registry with nil values
The current `NewRegistry` implementation allows for nil values in the
map, but this leads to subtle bugs when using the registry. This patch
enforces non-nil values by introducing additional checks before we
insert the value into the map.

The change unfortunately breaks a lot of tests and therefore it is
necessary to create additional mock: distro.

The new mock is used instead of the previous "real" implementation,
which used to contain nil values.
2020-02-19 23:37:45 +01:00
Jacob Kozol
fae51095d2 distro: add fedora 32 support
Images can be built for fedora 32. The pipeline generation and distro
 tests are based off of the fedora 30 ones. Repository information has
 also been added for the fedora 32 repos.
2020-02-19 14:18:58 +01:00
Jacob Kozol
bbbd9b9193 distro: add fedora 31 support
Images can be built for fedora 31. The pipeline generation and distro
tests are based off of the fedora 30 ones. Repository information has
also been added for the fedora 31 repos.
2020-02-18 12:34:17 +01:00
Tom Gundersen
2d675ea51b distro: add a Sources method
This outputs the sources needed for the pipeline generated for the
distro. At the moment no pipelines require sources, and so this
always returns the empty list.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-02-14 14:43:27 +01:00
Tom Gundersen
fefe5db4ce distro/pipeline: take depsolved packages as argument
This is unused for now, but will allow us to generate pipelines with
the pre-depsolved NEVRAs, so osbuild does not need to depsolve again.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-02-14 14:43:27 +01:00
Tom Gundersen
7f0f6c001e distro: expose BuildPackages
We want to be able to include the build packages when depsolving a
blueprint, so expose the list.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-02-14 14:43:27 +01:00
Tom Gundersen
42e8ca19be distro: expose BasePackages
We want to be able to include the base packages when depsolving a
blueprint, so expose the list.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-02-14 14:43:27 +01:00
Tom Gundersen
f0f2e80a97 distro: expose the ModulePlatfromID
This is needed for depsolving, so expose it from the distro package
so it can be passed to dnf-json (and not only to osbuild) as that does
depsolving too.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-02-14 14:43:27 +01:00
Tom Gundersen
d133454d91 pipeline: rename package to osbuild
Rename the package from `pipeline` to `osbuild` to reflect that it
will no longer be specific to pipelines, but rather covers all
osbuild datatypes.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-02-14 14:43:27 +01:00
Ondřej Budai
3ff88c336f fix ineffectual assignments found by golangci-lint 2020-02-14 12:41:09 +01:00
Martin Sehnoutka
edf081153a distro: remove previous implementation of testing distro
See internal/distro/fedoratest for a replacement
2020-02-12 11:17:26 +01:00
Martin Sehnoutka
d02bac15fe distro: use the new types in distro packages
Each distro is now represented by a strongly typed value but it can also
provide its name as a string.
2020-02-12 11:17:26 +01:00
Martin Sehnoutka
73994eb06d distro/fedoratest: create new testing distribution
The problem with the previous one was that we used artifical image
types, architectures, and distribution but that is not possible any more
because we don't want to use plain strings any more. This commit
introduces a new testing distro which uses proper, existing types.
2020-02-12 11:17:26 +01:00
Jacob Kozol
abb5469b30 distro: get proper image size from distro
When a user does not define the image size for a compose the default
image size of that image type is used. In order to properly store the
compose's image size even if the default is used the store calls the
distro function GetSizeForOutputType. This function accepts an output
format and image size. If the image size is 0 then the default
value for the output format will be returned. Also, for vhd images the
size must be rounded. This is now handled in the distro function instead
of the api.
2020-02-07 14:49:15 +01:00
Jacob Kozol
a86e697171 distro: add custom image size
When creating a pipeline the assembler includes an image size. This
image size can be set when creating the pipeline but if it is 0 then a
default image size will be used. The default is 2 GB except for ami
images which are 6 GB.
2020-01-23 00:57:31 +01:00
Tom Gundersen
4919ef1271 repositories: install the base repositories in the filesystem
During development of a new distro, we need to test composer against
nightly or beta repositories, but we cannot ship composer itself
with the nightly repository information hardcoded in. At the same
time, we want to distinguish between the system repositories of the
host and the repositories we use to generate images (the host may not
use the same distro/version/architecture as the target, and it may
include custom repositories that the target should not).

We therefore ship per distro repository information that can be
overriden (typically in testing) by dropping files in /etc.

For now use the latest nightlies for RHEL-8.2, we may want to
replace these with the official mirrors for GA eventually.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-01-03 22:35:52 +01:00
Tom Gundersen
2059f9500b distro/test: flesh out the test distro abit and use it for testing
We were using fedora-30 as a test-distro and tar as test-output, but
that causes lots of churn in the tests when we refactor things. Use
the test distro instead, when generic functionality is being tested
and restrict testing of the individual distros to the distro-specific
tests.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-01-03 22:35:52 +01:00
Tom Gundersen
5fcdd4bea5 distro: don't use a global variable to store the distro register
Introduce a DistroRegister object. For now this does not introduce
any functional changes, as the object is always instantited to be
the same. However, in follow-up patches it will get options.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2020-01-03 22:35:52 +01:00
Martin Sehnoutka
029cd9260f Tweak the Fedora vhd output format to boot in Azure 2020-01-02 12:57:27 +01:00
Lars Karlitski
73641ea904 distro: rename openstack and vmdk images to match lorax 2019-12-16 01:32:41 +01:00
Lars Karlitski
8c4ee795f7 distro: add Distro.Name()
Right now, there is no way to get at the name from a Distro instance.
We will need this to include the distro's name in the job we pass to the
worker, for instance.
2019-12-15 22:05:31 +01:00
Tom Gundersen
ba32f58ac8 distro/rhel82/qcow2: bring package set in line with official guest image
Also hook up the tast case to verify that the image boots.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-15 21:41:33 +01:00
Tom Gundersen
5e41526d16 distro/rhel82: add aarch64 support
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
b18c66f9fb distro/rhel82: add UEFI support
This follows the style used in fedor-30.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
55cc82d545 distro/fedora-30: add aarch64 support
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
db300cb481 distro/fedora-30: add UEFI support
On architectures that require EFI, we must create the ESP partition
and use a GPT partition table. We must also install either the UEFI
or the legacy version of GRUB2 in the image.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
8c1f194fc1 distro/fedora-30: set the module_platform_id
Make sure we don't fallback to looking at /etc/os-release.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
18971bd8ef pipeline/stage/grub2: support explicit legacy/uefi API
This allows legacy bootloader and UEFI bootloaders to be
enabled/disabled independently of each other.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
4c892d6d32 pipeline/assembler/qemu: rework partition table handling
Move to the new options format, allowing more flexible partition
tables. The pipeline changes, but the result should be the same.

This requires a yet-to-be-released version of osbuild.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
dbdac222a5 pipeline/stage/fstab: represent UUID as a string
These may not be strictly speaking well-formed UUID. Some filesystems
use other kinds of identifiers.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
506b012f08 distro: take architecture as an argument
Allow bootloader specific packages to be defined per architecture,
and allow repositories to depend on the architecture.

This does not altert he pipelines we produce, part from the ami
image now contains the grub2-pc package, rather than the grub2
package. This should make no difference.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
b034117e62 distro: sort the packages passed to the dnf stage
This should not have any effect apart from hiding any internal changes
to the way we compile the lists.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 17:23:31 +01:00
Tom Gundersen
6e2731deb8 distro/rhel82: fix the repositories
Fix these while we develop so we are guaranteed all tests run against
the same packages.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Tom Gundersen
e3be97990b distro/rhel82: include dracut-config-generic
We must make sure not to depend on the modules loaded in the host
kernel. This makes the initrd the same between different hosts.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Tom Gundersen
85ff8bcec2 distro/rhel82: exclude the timedatex pacakge for now
As for other output types, there is an SELinux bug that makes this
uninstallable.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Tom Gundersen
5b9d85dece distro/rhel82/vhd: drop stale comments
Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-14 13:08:35 +01:00
Lars Karlitski
b60bb1d011 distro: pass compression argument to the tar assembler 2019-12-13 12:21:47 +01:00
Lars Karlitski
42f46a5992 distro/rhel82: make openstack's package list constistent with lorax 2019-12-13 00:14:07 +01:00
Martin Sehnoutka
b7b60363dc Modify RHEL8.2 VHD image to boot in Azure 2019-12-12 13:52:21 +01:00
Tom Gundersen
d074395474 distro/rhel-8.2: update VHD images to be closer to lorax
The current one was taken from fedora and that failed both due to an
SELinux bug in RHEL8.2, but also because the list of packages was
wrong for the distro.

We don't enable WAAgent, which we probably should. And there are a few
other config tweaks to cloud-init and WAAgent that we skip too.

Signed-off-by: Tom Gundersen <teg@jklm.no>
2019-12-12 13:52:21 +01:00
Lars Karlitski
2b42612336 distro: insert custom sources into pipeline
This is an additional argument on `distro.Pipeline` right now. In the
future, we probably want this per-blueprint.
2019-12-11 21:19:32 +00:00