Commit graph

291 commits

Author SHA1 Message Date
Achilleas Koutsou
8d64cb9665 distro/rhel85: const strings for package set keys
const string keys make the names easier to reuse or change and is less
error prone.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
5909ca202d disk: boot partition finder as partitiontable method
Moved the function that searches for the boot partition index to the
PartitionTable struct as a method. The method returns -1 if not found
and it's now the responsibility of the caller to handle the case where
it is not found.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
cba720e63f rhel85: depsolve blueprint packages separately
To avoid packages specified in a blueprint from conflicting with exclude
lists, we depsolve blueprint packages separately and pass them into the
Manifest generator under the new "blueprint" package set key.

This approach has the added benefit that dependencies of packages
specified in the blueprint are not subject to exclusion in addition to
the explicitly named packages.

The OS pipeline which installs the packages for the base system merges
the two package sets before running the RPM stage. The signature of the
function is changed to explicitly require blueprint packages be
specified (though `nil` or empty slice is valid).

The kernel selection test is adapted to merge the package sets before
counting kernel package.

Adaptation of changes in
https://github.com/osbuild/osbuild-composer/pull/1349
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
190f473b04 distro/rhel85: update unit tests 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
d6ec5aa59e distro/rhel85: update ami package set
Based on http://git.app.eng.bos.redhat.com/git/spin-kickstarts.git/tree/rhel8/rhel-8.5-ec2.ks
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
b8d34d7679 distro/rhel85: introduce the ami image type
This is a copy of the ami image type from RHEL 8.4.
Small modifications are necessary.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
1f005a55b9 distro/rhel85: don't always install bootloader
Don't install bootloader unless `platform` (imageType.arch.legacy) is
defined.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
9c3c956655 distro/rhel85: aarch64 uefi: true 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
3f0f4ffab9 distro/rhel85: simplify edge types and package sets
- Cleaned up distro-specific edge package sets
- Added edge package set merging in PackageSets() function
- Edge image type definitions are no longer arch specific, just like the
  other image types
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
78920efc36 distro/rhel85: partition table fixes
- Automatically find /boot partition (fallback to root)
- Partition sizes need a bit of a shrink
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
876a67ccb0 distro/rhel85: introduce the openstack image type 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
ceba83dcca distro/rhel85: introduce the vmdk image type 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
2ef462bdaa distro/rhel85: add dhcp-client to vhd packages
Is no longer installed by default but is required to boot in Azure
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
1ed9008785 distro/rhel85: introduce the vhd image type 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
3eae16badb distro/rhel85: remove installer type for aarch64
The installer image type needs package definitions and testing before it
can be included in aarch64.
Removing for now.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
f09bf063c8 distro/rhel85: tidy up package sets
- Move all package set definitions to package_sets.go file.
- Do not append image package sets to arch and distro sets; this happens
  automatically in PackageSets() method.
- Fix architecture-specific package sets (boot and build where
  available).
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
dec74dba32 distro/rhel85: add image and qcow pipelines
Live image pipeline: Creates the live image in a file through a loopback
device.

Stages:
- truncate: create the file to hold the image
- sfdisk: partition the device
- mkfs.fat, mkfs.xfs: create the filesystems
- copy: copy the tree from the previous pipeline (the OS pipeline) into
  the directories where the partitions are mounted
- grub2.inst: install the bootloader

QEMU pipeline: Convert the live image from the previous pipeline to a
qcow2 image.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
4cf26bb628 distro/rhel85: split coreStages() implementation
coreStages() was meant to produce the stages that are common between all
image types. Since there are too many exceptions and differences between
traditional and edge image types, keep the two pipelines separately
instead.

Moved SELinux stages to the end of each pipeline.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
39371b01ba distro/rhel85: document selinuxStageOptions arg
Renamed the argument to have clearer (and correct) meaning and added a
function docstring to describe the purpose.

The argument is set to 'true' in the build pipeline.
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
708de084aa distro/rhel85: add grub2 stage options 2021-07-24 15:54:00 +02:00
Achilleas Koutsou
bf61ca422f distro/rhel85: introduce the qcow2 image type
- distro definition
- qcow2 package set
- pipeline generation function (unfinished)
2021-07-24 15:54:00 +02:00
Achilleas Koutsou
fd4a8387f7 distro/rhel85: partition table generator function
Copied from RHEL 8.4
2021-07-24 15:54:00 +02:00
Ondřej Budai
5ae6203d65 distro: drop Fedora 32
It's EOL.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-07-05 11:16:08 +02:00
Achilleas Koutsou
5f53b3865f rhel85: append "boot" package sets to "packages"
If an image type is marked "bootable", append the package set with  name
"boot".
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
e8dd5b3248 rhel85: move package sets to separate file
Package sets shared between multiple image types are statically defined
in functions.
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
49df4a70af rhel85: add base build package set to distribution 2021-07-01 12:48:32 +02:00
Achilleas Koutsou
f1e805658c rhel85: distro and arch package sets
Add package sets to the top level distribution and the
architectures. Merge package sets when requested through
`imageType.PackageSets()`.
This allows us to have "cascading" package sets:
    distro > arch > image type

Previously this was only supported for the build packages. The idea is
generalised based on the idea of arbitrary package sets.

Certain package sets have special meaning (for the distro):
- packages: blueprint packages and special option packages are merged
  into this one.
- build: is not expected to be defined on the image type so it is merged
  from the distro and arch explicitly.
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
38f5b23e87 rhel85: tar image type
Traditional tar image type.
Pipelines generation function for tar image type uses the same pipelines
and stages as the tar installer, but exports the OS image directly.
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
ec381fefb9 rhel85: generalise and simplify tar stage creation 2021-07-01 12:48:32 +02:00
Achilleas Koutsou
7e1ab365c8 rhel85: change kickstart file location
Global const for defining kickstart path for all installer types.
Kernel options added to point to path.

Kickstart file renamed to osbuild.ks for all types and placed in the
bootiso root.

Fixes #1363
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
32c69b3cf9 rhel85: add subscription stage to core pipeline 2021-07-01 12:48:32 +02:00
Achilleas Koutsou
242f6d3c44 rhel85: tar installer main pipeline
Define a core set of packages and stages shared by images.
2021-07-01 12:48:32 +02:00
Achilleas Koutsou
19d15af788 rhel85: new image type tar-installer 2021-07-01 12:48:32 +02:00
Ondřej Budai
e1e4a1f845 distro/rhel90: add s390utils-core to the build pipeline for s390x
A recent update to crypto-policies changed its requirements from:

Recommends: grubby

to:

Recommends: (grubby if kernel)

We don't install kernel in the build pipeline, thus grubby was now no longer
installed. This caused also s390utils-core to not be installed on s390x.
s390utils-core has to be in the build pipeline though because we use
/usr/sbin/zipl from it to install the bootloader to s390x images.

Long story short, images for s390x currently cannot be build because
/usr/sbin/zipl is no longer in the build pipeline. This commit fixes that
by explicitly adding s390utils-core to the package list for s390x.

I verified this fix on an actual s390x machine. The test manifest was
regenerated but it's actually still the same because we use an older
snapshot that doesn't contain the newest crypto-policies change.

Kudos to Tomáš Hozza for helping me out!

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-06-30 14:48:57 +02:00
Achilleas Koutsou
779103cb96 rhel85: add tests 2021-06-29 17:26:43 +01:00
Achilleas Koutsou
139c8eb3c4 Revert "rhel85: nolint package for unused private functions"
Pipeline generation functions are now used by the new image definitions.

This reverts commit cca084d5ce.
2021-06-29 17:26:43 +01:00
Achilleas Koutsou
f789f785a4 rhel85: rhel-edge image definitions
Image definitions added for all three edge types:
edge-commit
edge-container
edge-installer

Manifest pipelines generation methods attached to image types.
2021-06-29 17:26:43 +01:00
Ondřej Budai
9bcd219528 distro/rhel90: make selinux stage the last one
There are currently some context mismatches in the image because of the new
files from the sysconfig stage. Let's move the selinux stage to the end so
we fix them.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-06-29 13:56:26 +02:00
Ondřej Budai
4c16b2f77e distro/rhel90: include qemu-guest-agent in qcow2
Fixes COMPOSER-990

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-06-29 13:56:26 +02:00
Ondřej Budai
c37705e7d5 distro/rhel90: move all package lists to a separate file
There's currently the Package Deletion Process going on in CentOS Stream 9.
When a maintainer wants to get rid of a package, several steps must be taken
before the package is completely removed from all places. We want to include
osbuild-composer in the process so packages that are used here are not deleted
without us noticing it. Thus, maintainers going through the process should be
able to easily see all packages that osbuild-composer includes in images.

This commit moves all package lists from rhel90/distro to rhel90/packages
to allow CentOS Stream maintainers to easily packages required by composer.

Not a functional change.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-06-29 13:56:26 +02:00
Achilleas Koutsou
dce3e1de9e distro: test manifest version detection 2021-06-29 09:33:05 +01:00
Achilleas Koutsou
e87394fea1 distro: method for detecting manifest version
Small helper method for distro.Manifest that can detect the schema
version of the Manifest.
2021-06-29 09:33:05 +01:00
Martin Sehnoutka
71233fd860 distro: add fedora 34 and 35 aliases to f33
composer doesn't support f34 or f35, but it should be possible to build
these even with the f33 distro definition. Introduce f34 and f35 repos
and aliases for f33.
2021-06-05 20:31:45 +02:00
Ondřej Budai
a19b883727 distro/rhel90: enable cloud-init
cloud-init was enabled explicitly in the image-factory kickstart and thus we
need to explicitly enable it too.

Fixes: rhbz#1960309
Fixes: COMPOSER-920
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2021-05-17 14:55:50 +02:00
Tomas Hozza
fba9fe1072 Make the Distroregistry FromHost() return distro with correct name
Composer does not have 1:1 mapping of what can be the Host Distro name
and the names of supported distributions held in the Distroregistry.

The fact that the host distro `Name()` method as passed to the Weldr API
does not return the same name as what is used as distro name for
repository definitions. This makes it hard to use `distro.Distro` and
`distro.Arch` directly and rely on the values returned by them as their
name.

Add `New*HostDistro()` to all distro definitions, accepting the name
that should be returned by the distro's `Name()` method. This is useful
mainly if the host distro is Beta or Stream variant of the distro.

Change the distroregistry.Registry to contain host distro as a separate
value set when creating it using `New()` function. This value is
returned by `Registry.FromHost()` method. Determining the host distro is
handled by the `NewDefault()` function. Move the distro name mangling to
distroregistry package. Add relevant unit tests.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-05-14 15:43:00 +02:00
Tomas Hozza
f7f064274a Tests: remove fedoratest and replace it with test_distro
fedoratest was yet another dummy distribution used by unit tests. After
the rework of test_distro, there is no reason to not use it as the only
distro implementation for testing purposes.

Remove fedoratest distro and replace it with test_distro in all affected
tests.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-05-14 15:43:00 +02:00
Tomas Hozza
e5dd45b71c Extend the "Test Distro" definition and modify affected tests
Extend the "Test Distro" implementation and definition to contain two
architectures and make the second architecture contain two image types.
Add New2() function returning another "Test Distro".

Modify the `internal/store` unit tests to reflect changes done to the
"Test Distro".

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2021-05-14 15:43:00 +02:00
Achilleas Koutsou
cca084d5ce rhel85: nolint package for unused private functions
Package contains unexported helper functions for future implementations
of pipelines.
2021-05-11 12:10:09 +02:00
Achilleas Koutsou
e8a1509d46 rhel84: only download parent commit for installer
The parent commit would be added to the sources unconditionally. This
is only necessary for the edge-installer image type.

This doesn't technically change the build behaviour of an existing
distro and image type.  It simply avoids unnecessarily downloading an
ostree commit when only the ref is needed.
It does change the 'sources' section of the manifest however.
2021-05-11 12:10:09 +02:00
Achilleas Koutsou
a49ba1e672 rhel84: minor typo fix 2021-05-11 12:10:09 +02:00