The same test is run in distro/distro_test.go. The redundancy was probably
caused by a bitrot in several commits.
I decided to remove the test from distro implementations to reduce the amount
of duplicated code.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
My goal is to add a method to distroregistry to return Registry with
all supported distributions. This way, all supported distributions
would be defined only on one place.
To achieve this, the Registry must live outside the distro package
because the distro implementation depends on it and this would create
a circular dependency unsupported by Go.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
This replaces Packages() and BuildPackages() by returning a map of
package sets, the semantics of which is up to the distro to define.
They are meant to be depsolved and the result returned back as a
map to Manifest(), with the same keys.
No functional change.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Use en_US.UTF-8 as default for LANG, which is what previously was
used and is also needed to properly work on non-us/latin setups[1].
In the customization tests, use a different value than the default
one to check that the customization does in fact work.
[1] http://git.app.eng.bos.redhat.com/git/spin-kickstarts.git/tree/rhel8/rhel-8.2-kvm-x86_64.ks#n4
Co-authored-by: Achilleas Koutsou <achilleas@koutsou.net>
Explicitly set the kernel to boot into.
Also change the blueprint/kernenl handling:
Rather than only falling back to the default kernel name for
getting the package list, let GetKernel() always return the
correct result so we can rely on this being consistent.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Rather than setting this automagically, expose it to the caller. For
now the only caller we have simply passes it back in, so this is a
noop.
In follow-up commits this will be used to resolve the parent commit.
This is tested by verifying that the generated manifests do not
change.
Signed-off-by: Tom Gundersen <teg@jklm.no>
We aim at shrinking our deps eventually but we need subman for the time
being. This patch basically un-exclude subman which was introduced by
https://github.com/osbuild/osbuild-composer/pull/893
Signed-off-by: Antonio Murdaca <runcom@linux.com>
The kernel now comes from the blueprint packages even when it's not
specified. Removing from the base packages of the image types avoids
duplication and allows for alternative kernels to be specified without
also including the default.
The latter is necessary for RHEL for Edge and Fedora IoT images (ostree
commits) that fail to build when multiple kernels are installed.
ImageType tests modified to fix expected package order.
s390x isn't supported on Centos.
rhel-commit-edge sounds just wrong for Centos. We can revert this change any
time. The thing is that I wasn't able to find something like CentOS IoT and
we don't want to be in a position of defining a new distribution spin.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
I'm sorry, I need to extend this condition and my brain isn't powerful enough
to reason about complex negative conditions.
Not a functional change.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
The image definition is shared with the latest RHEL 8.y one (8.4 currently).
I expect that we the introduction of 8.5 support, we point the centos 8
distro at it.
The test repositories and manifests use the official CentOS composes. From
what I can tell, they are persistent. This is not guaranteed though, so we
might need to switch to RPMRepo at some point.
The "classic" CentOS 8 should also be buildable but due to the chicken and egg
issue (this commit will get into Centos "8.4" but Centos "8.4" isn't a thing
yet), we cannot test it and therefore it might be broken.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Test for each distro that runs through all architecture - image type
combinations and calls the Manifest() method with a kernel boot option
customization and checks if the ostree image types produce the expected
error.
Kernel boot parameters have no effect on ostree type images (Fedora IoT
and RHEL for Edge). Catch this and fail early in the pipeline creation
and communicate the issue to the user.
Modify RHEL 8.3 and 8.4 KVM guest images definition to produce osbuild
manifest with `org.osbuild.rhsm` stage to disable both RHSM DNF plugins
(`product-id` and `subscription-manager`).
Update `/docs/news/unreleased/osbuild-rhsm-stage.md` to note that RHEL
8.3 and 8.4 qcow2 image definitions are updated to disable RHSM DNF
plugins by default.
Enhance `tools/image-info` tool to add RHSM-specific section to its
output in case RHSM DNF plugins configurations exist in the tree.
Regenerate all RHEL image test cases affected by the patch set.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The org.osbuild.sysconfig stage is now supported. Config updates can be
made to the kernel and network files. Currently, the same values are
used for all image types in rhel84. The image-info script is updated to
allow testing the sysconfig info.
The last imagefactory nightly did not contain dnf-plugin-spacewalk,
fwupd, nss, or udisks2. These packages are now excluded. The
udisks2.service and mdmonitor.service are no longer enabled. Also, the
fwupd-refresh, mdcheck_continue, mdcheck_start, and mdmonitor-oneshot
timers are no longer listed as disabled services.
The packages in the last imagefactory nightly differ from ours. The
following packages are now added:
oddjob
oddjob-mkhomedir
psmisc
authselect-compat
rng-tools
dbxtool
Also, the rngd and nfs-convert services are enabled.
An image only had a systemd stage added if its blueprint contained
services or if its image type contained enabled services. The systemd
stage is now also added if the image type contains disabled services or
a default target.
The RHEL 8.4 qcow2 image type now specifies the multi-user default target.
In order to test this the image-info tool now includes the default
target in its output. Image test manifests are updated to include this
change.
RHEl 8.4 guest images need to have the default timezone of EST/EDT
unless the user specifies one in their blueprint. New York is a major
location for this timezone.
Imagine this situation: You have a RHEL system booted from an image produced
by osbuild-composer. On this system, you want to use osbuild-composer to
create another image of RHEL.
However, there's currently something funny with partitions:
All RHEL images built by osbuild-composer contain a root xfs partition. The
interesting bit is that they all share the same xfs partition UUID. This might
sound like a good thing for reproducibility but it has a quirk.
The issue appears when osbuild runs the qemu assembler: it needs to mount all
partitions of the future image to copy the OS tree into it.
Imagine that osbuild-composer is running on a system booted from an imaged
produced by osbuild-composer. This means that its root xfs partition has this
uuid:
efe8afea-c0a8-45dc-8e6e-499279f6fa5d
When osbuild-composer builds an image on this system, it runs osbuild that
runs the qemu assembler at some point. As I said previously, it will mount
all partitions of the future image. That means that it will also try to
mount the root xfs partition with this uuid:
efe8afea-c0a8-45dc-8e6e-499279f6fa5d
Do you remember this one? Yeah, it's the same one as before. However, the xfs
kernel driver doesn't like that. It contains a global table[1] of all xfs
partitions that forbids to mount 2 xfs partitions with the same uuid.
I mean... uuids are meant to be unique, right?
This commit changes the way we build RHEL 8.4 images: Each one now has a
unique uuid. It's now literally a unique universally unique identifier. haha
[1]: a349e4c659/fs/xfs/xfs_mount.c (L51)
Now that we have an abstract partition table definition, we can use it to
generate org.osbuild.fstab stage options.
This is extremely nice because it removes magic contains.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Using osbuild.QEMUAssemblerOptions to encode a partition table was weird.
This commit introduces a disk package that contains data types for defining
partition tables. Also, there's a handy function to convert the abstact
partition table to osbuild.QEMUAssemblerOptions.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Previously, the partition table definition was defined inside an assembler.
This has an issue though: The partitions and filesystems are needed at several
other places, e.g. grub2 stage and fstab stage. As the partition table was
basically hardcoded, this didn't matter - we could just use constants
in these stages. Not ideal but it worked.
This commit changes the behaviour: A partition table is firstly created and
then it's passed to the assembler function where complete assembler options
are created out of it.
To make this change as small as possible, osbuild.QEMUAssemblerOptions type
is used to encode the partition table for now.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
require.JSONEqf cannot handle diffs of such a big entity as a manifest is.
It just prints an empty string.
This commit unmarshalls the manifests instead and then uses the cmp library
to make a very nice and readable diff.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
The kernel options are updated to remove the read only option "ro" from
the qcow2 and amazon image types. Also, the qcow2's kernel options are
updated to only set console=ttyS0 once. It was declared twice which is
redundant so now it is set for both tty0 and ttyS0.
rng-tools is no longer included as a package in the RHEL 8.4 image. This
package is both removed from being an included package and also
specifically declared as excluded. The test manifests are updated.
The RHEL 8.4 specific runner was introduced in osbuild 22, released
on the 8th of October 2020. It should by now be in relevant Fedora
releases and RHEL 8.4.
Adapt the corresponding test cases.
Hybrid boot, i.e. MBR+UEFI, is only supported on x86_64, but not
on aarch64, which only supports UEFI. Therefore, disable legacy,
i.e. MBR, boot for aarch64 and remove the corresponding BIOS boot
partition.
Since hybrid boot is always used on x86_64, the "uefi" parameter
is removed completely, together with the "legacy only" option on
x86_64. Thus we are left with a simple one to one mapping of
architecture to qemu assembler stage options in `qemuAssembler`.
These packages are architecture-specific. Including them in the generic
package list broke image builds for everything but x86_64.
The corresponding architecture-dependant packages were already included
in `arch.bootloaderPackages` (some via dependencies).
Fixes#1103
cockpit-composer can now build rhel 8.4 images. Our distro name for
rhel 8.4 is rhel-84 unlike prior rhel releases which fall
under the umbrella name rhel-8. rhel 8.4 still uses the same
repos as the rest of the rhel 8 releases but points to a different
nightly repo for testing purposes. Test cases are added. The changes
between rhel 8.3 and 8.4 are as follows:
There is now a hybrid boot partition scheme for x86_64. x86_64 images
now use uefi boot and have 3 gpt partitions: a small unformated
partition for mbr compatibility, an efi boot partition of type vfat, and
a root partition of type xfs. The packages grub2-efi-x64 and shim-x64
are added as bootloader packages for all x86_64 images.
For qcow2 images ro is added as a kernel option and the following
packages are added (+) or removed (-):
+ dosfstools
+ efi-filesystem
+ efivar
+ efivar-libs
+ grub2-efi-x64
+ shim-x64
- rhn-client-tools
- rhnlib
- rhnsd
- rhn-setup
Attempt to clarify the structure of our tests. Each test case is now
encapsulated in a script in `test/cases`. Each of these scripts should
be runnable on a pristine machine and be independent of each other. It
is up to the test-orchestractor to decide if they should be run
consequtively instance, or in parallel on separate instances. Each
script can execute several tests and call whatever helper binaries
is desired. However, each case should be assumed to always run as one.
We no longer release into F31, and the right specfile was anyway not
being tested.
This allows us to remove a workaround that updates the VMs during
deploy, and other fedora-31 specific hacks.
No tests should be run directly from git, but should rather be installed
onto the test system using rpm and run from there. This moves towards
unifying our two types of test cases.
The new structure of is now:
`test/cmd`: the executors, one for each test-case. This is installed
into `/usr/libexec/test/osbuild-composer`.
`test/data`: data and config used by the tests. This is installed into
`/usr/share/tests/osbuild-composer`.
`schutzbot`: configuration of the actual test run. In particular, this
is where the distros and repositories to test against are
configured.
This is very much still work-in-progress, and is only the first step
towards simplifying schutzbot. Apart from moving files around, this
should be a noop.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Fedora 33 images can now be built and test cases are added for the new
images. The fedora 33 qcow2 and vmdk images are based off of the
official images and their kickstarters found here:
https://pagure.io/fedora-kickstarts. The fedora 33 iot image is based
off of the the config found here: https://pagure.io/fedora-iot/ostree.
The openstack, azure, and amazon image types have changes made to them
based off of the changes made to the qcow2. The changes between fedora
32 and fedora 33 are as follows:
Grub now loads its kernel command line options from
etc/kernel/cmdline, /usr/lib/kernel/cmdline, and /proc/cmdline instead
of from grub env. This is addressed by adding kernelCmdlineStageOptions
to use osbuild's kernel-cmdline stage to set these options. Alongside
`ro biosdevname=0 net.ifnames=0`, we also set `no_timer_check
console=tty1 console=ttyS0,115200n8` per what is set in the official
qcow2. For azure and amazon, the kernelOptions are still set as they
were in fedora 32.
The timezone is now set to UTC if a user does not set a timezone in the
blueprint customizations. Also, the hostname is set to
localhost.localdomain if the hostname isn't set in the blueprint.
Finally, the following packages have been removed:
polkit
geolite2-city
geolite2-country
zram-generator-defaults
We now have greenboot available so update the packages and services
also add exclude sections as subscription-manager is for some reason
getting pulled in which brings dnf and other deps we explicitly don't
want.
Signed-off-by: Peter Robinson <pbrobinson@gmail.com>
[teg: minor fixes and squash several fixup commits]
