Commit graph

334 commits

Author SHA1 Message Date
Jakub Rusz
e500bfbd6f test/manifests: renegerate fedora-35 manifests 2022-07-14 12:00:50 +03:00
Tom Gundersen
6ae65f1951 test/data/manifests: regenerate to drop build packages
With the more fine-grained build package set, different images will use
different build pipelines and each of them will be smaller.

We don't currently cache build pipelines so there is no downside to this. Even
when we start caching the difference between having one build pipeline per
image and one shared one is minimal at scale. This will still benefit users
doing one-off builds on-prem.

Most importantly, this tracks things correctly, making pipelines more
composable without having to maintain a global list of dependencies.
2022-07-12 08:19:57 +01:00
Achilleas Koutsou
f176326b71 test: update RHEL 8.4 manifests and image info
Regenerated all RHEL 8.4 manifests and image info with new definitions.
Edge image types renamed from `rhel-edge-*` to `edge-*`.
2022-07-08 09:30:34 +01:00
Achilleas Koutsou
a8b0a5f0bd test: convert all RHEL 8.4 v1 manifests to v2
Converted by loading them through the manifest parser in osbuild and
formatting them through v2 before dumping.
These are not the "real" manifests that osbuild-composer would generate,
but they will make it a tiny bit easier to compare and detect changes in
the distro definition when it's moved to the common rhel8 package.
2022-07-08 09:30:34 +01:00
fkolwa
14fd1fe279 Update RHEL90-91 repo config and and 9.1 test cases 2022-07-07 14:28:59 +02:00
Tom Gundersen
39c3d6ec35 manifest/os: move fs tool selection into the pipeline
Always include the tools for all the filesystem types in the partition table. There may be
usecases for having additional ones, for instance if the partition table is not known, but
this gives us a minimal baseline.

This includes dosfstools in images that have a vfat partition but did not include the tools.
2022-07-07 12:00:56 +01:00
Tom Gundersen
03fc2d5bfd distro/fedora: install kernel and bootloader in ostree commits
This is a partial revert of 006c5b26, where kernel settings and bootloaders were only
installed on bootable systems.

However, ostree-based systems need the ability to pick up kernels and bootloaders from
the commit to install on the instance, so make this conditional on being bootable or an
ostree commit. This is probably an indication that we need a different abstraction.
2022-07-05 22:39:40 +01:00
Tom Gundersen
a45caf5756 distro/fedora: image type should be more specific than arch
Currently an image type could override the boot loader in the architecture, but we should
not allow an image type to select a bootloader type the architecture does not support. So
only in case the architecture supports hybrid boot do we allow the image type to select
one of the two types.

This enables UEFI on AMIs for aarch64.
2022-07-05 22:39:40 +01:00
Tom Gundersen
ba31887ff2 fedora/pipelines/os: minor refactoring
Collect partition tables / boot loaders / kernels together and
make them conditional on the system being bootable.

As a side-effect, we no longer install the grub2 modules in ostree commits.
2022-07-05 12:15:50 +02:00
Tom Gundersen
b0b5a48c5f manifest/os: don't change the name based on ostree or not
Let the name reflect the pipeline type, and keep it the same whether or not
the contents is meant to be an OSTree commit or not.
2022-07-05 12:15:50 +02:00
Tom Gundersen
83e2060976 distro/fedora: drop package set name translations
No longer name the packageSetChains after the package set, but
keep them named after the pipelines. This should be a
non-functional change as dnf-json does not care about what the
chains are called, only that the names are unique.
2022-07-04 23:04:29 +01:00
Tom Gundersen
e6bc7dea18 manifests/fedora36/qcow2: regenerate with correct config
The current ones did not have the 'core' compsgroup set. Unsure how
this could happen.
2022-07-04 23:04:29 +01:00
Tom Gundersen
d8f540c5cf distro/fedora/iotInstaller: drop unused package set
The IoT installer does not use the iotCommit package set. Drop it.
2022-07-04 23:04:29 +01:00
Achilleas Koutsou
8676d3342d manifests: generate rhel-8
Manifests for rhel-8 (without minor version).  These are identical to
rhel-86.
2022-07-01 23:52:13 +01:00
Achilleas Koutsou
cc5d32169d distro/rhel8: add rhel-8 alias to rhel-86
Add a plain `rhel-8` alias as the default distribution name and version
for the `rhel8` package.  The `rhel-86` distro is still available via
the NewRHEL86() constructor.  These two distributions are identical.

Repositories
------------

The rhel-8 repositories (repositories/rhel-8.json) are now set to the
CDN repositories with no minor version:
https://cdn.redhat.com/content/dist/rhel8/8/...

The rhel-8 test repositories (test/data/repositories/rhel-8.json) were
already set to the plain `8` repositories.  The Google repos have been
added.

The test case generator repositories used for `rhel-8` are the rpmrepo
snapshots as for rhel-86.
2022-07-01 23:52:13 +01:00
Achilleas Koutsou
909d9de89e manifests: regenerate RHEL 8.6 customised qcow2 for ppc64le and s390x
They were missing the core group in the compose request.
2022-07-01 23:52:13 +01:00
Achilleas Koutsou
723c34984c distro: remove rhel8
The rhel8 package represented RHEL 8.3, which is EOL.
The current rhel86 package will be renamed to rhel8 and be responsible
for building all RHEL 8 minor versions.
2022-07-01 23:52:13 +01:00
Achilleas Koutsou
444b8e33c9 distro: don't enable fdo-client service on RHEL 8.5
RHEL 8.5 doesn't have FDO and it's not installed, but we were enabling
the service for it in edge.
2022-06-30 18:34:45 +02:00
Christian Kellner
54ae069b94 distro: remove rhel 9 beta
This was a stop-gap until the actual rhel 9 distro was created. It
is in a sad state, quite broken and shout not be used by anybody.
Put it out of its misery.
2022-06-30 14:15:33 +02:00
Juan Abia
8f9bb1bc01 tag azure resources with gitlab-ci-test
cloud cleaner removes old enough resources with 'gitlab-ci-test' tag.
2022-06-30 13:46:15 +02:00
Achilleas Koutsou
436d8f9b43 distro/rhel86: add RHEL 8.5 variant
Define the distribution strings for RHEL 8.5 in distro/rhel86 and add
constructors.  Remove the old 8.5 from the distro registry and use the
new constructors.

Composer can now build RHEL 8.5 image-installer on aarch64, which wasn't
supported before.

RHEL 8.5 manifests have changed to minimise the differences from 8.6.
Some changes are fixes made in 8.6 but never backported to 8.5 because
of our (older) policy of not changing definitions after the release of a
distro.
Other changes are non-functional (e.g., stage or package order).

See the list below for the source of each change.

Manifest changes:
- Stage order changed for org.osbuild.systemd-logind and
  org.osbuild.rhsm.
- org.osbuild.grub2 options: config.default = "saved"
  Reverted 111cd8871f
- Partition sizes: RHEL 8.5 had extra arbitrarily sized padding for the
  header.  Now all partitions are sized to fit headers exactly.
  Original change at b7abef54e8.
- SELinux set to permissive in Anaconda.  This was changed in RHEL 8.6
  and 9.0 but never backported to 8.5.
  See a7fbe916b7.
- Installer isolevel set to 3.  Like above, this was changed in
  8.6 and 9.0.
  Original change at d8d161480e.
- Specify a remote for edge deployments.
  Original change at b18b4e80a0.
2022-06-30 11:14:21 +02:00
Achilleas Koutsou
56e6208fcf distro/rhel86: imagetypes and packages per RHEL version
Added utility function for comparing RHEL version strings.

Conditions added:
- greenboot subpackages were changed between RHEL 8.5 and RHEL 8.6.
- fido client packages aren't available in RHEL prior to 8.6.
- the ec2 SAP image type is not supported in RHEL prior to 8.6.
- the edge-simplified-installer and edge-raw-image image types are not
  supported in RHEL prior to 8.6.
    - They were previously supported in 8.5 without FDO support, but now
      it's dropped from 8.5 completely.
2022-06-30 11:14:21 +02:00
Achilleas Koutsou
5ac0ef2f77 test: update RHEL 8.5 manifests
Package changes from RHUI repository updates.
2022-06-30 11:14:21 +02:00
Christian Kellner
c5a5da5f56 distro/fedora: add container image type
Add a container image type that is based on the existing fedora
container image. There is a delta in terms of the configuration
because osbuild does not yet provide all the neccessary means,
but the package set is already very close.
2022-06-29 10:02:46 +02:00
Jakub Rusz
d136a075bc manifests: regenerate RHEL-8.6 qcow2 test cases 2022-06-29 10:15:50 +03:00
Jakub Rusz
0201b0bff9 tests: Update compose version in Upgrade test 2022-06-21 10:40:58 +02:00
Jakub Rusz
2eee0632d6 manifests: regenerate rhel-91-gce-boot
We don't want to do gpg check for the google repositories.
2022-06-21 10:40:58 +02:00
Alexander Todorov
0d3ef8817a Update repositories in repo-*.json files 2022-06-21 10:40:58 +02:00
Christian Kellner
c38fcb128c distro/rhel7: add support for azure images
Add support for building Azure images, including RHUI based ones for
the private market place.
2022-06-21 00:06:26 +02:00
Christian Kellner
06e05df620 distro: add support for building a rhel 7 (qcow2)
Based on the RHEL 8.6 pipelines, needs a special buildroot with two
extra packages: python3-iniparse and python3-PyYAML.
Only x86_64 support for now.
2022-06-21 00:06:26 +02:00
Major Hayden
26b93f8f25 Blacklist amdgpu module on Azure images
The `amdgpu` module causes issues on certain GPU-enabled instances
on Azure and it must not be loaded by default.
Modules are sorted alphabetically.

Signed-off-by: Major Hayden <major@redhat.com>
Co-Authored-By: Christian Kellner <christian@redhat.com>
2022-06-09 14:18:45 +01:00
Christian Kellner
a1306a122a distro/rhel90: remove skx_edac, intel_cstate from denylist again
In commit 5c1530e we disabled `skx_edac` and `intel_cstate` but 
after further consultation with Prarit Bhargava it was agreed that 
for RHEL 9 we should indeed allow them.
2022-06-06 08:07:26 +01:00
Achilleas Koutsou
35e369f5c7 test: add test repository metadata
Metadata files for CS9 BaseOS x86_64.
This will be served by a temporary webserver for dnf-json unit testing.
2022-06-01 11:36:52 +01:00
Christian Kellner
c039a91b61 distro/rhel90: enable and configure NetworkManager-cloud-setup
Package was already installed, but we needed to enable the timer and
service and set the correct env variable via a drop-in to enable the
Azure cloud.
2022-05-31 10:22:22 +01:00
Christian Kellner
5c1530ee53 distro/rhel90: blacklist skx_edac,intel_cstate kernel modules on azure
Disabled by the MSFT images, we follow suit (really means we don't
exactly know why and should find out).
2022-05-31 10:22:22 +01:00
Christian Kellner
3b798edecb distro/rhel86: install and enable NetworkManager-cloud-setup
Install the package, enable timer and service and set the correct
env variable via a drop-in to enable the Azure cloud.
2022-05-31 10:22:22 +01:00
Christian Kellner
dc0ee05bc3 distro/rhel86: blacklist skx_edac,intel_cstate kernel modules on azure
Disabled by the MSFT images, we follow suit (really means we don't
exactly know why and should find out).
2022-05-31 10:22:22 +01:00
Christian Kellner
921c67cf1b distro/rhel90: compress azure-rhui images
Those images are forced to be 64GiB in size but mostly consist of zeros.
This makes them hard to handle, e.g. uploading to brew takes a forever.
The vhdPipelines is converted to a function returning the pipelinesFunc
and it has a single argument `compress` that will add the compression
pipeline bits if `true`. Will return exactly the old pipeline in case
of `false`.
2022-05-27 18:19:51 +02:00
Christian Kellner
5c90abdd0a distro/rhel86: compress azure-rhui images
Those images are forced to be 64GiB in size but mostly consist of zeros.
This makes them hard to handle, e.g. uploading to brew takes a forever.
The vhdPipelines is converted to a function returning the pipelinesFunc
and it has a single argument `compress` that will add the compression
pipeline bits if `true`. Will return exactly the old pipeline in case
of `false`.
2022-05-27 18:19:51 +02:00
Achilleas Koutsou
cd49c932a2 test: add prominent message in test script cleanup functions
When a test script fails in CI, it's often difficult to pinpoint the
exact line in the log where the script failed and the cleanup() function
(trapped on EXIT) begins.

Adding a prominent line (with greenprint where available) at the start
of the cleanup function will make reading logs of failed jobs a lot
easier.
2022-05-25 22:10:27 +02:00
Christian Kellner
4c7bf735fe distro/rhel90: install nm-cloud-setup for azure-rhui
Install the "NetworkManager-cloud-setup" on Azure Marketplace images.
2022-05-23 11:02:18 +02:00
Christian Kellner
4e9e438b75 distro/rhel90: add support for azure marketplace
Add support for building images for the Azure marketplace: add a
new image type "azure-rhui" that can be used to build images
tailored to the Azure marketplace.
This code is based on the corresponding image type in 8.6.

NB: does not have systemd-resovled (following RHEL 9 defaults)
2022-05-19 11:22:47 +02:00
Christian Kellner
6e2cb208bf test/data/manifests: regenerate rhel-91
It seems rhel-91 qcow2 customize images are out of sync because commit
2beb707 removed the core group from the `format-request-map.json` and
some these said manifests were generated between that commit and the
one that added it back 1ff36bce9.
2022-05-19 11:22:47 +02:00
Xiaofeng Wang
a6e2755fad test: Add running podman with non-root test
Bug BZ#2078937 has been fixed by osbuild PR#1013. Test should be
updated to test the fix and avoid regression
2022-05-17 21:25:49 +08:00
Christian Kellner
5983c295b3 distro/rhel86: ignore SRIOV interface via new udev rule on azure-rhui
Add a new udev rule that ignores the SRIOV network interface. See the
supplied comment for details why.
2022-05-16 15:46:46 +02:00
Juan Abia
99649ee142 generate-all-test-cases: generate all manifests
regenerate all manifests without image-info and add new ones
2022-05-13 21:01:37 +03:00
Jordi Gil
b6f61d26ea distro/fedora: extend kernel options for openstack, qcow2, vhd and vmdk types
* Use the same value from AMI to these image types:

"ro no_timer_check console=ttyS0,115200n8 biosdevname=0 net.ifnames=0"

* Regenerate manifests
2022-05-09 12:25:21 +02:00
Jordi Gil
69095f5034 distro/fedora: produce VMDK as stream-optimized
Added code in fedora/pipelines.go to add the subformat field in the
manifests

Added manifests for f34 and f35 for x86_64 only (image type not
available in aarch64)
2022-05-09 12:25:21 +02:00
Jordi Gil
3a079eec26 distro/fedora: Regenerated manifests
* IoT image types now correctly point to the fedora-identity-iot package
* QCOW2, VMDK and OCI types use Fedora Cloud as identity package
* Changed default target for AMI from graphical.target to multi-user.target. This matches the behaviour with the RHEL types, which all target the multi-user.
* Readded the image-info field for some manifests which was missing due to issues regenerating the manifests.
2022-05-09 12:25:21 +02:00
Jordi Gil
b19b824e38 distro/fedora: Regenerated manifests for f34-aarch and f35-x86_64 that include the image-info field 2022-05-09 12:25:21 +02:00