Commit graph

315 commits

Author SHA1 Message Date
Juan Abia
8f9bb1bc01 tag azure resources with gitlab-ci-test
cloud cleaner removes old enough resources with 'gitlab-ci-test' tag.
2022-06-30 13:46:15 +02:00
Achilleas Koutsou
436d8f9b43 distro/rhel86: add RHEL 8.5 variant
Define the distribution strings for RHEL 8.5 in distro/rhel86 and add
constructors.  Remove the old 8.5 from the distro registry and use the
new constructors.

Composer can now build RHEL 8.5 image-installer on aarch64, which wasn't
supported before.

RHEL 8.5 manifests have changed to minimise the differences from 8.6.
Some changes are fixes made in 8.6 but never backported to 8.5 because
of our (older) policy of not changing definitions after the release of a
distro.
Other changes are non-functional (e.g., stage or package order).

See the list below for the source of each change.

Manifest changes:
- Stage order changed for org.osbuild.systemd-logind and
  org.osbuild.rhsm.
- org.osbuild.grub2 options: config.default = "saved"
  Reverted 111cd8871f
- Partition sizes: RHEL 8.5 had extra arbitrarily sized padding for the
  header.  Now all partitions are sized to fit headers exactly.
  Original change at b7abef54e8.
- SELinux set to permissive in Anaconda.  This was changed in RHEL 8.6
  and 9.0 but never backported to 8.5.
  See a7fbe916b7.
- Installer isolevel set to 3.  Like above, this was changed in
  8.6 and 9.0.
  Original change at d8d161480e.
- Specify a remote for edge deployments.
  Original change at b18b4e80a0.
2022-06-30 11:14:21 +02:00
Achilleas Koutsou
56e6208fcf distro/rhel86: imagetypes and packages per RHEL version
Added utility function for comparing RHEL version strings.

Conditions added:
- greenboot subpackages were changed between RHEL 8.5 and RHEL 8.6.
- fido client packages aren't available in RHEL prior to 8.6.
- the ec2 SAP image type is not supported in RHEL prior to 8.6.
- the edge-simplified-installer and edge-raw-image image types are not
  supported in RHEL prior to 8.6.
    - They were previously supported in 8.5 without FDO support, but now
      it's dropped from 8.5 completely.
2022-06-30 11:14:21 +02:00
Achilleas Koutsou
5ac0ef2f77 test: update RHEL 8.5 manifests
Package changes from RHUI repository updates.
2022-06-30 11:14:21 +02:00
Christian Kellner
c5a5da5f56 distro/fedora: add container image type
Add a container image type that is based on the existing fedora
container image. There is a delta in terms of the configuration
because osbuild does not yet provide all the neccessary means,
but the package set is already very close.
2022-06-29 10:02:46 +02:00
Jakub Rusz
d136a075bc manifests: regenerate RHEL-8.6 qcow2 test cases 2022-06-29 10:15:50 +03:00
Jakub Rusz
0201b0bff9 tests: Update compose version in Upgrade test 2022-06-21 10:40:58 +02:00
Jakub Rusz
2eee0632d6 manifests: regenerate rhel-91-gce-boot
We don't want to do gpg check for the google repositories.
2022-06-21 10:40:58 +02:00
Alexander Todorov
0d3ef8817a Update repositories in repo-*.json files 2022-06-21 10:40:58 +02:00
Christian Kellner
c38fcb128c distro/rhel7: add support for azure images
Add support for building Azure images, including RHUI based ones for
the private market place.
2022-06-21 00:06:26 +02:00
Christian Kellner
06e05df620 distro: add support for building a rhel 7 (qcow2)
Based on the RHEL 8.6 pipelines, needs a special buildroot with two
extra packages: python3-iniparse and python3-PyYAML.
Only x86_64 support for now.
2022-06-21 00:06:26 +02:00
Major Hayden
26b93f8f25 Blacklist amdgpu module on Azure images
The `amdgpu` module causes issues on certain GPU-enabled instances
on Azure and it must not be loaded by default.
Modules are sorted alphabetically.

Signed-off-by: Major Hayden <major@redhat.com>
Co-Authored-By: Christian Kellner <christian@redhat.com>
2022-06-09 14:18:45 +01:00
Christian Kellner
a1306a122a distro/rhel90: remove skx_edac, intel_cstate from denylist again
In commit 5c1530e we disabled `skx_edac` and `intel_cstate` but 
after further consultation with Prarit Bhargava it was agreed that 
for RHEL 9 we should indeed allow them.
2022-06-06 08:07:26 +01:00
Achilleas Koutsou
35e369f5c7 test: add test repository metadata
Metadata files for CS9 BaseOS x86_64.
This will be served by a temporary webserver for dnf-json unit testing.
2022-06-01 11:36:52 +01:00
Christian Kellner
c039a91b61 distro/rhel90: enable and configure NetworkManager-cloud-setup
Package was already installed, but we needed to enable the timer and
service and set the correct env variable via a drop-in to enable the
Azure cloud.
2022-05-31 10:22:22 +01:00
Christian Kellner
5c1530ee53 distro/rhel90: blacklist skx_edac,intel_cstate kernel modules on azure
Disabled by the MSFT images, we follow suit (really means we don't
exactly know why and should find out).
2022-05-31 10:22:22 +01:00
Christian Kellner
3b798edecb distro/rhel86: install and enable NetworkManager-cloud-setup
Install the package, enable timer and service and set the correct
env variable via a drop-in to enable the Azure cloud.
2022-05-31 10:22:22 +01:00
Christian Kellner
dc0ee05bc3 distro/rhel86: blacklist skx_edac,intel_cstate kernel modules on azure
Disabled by the MSFT images, we follow suit (really means we don't
exactly know why and should find out).
2022-05-31 10:22:22 +01:00
Christian Kellner
921c67cf1b distro/rhel90: compress azure-rhui images
Those images are forced to be 64GiB in size but mostly consist of zeros.
This makes them hard to handle, e.g. uploading to brew takes a forever.
The vhdPipelines is converted to a function returning the pipelinesFunc
and it has a single argument `compress` that will add the compression
pipeline bits if `true`. Will return exactly the old pipeline in case
of `false`.
2022-05-27 18:19:51 +02:00
Christian Kellner
5c90abdd0a distro/rhel86: compress azure-rhui images
Those images are forced to be 64GiB in size but mostly consist of zeros.
This makes them hard to handle, e.g. uploading to brew takes a forever.
The vhdPipelines is converted to a function returning the pipelinesFunc
and it has a single argument `compress` that will add the compression
pipeline bits if `true`. Will return exactly the old pipeline in case
of `false`.
2022-05-27 18:19:51 +02:00
Achilleas Koutsou
cd49c932a2 test: add prominent message in test script cleanup functions
When a test script fails in CI, it's often difficult to pinpoint the
exact line in the log where the script failed and the cleanup() function
(trapped on EXIT) begins.

Adding a prominent line (with greenprint where available) at the start
of the cleanup function will make reading logs of failed jobs a lot
easier.
2022-05-25 22:10:27 +02:00
Christian Kellner
4c7bf735fe distro/rhel90: install nm-cloud-setup for azure-rhui
Install the "NetworkManager-cloud-setup" on Azure Marketplace images.
2022-05-23 11:02:18 +02:00
Christian Kellner
4e9e438b75 distro/rhel90: add support for azure marketplace
Add support for building images for the Azure marketplace: add a
new image type "azure-rhui" that can be used to build images
tailored to the Azure marketplace.
This code is based on the corresponding image type in 8.6.

NB: does not have systemd-resovled (following RHEL 9 defaults)
2022-05-19 11:22:47 +02:00
Christian Kellner
6e2cb208bf test/data/manifests: regenerate rhel-91
It seems rhel-91 qcow2 customize images are out of sync because commit
2beb707 removed the core group from the `format-request-map.json` and
some these said manifests were generated between that commit and the
one that added it back 1ff36bce9.
2022-05-19 11:22:47 +02:00
Xiaofeng Wang
a6e2755fad test: Add running podman with non-root test
Bug BZ#2078937 has been fixed by osbuild PR#1013. Test should be
updated to test the fix and avoid regression
2022-05-17 21:25:49 +08:00
Christian Kellner
5983c295b3 distro/rhel86: ignore SRIOV interface via new udev rule on azure-rhui
Add a new udev rule that ignores the SRIOV network interface. See the
supplied comment for details why.
2022-05-16 15:46:46 +02:00
Juan Abia
99649ee142 generate-all-test-cases: generate all manifests
regenerate all manifests without image-info and add new ones
2022-05-13 21:01:37 +03:00
Jordi Gil
b6f61d26ea distro/fedora: extend kernel options for openstack, qcow2, vhd and vmdk types
* Use the same value from AMI to these image types:

"ro no_timer_check console=ttyS0,115200n8 biosdevname=0 net.ifnames=0"

* Regenerate manifests
2022-05-09 12:25:21 +02:00
Jordi Gil
69095f5034 distro/fedora: produce VMDK as stream-optimized
Added code in fedora/pipelines.go to add the subformat field in the
manifests

Added manifests for f34 and f35 for x86_64 only (image type not
available in aarch64)
2022-05-09 12:25:21 +02:00
Jordi Gil
3a079eec26 distro/fedora: Regenerated manifests
* IoT image types now correctly point to the fedora-identity-iot package
* QCOW2, VMDK and OCI types use Fedora Cloud as identity package
* Changed default target for AMI from graphical.target to multi-user.target. This matches the behaviour with the RHEL types, which all target the multi-user.
* Readded the image-info field for some manifests which was missing due to issues regenerating the manifests.
2022-05-09 12:25:21 +02:00
Jordi Gil
b19b824e38 distro/fedora: Regenerated manifests for f34-aarch and f35-x86_64 that include the image-info field 2022-05-09 12:25:21 +02:00
Jordi Gil
6593e30d63 distro/fedora: Regenerated test manifests using V2 API 2022-05-09 12:25:21 +02:00
Jordi Gil
40d5950f1e distro/fedora: Set default target for ami/ec2 image type to multi-user and remove extra console entry in kernel opts 2022-05-09 12:25:21 +02:00
Jordi Gil
1d0e5c3410 distro/fedora: changed x86_64 bootType to hybrid 2022-05-09 12:25:21 +02:00
Jordi Gil
96d9e63b77 distro/fedora: capitalized IoT in fedora 2022-05-09 12:25:21 +02:00
Jordi Gil
00a84d1fdc distro: allow user and group customizations for iot-installer 2022-05-09 12:25:21 +02:00
Jordi Gil
2beb707def distro/fedora: regenerated test manifests 2022-05-09 12:25:21 +02:00
Tomas Hozza
be6675d54e distro(all): produce VMDK as stream-optimized
Modify pipelines in all distro definitions to produce stream-optimized VMDK
image.

Regenerate all VMDK test cases.

Bump worker dependency on osbuild to the version supporting VMDK
subformat in both QEMU assembler and stage
2022-05-04 16:22:29 +02:00
Ondřej Budai
082da070d1 test: generate manifests for rhel-87 and rhel-91
These are without image-info as discussed in the PR.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-05-03 18:13:28 +02:00
Ondřej Budai
df011dbb06 tests: add rhel-87 repositories overrides
This is a bare minimum for our downstream testsuite to pass (otherwise
it will fail on non-existing 8.7 CDN repositories).

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-05-03 18:13:28 +02:00
Ondřej Budai
8575620591 distro: add an alias for rhel-91
We would benefit from having support for 9.1 downstream so let's add it in
the form of an alias. This is a bare minimum for having a proper 9.1 support.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-05-03 18:13:28 +02:00
Christian Kellner
98b491fcac distro/rhel85: automatically convert to LVM on fs customizations
Whenever we create a new mountpoint due to a user customization,
ensure the layout uses LVM, i.e. convert plain layouts to it, if
needed. This does not apply to rpm-ostree based systems.
Add "lvm2" to the build pipeline and thus generate new manifests
and image infos.
Adjust the existing tests that assumed we can not create more
than 4 partitions on mbr layouts, since that is now not true
anymore.
This is a port from rhel86, commit 63aa155

The change in osPipeline() is required now to fix the Prefix for the
bootloader specification when LVM is used. The unspecified Prefix, which
was previously used for all cases, defaults to "/boot". When the layout
is converted to LVM, a boot partition is created and the BLS Prefix
should be set to "".

In the case where we don't have a partition table, the BLS stage is not
needed, but it was done unconditionally before, so keep the default
image definitions unchanged.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-04-30 12:37:50 +01:00
Achilleas Koutsou
d357f92f91 test: add fs customization to qcow2 test case
Added a filesystem customization to the qcow2 test case to test that the
filesystem is converted to an LVM layout.
Set overrides for distros that don't support fs customizations.
2022-04-29 08:55:36 +02:00
Tomas Hozza
81fc032a45 RHEL-90: don't pass blueprint package set separately to pipelines
Blueprint package set is now depsolved together with the OS package set
in a chain. The result is stored in the package specs sets under the OS
package set name.

In reality, the code was able to handle a `nil` package specs to be
passed to pipelines, however some parts were looking for the kernel
version in the blueprint package specs, which would be a bug.

Regenerated affected image test cases.
2022-04-28 14:42:49 +02:00
Tomas Hozza
e6a20699f9 RHEL-90beta: don't pass blueprint package set separately to pipelines
Blueprint package set is now depsolved together with the OS package set
in a chain. The result is stored in the package specs sets under the OS
package set name.

In reality, the code was able to handle a `nil` package specs to be
passed to pipelines, however some parts were looking for the kernel
version in the blueprint package specs, which would be a bug.

Regenerated affected image test cases.
2022-04-28 14:42:49 +02:00
Tomas Hozza
c26a1839d1 RHEL-86: don't pass blueprint package set separately to pipelines
Blueprint package set is now depsolved together with the OS package set
in a chain. The result is stored in the package specs sets under the OS
package set name.

In reality, the code was able to handle a `nil` package specs to be
passed to pipelines, however some parts were looking for the kernel
version in the blueprint package specs, which would be a bug.

Regenerated affected image test cases.
2022-04-28 14:42:49 +02:00
Tomas Hozza
4c707663f2 RHEL-85: don't pass blueprint package set separately to pipelines
Blueprint package set is now depsolved together with the OS package set
in a chain. The result is stored in the package specs sets under the OS
package set name.

In reality, the code was able to handle a `nil` package specs to be
passed to pipelines, however some parts were looking for the kernel
version in the blueprint package specs, which would be a bug.

Regenerated affected image test cases.
2022-04-28 14:42:49 +02:00
Achilleas Koutsou
0f0b2072d5 test: update test manifests
Update GPG keys in all RHEL 8.x test manifests.
2022-04-28 09:50:27 +02:00
Achilleas Koutsou
81c62204b9 repositories: update gpg key for all RHEL 8.x repos
Follow-up to 60db6ad06f

The SHA-1 key is no longer supported in RHEL 9.0.  This isn't a problem
for RHEL 8.x in general, but it prevents cross building RHEL 8.x images
on RHEL 9.0, since the host (RHEL 9.0) rpm and openssl cannot import the
older keys and we fail to bootstrap the build root for the new image if
the source repositories use SHA-1 keys.

Related rhbz#2058497 (Comment 18).

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2022-04-28 09:50:27 +02:00
Achilleas Koutsou
389b7c1848 Unify formatting for all repository json files
- 2 space indent
- lists on multiple lines
- newlines at EOF

This was accomplished by simply running each file through `jq` with no
arguments.
It is also equivalent to Python's `json.dump(..., indent=2)` plus the
added newline.
2022-04-28 09:50:27 +02:00