Add the rhui-azure rpmrepo snapshots to aarch64 repositories.
The repos are labelled as x86_64 but they're actually 'noarch' and we
name them x86_64 when creating the snapshots.
Co-Authored-By: Christian Kellner <christian@kellner.me>
The previous value for parent was not a valid commit ID. In a regular
compose request, the parent ref is resolved before being added as a
source in the manifest. However, when building test manifests, since
the values aren't real, we don't resolve the ostree refs, meaning that
the test manifests weren't valid — they failed validity checks if run in
osbuild.
Replaced the fake parent ref with a fake parent commit ID
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff".
The manifests still aren't buildable, since they don't point to a valid
ostree repository or commit, but now they're at least valid (in the
technical sense).
Add manifests for container embedding in edge-commits and qcow2s.
Limit qcow2 container embedding to RHEL 9.1 for now since it's not
supported on Fedora or RHEL 8 yet.
It needs to be listed in format-request-map.json, and the repo snapshots
required to depsolve the packages need to be added to repos.json
NOTE: The snapshots are from https://github.com/osbuild/rpmrepo/
There was a missing `overrides` property in the `azure-rhui` request
definition, which resulted in:
```
$ ./tools/test-case-generators/generate-test-cases --distro rhel-90
--arch x86_64 --image-types azure-rhui --store /dev/null --output
/home/thozza/devel/osbuild-composer/test/data/manifests
--keep-image-info
Traceback (most recent call last):
File "/home/thozza/devel/osbuild-composer/./tools/test-case-generators/generate-test-cases", line 176, in <module>
main(args.distro, args.arch, args.image_types, args.keep_image_info, args.store, args.output)
File "/home/thozza/devel/osbuild-composer/./tools/test-case-generators/generate-test-cases", line 153, in main
if distro in test_case_request["overrides"]:
KeyError: 'overrides'
```
Fedora 34 is EOL, let's remove all traces of it, including:
- distro definition
- repositories (and test one)
- test manifests
- special package set rules
- hacks from the spec file
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
Add a container image type that is based on the existing fedora
container image. There is a delta in terms of the configuration
because osbuild does not yet provide all the neccessary means,
but the package set is already very close.
overrides where needed for `qcow2` and `simplified-image-installer` images on specific distros. Also some repos needed to be updated to newer versions.
Do not apply the user customizations on edge-installer on RHEL-85/90beta,
since they are not supported there yet. The way we generate image test
cases from the `format-request-map.json` makes even the customized image
types being generated for all distributions automatically.
Removed in 2beb707def, possibly
accidentally.
The affected manifests were not regenerated based on this change so
they all already contain the core group.
Added a filesystem customization to the qcow2 test case to test that the
filesystem is converted to an LVM layout.
Set overrides for distros that don't support fs customizations.
Add the `gce-rhui` image type intended for Google Compute Engine. The image
uses Google's RHUI infrastructure to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Followup from, f34380d5b5 and
3a1765a5a8, copied to the rest of the RHEL
distro definitions.
For now, these customizations have no effect on the manifest.
The new `with-users` variants of the edge-installer test cases include
the user customizations in the blueprint, but the manifests are
(currently) the same as the corresponding base cases.
Add support for building images for the Azure marketplace: add a
new image type "azure-rhui" that can be used to build images
tailored to the Azure marketplace.
Add two sample manifests for 8.5 and 8.6, but note that even the
8.5 is using the 8.6 distro definitions. Also no image-info is
included since `image-info` cannot (yet) handle LVM setups and
the azure marketplace images use the LVM setup.
Adds ostree params to the request to generate a test manifest for
edge-installers and necessary customisations in the blueprint for the
edge-simplified installer. The manifest is not buildable but works for
checking for changes in the pipeline and packages for the installers.
Add F34 and F35 image test cases for all remaining image types, which
were previously not tested. With this PR, image test cases are now
generated for all image types on all architectures as supported by the
Fedora distro definition.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add a new option `no-image-info`, which can be specified for an image
type in `format-request-map.json` in case the image can not be analyzed
using `image-info` and thus it should not be run as part of generating
the image test case.
This is the case for e.g. the `image-installer` image type in RHEL-8.6,
which effectively breaks generating all image test cases for RHEL-8.6
due to `image-info` failing to analyze the resulting image.
This change will allow us to consistently generate image test cases
(manifests only) for all image types, which were so far not covered by
the image testing.
Modify the `generate-test-cases` script to recognize the `no-image-info`
option in the `format-request-map.json`
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The image is not available via Weldr API, because it requires RHUI
client RPMs.
The content and configuration is based on RHEL-8.6 EC2 SAP image, since
there is no definition for the RHEL-9 SAP image yet.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Generating image test case with all possible blueprint customizations is
supported only for `qcow2` image type. Generating `qcow2` images with
customizations was not a lot of fun. One had to use a special CLI option
for this case.
To streamline the generation of image test cases, drop the
`--with-customizations` option from the `generate-test-cases` script and
move the functionality to `format-request-map.json` by defining a new
`qcow2-customize` test case. This is very similar to what `*edge-rt`
test case already uses. This mean that when the test case for `qcow2`
image type is being generated, actually two test cases will be
generated. The filesystem customizations since RHEL-8.5 are handled
through distro-specific `overrides`.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Some of the `overrides` specified in `format-request-map.json` were
redundant, because the same blueprint customizations were already
specified in the image compose-request. Thus these values had no
additional values.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Some image type test cases require additional repositories, which are
not available for all architectures. However, when an image type test
case is specified in the `format-request-map.json`, it is generated on any
architecture.
This behavior is creating issues when generating `*edge-commit` image
type test cases. This is because the `format-request-map.json` contains one
additional definition for `*edge-commit-rt`, which includes `kernel-rt`
package. However repositories with this package are available only for
x86_64. Therefore, when generating image test cases for `*edge-commit`,
the `generate-test-cases` script always generates two test cases,
but the generation of `*edge-commit-rt` always fails on non-x86_64
architectures.
Add a new optional member to the image type test case object in
`format-request-map.json`, called `supported_arches`. Its value is a
list of strings, specifying the supported architectures of the image
type test case. In case the member is not specified, the image test
case is supported on any architecture.
Extend the `generate-test-cases` script to skip image type test case
generation in case the case has the `supported_arches` specified and the
requested architecture is not in the list.
Fix#1478
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Renamed tar-installer to image-installer.
This is a more appropriate name:
- It disassociates the image type from the "tar" image type. The two
should not be perceived to be connected.
- It's more descriptive. The format of the payload (tar) isn't relevant
to the purpose of the image type.
Added to format-request-map and generated manifest.
No image-info since ISOs aren't supported by the tool. The manifest is
useful only for detecting changes in the image type definition.
Add new image type definitions `ec2` and `ec2-ha` representing the
official RHEL ec2 image types.
Add a `xzArchivePipeline()`, which returns a pipeline producing a XZ
archive from a file produced by a different pipeline.
Add rpmrepo snapshots for `rhui` and `ha` repositories used to generate
image test cases. `rhui` is used by the `ec2` image and it is available
on x86_64 and aarch64 architectures. `ha` is used by the `ec2-ha` image
and it is available only for x86_64.
The new image type definitions are currently not used by any
API test case.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
We have integration tests that boot most of the image types, let
the image tests focus on the actual content of the images, shaving
off considerable CI time.
The only missing one is OpenStack (which should be ported over) and
tar (which we need to define the integration test for).
Signed-off-by: Tom Gundersen <teg@jklm.no>
Two new test cases added to format-request-map and test cases are
generated.
1. kernel-rt for RHEL images:
Requires new package repositories for RHEL 8.3 and 8.4.
Creates an OSTree commit with the `kernel-rt` as a customization.
2. kernel-debug for Fedora images:
kernel-rt isn't included in the official fedora repositories. Using
kernel-debug at least tests the feature with the fedora-iot-commit type.
Remove 'subscription' customization from 'format-request-map.json',
because it currently has no effect on image test cases. The reason is
that this customization of image is possible only via Cloud API in
composer and setting imageOptions. However image test cases currently
support image customization only via Blueprint customizations.
Update repo links in 'repos.json' to the latest rpmrepo snapshots.
Regenerate all image test cases affected by updates to 'repos.json'
and 'format-request-map.json'. This includes all `rhel-8` and `rhel-84`
image test cases.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
This commit does several things:
1) Changes the Fedora 33 repos in the test case generator from development
to release ones.
2) Fixes format-request-map.json so we can generate fedora-iot-commit
"images".
3) Regenerates all the cases.
Although the vhdx format is smaller than raw, AWS often has some issues
booting instances from an imported vhdx snapshot. The raw format seems
to boot consistently at AWS.
Fixes#754.
Signed-off-by: Major Hayden <major@redhat.com>
Allow individual test-cases or sub-sets of test-cases to be generated
more easily.
We allow explicit skipping of image-info generation (and hence the
osbuild run), and also individual image types to be specified.
Also drop distros and image types that are no longer supported.
Signed-off-by: Tom Gundersen <teg@jklm.no>
Previously, vhd images were tested using QEMU. This commit changes that to
boot them in the actual Azure infrastructure.
Azure VMs have quite a lot of dependencies - a network interface, a virtual
network, a network security group, a public ip address and a disk. Azure CLI
and Azure Portal handle the creation of all these resources internally.
However, when using the API, the caller is responsible to create all these
resources before creating an actual VM.
To handle the creation of all the resources in the right order, a deployment
is used. A deployment is a set of resources defined in a JSON document.
It can optionally take parameters to customize each deployment. After the
deployment is finished, the VM is up and ready to be tested using SSH.
Sadly, the deployments are a bit hard to clean-up. One would expect that
deleting a deployment removes all the deployed resources. However, it doesn't
work this way and therefore it's needed to clean up all resources "manually".
For this reason, our deployment sets a unique tag on all the resources created
by the deployment. After this test is finished, the API is queried for all
the resources with the tag and then, they're deleted in the right order.
Prior this commit the ami image type produced raw.xz images. This was bad for
two reasons:
- The upload was broken because AWS doesn't support tar.xz format
- XZ compression is terribly slow
This commit changes the format to vhdx, which is supported by AWS and also
quite quick. See https://github.com/osbuild/osbuild-composer/issues/257
why vhdx was chosen.
Fixes#257
Prior this commit the ami images were tested locally using qemu. This does
not reflect at all how they're used in practice. This commit introduces
the support for running them in the actual AWS. Yay!
The structure of code reflects that we want to switch to osbuild-composer
to build the images soon.
