Commit graph

188 commits

Author SHA1 Message Date
Achilleas Koutsou
fb4309acf4 test: add repos and test manifests for Fedora 38 2022-09-13 16:06:19 +01:00
Achilleas Koutsou
eb731e031d test: generate new manifests
Add new image type to format request map and generate manifests.
2022-09-13 16:06:19 +01:00
Ondřej Budai
e2a298723e ci: enable fedora 37 tests
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-13 16:06:19 +01:00
Tomas Hozza
735c6ec9c3 generate-all-test-cases: make output default to test/data/manifests 2022-08-17 11:59:39 +02:00
Alexander Todorov
9cf8054d21 Add fedora-36.json repo definitions 2022-07-21 12:19:38 +03:00
Ondřej Budai
3e25f5ef76 remove all traces of fedora 34
Fedora 34 is EOL, let's remove all traces of it, including:

- distro definition
- repositories (and test one)
- test manifests
- special package set rules
- hacks from the spec file

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-07-18 20:38:33 +02:00
Achilleas Koutsou
577ea9eb95 format-request-map: minor fixes and updates
- azure-rhui filename fix (.xz)
- azure-rhui doesn't need distro override
- minor formatting fixes
2022-07-08 09:30:34 +01:00
Achilleas Koutsou
0adb742ede test-case-generators/repos: new repos for RHEL 8.4
Added unversioned (el8, no minor version) repositories for RHEL 8.4
that provide packages for building ec2 and azure-rhui image types.

Added new repo snapshots to RHEL 8.4: ha, sap, and saphana
2022-07-08 09:30:34 +01:00
Achilleas Koutsou
6a4e46e51a test-case-generators/repos: rhel-edge-commit -> edge-commit
With the merging of 8.4 into the main rhel8 package, the name
'rhel-edge-commit' is no longer the primary name for the image type.
More generally, the 'rhel-' prefix doesn't appear in the main name for
any image type anymore.
2022-07-08 09:30:34 +01:00
fkolwa
14fd1fe279 Update RHEL90-91 repo config and and 9.1 test cases 2022-07-07 14:28:59 +02:00
Achilleas Koutsou
cc5d32169d distro/rhel8: add rhel-8 alias to rhel-86
Add a plain `rhel-8` alias as the default distribution name and version
for the `rhel8` package.  The `rhel-86` distro is still available via
the NewRHEL86() constructor.  These two distributions are identical.

Repositories
------------

The rhel-8 repositories (repositories/rhel-8.json) are now set to the
CDN repositories with no minor version:
https://cdn.redhat.com/content/dist/rhel8/8/...

The rhel-8 test repositories (test/data/repositories/rhel-8.json) were
already set to the plain `8` repositories.  The Google repos have been
added.

The test case generator repositories used for `rhel-8` are the rpmrepo
snapshots as for rhel-86.
2022-07-01 23:52:13 +01:00
Christian Kellner
c5a5da5f56 distro/fedora: add container image type
Add a container image type that is based on the existing fedora
container image. There is a delta in terms of the configuration
because osbuild does not yet provide all the neccessary means,
but the package set is already very close.
2022-06-29 10:02:46 +02:00
Jakub Rusz
2eee0632d6 manifests: regenerate rhel-91-gce-boot
We don't want to do gpg check for the google repositories.
2022-06-21 10:40:58 +02:00
Christian Kellner
06e05df620 distro: add support for building a rhel 7 (qcow2)
Based on the RHEL 8.6 pipelines, needs a special buildroot with two
extra packages: python3-iniparse and python3-PyYAML.
Only x86_64 support for now.
2022-06-21 00:06:26 +02:00
Achilleas Koutsou
c1f7003e12 genall: move to cmd/ and rename to gen-manifests 2022-06-01 11:36:52 +01:00
Achilleas Koutsou
7a70a5e69b dnfjson: drop repo checksums
The repository checksums in the response from dnf-json aren't used
anywhere.  Since we're making changes to dnf-json and depsolving, now is
a good opportunity to drop them completely.
2022-06-01 11:36:52 +01:00
Achilleas Koutsou
0ef4e7ee84 genall.go: create all test manifests
- Standalone executable for generating all test manifests in parallel.
- Command line flags:
  - Output directory (-output)
  - Number of concurrent workers (-workers)
- Collects list of image types from the distro list and reads:
  - tools/test-case-generators/repos.json for repositories
  - tools/test-case-generators/format-request-map.json for
    customizations
- Prints progress (finished/total)
- Collects errors and failures and prints them after all jobs are
  finished
2022-06-01 11:36:52 +01:00
Achilleas Koutsou
b1ce6e18d5 tools: remove dnf-json socket start function
ensure_dnf_json() function no longer needed.
2022-06-01 11:36:52 +01:00
Christian Kellner
4e9e438b75 distro/rhel90: add support for azure marketplace
Add support for building images for the Azure marketplace: add a
new image type "azure-rhui" that can be used to build images
tailored to the Azure marketplace.
This code is based on the corresponding image type in 8.6.

NB: does not have systemd-resovled (following RHEL 9 defaults)
2022-05-19 11:22:47 +02:00
Christian Kellner
8ee19af1d0 test-case-generators/repos: remove duplicated rhel-91 block
We really only can have one. The one that was used for the generation
of the manifests is kept and the other one removed (although it has
newer repositories).
2022-05-19 11:22:47 +02:00
Juan Abia
967c8734a1 test-case-generators: update repos and add img types overrides
overrides where needed for `qcow2` and `simplified-image-installer` images on specific distros. Also some repos needed to be updated to newer versions.
2022-05-13 21:01:37 +03:00
Juan Abia
a87193369d update repos.json
fix typos and add new repos
2022-05-13 21:01:37 +03:00
Tomas Hozza
1aabc1870d Image tests: don't use customizations for rhel-85/90beta edge-installer
Do not apply the user customizations on edge-installer on RHEL-85/90beta,
since they are not supported there yet. The way we generate image test
cases from the `format-request-map.json` makes even the customized image
types being generated for all distributions automatically.
2022-05-13 21:01:37 +03:00
Tomas Hozza
5be81326eb generate-all-test-cases: capture manifests output
Capture stdout and stderr output when running generate-test-casesin the manifests command.
This is helpful for debugging test case generation failures.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-05-13 21:01:37 +03:00
Tomas Hozza
40c095a850 Tools: fetch image test case generation matrix from composer
Add a simple tool `osbuild-composer-image-definitions` which dumps the
matrix of all distributions, architectures and image types names
supported by composer as a JSON to the stdout.

Default to fetching the image test case generation matrix directly from
composer. This eliminates the need to update a JSON source file with
this information every time a new distro or image type are added to
composer.

Delete the previously used JSON source file with the image test case
generation matrix.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-05-13 21:01:37 +03:00
Tomas Hozza
035b616308 test-case-generators: add request definitions for edge-raw-image
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-05-13 21:01:37 +03:00
Tomas Hozza
97cb091ef7 test-case-generators: add RHEL-87/91 and F36 repos
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-05-13 21:01:37 +03:00
Achilleas Koutsou
1ff36bce9a tools/format-request-map: add 'core' group to qcow2 customize
Removed in 2beb707def, possibly
accidentally.
The affected manifests were not regenerated based on this change so
they all already contain the core group.
2022-05-13 13:25:34 +02:00
Jordi Gil
00a84d1fdc distro: allow user and group customizations for iot-installer 2022-05-09 12:25:21 +02:00
Jordi Gil
2beb707def distro/fedora: regenerated test manifests 2022-05-09 12:25:21 +02:00
Ondřej Budai
c811860771 test-case-generators: add rhel-87 and rhel-91
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-05-03 18:13:28 +02:00
Christian Kellner
98b491fcac distro/rhel85: automatically convert to LVM on fs customizations
Whenever we create a new mountpoint due to a user customization,
ensure the layout uses LVM, i.e. convert plain layouts to it, if
needed. This does not apply to rpm-ostree based systems.
Add "lvm2" to the build pipeline and thus generate new manifests
and image infos.
Adjust the existing tests that assumed we can not create more
than 4 partitions on mbr layouts, since that is now not true
anymore.
This is a port from rhel86, commit 63aa155

The change in osPipeline() is required now to fix the Prefix for the
bootloader specification when LVM is used. The unspecified Prefix, which
was previously used for all cases, defaults to "/boot". When the layout
is converted to LVM, a boot partition is created and the BLS Prefix
should be set to "".

In the case where we don't have a partition table, the BLS stage is not
needed, but it was done unconditionally before, so keep the default
image definitions unchanged.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-04-30 12:37:50 +01:00
Tomas Hozza
8ee90334ba generate-all-test-cases: add manifests command
Add `manifests` command for generating image test cases without
`image-info` report, so only manifests.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-29 12:13:00 +02:00
Achilleas Koutsou
d357f92f91 test: add fs customization to qcow2 test case
Added a filesystem customization to the qcow2 test case to test that the
filesystem is converted to an LVM layout.
Set overrides for distros that don't support fs customizations.
2022-04-29 08:55:36 +02:00
Achilleas Koutsou
81c62204b9 repositories: update gpg key for all RHEL 8.x repos
Follow-up to 60db6ad06f

The SHA-1 key is no longer supported in RHEL 9.0.  This isn't a problem
for RHEL 8.x in general, but it prevents cross building RHEL 8.x images
on RHEL 9.0, since the host (RHEL 9.0) rpm and openssl cannot import the
older keys and we fail to bootstrap the build root for the new image if
the source repositories use SHA-1 keys.

Related rhbz#2058497 (Comment 18).

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2022-04-28 09:50:27 +02:00
Achilleas Koutsou
389b7c1848 Unify formatting for all repository json files
- 2 space indent
- lists on multiple lines
- newlines at EOF

This was accomplished by simply running each file through `jq` with no
arguments.
It is also equivalent to Python's `json.dump(..., indent=2)` plus the
added newline.
2022-04-28 09:50:27 +02:00
Tomas Hozza
f45bdf3fb3 Don't pass GPG keys for GCP repos and don't verify signatures on el9/c9s
Google repositories use RSA/SHA1 for signing packages. However the SHA1
has been disabled by default on el9/c9s. Since osbuild-composer imports
GPG keys specified in the repository definition unconditionally, this
creates issues when installing rpms signed with the key by osbuild [1].

Remove GPG keys in all el9/c9s GCP repo definitions and disable GPG
signature verification until [2] is resolved.

[1] https://github.com/osbuild/osbuild/issues/991
[2] https://issuetracker.google.com/issues/223626963

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Tomas Hozza
e7f6b95032 RHEL-84: add gce-rhui image type
Add the `gce-rhui` image type intended for Google Compute Engine. The image
uses Google's RHUI infrastructure to access Red Hat content.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Tomas Hozza
795294d98f RHEL-84: add gce image type
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Tomas Hozza
77b5ef459d RHEL-85: add gce-rhui image type
Add the `gce-rhui` image type intended for Google Compute Engine. The image
uses Google's RHUI infrastructure to access Red Hat content.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Tomas Hozza
6ffe63442e RHEL-85: add gce image type
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Tomas Hozza
0219ad1f43 RHEL-86: add gce-rhui image type
Add the `gce-rhui` image type intended for Google Compute Engine. The image
uses Google's RHUI infrastructure to access Red Hat content.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Tomas Hozza
db75c39c4d RHEL-86: add gce image type
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Tomas Hozza
5d27b7c784 RHEL-90: add gce image type
Add the `gce` image type intended for Google Compute Engine. The image
is BYOS - bring your own subscription and requires registering in order
to access Red Hat content.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-04-14 19:07:31 +01:00
Achilleas Koutsou
15243f1dc5 distro: allow user and group customizations for edge-installer
Followup from, f34380d5b5 and
3a1765a5a8, copied to the rest of the RHEL
distro definitions.

For now, these customizations have no effect on the manifest.

The new `with-users` variants of the edge-installer test cases include
the user customizations in the blueprint, but the manifests are
(currently) the same as the corresponding base cases.
2022-04-06 12:09:09 +02:00
Achilleas Koutsou
11be84c225 test: add RHEL 8.4 rhel-edge-installer test case
Include RHEL 8.4 rhel-edge-installer in the test cases.
2022-04-06 12:09:09 +02:00
Achilleas Koutsou
0554da5a4d distro-arch-imagetype-map: change indent to 2 spc
Change indent to match the rest of the json files in the project.
2022-04-06 12:09:09 +02:00
Ondřej Budai
fd49ed9a07 repositories: update key for RHEL 9.0
RHEL 9.0 will ship a brand new auxiliary key. Let's use it everywhere in our
RHEL 9 stuff. Taken from current RHEL 9.0's redhat-release package.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-04-01 19:22:23 +02:00
Christian Kellner
45b1fc3cd1 osbuild2: honor GPG key setting for rpm inputs
We should honour `pkg.CheckGPG` when creating the file inputs for the
rpm stage. This was lost in the transition from v1 to v2 manifests.

Regenerate image test manifests.

Co-authored-by: Tomas Hozza <thozza@redhat.com>
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-03-30 14:32:28 +02:00
Achilleas Koutsou
3ac1f940a4 format-request-map: add installers with users
Add two new test cases:
- image-installer-with-users
- edge-installer-with-users
2022-03-28 14:08:57 +02:00