Commit graph

5336 commits

Author SHA1 Message Date
Jakub Rusz
004c4e8acc Gitlab CI: Run libvirt.sh tests on GCP
We can now use GCP runners in our CI. Let's test them out by running all
of libvirt tests for starters.
2023-02-20 14:24:09 +01:00
Sanne Raymaekers
96b42b5791 internal/worker: add String to clienterrors.Error struct
This should make the logging better when errors have another error
struct.

Fixes #3272
2023-02-16 00:55:34 +01:00
Juan Abia
5a62453190 Remove cloud cleaner
cloud cleaner is now being executed in it's own repo.
2023-02-15 17:23:35 +01:00
Juan Abia
197cd47e0b test/cases: move CIV options into a variable
In order to provide different options when running CIV from it's own CI,
move them to the variable CIV_OPTIONS
2023-02-15 17:23:35 +01:00
Thomas Lavocat
be48279af8 schutzfile: update OSBuild's ref
Image-info needs to have support for mounting the devices with their
respective mount options. Otherwise it will fail schema validation.
2023-02-14 14:37:10 +01:00
Thomas Lavocat
aba981c508 tools/image-info: bad syntax on rhel8 python
We can't use:

with (a() as b, c() as d):
    pass

On older versions of python.
2023-02-14 14:37:10 +01:00
Thomas Lavocat
15d80a0cd1 tools/image-info: cosmetics, autopep8 2023-02-14 14:37:10 +01:00
Thomas Lavocat
36d890d452 toos/image-info: use SECTOR-SIZE
Instead of numerical value
2023-02-14 14:37:10 +01:00
Thomas Lavocat
0324e02659 tools/image-info: support iso
Add the support for ISOs. Mounting then as a loopack device and then
accessing the content as a tarball
2023-02-14 14:37:10 +01:00
Thomas Lavocat
d0be1ec5a4 tools/image-info: validate with osbuild's schemas
Before we were invoking osbuild's stages/devices/mounter directly
without taking the time to validate that the options sent were actually
valid and supported.
This commit adds the support of the validation schema into image-info so
that we're sure we don't mess with the internals when we call them.
2023-02-14 14:37:10 +01:00
Thomas Lavocat
ffd45d635a tools/image-info: mount with the mount options
Before the new version of image-info was ignoring the mount options for
the devices leading to errors in the resulting json.
2023-02-14 14:37:10 +01:00
Thomas Lavocat
62e6bed23d tools/image-info: clean placeholder try block
Was only there to ease diffing previous commit.
2023-02-14 14:37:10 +01:00
Thomas Lavocat
31e3729236 tools/image-info: use osbuild's internal to mount
On the newest versions of rhel 92, 88, a change in behavior makes the
previous version of image-info failing to mount loopback devices. We've
tracked down this error to be a race condition on udev, yet without
understanding what changed for now.

Osbuild had for some time already a cleaner way to mount partitions.
osbuild has some machinery to opt out of block device handling in udev
48a4419705/devices/org.osbuild.loopback (L69)
Using this fixes the issue at hand.

This changes the way we need to mount all the partitions, including the
LVM ones. This new mechanism might also pave the way to include lusks fs.
2023-02-14 14:37:10 +01:00
Thomas Lavocat
aaddca445d tools/image-info: fix mypy errors 2023-02-14 14:37:10 +01:00
Antonio Murdaca
71ca1d0e9a regenerate manifests
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-02-14 12:07:23 +01:00
Antonio Murdaca
6afab3dff2 rhel9: fix conditionals for sysroot.readonly enablement
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-02-14 12:07:23 +01:00
Antonio Murdaca
1b9e008c29 rhel8: remove sysroot.readonly functionality
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-02-14 12:07:23 +01:00
Brian C. Lane
6846f59ff4 manifest: Add tests for subscriptions
There are 3 different ways for subscriptions to be setup, test the
pipeline to make sure it includes the correct commands, and check the
package set chain to make sure it includes the packages needed to
support the selected subscription mode.
2023-02-09 09:41:15 +01:00
Brian C. Lane
6944d15293 cloudapi: Add a test for the subscription rhc flag
This makes sure that the new flag is accepted by the API endpoint.
2023-02-09 09:41:15 +01:00
Brian C. Lane
b0e388d115 cloudapi: Add subscription option for rhc
When rhc is selected it will install the required packages, register
using rhc and always enable insights.

When rhc is not selected it will use subscription manager for
registration, and optionally enable insights. Also installing required
packages.
2023-02-09 09:41:15 +01:00
Jakub Rusz
3cdfa9d7f0 internal/prometheus: add more buckets for job durations
We were hitting the limit on stage, let's increase it.
2023-02-08 12:33:10 +01:00
Diaa Sami
20c6fad7c2 osbuild-worker/koji: Add logging for koji requests/responses 2023-02-08 11:40:34 +01:00
schutzbot
4e574b6e99 Post release version bump
[skip ci]
2023-02-08 08:25:30 +00:00
Antonio Murdaca
b8ddbd3031 internal/distro/rhel9: enable ignition on 9+
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-02-07 13:30:22 +01:00
Antonio Murdaca
5ab00987cd test/cases: add ignition test to edge-raw-image
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-02-07 13:30:22 +01:00
Antonio Murdaca
b3f1b0edc8 ignition: drop embedded provisioning url
Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-02-07 13:30:22 +01:00
Ondřej Budai
2b1facb44d distro/rhel8: don't install missing MSFT key into azure-sap-rhui
The rhui-azure-rhel8-sap-ha package is currently missing
the /etc/pki/rpm-gpg/RPM-GPG-KEY-microsoft-azure-release key.
This makes the image type unbuildable, which causes some of our tests
to fail.

Overlay the generic RHUI config, so the missing key isn't imported.
See CLOUDX-336 for more information.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-02-07 13:16:12 +01:00
Ondřej Budai
bd5bffc8cf distro/rhel8: make sap image config methods private
They are not used in a different package, so they can be private.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-02-07 13:16:12 +01:00
dependabot[bot]
c2a290ef76 build(deps): bump github.com/labstack/echo/v4 from 4.9.0 to 4.10.0
Bumps [github.com/labstack/echo/v4](https://github.com/labstack/echo) from 4.9.0 to 4.10.0.
- [Release notes](https://github.com/labstack/echo/releases)
- [Changelog](https://github.com/labstack/echo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/labstack/echo/compare/v4.9.0...v4.10.0)

---
updated-dependencies:
- dependency-name: github.com/labstack/echo/v4
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-07 09:03:10 +01:00
dependabot[bot]
ff86f29120 build(deps): bump github.com/openshift-online/ocm-sdk-go
Bumps [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go) from 0.1.287 to 0.1.315.
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.287...v0.1.315)

---
updated-dependencies:
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-07 09:02:42 +01:00
Xiaofeng Wang
b4f15f13e3 test: Add minimal-raw image test
The minimal-raw image is for Fedora only and feature merged by
PR #3181
2023-02-06 17:58:05 +01:00
Sanne Raymaekers
8f9f0d03c5 repositories: fix rhel-90 repositories
The rhel-90 repos should point to the rhel-90 minor version repos, the
rhel-9 repos should point to the rolling release.
2023-02-06 17:35:54 +01:00
Antonio Murdaca
e34b88d6e4 .github/workflows: pin golangci-lint to v1.51.0
The new "latest" aka v1.51.1 started throwing unused errors during runs,
blocking all the other tests, to quickly unblock that until a proper
fix/rewrite is found, pin the version to the previous one.

Signed-off-by: Antonio Murdaca <antoniomurdaca@gmail.com>
2023-02-06 11:04:06 +01:00
Ondřej Budai
9633dce3bb osbuild-pipeline: add support for RHSM when defining repos
I use this tool quite a lot and I often want to use the CDN content, so
I would very much appreciate RHSM support. :)

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-02-06 10:02:04 +01:00
Achilleas Koutsou
d877b7419e osbuild-dnf-json-tests: remove distro loop
We only test one distro, CentOS 9, so there's no need to loop and spawn
a new t.Run().
2023-02-02 19:01:20 +01:00
Brian C. Lane
356f261bcd dnf-json: disable pylint warnings
The _dnfrepo is full of branches, turn off warning.

The keyfile doesn't use 'with' because the file needs to remain
available when the function exits. Cleanup of persistdir will clean up
the temporary file used for the key.
2023-02-01 10:27:58 +01:00
Brian C. Lane
d5e666cc9c tests: Add tests for new repo gpg support
Add testing for gpgkeys and check_repogpg.
2023-02-01 10:27:58 +01:00
Brian C. Lane
c2577eaea8 Add gpgkey and check_repogpg support to dnf-json
This allows verification of repository metadata signatures.

The gpgkeys field is a list of key urls, or the gpg key itself, starting
with '-----BEGIN PGP PUBLIC KEY BLOCK-----'. These will be written to a
temporary file, and that file:// url will be passed to dnf.
2023-02-01 10:27:58 +01:00
Brian C. Lane
d12447408b Change RepoConfig.GPGKey to an array of key strings
DNF supports more than one GPG key. It is possible that one may be used for
signing packages, and another to sign the repository metadata. This
renamed GPGKey to GPGKeys internally. It does not change the on-disk
repository json format.
2023-02-01 10:27:58 +01:00
Brian C. Lane
a47375a10d Rename gpgkey_urls to gpgkeys
This has never been supported in osbuild-composer, so it is safe to
rename in preparation for adding repo metadata gpg signature checking.
2023-02-01 10:27:58 +01:00
Achilleas Koutsou
7a1d7ae1d7 test: update installer manifests for RHEL 9.x and CS9
Dracut modules and drivers fixed for the dracut stage in all RHEL 9.x
and CS9 Anaconda installers (image-installer and edge-installer).
2023-01-31 11:03:26 +01:00
Achilleas Koutsou
2f96cc7051 manifest: fix dracut modules and drivers for Anaconda installers
When the image definitions were updated to the new framework, I failed
to update the dracut modules for Anaconda installers to match the
existing ones.

The changes in the manifest are at commit
c4af0a1886.
The nvdimm module and the additional drivers were removed.

The nvdimm module in particular is required for http boot but should
only be specified for RHEL 9.  In RHEL 8 it is part of the default set
of modules.

See 02bb7a0b4f and
dc95382ba3 for the original commits that
introduced these changes.
2023-01-31 11:03:26 +01:00
Irene Diez
1dfc82ae4a test/data/manifests: regen for edge container + commit
Signed-off-by: Irene Diez <idiez@redhat.com>
2023-01-30 19:26:44 +01:00
Irene Diez
3b62e15d01 internal/distro/rhel8/edge: add sos package for rhel8.6+ and CS8
Similarly to the change made for rhel9, adding the sos package
gives users a built-in way to gather system logs and debug info.

Signed-off-by: Irene Diez <idiez@redhat.com>
2023-01-30 19:26:44 +01:00
Irene Diez
18aee4e465 internal/distro/rhel9/edge: add sos package
Adds the sos package to the edge commit package set so that
users have built-in way to gather system logs and debug info.

Signed-off-by: Irene Diez <idiez@redhat.com>
2023-01-30 19:26:44 +01:00
Achilleas Koutsou
2bdf6c724f distro/rhel8: update distro unit tests 2023-01-30 17:40:18 +01:00
Achilleas Koutsou
9706de4f44 test: generate new manifests: RHEL 8 Azure aarch64 2023-01-30 17:40:18 +01:00
Achilleas Koutsou
5e27e01e5e test-case-generators: enable rhel8 azure aarch64 manifest generation
Add the rhui-azure rpmrepo snapshots to aarch64 repositories.
The repos are labelled as x86_64 but they're actually 'noarch' and we
name them x86_64 when creating the snapshots.
2023-01-30 17:40:18 +01:00
Achilleas Koutsou
2723554b2a distro/rhel8: add Azure aarch64 platform and include image types
The Azure image types require hyperv-daemons which isn't available on
RHEL 8 prior to 8.6.
2023-01-30 17:40:18 +01:00
Achilleas Koutsou
c3d5fd516f distro/rhel8: add partition table for aarch64 azure-rhui
Copy of the partition table for the same image type in RHEL 9.
2023-01-30 17:40:18 +01:00