Commit graph

6163 commits

Author SHA1 Message Date
Sanne Raymaekers
828d82e871 upload/azure: user newer resourcemanager packages
The older azure sdk and autorest packages are deprecated.

Signed-off-by: Sanne Raymaekers <sanne.raymaekers@gmail.com>
2024-01-16 14:25:58 +01:00
Brian C. Lane
d5e1bc28e9 tests: Run check-runners as part of the tests 2024-01-16 12:29:21 +01:00
Brian C. Lane
388aa22b94 Schutzfile: Remove unused repos as reported by ./tools/check-runners
This removes these unused repos:

fedora-37
rhel-8.6
rhel-8.7
rhel-8.9
rhel-9.0
rhel-9.1
rhel-9.3
2024-01-16 12:29:21 +01:00
Brian C. Lane
978c20d36f tools: Add helper script to check Schutzfile and gitlab-ci.yml runners
This script reads the RUNNER entries from .gitlab-ci.yml and the
snapshot repo names from Schutzfile. It then prints which runners don't
have entries in Schutzfile and which Schutzfile entries don't have
runners.

This is slightly complicated by the fact that centos-stream reports
itself as centos. So when checking for missing entries they are
considered to be the same thing. eg. If centos is a RUNNER and
centos-stream is in Schutzfile it doesn't report anything missing.
2024-01-16 12:29:21 +01:00
Brian C. Lane
e287138e2e cloudapi: Add testing for blueprint compose request
This tests to make sure the blueprint produced by the customizations
data and the blueprint data are identical.
2024-01-15 11:48:35 +01:00
Brian C. Lane
7438e29375 cloudapi: Create a compose using the blueprint data
If the request includes a blueprint (and not customizations) it uses
that blueprint for the compose.
2024-01-15 11:48:35 +01:00
Brian C. Lane
5b87877109 cloudapi: Add optional blueprint schema to compose request
This adds a 'blueprint' section to the compose request. It also
restricts it so that only 'blueprint' or 'customizations' can be
included, but not both. The goal is to move to using 'blueprint' for all
customizations so that there is a single consistent interface for the
clients.

Where the openapi schemas are the same between the two they have been
shared, but a few are different. They are created with 'Blueprint*' as
their name.

This also re-adds the SSHKey schema removed by commit
bfad6d50e1, it is used by the Blueprint
Customization.
2024-01-15 11:48:35 +01:00
Brian C. Lane
ea05f400e6 cloudapi: Listen on /run/cloudapi/api.socket
Add a local socket for communicating with the cloudapi. It is started by
osbuild-composer.socket and is located at /run/cloudapi/api.socket

cloudapi requests can be passed to it using curl like this:

curl -k --unix-socket /run/cloudapi/api.socket --header 'Content-Type: application/json' \
--data request.json http://localhost/api/image-builder-composer/v2/compose

A simple request.json looks like this:

{
  "distribution": "fedora-38",
  "image_request":
    {
      "architecture": "x86_64",
      "image_type": "guest-image",
      "upload_options": {},
      "repositories": [
          {
            "name": "fedora",
            "metalink": "https://mirrors.fedoraproject.org/metalink?repo=fedora-38&arch=x86_64",
            "check_gpg": false
          },
          {
            "name": "updates",
            "metalink": "https://mirrors.fedoraproject.org/metalink?repo=updates-released-f38&arch=x86_64",
            "check_gpg": false
          }
      ]
    }
}
2024-01-15 11:48:35 +01:00
Alexander Todorov
6735e74897 Invert wrong boolean condition in filesystem test
the conditionals have been introduced in
8960a51d2f
which states (emphasys mine):

> The new partition rules are **in** osbuild-composer v94 and higher.

but the condition was "not v94 or higher" and as a result it fails on
8.10 nightly with osbuild-composer v92 with:

failed to initialize osbuild manifest: The following custom mountpoints are not supported [\"/boot/firmware\" \"/foobar\"]

see https://gitlab.com/redhat/services/products/image-builder/ci/osbuild-composer/-/jobs/5905865402
2024-01-11 20:57:46 +04:00
Alexander Todorov
22febe2316 Skip filesystem.sh test job on Fedora-39
looks like there is a kernel bug and the patch is 6+ months old:
https://github.com/osbuild/osbuild-composer/pull/3820#issuecomment-1881726795
2024-01-10 15:47:42 +01:00
Michael Vogt
a93814c1a8 Schutzfile: bump version of osbuild to 5fc3b565 2024-01-10 15:47:42 +01:00
Michael Vogt
5f678b1d7b test: add debug output to filesystem.sh if mounting fails 2024-01-10 15:47:42 +01:00
Alexander Todorov
ba2f5a7ecb Fix a boolean condition in filesystem.sh 2024-01-10 15:47:42 +01:00
Alexander Todorov
bbb685e9a3 Add Fedora 39 repositories in Schutzfile
Note: Modular repositories have been retired:
c80be57977
2024-01-10 15:47:42 +01:00
Alexander Todorov
5033afa2e2 Update test runners to Fedora 39 2024-01-10 15:47:42 +01:00
Michael Vogt
973a8cf285 image-info: update for new "partition" option in mounts.Mount
In the osbuild PR#1501 [0] a new paramter "partition" for
mounts.Mount() was added.

This commit updates the code that it deals cleanly with the new
and the old API for `mount.Mount`.

[0] https://github.com/osbuild/osbuild/pull/1501
2024-01-10 09:38:31 +01:00
schutzbot
421273d1e0 Post release version bump
[skip ci]
2024-01-10 08:32:45 +00:00
Michael Vogt
96e3b727b3 schutzbot: add my "mvogt" SSH key 2024-01-08 15:55:38 +01:00
Tomáš Hozza
0169b7b905 Schutzfile: update osbuild SHA to v102
This should solve the CI failures on el8.10 due to fix the missing
`/etc/X11/xorg.conf.d/00-keyboard.conf` file when building images and
setting keyboard layout.

The osbuild v102 is currently blocked in RHEL CI, so workaround this
issue for now at least on `main` in the upstream.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-04 11:29:43 +01:00
schutzbot
f25bbda32d schutzfile: Update snapshots to 20240101 2024-01-04 11:29:43 +01:00
dependabot[bot]
623e761c45 build(deps): bump the go-deps group with 5 updates
Bumps the go-deps group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) | `1.49.7` | `1.49.13` |
| [github.com/labstack/echo/v4](https://github.com/labstack/echo) | `4.11.3` | `4.11.4` |
| [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go) | `0.1.388` | `0.1.390` |
| [github.com/osbuild/images](https://github.com/osbuild/images) | `0.26.0` | `0.28.0` |
| [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) | `1.17.0` | `1.18.0` |


Updates `github.com/aws/aws-sdk-go` from 1.49.7 to 1.49.13
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.49.7...v1.49.13)

Updates `github.com/labstack/echo/v4` from 4.11.3 to 4.11.4
- [Release notes](https://github.com/labstack/echo/releases)
- [Changelog](https://github.com/labstack/echo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/labstack/echo/compare/v4.11.3...v4.11.4)

Updates `github.com/openshift-online/ocm-sdk-go` from 0.1.388 to 0.1.390
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.388...v0.1.390)

Updates `github.com/osbuild/images` from 0.26.0 to 0.28.0
- [Release notes](https://github.com/osbuild/images/releases)
- [Commits](https://github.com/osbuild/images/compare/v0.26.0...v0.28.0)

Updates `github.com/prometheus/client_golang` from 1.17.0 to 1.18.0
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.17.0...v1.18.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/labstack/echo/v4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/osbuild/images
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-02 11:43:07 +01:00
schutzbot
df3d661178 Post release version bump
[skip ci]
2023-12-27 08:21:58 +00:00
Tomáš Hozza
cfc63297eb Test: delete "diff manifests" test case
This is not adding any value any more, since all image definitions are
defined in the osbuild/images repository and just vendored in this one.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-12-22 12:21:59 +01:00
Alexander Todorov
42a90914ab Surround composer-cli call with set +e/set -e as a workaround
because apparently calling nvrGreaterOrEqual inside a for loop doesn't
behave as expected and this fails on the second iteration.
2023-12-22 11:07:52 +01:00
Alexander Todorov
b47e741895 Add even more debug output
loop iterates overt centos-8 and then goes onto centos-9 and everything
fails silently
2023-12-22 11:07:52 +01:00
Alexander Todorov
8b91cf6060 Add debugging information for cross-distro.sh 2023-12-22 11:07:52 +01:00
Alexander Todorov
d6aff2f103 Adjust expected response based on osbuild-composer version 2023-12-22 11:07:52 +01:00
Ondřej Budai
7bcf8e5942 test-case-generators/repos: add required repos for azure-sap-rhui
This image requires a different RHUI repository and also the saphana one.

Similarly to the previous commit, we should probably remove the manifest diff
tests altogether, but let's just fix them for now.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-12-22 00:11:19 +00:00
Ondřej Budai
6a4b4b5ded dnfjson: fix repo hashes
20808e9848

changed the hash generation algorithm again, let's adjust the values.

In the future, we should probably mock this out, or change the test.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-12-22 00:11:19 +00:00
Ondřej Budai
807f249146 go.mod: bump images to v0.26.0
This is mainly needed in order to get the RHEL 9 SAP Azure images.
2023-12-22 00:11:19 +00:00
Ondrej Ezr
d5483ccfb0 test: Add nginx to the image request 2023-12-20 09:02:06 +01:00
Ondrej Ezr
4b3b942dde cloudapi: Add module_hotfixes flag
Allow passing module_hotfixes flag through the cloudapi.
This will enable depsolving on repositories that might be affected by modularity filtering.

Refs HMS-3202
2023-12-20 09:02:06 +01:00
Ondrej Ezr
61e6f75281 dnf-json: Allow passing module_hotfixes
Allow passing module_hotfixes as repo option,
it will allow disabling of modularity filtering per repository.
2023-12-20 09:02:06 +01:00
Diaa Sami
9b53d3e6a6 composer: update ident field used for Splunk
to maintain backward compatibility
2023-12-18 11:46:23 +01:00
Diaa Sami
2443048ecf composer: cleanup unused code related to fluentd sidecar
COMPOSER-2051
2023-12-18 11:46:23 +01:00
Xiaofeng Wang
3c3445c832 test: fix fdo mac address test issue 2023-12-15 18:56:11 +01:00
Mario Cattamo
a4ebf53ad8 test: add test for fdo interface in simplified installer 2023-12-14 22:21:09 +01:00
djach7
4870a836f3 cloudapi: enable 'fdo.di_mfg_string_type_mac_iface'
Adds implementation of the 'fdo.di_mfg_string_type_mac_iface' dracut
variable to allow simplified installer images to pass this value to the
manufacturing-client.service.
2023-12-14 22:21:09 +01:00
Achilleas Koutsou
6d57e01506 deps: update images to v0.24.0
Update the images dependency to v0.24.0

Includes the addition of the new FDO option
'di_mfg_string_type_mac_iface'.
2023-12-14 22:21:09 +01:00
Sanne Raymaekers
c6aa7d88d2 internal/weldr: specify architecture of compose
This is useful in environments with multi-arch remote workers.
Defaults to the host architecture.
2023-12-14 21:25:32 +01:00
Sanne Raymaekers
e24772dc57 worker/server: check if worker is available for architecture 2023-12-14 21:25:32 +01:00
Sanne Raymaekers
850e44589b worker/server: split out jobqueue call from PostWorker handler 2023-12-14 21:25:32 +01:00
Sanne Raymaekers
ac854b7cc8 pkg/jobqueue: add arch to worker 2023-12-14 21:25:32 +01:00
Xiaofeng Wang
8a8b1406fb test: update osbuild dependency sha for RHEL 9.4 2023-12-14 17:18:40 +08:00
Miguel Martín
991293a897 Generate FIPS compliant SSH keys
Generate FIPS compliant SSH keys required
for testing system FIPS mode support
2023-12-13 10:19:47 +01:00
Miguel Martín
38f9687cc1 Clean up libvirt network
Clean up libvirt network once the script has
finished successfully.
2023-12-13 10:19:47 +01:00
Miguel Martín
53955af7ca Do not run set-env-variables.sh
Do not run `set-env-variables.sh` to set ARCH and OS variables
to avoid the CI failing because of shellcheck lint problems.

Signed-off-by: Miguel Martín <mmartinv@redhat.com>
2023-12-13 10:19:47 +01:00
Miguel Martín
20c8892ec9 Do not harcode user's password hash in scripts
Generate user's password hash based on `EDGE_USER_PASSWORD`
variable instead of hardcoding it

Signed-off-by: Miguel Martín <mmartinv@redhat.com>
2023-12-13 10:19:47 +01:00
Miguel Martín
a0d357c66b Enable FIPS tests
Make it possible to test if system FIPS mode has been enabled
in the resulting images by running the scripts with a FIPS="true"
environment variable

Signed-off-by: Miguel Martín <mmartinv@redhat.com>
2023-12-13 10:19:47 +01:00
Miguel Martín
10ccb3b385 feature: check FIPS mode within the ostree images
Check if the os has the system FIPS mode enabled
when `fips="true"` is passed as an ansible variable.

Signed-off-by: Miguel Martín <mmartinv@redhat.com>
2023-12-13 10:19:47 +01:00