Commit graph

6181 commits

Author SHA1 Message Date
Tomáš Hozza
43e87632fb Drop common.CurrentArch() in favor of osbuild/images
Drop `common.CurrentArch()` implementation and use
`arch.Current().String()` from the osbuild/images instead.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
c4e5ab5aca Drop internal/common/distro.go in favor of osbuild/images
Drop `common.GetHostDistroName()` implementation and use
`distro.GetHostDistroName()` from the osbuild/images instead.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
fb1b27c0ef Move cleaning up old dnfjson cache dirs from composer to Weldr API
Previously, the old dnfjson cache dirs for unsupported distributions
were deleted in the osbuild-composer binary on startup. This is no
longer possible, since the supported distros are determined by loading
available repositories. Loading repositories happens in the Weldr API
constructor. Move the cleanup code there.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
625b1578fa Port osbuild/images v0.33.0 with dot-notation to composer
Update the osbuild/images to the version which introduces "dot notation"
for distro release versions.

 - Replace all uses of distroregistry by distrofactory.
 - Delete local version of reporegistry and use the one from the
   osbuild/images.
 - Weldr: unify `createWeldrAPI()` and `createWeldrAPI2()` into a single
   `createTestWeldrAPI()` function`.
 - store/fixture: rework fixtures to allow overriding the host distro
   name and host architecture name. A cleanup function to restore the
   host distro and arch names is always part of the fixture struct.
 - Delete `distro_mock` package, since it is no longer used.
 - Bump the required version of osbuild to 98, because the OSCAP
   customization is using the 'compress_results' stage option, which is
   not available in older versions of osbuild.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
f6ff8c40dd Delete cmd/osbuild-composer-image-definitions
This command is part of the osbuild/images repo, where all image
definitions live. Having it in the osbuild-composer repository does not
add any value.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
1943732f73 Delete cmd/osbuild-pipeline
This command was used by the original Python scripts used to generate
test manifests. These scripts were deleted, thus there is no need to
keep this command around any more.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
79d5ab483b tools: delete old Python manifest generator scripts
These are no longer used, let's delete them.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
e235805171 Delete cmd/osbuild-package-sets
This command is part of the osbuild/images repo, where all image
definitions live. Having it in the osbuild-composer repository does not
add any value.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
c1affa6188 Delete cmd/osbuild-playground
This command is part of the osbuild/images repo, where all image
definitions live. Having it in the osbuild-composer repository does not
add any value.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
schutzbot
34f4196677 Post release version bump
[skip ci]
2024-01-24 08:23:36 +00:00
Sanne Raymaekers
589bd5f1a1 tools/fedora-worker-packer: fix packer only/except
It should only build fedora, not everything except fedora.
2024-01-23 13:06:38 +01:00
Sanne Raymaekers
6a0c893a4d tools/appsre-packer: add script to build fedora worker images 2024-01-23 00:18:51 +01:00
Sanne Raymaekers
c7aad1b17d tools/app-sre-build-worker-packer: split out the CI script
To add the fedora workers built from AppSRE's infrastructure, we need a
combination that wasn't possible with the current matrix based on the CI
environment (jenkins vs gitlab).
2024-01-23 00:18:51 +01:00
Alexander Todorov
35c4d3efba Revert "test/cases: Add workaround for oscap firewalld rules"
b/c https://github.com/ComplianceAsCode/content/issues/11275 has been
fixed. This reverts commit c77b73a888.
2024-01-22 15:22:34 +04:00
djach7
4d241b684b edge: add iot-simplified-installer image type
Adds iot-simplified-installer image type to enable koji composes.

Signed-off-by: djach7 <djachimo@redhat.com>
2024-01-18 21:45:23 +01:00
Sanne Raymaekers
3631316e36 tools/build-rpms: fix getting the osbuild commit from Schutzfile
osbuild/osbuild-composer#3614 removed the rhel-8.6, the centos-stream-9
entry should be there for a while at least.
2024-01-16 16:26:01 +01:00
Sanne Raymaekers
edadbb3470 upload/azure: set hyperv generation
This is a required parameter in the new armcompute sdk.
2024-01-16 14:25:58 +01:00
Sanne Raymaekers
adc07299a8 upload/azure: set storage account minimum TLS version to 1.2 2024-01-16 14:25:58 +01:00
Sanne Raymaekers
828d82e871 upload/azure: user newer resourcemanager packages
The older azure sdk and autorest packages are deprecated.

Signed-off-by: Sanne Raymaekers <sanne.raymaekers@gmail.com>
2024-01-16 14:25:58 +01:00
Brian C. Lane
d5e1bc28e9 tests: Run check-runners as part of the tests 2024-01-16 12:29:21 +01:00
Brian C. Lane
388aa22b94 Schutzfile: Remove unused repos as reported by ./tools/check-runners
This removes these unused repos:

fedora-37
rhel-8.6
rhel-8.7
rhel-8.9
rhel-9.0
rhel-9.1
rhel-9.3
2024-01-16 12:29:21 +01:00
Brian C. Lane
978c20d36f tools: Add helper script to check Schutzfile and gitlab-ci.yml runners
This script reads the RUNNER entries from .gitlab-ci.yml and the
snapshot repo names from Schutzfile. It then prints which runners don't
have entries in Schutzfile and which Schutzfile entries don't have
runners.

This is slightly complicated by the fact that centos-stream reports
itself as centos. So when checking for missing entries they are
considered to be the same thing. eg. If centos is a RUNNER and
centos-stream is in Schutzfile it doesn't report anything missing.
2024-01-16 12:29:21 +01:00
Brian C. Lane
e287138e2e cloudapi: Add testing for blueprint compose request
This tests to make sure the blueprint produced by the customizations
data and the blueprint data are identical.
2024-01-15 11:48:35 +01:00
Brian C. Lane
7438e29375 cloudapi: Create a compose using the blueprint data
If the request includes a blueprint (and not customizations) it uses
that blueprint for the compose.
2024-01-15 11:48:35 +01:00
Brian C. Lane
5b87877109 cloudapi: Add optional blueprint schema to compose request
This adds a 'blueprint' section to the compose request. It also
restricts it so that only 'blueprint' or 'customizations' can be
included, but not both. The goal is to move to using 'blueprint' for all
customizations so that there is a single consistent interface for the
clients.

Where the openapi schemas are the same between the two they have been
shared, but a few are different. They are created with 'Blueprint*' as
their name.

This also re-adds the SSHKey schema removed by commit
bfad6d50e1, it is used by the Blueprint
Customization.
2024-01-15 11:48:35 +01:00
Brian C. Lane
ea05f400e6 cloudapi: Listen on /run/cloudapi/api.socket
Add a local socket for communicating with the cloudapi. It is started by
osbuild-composer.socket and is located at /run/cloudapi/api.socket

cloudapi requests can be passed to it using curl like this:

curl -k --unix-socket /run/cloudapi/api.socket --header 'Content-Type: application/json' \
--data request.json http://localhost/api/image-builder-composer/v2/compose

A simple request.json looks like this:

{
  "distribution": "fedora-38",
  "image_request":
    {
      "architecture": "x86_64",
      "image_type": "guest-image",
      "upload_options": {},
      "repositories": [
          {
            "name": "fedora",
            "metalink": "https://mirrors.fedoraproject.org/metalink?repo=fedora-38&arch=x86_64",
            "check_gpg": false
          },
          {
            "name": "updates",
            "metalink": "https://mirrors.fedoraproject.org/metalink?repo=updates-released-f38&arch=x86_64",
            "check_gpg": false
          }
      ]
    }
}
2024-01-15 11:48:35 +01:00
Alexander Todorov
6735e74897 Invert wrong boolean condition in filesystem test
the conditionals have been introduced in
8960a51d2f
which states (emphasys mine):

> The new partition rules are **in** osbuild-composer v94 and higher.

but the condition was "not v94 or higher" and as a result it fails on
8.10 nightly with osbuild-composer v92 with:

failed to initialize osbuild manifest: The following custom mountpoints are not supported [\"/boot/firmware\" \"/foobar\"]

see https://gitlab.com/redhat/services/products/image-builder/ci/osbuild-composer/-/jobs/5905865402
2024-01-11 20:57:46 +04:00
Alexander Todorov
22febe2316 Skip filesystem.sh test job on Fedora-39
looks like there is a kernel bug and the patch is 6+ months old:
https://github.com/osbuild/osbuild-composer/pull/3820#issuecomment-1881726795
2024-01-10 15:47:42 +01:00
Michael Vogt
a93814c1a8 Schutzfile: bump version of osbuild to 5fc3b565 2024-01-10 15:47:42 +01:00
Michael Vogt
5f678b1d7b test: add debug output to filesystem.sh if mounting fails 2024-01-10 15:47:42 +01:00
Alexander Todorov
ba2f5a7ecb Fix a boolean condition in filesystem.sh 2024-01-10 15:47:42 +01:00
Alexander Todorov
bbb685e9a3 Add Fedora 39 repositories in Schutzfile
Note: Modular repositories have been retired:
c80be57977
2024-01-10 15:47:42 +01:00
Alexander Todorov
5033afa2e2 Update test runners to Fedora 39 2024-01-10 15:47:42 +01:00
Michael Vogt
973a8cf285 image-info: update for new "partition" option in mounts.Mount
In the osbuild PR#1501 [0] a new paramter "partition" for
mounts.Mount() was added.

This commit updates the code that it deals cleanly with the new
and the old API for `mount.Mount`.

[0] https://github.com/osbuild/osbuild/pull/1501
2024-01-10 09:38:31 +01:00
schutzbot
421273d1e0 Post release version bump
[skip ci]
2024-01-10 08:32:45 +00:00
Michael Vogt
96e3b727b3 schutzbot: add my "mvogt" SSH key 2024-01-08 15:55:38 +01:00
Tomáš Hozza
0169b7b905 Schutzfile: update osbuild SHA to v102
This should solve the CI failures on el8.10 due to fix the missing
`/etc/X11/xorg.conf.d/00-keyboard.conf` file when building images and
setting keyboard layout.

The osbuild v102 is currently blocked in RHEL CI, so workaround this
issue for now at least on `main` in the upstream.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-04 11:29:43 +01:00
schutzbot
f25bbda32d schutzfile: Update snapshots to 20240101 2024-01-04 11:29:43 +01:00
dependabot[bot]
623e761c45 build(deps): bump the go-deps group with 5 updates
Bumps the go-deps group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) | `1.49.7` | `1.49.13` |
| [github.com/labstack/echo/v4](https://github.com/labstack/echo) | `4.11.3` | `4.11.4` |
| [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go) | `0.1.388` | `0.1.390` |
| [github.com/osbuild/images](https://github.com/osbuild/images) | `0.26.0` | `0.28.0` |
| [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) | `1.17.0` | `1.18.0` |


Updates `github.com/aws/aws-sdk-go` from 1.49.7 to 1.49.13
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.49.7...v1.49.13)

Updates `github.com/labstack/echo/v4` from 4.11.3 to 4.11.4
- [Release notes](https://github.com/labstack/echo/releases)
- [Changelog](https://github.com/labstack/echo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/labstack/echo/compare/v4.11.3...v4.11.4)

Updates `github.com/openshift-online/ocm-sdk-go` from 0.1.388 to 0.1.390
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.388...v0.1.390)

Updates `github.com/osbuild/images` from 0.26.0 to 0.28.0
- [Release notes](https://github.com/osbuild/images/releases)
- [Commits](https://github.com/osbuild/images/compare/v0.26.0...v0.28.0)

Updates `github.com/prometheus/client_golang` from 1.17.0 to 1.18.0
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.17.0...v1.18.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/labstack/echo/v4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/osbuild/images
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-02 11:43:07 +01:00
schutzbot
df3d661178 Post release version bump
[skip ci]
2023-12-27 08:21:58 +00:00
Tomáš Hozza
cfc63297eb Test: delete "diff manifests" test case
This is not adding any value any more, since all image definitions are
defined in the osbuild/images repository and just vendored in this one.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-12-22 12:21:59 +01:00
Alexander Todorov
42a90914ab Surround composer-cli call with set +e/set -e as a workaround
because apparently calling nvrGreaterOrEqual inside a for loop doesn't
behave as expected and this fails on the second iteration.
2023-12-22 11:07:52 +01:00
Alexander Todorov
b47e741895 Add even more debug output
loop iterates overt centos-8 and then goes onto centos-9 and everything
fails silently
2023-12-22 11:07:52 +01:00
Alexander Todorov
8b91cf6060 Add debugging information for cross-distro.sh 2023-12-22 11:07:52 +01:00
Alexander Todorov
d6aff2f103 Adjust expected response based on osbuild-composer version 2023-12-22 11:07:52 +01:00
Ondřej Budai
7bcf8e5942 test-case-generators/repos: add required repos for azure-sap-rhui
This image requires a different RHUI repository and also the saphana one.

Similarly to the previous commit, we should probably remove the manifest diff
tests altogether, but let's just fix them for now.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-12-22 00:11:19 +00:00
Ondřej Budai
6a4b4b5ded dnfjson: fix repo hashes
20808e9848

changed the hash generation algorithm again, let's adjust the values.

In the future, we should probably mock this out, or change the test.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-12-22 00:11:19 +00:00
Ondřej Budai
807f249146 go.mod: bump images to v0.26.0
This is mainly needed in order to get the RHEL 9 SAP Azure images.
2023-12-22 00:11:19 +00:00
Ondrej Ezr
d5483ccfb0 test: Add nginx to the image request 2023-12-20 09:02:06 +01:00
Ondrej Ezr
4b3b942dde cloudapi: Add module_hotfixes flag
Allow passing module_hotfixes flag through the cloudapi.
This will enable depsolving on repositories that might be affected by modularity filtering.

Refs HMS-3202
2023-12-20 09:02:06 +01:00