Commit graph

6213 commits

Author SHA1 Message Date
dependabot[bot]
232bce95b5 build(deps): bump actions/upload-artifact from 3 to 4
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 3 to 4.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-06 12:28:59 +01:00
dependabot[bot]
7b5c4eb526 build(deps): bump actions/setup-go from 4 to 5
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 4 to 5.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-06 12:26:58 +01:00
Tomáš Hozza
f6d35ac5ad Update osbuild/images to v0.35.0
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-02-06 12:20:08 +01:00
Sanne Raymaekers
44426bb48f templates/dashboards: add community stage service to orgs 2024-02-05 11:38:53 +01:00
Simon Steinbeiss
add35b86ce DEPLOYING/HACKING.md: Consistently use inline refs
The footnote or endnote style links sometimes don't work well with
DocuSaurus, i.e. they're not resolved correctly and lead to build
errors.
https://github.com/osbuild/osbuild.github.io/actions/runs/7776075785/job/21202788034?pr=36
2024-02-05 10:56:45 +01:00
Simon Steinbeiss
1d1b410654 HACKING/DEPLOYING.md: Markdown syntax fixes 2024-02-04 17:08:57 +01:00
Sanne Raymaekers
de548c36f3 pkg/jobqueue: fix worker status update query
The workers table should be updated, not the heartbeats. Currently every
worker is reregisterig every minute.
2024-02-02 15:24:57 +01:00
Achilleas Koutsou
3e31ebfbb1 osbuild-composer: add iot-bootable-container to weldr deny list
We only want to build this image type in Fedora Koji.
2024-02-02 10:34:41 +01:00
Achilleas Koutsou
bbbb3278f8 test: update fedora-39 test repositories
Required for bootc for iot-bootable-container.
2024-02-02 10:34:41 +01:00
Achilleas Koutsou
76a699653a test: add iot-bootable-container to api tests
The image type supports uploading to both s3 and container registries.
Right now we only care about getting it in Koji, so let's just add it to
the s3 upload section.

The container registry test also embeds containers in the image, which
the iot-bootable-container doesn't support (yet).
2024-02-02 10:34:41 +01:00
Achilleas Koutsou
52d4b196a3 cloudapi: enable iot-bootable-container image type 2024-02-02 10:34:41 +01:00
Tom Koscielniak
d97f54cc5c Add RHEL 9.3 and 8.9 GA to pipeline 2024-02-01 16:07:27 +01:00
dependabot[bot]
b461b1faf3 build(deps): bump the go-deps group with 10 updates
Bumps the go-deps group with 10 updates:

| Package | From | To |
| --- | --- | --- |
| [cloud.google.com/go/compute](https://github.com/googleapis/google-cloud-go) | `1.23.3` | `1.23.4` |
| [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) | `1.36.0` | `1.37.0` |
| [github.com/Azure/azure-sdk-for-go/sdk/azidentity](https://github.com/Azure/azure-sdk-for-go) | `1.4.0` | `1.5.1` |
| [github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5](https://github.com/Azure/azure-sdk-for-go) | `5.3.0` | `5.5.0` |
| [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) | `1.50.0` | `1.50.8` |
| [github.com/google/uuid](https://github.com/google/uuid) | `1.5.0` | `1.6.0` |
| [github.com/jackc/pgtype](https://github.com/jackc/pgtype) | `1.14.0` | `1.14.1` |
| [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go) | `0.1.390` | `0.1.398` |
| [github.com/osbuild/images](https://github.com/osbuild/images) | `0.33.0` | `0.34.0` |
| [google.golang.org/api](https://github.com/googleapis/google-api-go-client) | `0.160.0` | `0.161.0` |


Updates `cloud.google.com/go/compute` from 1.23.3 to 1.23.4
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/documentai/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/compute/v1.23.3...compute/v1.23.4)

Updates `cloud.google.com/go/storage` from 1.36.0 to 1.37.0
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.36.0...spanner/v1.37.0)

Updates `github.com/Azure/azure-sdk-for-go/sdk/azidentity` from 1.4.0 to 1.5.1
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.4.0...sdk/internal/v1.5.1)

Updates `github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5` from 5.3.0 to 5.5.0
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/resourcemanager/compute/armcompute/v5.3.0...sdk/resourcemanager/compute/armcompute/v5.5.0)

Updates `github.com/aws/aws-sdk-go` from 1.50.0 to 1.50.8
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.50.0...v1.50.8)

Updates `github.com/google/uuid` from 1.5.0 to 1.6.0
- [Release notes](https://github.com/google/uuid/releases)
- [Changelog](https://github.com/google/uuid/blob/master/CHANGELOG.md)
- [Commits](https://github.com/google/uuid/compare/v1.5.0...v1.6.0)

Updates `github.com/jackc/pgtype` from 1.14.0 to 1.14.1
- [Changelog](https://github.com/jackc/pgtype/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jackc/pgtype/compare/v1.14.0...v1.14.1)

Updates `github.com/openshift-online/ocm-sdk-go` from 0.1.390 to 0.1.398
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.390...v0.1.398)

Updates `github.com/osbuild/images` from 0.33.0 to 0.34.0
- [Release notes](https://github.com/osbuild/images/releases)
- [Commits](https://github.com/osbuild/images/compare/v0.33.0...v0.34.0)

Updates `google.golang.org/api` from 0.160.0 to 0.161.0
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.160.0...v0.161.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/compute
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azidentity
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/google/uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/jackc/pgtype
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/osbuild/images
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-01 12:23:01 +01:00
Jakub Rusz
db0e6c9643 Packer: change fedora-38 aarch64 ami
This ami is currently broken, switch to a slightly older one.
2024-01-31 10:11:50 +01:00
Jakub Rusz
9e0edb2f00 CI: update terraform SHA
We had to switch fedora-38-aarch64 image to an older one because the
current one was broken in AWS.
2024-01-31 10:11:50 +01:00
Sanne Raymaekers
e289b763e7 templates/packer: deal with unbound variables
Don't allow unbound variables, but for the variables that are used to
determine whether or not that part of the setup should continue, default
to empty/undefined.
2024-01-30 21:41:31 +01:00
Tomáš Hozza
0c0a758600 Packer: default to RHEL-9.3 Schutzfile config for rhel-9
This is necessary to get the proper osbuild SHA to install the latest
required osbuild version into the image.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
1fa72f262e test/api.sh/vsphere: specify the resource pool for image import
Specify the resource pool when importing the VMDK to the VCenter.
This should prevent the following error:

govc: default resource pool resolves to multiple instances, please specify

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
ae68358a0d worker/osbuild: provide more details and logs when osbuild build fails
Add any errors to job error details when an osbuild build fails.
Otherwise these won't show up in the worker log, which makes
debugging issues harder.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
0690b6f1ba cloudapi: pass depsolve job error details to the manifest job error
If a depsolve job fails, the error details were not passed as details to
the manifest job error details. This may help with debugging failures.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
e664d49541 Test/old-worker-new-composer: don't wait for composer indefinitely
The test would loop for the job timeout limit waiting for composer
container, in case the composer API does not come up for whatever
reason. Modify the test case to wait only for 12x 10s before failing.
In case the composer API does not come up, print the logs from the
composer container and exit with non-zero code.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
a33da152da Test/weldr: add test for dot-notation and distro aliases
Add test case for verifying that distros and repo configurations work
with and without the dot-notation in the on-prem scenario. Run the test
case on the latest RHEL 8 and 9 versions in development and verify the
default distro aliases behavior on them.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
2fe5e01d4d Test/api.sh: test distro aliases and names without dot-notation
Extend the `api.sh` to allow testing compose requests with distro name
which does not use dot-notation (specifically when the dot is removed
from the distro name as it used to be in the past). In addition to that,
allow also testing the distro alias using distro name without the minor
version in compose requests.

Enable these two new test variants in the CI.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
fb6acbbe83 test/data/repositories: update GA releases and add SAP repos
Update RHEL 8.9 and 9.3 repo definitions to use the CDN repos, since
these are already GA.

Add SAP repositories to all RHEL repo configs, to be able to build the
SAP image for testing purposes.

Fix minor issues found in repos (e.g. 8.8 RT repo pointing to 8.7,
etc.).

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
9252e2525c Test/azure.sh: don't use dot-notation for image name
Using the DISTRO_CODE with dot to separate major and minor version is
breaking cloud-image-val testing. Specifically, the tool is using
regular expression to search for an uploaded image, but the expression
is not expecting any dot in the version.

This will be fixed in [1], but merging it will take some time due to CIV
CI being currently broken. Due to this, workaround the problem for now
by making sure that the TEST_ID, which is used to construct the
IMAGE_NAME, does not contain any dot in the DISTRO_CODE.

[1] https://github.com/osbuild/cloud-image-val/pull/290

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
499cda4a57 Test/manifests: rename files to use dot-notation
Otherwise, the image tests will fail on globing for testing manifests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
66571784d1 Use dot-notation for repo configs and in test cases.
Where applicable, modify all repo config filenames to use a dot
to separate the release major and minor version. Modify test cases
to not remove dot from the distro version any more.

Existing tests will be extended (or new tests added) to explicitly test
backward compatibility and ensure that using old distro names without a
dot still works.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
c82c0ae6b5 Test/api.sh: pass TEST_MODULE_HOTFIXES as an env variable
While I was the one to push for TEST_MODULE_HOTFIXES to be passed as an
argument to `api.sh`, when it was implemented, this is turning our to be
impractical. I would like to extend `api.sh` to test distro names with
and without the dot to separate major and minor version and also to test
the ability to specify distro name explicitly (e.g. a distro alias).
Passing all of these new options and their combinations would make the
CLI go wild.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
17ad9c1165 Makefile/rpm: always copy over SPEC file
Any changes done to the SPEC file were not reflected in the
`rpmbuild/SPECS` directory across runs of `make rpm`. One had to delete
the SPEC file manually to be updated and used for the RPM build.

Mark the target as PHONY to ensure that the SPEC is always updated.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
8ba3426f12 composer: use distro name aliases from configuration
Register the distro name aliases from the configuration on the distro
factory.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
b79eaeb78e weldr: don't determine distro name from image type when depsolving
Weldr API used to determine the distro name from the image type, when it
was getting the repositories to use for depsolving and for the actual
depsolving (solver uses the distro name to namespace cache).

This used to be OK, but with the introduction of distro name aliases,
the distro name used to get the distro object may not be the same as the
name returned by the actual distro object. To preserve the current
behavior, the same name used to get the distro object should be used to
also get the repositories for depsolving and to namespace depsolving
cache.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
cdf57e5bc1 osbuild-composer/config: support specifying distro aliases
Add new configuration option `distro_aliases`, which is a map of
strings, allowing to specify distro name alias for supported
distributions.

Define aliases for RHEL major versions without the minor version
specified.

For now, the distro aliases map is not used by any API
implementation and it is ignored.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
43e87632fb Drop common.CurrentArch() in favor of osbuild/images
Drop `common.CurrentArch()` implementation and use
`arch.Current().String()` from the osbuild/images instead.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
c4e5ab5aca Drop internal/common/distro.go in favor of osbuild/images
Drop `common.GetHostDistroName()` implementation and use
`distro.GetHostDistroName()` from the osbuild/images instead.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
fb1b27c0ef Move cleaning up old dnfjson cache dirs from composer to Weldr API
Previously, the old dnfjson cache dirs for unsupported distributions
were deleted in the osbuild-composer binary on startup. This is no
longer possible, since the supported distros are determined by loading
available repositories. Loading repositories happens in the Weldr API
constructor. Move the cleanup code there.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
625b1578fa Port osbuild/images v0.33.0 with dot-notation to composer
Update the osbuild/images to the version which introduces "dot notation"
for distro release versions.

 - Replace all uses of distroregistry by distrofactory.
 - Delete local version of reporegistry and use the one from the
   osbuild/images.
 - Weldr: unify `createWeldrAPI()` and `createWeldrAPI2()` into a single
   `createTestWeldrAPI()` function`.
 - store/fixture: rework fixtures to allow overriding the host distro
   name and host architecture name. A cleanup function to restore the
   host distro and arch names is always part of the fixture struct.
 - Delete `distro_mock` package, since it is no longer used.
 - Bump the required version of osbuild to 98, because the OSCAP
   customization is using the 'compress_results' stage option, which is
   not available in older versions of osbuild.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
f6ff8c40dd Delete cmd/osbuild-composer-image-definitions
This command is part of the osbuild/images repo, where all image
definitions live. Having it in the osbuild-composer repository does not
add any value.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
1943732f73 Delete cmd/osbuild-pipeline
This command was used by the original Python scripts used to generate
test manifests. These scripts were deleted, thus there is no need to
keep this command around any more.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
79d5ab483b tools: delete old Python manifest generator scripts
These are no longer used, let's delete them.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
e235805171 Delete cmd/osbuild-package-sets
This command is part of the osbuild/images repo, where all image
definitions live. Having it in the osbuild-composer repository does not
add any value.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
Tomáš Hozza
c1affa6188 Delete cmd/osbuild-playground
This command is part of the osbuild/images repo, where all image
definitions live. Having it in the osbuild-composer repository does not
add any value.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-01-26 11:32:34 +01:00
schutzbot
34f4196677 Post release version bump
[skip ci]
2024-01-24 08:23:36 +00:00
Sanne Raymaekers
589bd5f1a1 tools/fedora-worker-packer: fix packer only/except
It should only build fedora, not everything except fedora.
2024-01-23 13:06:38 +01:00
Sanne Raymaekers
6a0c893a4d tools/appsre-packer: add script to build fedora worker images 2024-01-23 00:18:51 +01:00
Sanne Raymaekers
c7aad1b17d tools/app-sre-build-worker-packer: split out the CI script
To add the fedora workers built from AppSRE's infrastructure, we need a
combination that wasn't possible with the current matrix based on the CI
environment (jenkins vs gitlab).
2024-01-23 00:18:51 +01:00
Alexander Todorov
35c4d3efba Revert "test/cases: Add workaround for oscap firewalld rules"
b/c https://github.com/ComplianceAsCode/content/issues/11275 has been
fixed. This reverts commit c77b73a888.
2024-01-22 15:22:34 +04:00
djach7
4d241b684b edge: add iot-simplified-installer image type
Adds iot-simplified-installer image type to enable koji composes.

Signed-off-by: djach7 <djachimo@redhat.com>
2024-01-18 21:45:23 +01:00
Sanne Raymaekers
3631316e36 tools/build-rpms: fix getting the osbuild commit from Schutzfile
osbuild/osbuild-composer#3614 removed the rhel-8.6, the centos-stream-9
entry should be there for a while at least.
2024-01-16 16:26:01 +01:00
Sanne Raymaekers
edadbb3470 upload/azure: set hyperv generation
This is a required parameter in the new armcompute sdk.
2024-01-16 14:25:58 +01:00
Sanne Raymaekers
adc07299a8 upload/azure: set storage account minimum TLS version to 1.2 2024-01-16 14:25:58 +01:00