Commit graph

6765 commits

Author SHA1 Message Date
Sanne Raymaekers
2bdeede4b8 Revert "schutzbot/terraform: new fedora-41 runners"
This reverts commit 8485481c90.
2024-09-24 14:46:58 +02:00
Sanne Raymaekers
4cf488376d Revert "Schutzfile: add fedora-41"
This reverts commit 8ef5bac4b9.
2024-09-24 14:46:58 +02:00
Sanne Raymaekers
3f636467ff Revert "test/data: add fedora-41"
This reverts commit 9782abe184.
2024-09-24 14:46:58 +02:00
Sanne Raymaekers
9782abe184 test/data: add fedora-41 2024-09-23 14:30:27 -07:00
Sanne Raymaekers
8ef5bac4b9 Schutzfile: add fedora-41 2024-09-23 14:30:27 -07:00
Sanne Raymaekers
8485481c90 schutzbot/terraform: new fedora-41 runners 2024-09-23 14:30:27 -07:00
Sanne Raymaekers
75cd8ee780 .gitlab-ci.yml: add fedora-41 2024-09-23 14:30:27 -07:00
Sanne Raymaekers
9c68a82d2e repositories: add fedora-41 2024-09-23 14:30:27 -07:00
Tomáš Hozza
3607783339 provision.sh: ensure no repositories in the Service scenario
In the Service scenario, we should be testing that osbuild-composer
works fine, without any repository configurations being present on the
filesystem.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-23 18:51:39 +02:00
Tomáš Hozza
7437770352 composer: don't create RepoRegistry using reporegistry.New()
The `reporegistry.New()` has been enhanced to return an error, in case
there were no repositories loaded. This was to fix the situation in many
unit tests, which were previously not loading any repositories and
silently not running any tests.

This however broke our SaaS deployment, where we actually do not
configure any repositories on the filesystem. As a result,
osbuild-composer started to fail on it.

Workaround this situation in osbuild-composer by reverting to the old
behavior by loading the repo configs separately and then using the
loaded repos (which could be empty map) to initialize the RepoRegistry.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-23 18:51:39 +02:00
schutzbot
f001c05157 Post release version bump
[skip ci]
2024-09-20 15:15:08 +00:00
Tomáš Hozza
992bf257ec Test/repositories: remove the AUX GPG key from RHEL-8 repos
This is needed for GPG key import to work on RHEL-9, because the key
uses SHA-1. This results in the following error when importing the key
during the build of the build container:

"Signature not supported. Hash algorithm SHA1 not available."

We do not need the AUX key in our RHEL-8 repo definitions used for
testing.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
efc251fa02 CloudAPI: test /sboms endpoint for Koji composes
Extend the API unit test for Koji composes, to verify that the newly
added /sboms endpoint works correctly.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
cf79bf677b CloudAPI: test /sboms endpoint for regular composes
Extend the unit test for regular (non-Koji) composes, to verify that
 the newly added /sboms endpoint works correctly.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
6e8f0418a6 CloudAPI: add new /composes/{id}/sboms endpoint
Add a new /sboms API endpoint, for getting SBOM documents for a given
compose ID. The endpoint returns an array of SBOM documents for each
image built as part of the compose. For each image, there is an SBOM
document for each osbuild pipeline, which installs RPM packages. This is
usually one 'buildroot' and one 'image' pipeline.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
102d06774c CloudAPI: extend manifestJobResultsFromJobDeps() to also return JobInfo
Extend the `manifestJobResultsFromJobDeps()` function to also return the
manifest `JobInfo`. This will be useful to inspect the job dependencies
and eliminate the need to add a specialized function for getting only
the `JobInfo`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
0053a6187b CI/Koji: run Koji tests only on the latest RHEL-9 GA
None of our worker is running RHEL-8 any more. There's no value in
testing the Koji scenario on RHEL-8, RHEL-9 is fully sufficient.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
b50816a706 CI/Koji: test building of RHEL-10 on RHEL-9
This is what is currently happening on our Brew workers.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
dd7afa416c CI/Koji: test RHEL-8 builds on RHEL-9
We have been testing builds of RHEL-9 on RHEL-8 for the Koji use case.
However, all of our workers are now running the latest GA RHEL-9
version. Therefore we should flip the test and test building of RHEL-8
on RHEL-9.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
fd33a37366 Test/koji.sh: adjust for SBOM documents
Adjust the test case to cope with the SPDX SBOM documents uploaded to
the Koji. Also explicitly check that there is the expected number of
SBOM documents uploaded as the image build output.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
71a12742d4 Worker/osbuild/koji: upload SBOM documents
Extend the Koji target handling in the osbuild job, to also upload SBOM
documents attached to the related depsolve job result.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
1c7462b275 Worker/koji-finalize: import uploaded SBOM documents
If the Koji target result contains information about any uploaded SBOM
documents, import them to Koji as part of the finalize task.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
c109265abb Target/koji: extend the result struct with SBOM docs
Extend the Koji target result struct with an optional slice for uploaded
SBOM documents.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
4ae35a0ed9 Worker/osbuild: depend on depsolve job for Koji composes
For Koji composes, all files are uploaded to Koji as part of the osbuild
job (specifically as part of handling the Koji target). So in order to
be able to upload SBOM documents to Koji as part of Koji compose, the
osbuild job needs to to be able to access the depsolve job result, which
contains the SBOM documents. For this, the osbuild job must depend on
the depsolve job.

For Koji composes, make sure that osbuild job depends on the depsolve
job and set the DepsolveDynArgsIdx.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
f8d231d024 CloudAPI: request SBOM documents in depsolve jobs
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
4779e90e17 Worker/depsolve: add support for SBOM
Add support to the `DepsolveJob` for requesting SBOM documents and
returning the results from the job.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
0628ac9131 Worker/json: remove redundant comment
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
b931a9789b SPEC: bump the minimum osbuild version
This is needed for the SBOM support

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
7bdd036395 Update osbuild/images to v0.88.0
Adjust all paces that call `Solver.Depsolve()`, to cope with the changes
that enabled SBOM support.

Fix loading of testing repositories in the CloudAPI unit tests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
b2096c2963 Test/repositories: update el10 / c10s repmrepo snapshots
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
c84e9cb1c8 Schutzfile: update el10 / c10s rpmrepo snapshots
This is needed to be able to install osbuild RPM builds, because that
repo uses newer snapshots.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
e463ab7bf5 Schutzfile: update osbuild ref
Update the ref to the v129, which supports SBOM documents.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
schutzbot
99e77147af Post release version bump
[skip ci]
2024-09-18 08:15:27 +00:00
Achilleas Koutsou
b0c0e84142 test: replace openscap with compliance in rhsm facts
See https://github.com/osbuild/images/pull/914
2024-09-17 23:33:44 +02:00
Achilleas Koutsou
2a6fe9122f github: disable caching for golangci-lint-action
There seems to be a caching issue with the linter.  It's reporting go
dependency replacements when there are none.
2024-09-17 23:33:44 +02:00
Achilleas Koutsou
4248564a55 cloudapi: update expected image type test for gcp-rhui
gce-rhui is now gone from RHEL 9 [1] and the old name simply aliases to
gce.  gcp-rhui in the cloudapi now resolves to 'gce' in RHEL 9 and
'gce-rhui' in RHEL 8.

[1] https://github.com/osbuild/images/pull/857
2024-09-17 23:33:44 +02:00
Achilleas Koutsou
ec01c6908b blueprint: sshkey to users in images blueprint conversion
The sshkey customization in osbuild/images has been dropped.  In
osbuild-composer we maintain it for backwards compatibility, converting
each to a user customization, which is a superset of the sshkey.
2024-09-17 23:33:44 +02:00
Achilleas Koutsou
1f21f8e217 go.mod: update osbuild/images to v0.87.0
Update osbuild/images to include:
- blueprint: remove the sshkey customization
  (https://github.com/osbuild/images/pull/928).
- [RHEL-9] Drop RHSM and RHUI-specific config from Azure and EC2 images
  (COMPOSER-2308) (https://github.com/osbuild/images/pull/857).
2024-09-17 23:33:44 +02:00
Jakub Rusz
f58193dbeb test/cases: implement retry for ssh connection in api test
Fixing this by implementing a retry for the whole ssh connection
attempt.
2024-09-16 12:41:05 +02:00
Sanne Raymaekers
0273908fc7 tools/build-rpms: fix ec2 client initialisation
Fixes:
```
Error: Create EC2 instances  failed: 'ec2.ServiceResource' object has no attribute 'describe_images'
Traceback (most recent call last):
  File "/osbuild-composer/tools/build-rpms.py", line 218, in <module>
    stage_generate_rpms(cleanup_actions, args)
  File "/osbuild-composer/tools/build-rpms.py", line 175, in stage_generate_rpms
    create_ec2_instances, cleanup_actions, args, keyname)
  File "/osbuild-composer/tools/build-rpms.py", line 66, in stage
    ret = fun(*args)
  File "/osbuild-composer/tools/build-rpms.py", line 109, in create_ec2_instances
    img = ec2.describe_images(ImageIds=[arch_info[a]["ImageId"]])
AttributeError: 'ec2.ServiceResource' object has no attribute 'describe_images'
```
2024-09-11 17:31:13 +02:00
Sanne Raymaekers
72ed7db62d tools/build-rpms: increase rpm builder instance disk size
The instance created for rpm builds is running out of space on the
default 10G disk, let's double it.
2024-09-11 15:51:04 +02:00
Michael Vogt
3df26ed79c osbuild-worker: fix "crashing" on worker registration issues
When the osbuild worker cannot register itself with the server
on startup the worker will "crash". This is inconsistent with the
existing behavior in `workerHeartbeat()` which deals with connectivity
or other server issue gracefully and retries periodically.

To unify the behavior this commit changes the behavior and only
issues a `logrus.Warnf` instead of the previous `Falalf` when
the registration fails.

Co-authored-by: Florian Schüller <florian.schueller@redhat.com>
2024-09-10 16:19:47 +02:00
Jakub Rusz
751ad6af69 Update Fedora-40 repositories to branched ones 2024-09-10 12:35:19 +02:00
Sanne Raymaekers
d6031ae87a upload/azure: turn off public access on storage accounts
Users might have compliance policies on their azure accounts which
forbid public access on storage accounts.
2024-09-09 12:52:14 +02:00
Jakub Rusz
758982907a tests/cases: Rename upgrade test
And schedule on rhel-10
2024-09-06 14:38:57 +02:00
Jakub Rusz
15c8e83b94 test/cases: adapt upgrade to support rhel9to10 2024-09-06 14:38:57 +02:00
Florian Schüller
bb53f4833f internal/worker/client.go: refactor reading worker ID
Adds a helper function to the worker client instead of
redeclaring the same inline function.
2024-09-06 12:43:05 +02:00
Sanne Raymaekers
a6b87b58f2 test/api: check openscap facts when appropriate 2024-09-06 11:39:35 +02:00
Sanne Raymaekers
b21f24e751 cloudapi/v2: populate openscap related rhsm facts
Also adds the policy id to the blueprint, this doesn't have any effect
on the openscap step, it just puts in place the rhsm fact so instances
registered to insights will appear under that policy.
2024-09-06 11:39:35 +02:00
Sanne Raymaekers
387f971bf0 go.mod: update to osbuild/images@v0.83.0 2024-09-06 11:39:35 +02:00