Commit graph

6337 commits

Author SHA1 Message Date
Sanne Raymaekers
b4e361e093 osbuild-jobsite-builder: unpack source tarball in populate 2024-03-19 17:07:30 +01:00
Sanne Raymaekers
07598ea65c osbuild-jobsite-manager: pass store to builder 2024-03-19 17:07:30 +01:00
Jerry Uanino
551db639da Fixed link to developer guide 2024-03-16 10:41:18 +01:00
Ondřej Budai
e5853c9aa5 Remove rhel-10.0 alias from the openshift template
We now have a proper rhel-10.0 distribution, and this alias is clashing
with it, so we are seeing the following message in production:

failed to configure distro aliases: invalid aliases: ["alias 'rhel-10.0' masks an existing distro"]

Let's fix it by removing the alias, it's obviously not needed anymore.
2024-03-15 15:29:45 +01:00
Simon de Vlieger
babf80f060 deps: bump 2024-03-15 12:12:59 +01:00
Brian C. Lane
57ebfb4011 cloudapi: Use distro repos if none included in imageRequest
In order to support cloudapi blueprint requests from the cmdline using
composer-cli it needs to select the repositories based on the selected
distribution instead of requiring the user to include them with the
request.

If the image request includes repositories they are used, which matches
the current behavior. If the repository list is empty it will use the
distribution name to select from the repositories shipped with
osbuild-composer.
2024-03-11 03:04:54 -07:00
Brian C. Lane
01ba674cac cloudapi: Pass the RepoRegistry to the cloudapi Server 2024-03-11 03:04:54 -07:00
Brian C. Lane
b8967d53bc Move RepoRegistry setup into Composer
This is so that both the weldr and cloud api's can use it as the source
of their repositories.
2024-03-11 03:04:54 -07:00
Tomáš Hozza
f660ed65b6 Packer: remove escaped newline from worker config
If the /tmp/cloud_init_vars contained OSBUILD_EXECUTOR_CLOUDWATCH_GROUP
variable set, the worker configuration file would contain a line with
escaped newline character at the end of the value configuring
`cloudwatch_group` for the `osbuild_executor`. This makes the worker
fail to start when loading the configuration.

Remove the newline from the value appended to the worker config by the
initialization script.

Fix #4001

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-11 10:46:57 +01:00
Tomáš Hozza
1659d190b0 Packer: set 'cloudwatch_group' in worker config if provided
Set the 'cloudwatch_group' value in the worker configuration if provided
in /tmp/cloud_init_vars, so that it is used by the worker when spinning
up an osbuild-executor instance.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-08 13:13:44 +01:00
Tomáš Hozza
e7743f17ec Worker: allow configuring executor CloudWatch group
We need the ability to use different CloudWatch group for the
osbuild-executor on Fedora workers in staging and production
environment.

Extend the worker confguration to allow configuring the CloudWatch group
name used by the osbuild-executor. Extend the secure instance code to
instruct cloud-init via user data to create /tmp/cloud_init_vars file
with the CloudWatch group name in the osbuild-executor instance, to make
it possible for the executor to configure its logging differently based
on the value.

Cover new changes by unit tests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-08 13:13:44 +01:00
Brian C. Lane
ceddabc395 osbuild-worker: Convert osbuild raw json error to a string for logging
Without this you get an array of bytes in the logs. Convert the raw json
output from osbuild to a string so there is some hope that it is
readable.
2024-03-08 01:17:00 +01:00
Simon de Vlieger
b89293ea94 ci: add a PR best practices check 2024-03-07 19:46:11 +01:00
Tomáš Hozza
85753c3617 Test/Koji: test only combinations that we run in the service
Test `koji.sh` with cloud upload on the latest RHEL-8.9 and RHEL9.3 GA
releases, which we use on our Brew workers.

Test the `koji.sh` only on the latest RHEL-8.9, RHEL-9.3 and Fedora
releases, which we use on our Brew and Koji workers.

There's no value in testing this case on CentOS Stream, RHEL EUS
releases or nightly composes.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-07 17:44:54 +01:00
Tomáš Hozza
7c42924ffa Packer: don't source non-existing file in worker_executor.sh
The /tmp/cloud_init_vars is not created on the worker executor, so
sourcing it will make the script fail. Comment the line out, until we
change the worker implementation to inject this file into the worker
executor using cloud-init.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-07 13:31:55 +01:00
Sanne Raymaekers
86e19bf198 templates/packer: set up vector correctly in the executor 2024-03-07 13:15:36 +01:00
Sanne Raymaekers
ef44f22d89 templates/packer: vector expects a yaml configuration now
When switching to the new vector repos, the version was also bumped. The
newer versions expect a yaml config.
2024-03-07 07:32:47 +01:00
Sanne Raymaekers
aa156028b5 templates/packer: make worker_config script executable 2024-03-06 20:38:36 +01:00
Simon Steinbeiss
bdc973d085 test/api.sh: Drop distro aliases and dot-notation
This reverts commit 2fe5e01d4d with the
exception of the changes in tools/provision.sh.
This functionality is covered by unit tests in osbuild/images:
aaa53adc3c/pkg/distrofactory/distrofactory_test.go (L9)
2024-03-06 13:28:30 +01:00
Tomáš Hozza
f02fe86366 Packer: make osbuild-executor worker config more configurable
We need to use custom IAM policy name used by the worker for
osbuild-executor on Fedora workers (in prod vs. stage). And we have the
same requirement for the CloudWatch log group used by the
osbuild-executor.

Modify the Ansible playbook used by Packer to use the values from
/tmp/cloud_init_vars if set and defaulting to the current values if not
set.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-06 12:02:56 +01:00
Achilleas Koutsou
e8a3aac989 test: update minimal-raw test
Update minimal-raw test with new artifact filename.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2024-03-06 10:17:28 +01:00
Achilleas Koutsou
525027726e deps: update images to v0.43.0 2024-03-06 10:17:28 +01:00
schutzbot
75d23d7c98 Post release version bump
[skip ci]
2024-03-06 08:14:12 +00:00
Alexander Todorov
4db594dac7 Skip image_installer tests b/c already tested in installers.sh 2024-03-06 09:09:39 +01:00
Alexander Todorov
d0237daf75 Create a custom group via blueprint in installers.sh test script 2024-03-06 09:09:39 +01:00
Alexander Todorov
d7388fdc66 Remove already tested manifests from test-case-generators
and remove redundant skips from image_tests.sh
2024-03-06 09:09:39 +01:00
Alexander Todorov
eb5db0bd2c Skip all edge image types, already covered in Edge CI 2024-03-06 09:09:39 +01:00
Alexander Todorov
a74756e4fb Skip gce image type, already covered in gcp.sh 2024-03-06 09:09:39 +01:00
Alexander Todorov
c25db57d7d Skip image types covered in azure.sh
namely vhd-boot
2024-03-06 09:09:39 +01:00
Alexander Todorov
c67add4a6e Skip image types already covered in vmware.sh
namely ova-boot|vmdk-boot
2024-03-06 09:09:39 +01:00
Alexander Todorov
20a36964a9 Skip Image Tests for image types already covered in osbuild/images
namely ami-boot|edge_ami|ec2-boot|ec2_ha|ec2_sap

Related: COMPOSER-2127
2024-03-06 09:09:39 +01:00
Simon Steinbeiss
362caacd1f tests: Drop bigiso test (HMS-3710)
This test is being migrated to osbuild/images.
2024-03-06 01:14:57 +01:00
Jakub Rusz
eec2cf62e1 CI: Drop SonarQube in favor of Snyk 2024-03-06 00:11:46 +01:00
Sanne Raymaekers
9004ca6310 Revert "packer: pin the vector version"
This reverts commit 5315264f2e.

This got fixed in 0.22.1.
2024-03-05 19:10:28 +01:00
Sanne Raymaekers
cad29f93a7 templates/packer: migrate vector to new repos
See https://vector.dev/highlights/2023-11-07-new-linux-repos/ for more
information.
2024-03-05 19:10:28 +01:00
Simon de Vlieger
073e304978 images: update dependency
Update the images dependency to get the new version gates for Fedora.
2024-03-04 16:20:26 +01:00
Tom Koscielniak
b4dcfc4480 Remove libvirt workaround 2024-03-04 15:39:08 +01:00
dependabot[bot]
ef07f2b81f build(deps): bump golangci/golangci-lint-action from 3 to 4
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 3 to 4.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-04 15:01:41 +01:00
dependabot[bot]
1bd19ab75b build(deps): bump stackrox/kube-linter-action from 1.0.4 to 1.0.5
Bumps [stackrox/kube-linter-action](https://github.com/stackrox/kube-linter-action) from 1.0.4 to 1.0.5.
- [Release notes](https://github.com/stackrox/kube-linter-action/releases)
- [Commits](https://github.com/stackrox/kube-linter-action/compare/v1.0.4...v1.0.5)

---
updated-dependencies:
- dependency-name: stackrox/kube-linter-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-04 15:00:30 +01:00
Alexander Todorov
8f28da5214 Check for unbound variables in cleanup() 2024-03-04 12:49:56 +01:00
Alexander Todorov
e4b514ca3a Don't trap the EXIT signal twice
killing the worker journal via EXIT signal prevents the cleanup()
function from executing!

NOTE: this is a problem in other scripts as well and needs to be
refactored there too!
2024-03-04 12:49:56 +01:00
Alexander Todorov
97c0c21513 Test images in OpenShift Virtualization 2024-03-04 12:49:56 +01:00
Sanne Raymaekers
09350d0c59 .gitlab-ci.yml: add executor test 2024-03-01 19:20:51 +01:00
Sanne Raymaekers
6602acda6b test/cases/worker-executor: test the aws.ec2 osbuild-executor
Sets up the executor over ssh and waits until completed.
2024-03-01 19:20:51 +01:00
Sanne Raymaekers
55c737e4f9 schutzbot/terraform: support instance profiles on aws runners 2024-03-01 19:20:51 +01:00
Sanne Raymaekers
040eec4089 osbuild-worker: allow adding key to aws.ec2 executor
This is useful during testing to set up the executor machine.
2024-03-01 19:20:51 +01:00
Gianluca Zuccarelli
c480d79e95 cloudapi: enable masked services
Extend the cloudapi to accept masked systemd services as a
customization.
2024-02-29 20:57:39 +01:00
Gianluca Zuccarelli
f6b76cce31 Update osbuild/images to v0.41.0
Multiple blueprint fixes:

- Extend the blueprint service customizations to accept services to be
  masked.

- The `storage-path` and `container-transport` fields were removed in
  imagees 41.0 in order to simplify the way local storage containers are
  handled.
2024-02-29 20:57:39 +01:00
Brian C. Lane
4e504f7905 README: Fix reference to developer guide 2024-02-29 10:56:03 +01:00
Mario Cattamo
e6a7cb2544 ci: skip ostree-remount check in CS9 due to bug RHEL-25249 2024-02-27 17:11:03 +01:00