Commit graph

6351 commits

Author SHA1 Message Date
Sanne Raymaekers
dabe63cb78 osbuild-jobsite: increase populate timeout
30 seconds can be a bit too little for the entire store.
2024-03-20 21:58:17 +01:00
Sanne Raymaekers
314ed4b527 cloud/awscloud: allow internet access on secure instance again
The executor is timing out and there are no logs. This will require some
further work. Remove the restriction for now.
2024-03-20 14:58:25 +01:00
Simon de Vlieger
8653f614f0 repositories: remove rhel-8-beta
This is an invalid repository name.

Signed-off-by: Simon de Vlieger <cmdr@supakeen.com>
2024-03-20 12:30:20 +01:00
Achilleas Koutsou
3633528613 worker: drop compatibility test for DepsolveJob serialisation
This reverts commit c8ce3e4428
2024-03-20 12:12:12 +01:00
Achilleas Koutsou
80f8fa051c worker: drop backwards compatibility for DepsolveJob serialisation
We've kept this around a lot longer than we should have.

This reverts commit 94c7fda779.
2024-03-20 12:12:12 +01:00
Achilleas Koutsou
3e0f8a7ac3 test: modify the kickstart file in nightly pipelines
Nightly pipelines test against older osbuild-composer versions so we
need to still modify the kickstart file until the feature reaches the
nightly composes.
2024-03-20 11:45:08 +01:00
Achilleas Koutsou
8f78171f83 cloudapi: add installer customizations to the API
Adding both as a traditional compose request customization and in the
new blueprint customizations.
2024-03-20 11:45:08 +01:00
Achilleas Koutsou
eccdcfcab0 test: use new installer customizations
Use the new installer customizations to build and boot an unattended
installation and add the users to the sudoers with NOPASSWD enabled.

This is added both to he base installer test (installers.sh) and the
ostree test (ostree-ng.sh).  For the ostree test, we keep the mkksiso
step to retain the remote configuration change.  The modksiso() function
is changed to explicitly read the osbuild.ks file since we don't want to
modify the osbuild-base.ks file and we want to ensure that the kernel
command line argument points to the osbuild.ks file.
2024-03-20 11:45:08 +01:00
Achilleas Koutsou
bd57d95e49 blueprint: add installer customizations 2024-03-20 11:45:08 +01:00
schutzbot
535ff2ef2d Post release version bump
[skip ci]
2024-03-20 08:12:49 +00:00
Sanne Raymaekers
cda94f4f62 templates/packer: don't subscribe executor
All the required sources will be proxied.
2024-03-19 17:07:30 +01:00
Sanne Raymaekers
4e2d990af8 test/worker-executor: internet access during executor setup 2024-03-19 17:07:30 +01:00
Sanne Raymaekers
79b5b736e9 cloud/awscloud: restrict network egress for secure instance
The security instance should no longer have any internet access.
2024-03-19 17:07:30 +01:00
Sanne Raymaekers
56bdcf1cdc aws.ec2: prep sources before starting build 2024-03-19 17:07:30 +01:00
Sanne Raymaekers
b4e361e093 osbuild-jobsite-builder: unpack source tarball in populate 2024-03-19 17:07:30 +01:00
Sanne Raymaekers
07598ea65c osbuild-jobsite-manager: pass store to builder 2024-03-19 17:07:30 +01:00
Jerry Uanino
551db639da Fixed link to developer guide 2024-03-16 10:41:18 +01:00
Ondřej Budai
e5853c9aa5 Remove rhel-10.0 alias from the openshift template
We now have a proper rhel-10.0 distribution, and this alias is clashing
with it, so we are seeing the following message in production:

failed to configure distro aliases: invalid aliases: ["alias 'rhel-10.0' masks an existing distro"]

Let's fix it by removing the alias, it's obviously not needed anymore.
2024-03-15 15:29:45 +01:00
Simon de Vlieger
babf80f060 deps: bump 2024-03-15 12:12:59 +01:00
Brian C. Lane
57ebfb4011 cloudapi: Use distro repos if none included in imageRequest
In order to support cloudapi blueprint requests from the cmdline using
composer-cli it needs to select the repositories based on the selected
distribution instead of requiring the user to include them with the
request.

If the image request includes repositories they are used, which matches
the current behavior. If the repository list is empty it will use the
distribution name to select from the repositories shipped with
osbuild-composer.
2024-03-11 03:04:54 -07:00
Brian C. Lane
01ba674cac cloudapi: Pass the RepoRegistry to the cloudapi Server 2024-03-11 03:04:54 -07:00
Brian C. Lane
b8967d53bc Move RepoRegistry setup into Composer
This is so that both the weldr and cloud api's can use it as the source
of their repositories.
2024-03-11 03:04:54 -07:00
Tomáš Hozza
f660ed65b6 Packer: remove escaped newline from worker config
If the /tmp/cloud_init_vars contained OSBUILD_EXECUTOR_CLOUDWATCH_GROUP
variable set, the worker configuration file would contain a line with
escaped newline character at the end of the value configuring
`cloudwatch_group` for the `osbuild_executor`. This makes the worker
fail to start when loading the configuration.

Remove the newline from the value appended to the worker config by the
initialization script.

Fix #4001

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-11 10:46:57 +01:00
Tomáš Hozza
1659d190b0 Packer: set 'cloudwatch_group' in worker config if provided
Set the 'cloudwatch_group' value in the worker configuration if provided
in /tmp/cloud_init_vars, so that it is used by the worker when spinning
up an osbuild-executor instance.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-08 13:13:44 +01:00
Tomáš Hozza
e7743f17ec Worker: allow configuring executor CloudWatch group
We need the ability to use different CloudWatch group for the
osbuild-executor on Fedora workers in staging and production
environment.

Extend the worker confguration to allow configuring the CloudWatch group
name used by the osbuild-executor. Extend the secure instance code to
instruct cloud-init via user data to create /tmp/cloud_init_vars file
with the CloudWatch group name in the osbuild-executor instance, to make
it possible for the executor to configure its logging differently based
on the value.

Cover new changes by unit tests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-08 13:13:44 +01:00
Brian C. Lane
ceddabc395 osbuild-worker: Convert osbuild raw json error to a string for logging
Without this you get an array of bytes in the logs. Convert the raw json
output from osbuild to a string so there is some hope that it is
readable.
2024-03-08 01:17:00 +01:00
Simon de Vlieger
b89293ea94 ci: add a PR best practices check 2024-03-07 19:46:11 +01:00
Tomáš Hozza
85753c3617 Test/Koji: test only combinations that we run in the service
Test `koji.sh` with cloud upload on the latest RHEL-8.9 and RHEL9.3 GA
releases, which we use on our Brew workers.

Test the `koji.sh` only on the latest RHEL-8.9, RHEL-9.3 and Fedora
releases, which we use on our Brew and Koji workers.

There's no value in testing this case on CentOS Stream, RHEL EUS
releases or nightly composes.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-07 17:44:54 +01:00
Tomáš Hozza
7c42924ffa Packer: don't source non-existing file in worker_executor.sh
The /tmp/cloud_init_vars is not created on the worker executor, so
sourcing it will make the script fail. Comment the line out, until we
change the worker implementation to inject this file into the worker
executor using cloud-init.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-07 13:31:55 +01:00
Sanne Raymaekers
86e19bf198 templates/packer: set up vector correctly in the executor 2024-03-07 13:15:36 +01:00
Sanne Raymaekers
ef44f22d89 templates/packer: vector expects a yaml configuration now
When switching to the new vector repos, the version was also bumped. The
newer versions expect a yaml config.
2024-03-07 07:32:47 +01:00
Sanne Raymaekers
aa156028b5 templates/packer: make worker_config script executable 2024-03-06 20:38:36 +01:00
Simon Steinbeiss
bdc973d085 test/api.sh: Drop distro aliases and dot-notation
This reverts commit 2fe5e01d4d with the
exception of the changes in tools/provision.sh.
This functionality is covered by unit tests in osbuild/images:
aaa53adc3c/pkg/distrofactory/distrofactory_test.go (L9)
2024-03-06 13:28:30 +01:00
Tomáš Hozza
f02fe86366 Packer: make osbuild-executor worker config more configurable
We need to use custom IAM policy name used by the worker for
osbuild-executor on Fedora workers (in prod vs. stage). And we have the
same requirement for the CloudWatch log group used by the
osbuild-executor.

Modify the Ansible playbook used by Packer to use the values from
/tmp/cloud_init_vars if set and defaulting to the current values if not
set.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-06 12:02:56 +01:00
Achilleas Koutsou
e8a3aac989 test: update minimal-raw test
Update minimal-raw test with new artifact filename.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2024-03-06 10:17:28 +01:00
Achilleas Koutsou
525027726e deps: update images to v0.43.0 2024-03-06 10:17:28 +01:00
schutzbot
75d23d7c98 Post release version bump
[skip ci]
2024-03-06 08:14:12 +00:00
Alexander Todorov
4db594dac7 Skip image_installer tests b/c already tested in installers.sh 2024-03-06 09:09:39 +01:00
Alexander Todorov
d0237daf75 Create a custom group via blueprint in installers.sh test script 2024-03-06 09:09:39 +01:00
Alexander Todorov
d7388fdc66 Remove already tested manifests from test-case-generators
and remove redundant skips from image_tests.sh
2024-03-06 09:09:39 +01:00
Alexander Todorov
eb5db0bd2c Skip all edge image types, already covered in Edge CI 2024-03-06 09:09:39 +01:00
Alexander Todorov
a74756e4fb Skip gce image type, already covered in gcp.sh 2024-03-06 09:09:39 +01:00
Alexander Todorov
c25db57d7d Skip image types covered in azure.sh
namely vhd-boot
2024-03-06 09:09:39 +01:00
Alexander Todorov
c67add4a6e Skip image types already covered in vmware.sh
namely ova-boot|vmdk-boot
2024-03-06 09:09:39 +01:00
Alexander Todorov
20a36964a9 Skip Image Tests for image types already covered in osbuild/images
namely ami-boot|edge_ami|ec2-boot|ec2_ha|ec2_sap

Related: COMPOSER-2127
2024-03-06 09:09:39 +01:00
Simon Steinbeiss
362caacd1f tests: Drop bigiso test (HMS-3710)
This test is being migrated to osbuild/images.
2024-03-06 01:14:57 +01:00
Jakub Rusz
eec2cf62e1 CI: Drop SonarQube in favor of Snyk 2024-03-06 00:11:46 +01:00
Sanne Raymaekers
9004ca6310 Revert "packer: pin the vector version"
This reverts commit 5315264f2e.

This got fixed in 0.22.1.
2024-03-05 19:10:28 +01:00
Sanne Raymaekers
cad29f93a7 templates/packer: migrate vector to new repos
See https://vector.dev/highlights/2023-11-07-new-linux-repos/ for more
information.
2024-03-05 19:10:28 +01:00
Simon de Vlieger
073e304978 images: update dependency
Update the images dependency to get the new version gates for Fedora.
2024-03-04 16:20:26 +01:00