Commit graph

6368 commits

Author SHA1 Message Date
Simon Steinbeiss
41a532291c README: Update Matrix link 2024-04-10 11:55:09 +02:00
Simon Steinbeiss
006c7a6742 README: Add osbuild-composer's principles 2024-04-10 11:55:09 +02:00
Sanne Raymaekers
484c82ce55 templates/packer: set http(s)_proxy environment variable in unit 2024-04-10 10:03:43 +02:00
dependabot[bot]
00ebf7c5f0 build(deps): bump github.com/docker/docker
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 24.0.7+incompatible to 24.0.9+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v24.0.7...v24.0.9)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-08 20:40:15 +02:00
Ondřej Budai
0cc7cc99e6 Disable F40 CI (CVE-2024-3094 response) 2024-04-05 16:27:50 +02:00
schutzbot
63473c8e79 Post release version bump
[skip ci]
2024-04-03 08:13:59 +00:00
Sanne Raymaekers
c8130d0689 templates/packer: support ldap service account for repo mtls conf
The secret needs 3 fields, the cert, key and baseurl for the
repository. The CA is optional.
2024-03-29 20:45:05 +01:00
Sanne Raymaekers
53f77368fd osbuild-worker: add support for mtls dnf repo secrets 2024-03-29 14:46:54 +01:00
Alexander Todorov
dfed9114ff Execute OpenShift Virtualization tests only on RHEL 9.x nightly
reduce the number of overall test executions b/c resources in this
environment are severely limited.
2024-03-28 12:21:57 +02:00
Sanne Raymaekers
3be2c92a1b internal/blueprint: add ExpireDate to UserCustomization
Just to make osbuild-composer compile, this field is not functional yet.
2024-03-28 11:11:13 +01:00
Sanne Raymaekers
1ce72071b4 go.mod: bump osbuild/images to 51 2024-03-28 11:11:13 +01:00
Achilleas Koutsou
9aa80c25bc test: drop oscap test script and gitlab job
The test is being migrated to osbuild/images.
See https://github.com/osbuild/images/pull/502

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2024-03-27 17:20:37 +01:00
Brian C. Lane
d3fc53bdec cloudapi: Override the request distro with the blueprint distro
When the blueprint sets a specific distribution it should be used
instead of the distribution from the compose request.

Includes a test to make sure it uses repositories from the blueprint,
not the request.
2024-03-22 19:12:24 +01:00
Sanne Raymaekers
f311adf35d osbuild-jobsite-builder: disable http.Server timeouts
Let's just rely on the timeouts surrounding the whole step.
2024-03-21 17:08:07 +01:00
Sanne Raymaekers
d3074fc265 osbuild-jobsite-manager: clean up store archive after use 2024-03-21 15:20:52 +01:00
Sanne Raymaekers
bc17204001 osbuild-jobsite-manager: reseek when necessary 2024-03-21 15:20:52 +01:00
Sanne Raymaekers
c19e9fb3b9 osbuild-jobsite-manager: close writer before sending the store 2024-03-21 15:20:52 +01:00
Sanne Raymaekers
dabe63cb78 osbuild-jobsite: increase populate timeout
30 seconds can be a bit too little for the entire store.
2024-03-20 21:58:17 +01:00
Sanne Raymaekers
314ed4b527 cloud/awscloud: allow internet access on secure instance again
The executor is timing out and there are no logs. This will require some
further work. Remove the restriction for now.
2024-03-20 14:58:25 +01:00
Simon de Vlieger
8653f614f0 repositories: remove rhel-8-beta
This is an invalid repository name.

Signed-off-by: Simon de Vlieger <cmdr@supakeen.com>
2024-03-20 12:30:20 +01:00
Achilleas Koutsou
3633528613 worker: drop compatibility test for DepsolveJob serialisation
This reverts commit c8ce3e4428
2024-03-20 12:12:12 +01:00
Achilleas Koutsou
80f8fa051c worker: drop backwards compatibility for DepsolveJob serialisation
We've kept this around a lot longer than we should have.

This reverts commit 94c7fda779.
2024-03-20 12:12:12 +01:00
Achilleas Koutsou
3e0f8a7ac3 test: modify the kickstart file in nightly pipelines
Nightly pipelines test against older osbuild-composer versions so we
need to still modify the kickstart file until the feature reaches the
nightly composes.
2024-03-20 11:45:08 +01:00
Achilleas Koutsou
8f78171f83 cloudapi: add installer customizations to the API
Adding both as a traditional compose request customization and in the
new blueprint customizations.
2024-03-20 11:45:08 +01:00
Achilleas Koutsou
eccdcfcab0 test: use new installer customizations
Use the new installer customizations to build and boot an unattended
installation and add the users to the sudoers with NOPASSWD enabled.

This is added both to he base installer test (installers.sh) and the
ostree test (ostree-ng.sh).  For the ostree test, we keep the mkksiso
step to retain the remote configuration change.  The modksiso() function
is changed to explicitly read the osbuild.ks file since we don't want to
modify the osbuild-base.ks file and we want to ensure that the kernel
command line argument points to the osbuild.ks file.
2024-03-20 11:45:08 +01:00
Achilleas Koutsou
bd57d95e49 blueprint: add installer customizations 2024-03-20 11:45:08 +01:00
schutzbot
535ff2ef2d Post release version bump
[skip ci]
2024-03-20 08:12:49 +00:00
Sanne Raymaekers
cda94f4f62 templates/packer: don't subscribe executor
All the required sources will be proxied.
2024-03-19 17:07:30 +01:00
Sanne Raymaekers
4e2d990af8 test/worker-executor: internet access during executor setup 2024-03-19 17:07:30 +01:00
Sanne Raymaekers
79b5b736e9 cloud/awscloud: restrict network egress for secure instance
The security instance should no longer have any internet access.
2024-03-19 17:07:30 +01:00
Sanne Raymaekers
56bdcf1cdc aws.ec2: prep sources before starting build 2024-03-19 17:07:30 +01:00
Sanne Raymaekers
b4e361e093 osbuild-jobsite-builder: unpack source tarball in populate 2024-03-19 17:07:30 +01:00
Sanne Raymaekers
07598ea65c osbuild-jobsite-manager: pass store to builder 2024-03-19 17:07:30 +01:00
Jerry Uanino
551db639da Fixed link to developer guide 2024-03-16 10:41:18 +01:00
Ondřej Budai
e5853c9aa5 Remove rhel-10.0 alias from the openshift template
We now have a proper rhel-10.0 distribution, and this alias is clashing
with it, so we are seeing the following message in production:

failed to configure distro aliases: invalid aliases: ["alias 'rhel-10.0' masks an existing distro"]

Let's fix it by removing the alias, it's obviously not needed anymore.
2024-03-15 15:29:45 +01:00
Simon de Vlieger
babf80f060 deps: bump 2024-03-15 12:12:59 +01:00
Brian C. Lane
57ebfb4011 cloudapi: Use distro repos if none included in imageRequest
In order to support cloudapi blueprint requests from the cmdline using
composer-cli it needs to select the repositories based on the selected
distribution instead of requiring the user to include them with the
request.

If the image request includes repositories they are used, which matches
the current behavior. If the repository list is empty it will use the
distribution name to select from the repositories shipped with
osbuild-composer.
2024-03-11 03:04:54 -07:00
Brian C. Lane
01ba674cac cloudapi: Pass the RepoRegistry to the cloudapi Server 2024-03-11 03:04:54 -07:00
Brian C. Lane
b8967d53bc Move RepoRegistry setup into Composer
This is so that both the weldr and cloud api's can use it as the source
of their repositories.
2024-03-11 03:04:54 -07:00
Tomáš Hozza
f660ed65b6 Packer: remove escaped newline from worker config
If the /tmp/cloud_init_vars contained OSBUILD_EXECUTOR_CLOUDWATCH_GROUP
variable set, the worker configuration file would contain a line with
escaped newline character at the end of the value configuring
`cloudwatch_group` for the `osbuild_executor`. This makes the worker
fail to start when loading the configuration.

Remove the newline from the value appended to the worker config by the
initialization script.

Fix #4001

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-11 10:46:57 +01:00
Tomáš Hozza
1659d190b0 Packer: set 'cloudwatch_group' in worker config if provided
Set the 'cloudwatch_group' value in the worker configuration if provided
in /tmp/cloud_init_vars, so that it is used by the worker when spinning
up an osbuild-executor instance.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-08 13:13:44 +01:00
Tomáš Hozza
e7743f17ec Worker: allow configuring executor CloudWatch group
We need the ability to use different CloudWatch group for the
osbuild-executor on Fedora workers in staging and production
environment.

Extend the worker confguration to allow configuring the CloudWatch group
name used by the osbuild-executor. Extend the secure instance code to
instruct cloud-init via user data to create /tmp/cloud_init_vars file
with the CloudWatch group name in the osbuild-executor instance, to make
it possible for the executor to configure its logging differently based
on the value.

Cover new changes by unit tests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-08 13:13:44 +01:00
Brian C. Lane
ceddabc395 osbuild-worker: Convert osbuild raw json error to a string for logging
Without this you get an array of bytes in the logs. Convert the raw json
output from osbuild to a string so there is some hope that it is
readable.
2024-03-08 01:17:00 +01:00
Simon de Vlieger
b89293ea94 ci: add a PR best practices check 2024-03-07 19:46:11 +01:00
Tomáš Hozza
85753c3617 Test/Koji: test only combinations that we run in the service
Test `koji.sh` with cloud upload on the latest RHEL-8.9 and RHEL9.3 GA
releases, which we use on our Brew workers.

Test the `koji.sh` only on the latest RHEL-8.9, RHEL-9.3 and Fedora
releases, which we use on our Brew and Koji workers.

There's no value in testing this case on CentOS Stream, RHEL EUS
releases or nightly composes.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-07 17:44:54 +01:00
Tomáš Hozza
7c42924ffa Packer: don't source non-existing file in worker_executor.sh
The /tmp/cloud_init_vars is not created on the worker executor, so
sourcing it will make the script fail. Comment the line out, until we
change the worker implementation to inject this file into the worker
executor using cloud-init.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-07 13:31:55 +01:00
Sanne Raymaekers
86e19bf198 templates/packer: set up vector correctly in the executor 2024-03-07 13:15:36 +01:00
Sanne Raymaekers
ef44f22d89 templates/packer: vector expects a yaml configuration now
When switching to the new vector repos, the version was also bumped. The
newer versions expect a yaml config.
2024-03-07 07:32:47 +01:00
Sanne Raymaekers
aa156028b5 templates/packer: make worker_config script executable 2024-03-06 20:38:36 +01:00
Simon Steinbeiss
bdc973d085 test/api.sh: Drop distro aliases and dot-notation
This reverts commit 2fe5e01d4d with the
exception of the changes in tools/provision.sh.
This functionality is covered by unit tests in osbuild/images:
aaa53adc3c/pkg/distrofactory/distrofactory_test.go (L9)
2024-03-06 13:28:30 +01:00