Commit graph

262 commits

Author SHA1 Message Date
Sanne Raymaekers
484c82ce55 templates/packer: set http(s)_proxy environment variable in unit 2024-04-10 10:03:43 +02:00
Sanne Raymaekers
c8130d0689 templates/packer: support ldap service account for repo mtls conf
The secret needs 3 fields, the cert, key and baseurl for the
repository. The CA is optional.
2024-03-29 20:45:05 +01:00
Sanne Raymaekers
cda94f4f62 templates/packer: don't subscribe executor
All the required sources will be proxied.
2024-03-19 17:07:30 +01:00
Ondřej Budai
e5853c9aa5 Remove rhel-10.0 alias from the openshift template
We now have a proper rhel-10.0 distribution, and this alias is clashing
with it, so we are seeing the following message in production:

failed to configure distro aliases: invalid aliases: ["alias 'rhel-10.0' masks an existing distro"]

Let's fix it by removing the alias, it's obviously not needed anymore.
2024-03-15 15:29:45 +01:00
Tomáš Hozza
f660ed65b6 Packer: remove escaped newline from worker config
If the /tmp/cloud_init_vars contained OSBUILD_EXECUTOR_CLOUDWATCH_GROUP
variable set, the worker configuration file would contain a line with
escaped newline character at the end of the value configuring
`cloudwatch_group` for the `osbuild_executor`. This makes the worker
fail to start when loading the configuration.

Remove the newline from the value appended to the worker config by the
initialization script.

Fix #4001

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-11 10:46:57 +01:00
Tomáš Hozza
1659d190b0 Packer: set 'cloudwatch_group' in worker config if provided
Set the 'cloudwatch_group' value in the worker configuration if provided
in /tmp/cloud_init_vars, so that it is used by the worker when spinning
up an osbuild-executor instance.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-08 13:13:44 +01:00
Tomáš Hozza
7c42924ffa Packer: don't source non-existing file in worker_executor.sh
The /tmp/cloud_init_vars is not created on the worker executor, so
sourcing it will make the script fail. Comment the line out, until we
change the worker implementation to inject this file into the worker
executor using cloud-init.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-07 13:31:55 +01:00
Sanne Raymaekers
86e19bf198 templates/packer: set up vector correctly in the executor 2024-03-07 13:15:36 +01:00
Sanne Raymaekers
ef44f22d89 templates/packer: vector expects a yaml configuration now
When switching to the new vector repos, the version was also bumped. The
newer versions expect a yaml config.
2024-03-07 07:32:47 +01:00
Sanne Raymaekers
aa156028b5 templates/packer: make worker_config script executable 2024-03-06 20:38:36 +01:00
Tomáš Hozza
f02fe86366 Packer: make osbuild-executor worker config more configurable
We need to use custom IAM policy name used by the worker for
osbuild-executor on Fedora workers (in prod vs. stage). And we have the
same requirement for the CloudWatch log group used by the
osbuild-executor.

Modify the Ansible playbook used by Packer to use the values from
/tmp/cloud_init_vars if set and defaulting to the current values if not
set.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-06 12:02:56 +01:00
Sanne Raymaekers
9004ca6310 Revert "packer: pin the vector version"
This reverts commit 5315264f2e.

This got fixed in 0.22.1.
2024-03-05 19:10:28 +01:00
Sanne Raymaekers
cad29f93a7 templates/packer: migrate vector to new repos
See https://vector.dev/highlights/2023-11-07-new-linux-repos/ for more
information.
2024-03-05 19:10:28 +01:00
Tomáš Hozza
e561ba0854 templates/composer: set DISTRO_ALIASES for composer
Set the RHEL release names without the minor version to point to the
latest GA release. Set the 'rhel-10.0' to the latest RHEL-9 minor
release in development, so that one can start building RHEL-10 images
without referencing RHEL-9.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-02-21 12:06:33 +01:00
Sanne Raymaekers
1abd887c23 templates/packer: rename executor log group
In app-interface the output resource names need to be unique, and the
log group name is already shared with the role.
2024-02-16 11:46:03 +01:00
Sanne Raymaekers
3400c5c20d templates/packer: setup vector in osbuild-executor 2024-02-15 12:05:17 +01:00
Sanne Raymaekers
a9010479e3 templates/packer: set -builder-path to /var/cache/osbuild-builder
The builder uses `/run/osbuild` as a default path for this argument. Yet
this directory doesn't exist when the builder writes the manifest. But
osbuild should own this directory, not the builder.

Furthermore `/run` is a tmpfs, so the executor might run into memory
issues if we use `/run` as the store and output directory (on the "host"
workers these are in `/var/cache`).

While `/tmp` might seem like a good candidate on RHEL, it's a tmpfs on
Fedora, so it's also to be avoided.
2024-02-15 01:02:10 +01:00
Sanne Raymaekers
04ce056b6a templates/packer: let the executor listen on all interfaces 2024-02-14 22:55:54 +01:00
Sanne Raymaekers
f03dac5834 templates/packer: don't subscribe fedora executor 2024-02-14 13:45:52 +01:00
Sanne Raymaekers
d5fd1bbbf0 templates/packer: add worker-executor service to image 2024-02-14 13:45:52 +01:00
Sanne Raymaekers
a9066ee793 templates/packer: remove worker_builder script from init service
This script doesn't actually exist.
2024-02-14 13:45:52 +01:00
Sanne Raymaekers
d0caac9d69 templates/packer: use aws.ec2 osbuild executor 2024-02-14 09:54:11 +01:00
Sanne Raymaekers
4111ca4e0e templates/packer: add worker-executor service 2024-02-14 09:54:11 +01:00
Diaa Sami
c9c51613a4 composer: glitchtip integration 2024-02-13 14:57:57 +01:00
Sanne Raymaekers
44426bb48f templates/dashboards: add community stage service to orgs 2024-02-05 11:38:53 +01:00
Jakub Rusz
db0e6c9643 Packer: change fedora-38 aarch64 ami
This ami is currently broken, switch to a slightly older one.
2024-01-31 10:11:50 +01:00
Sanne Raymaekers
e289b763e7 templates/packer: deal with unbound variables
Don't allow unbound variables, but for the variables that are used to
determine whether or not that part of the setup should continue, default
to empty/undefined.
2024-01-30 21:41:31 +01:00
Diaa Sami
6cfa26399f composer: use logrus hook instead of k8s sidecar
for splunk log forwarding
Fixes COMPOSER-2051
2023-11-28 12:42:00 +01:00
Gianluca Zuccarelli
3fe36d0012 templates/packer: configure pulp creds on startup 2023-11-07 10:48:00 +01:00
Sanne Raymaekers
bf3ff40a65 dashboards: drop interval from composer dashboard and fix slo
The latency budget remaining used $__range instead of the 28d constant.
2023-10-03 11:48:37 +02:00
Sanne Raymaekers
f05a5b59f3 dashboards: drop API section from worker job stats dashboard
Renames the worker dashboard to worker job stats dashboard.

Drops the interval variable and relies solely on $__range and
$__rate_interval.
2023-10-03 11:48:37 +02:00
Sanne Raymaekers
1475e216d2 dashboards: add worker api dashboard
Also this one is made without a separate interval variable, instead
relying on $__rate_interval and $__interval.
2023-10-03 11:48:37 +02:00
Sanne Raymaekers
33f9a6726e dashboards: fix composer dash request rate errors 2023-10-02 18:50:37 +02:00
Sanne Raymaekers
9d7159dab3 templates/packer: retry subscribtion 2023-09-25 11:56:42 +02:00
Sanne Raymaekers
0dc1a01077 templates/packer: configure oracle cloud credentials on startup 2023-09-22 09:55:48 +02:00
Sanne Raymaekers
3a9bcded32 templates/composer: fix cpu request/limits
The fluentd sidecar had the same request/limit as the service container,
and the migrate init-container had the fluentd request/limit. It should
be the other way round.
2023-09-21 12:41:06 +02:00
Sanne Raymaekers
5bb9d414a2 templates/compose: add startingDeadlineSeconds to maintenance job
The job won't run if it doesn't get scheduled within 30 minutes. This
prevents the job running multiple times in a row if it didn't get
scheduled, for instance due to resource limits.
2023-09-21 12:41:06 +02:00
Sanne Raymaekers
e0b2455acf templates/composer: parameterise maintenance job cpu req/limit 2023-09-21 11:11:35 +02:00
Sanne Raymaekers
715bdba1bf dashboards/worker: default to showing the past 6 hours
The worker dashboards contains slow queries, running these on 28 days of
data take a very long time (and they often time out).
2023-08-24 17:01:23 +02:00
Ondřej Budai
ba417dbf3d packer: use gp3 volumes
GP3 is cheaper than GP2, let's switch to it for storing our images:
https://fedoraproject.org/wiki/Changes/CloudEC2gp3

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-07-21 12:20:47 +02:00
Ondřej Budai
b461e403ef packer: move Fedora to 38
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-07-21 12:20:47 +02:00
Sanne Raymaekers
a2c07ea83a templates/dashboards: rework composer dashboard
splits the board into 3 sections:
- SLO
- API throughput
- API latency

It's also possible to filter by tenant. And some colours were adjusted
to improve readability.
2023-06-30 11:06:51 +02:00
Sanne Raymaekers
170feba87b templates/packer: use RH SSO for the default token endpoint
MAS SSO (identity.api.openshift.com) was deprecated, RH SSO should be
the default.
2023-06-29 11:32:44 +02:00
Sanne Raymaekers
38093100e3 templates/composer: No longer accept MAS SSO 2023-06-29 11:32:44 +02:00
Diaa Sami
8398f27742 internal/cloudapi: additional prometheus listener
Listening on another port, while keeping the existing endpoint until
transition is complete
2023-06-07 17:05:32 +02:00
Sanne Raymaekers
53198bed6e templates/composer: fix fluentd requests/limits
No separate request for memory was defined in #3472, only cpu
request/limit.
2023-06-05 16:16:18 +02:00
Sanne Raymaekers
3faab2f102 templates/composer: add separate CPU request/limit for sidecar 2023-06-05 11:51:36 +02:00
Sanne Raymaekers
0ddbee11cd templates/composer: parametrise replicas 2023-06-05 11:51:36 +02:00
Ondřej Budai
dce2ced50b packer: bump the amazon plugin to 1.2.3
Since the previous commit removed the associate_public_ip_address, we should
not be hitting the new behaviour introduced in 1.2.3, thus everything will
hopefully work as before.
2023-05-05 11:07:05 +02:00
Ondřej Budai
a2a5618149 packer: remove associate_public_ip_address
The documentation for this option says the following:

> If using a non-default VPC, public IP addresses are not provided by default.
> If this is true, your new instance will get a Public IP. default: unset

We don't specify a VPC in the packer build, thus we are using the default
one. Therefore, I don't think we actually need this option as it's useful
only for non-default VPCs.

See
https://developer.hashicorp.com/packer/plugins/builders/amazon/ebs#run-configuration

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-05-05 11:07:05 +02:00