Commit graph

80 commits

Author SHA1 Message Date
Tomáš Hozza
520b94e24a templates/packer: stop setting up Pulp credentials
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-08-12 19:42:51 +02:00
Florian Schüller
0eaef83e26 templates/packer: avoid errors in worker-executor startup
When the worker executor starts up, many error messages and warnings are
shown in the system logs, worker-initialization.service should actually
not run at all. The service crashes and functionally that's fine, but
it just messes up the log, raises questions and can be avoided by just
not running it.
2025-07-24 08:38:55 +02:00
Sanne Raymaekers
7dea1bcd01 many: remove jobsite code
This was replaced by `cmd/osbuild-worker-executor`.
2025-06-25 03:01:59 +02:00
Sanne Raymaekers
cedc351bbd templates/packer: fix installing rpms from copr
There are now 2 colons present, one separating the epoch and the
version, and one before the comment.
2025-06-20 21:57:04 +02:00
Sanne Raymaekers
c3cb3ba785 templates/packer: set wanted-by to cloud-init.target
The `cloud-init.target` in 9.6 has `After=multi-user.target` in its unit
config. The worker initialization service was set to run before
`multi-user.target`, but after `cloud-final.service`. This created an
impossible situation and systemd just disabling the initialization
service.

So this changes:
`multi-user.target -> worker-*.service -> cloud-final.service -> multi-user.target`
to
`cloud-init.target -> worker-*.service -> cloud-final.service -> multi-user.target`.

Thus resolving the loop.
2025-05-14 21:01:39 +02:00
Sanne Raymaekers
0ef0ae7c97 templates/packer: allow setting executor type in worker config
Currently the worker images always have to use aws.ec2, this way we can
use the host executor for fedora.
2024-10-28 10:51:34 +01:00
Florian Schüller
8d24dcfbde osbuild-worker: add CHANNEL to worker logs
aka "the deployment channel" like "staging" or "production"
2024-08-28 16:41:07 +02:00
Sanne Raymaekers
af73f2eccf templates/packer: make set_executor_hostname executable
Prevents `worker-executor.service: Failed at step EXEC spawning
/usr/local/libexec/worker-initialization-scripts/set_executor_hostname.sh:
Permission denied`.
2024-06-26 10:56:57 +02:00
Sanne Raymaekers
2a621521a8 osbuildexecutor/aws.ec2: set hostname of executor via cloud-init
This way much more of the journal will be captured under the new
hostname.
2024-06-25 10:58:10 +02:00
Sanne Raymaekers
7d7bce76c0 templates/packer: use osbuild-worker-executor 2024-06-12 11:36:30 +02:00
Sanne Raymaekers
4629a31f22 templates/packer: use python3.10 on fedora
Ansible on fedora 40 seems broken, the default python 3.12 interpreter
doesn't work, 3.10 works but then the dnf module breaks.

Use 3.10 and stop using the dnf module.
2024-05-31 13:55:58 +02:00
Sanne Raymaekers
22e15da73c templates/packer: use import_tasks instead of include_tasks
The tags don't get inherited through the dynamic `include_tasks`
command. Use `import_tasks` to preserve the tags.
2024-05-31 13:55:58 +02:00
Sanne Raymaekers
13aae7d532 templates/packer: invert tag logic
With the rpmcopy or rpmrepo_osbuild tags, the `Install worker rpm` stage
got skipped on RHEL and CI. Invert the tag logic and use `--tags`
instead of `--skip-tags`.
2024-05-21 09:40:11 +02:00
Sanne Raymaekers
592308f7af templates/packer/ansible: add task to install rpms from copr
Split the rpmrepo tasks in osbuild and composer. With copr we'll use
osbuild from rpmrepo, because the osbuild copr rpms disappear too
quickly.
2024-05-07 13:57:48 +02:00
Sanne Raymaekers
49566b7ce4 templates/packer: add failure script
In case the service failed, set the instance to unhealthy.
2024-05-02 13:34:47 +02:00
Sanne Raymaekers
3df0c3a631 templates/packer: fix proxy config in ldap service account init
The proxy is set to "null" currently.
2024-04-23 22:13:17 +02:00
Sanne Raymaekers
677e30cc68 templates/packer: add proxy 2024-04-17 16:17:57 +02:00
Sanne Raymaekers
18db445745 Revert "templates/packer: set http(s)_proxy environment variabl…"
This reverts commit 484c82ce55.

The AWS sdk fails to get the instance identity document when the proxy
is configured. The proxy will need to be configured explicitly for the
depsolve job and osbuild (sources) job.
2024-04-17 16:17:57 +02:00
Sanne Raymaekers
484c82ce55 templates/packer: set http(s)_proxy environment variable in unit 2024-04-10 10:03:43 +02:00
Sanne Raymaekers
c8130d0689 templates/packer: support ldap service account for repo mtls conf
The secret needs 3 fields, the cert, key and baseurl for the
repository. The CA is optional.
2024-03-29 20:45:05 +01:00
Sanne Raymaekers
cda94f4f62 templates/packer: don't subscribe executor
All the required sources will be proxied.
2024-03-19 17:07:30 +01:00
Tomáš Hozza
f660ed65b6 Packer: remove escaped newline from worker config
If the /tmp/cloud_init_vars contained OSBUILD_EXECUTOR_CLOUDWATCH_GROUP
variable set, the worker configuration file would contain a line with
escaped newline character at the end of the value configuring
`cloudwatch_group` for the `osbuild_executor`. This makes the worker
fail to start when loading the configuration.

Remove the newline from the value appended to the worker config by the
initialization script.

Fix #4001

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-11 10:46:57 +01:00
Tomáš Hozza
1659d190b0 Packer: set 'cloudwatch_group' in worker config if provided
Set the 'cloudwatch_group' value in the worker configuration if provided
in /tmp/cloud_init_vars, so that it is used by the worker when spinning
up an osbuild-executor instance.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-08 13:13:44 +01:00
Tomáš Hozza
7c42924ffa Packer: don't source non-existing file in worker_executor.sh
The /tmp/cloud_init_vars is not created on the worker executor, so
sourcing it will make the script fail. Comment the line out, until we
change the worker implementation to inject this file into the worker
executor using cloud-init.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-07 13:31:55 +01:00
Sanne Raymaekers
86e19bf198 templates/packer: set up vector correctly in the executor 2024-03-07 13:15:36 +01:00
Sanne Raymaekers
ef44f22d89 templates/packer: vector expects a yaml configuration now
When switching to the new vector repos, the version was also bumped. The
newer versions expect a yaml config.
2024-03-07 07:32:47 +01:00
Sanne Raymaekers
aa156028b5 templates/packer: make worker_config script executable 2024-03-06 20:38:36 +01:00
Tomáš Hozza
f02fe86366 Packer: make osbuild-executor worker config more configurable
We need to use custom IAM policy name used by the worker for
osbuild-executor on Fedora workers (in prod vs. stage). And we have the
same requirement for the CloudWatch log group used by the
osbuild-executor.

Modify the Ansible playbook used by Packer to use the values from
/tmp/cloud_init_vars if set and defaulting to the current values if not
set.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-03-06 12:02:56 +01:00
Sanne Raymaekers
9004ca6310 Revert "packer: pin the vector version"
This reverts commit 5315264f2e.

This got fixed in 0.22.1.
2024-03-05 19:10:28 +01:00
Sanne Raymaekers
cad29f93a7 templates/packer: migrate vector to new repos
See https://vector.dev/highlights/2023-11-07-new-linux-repos/ for more
information.
2024-03-05 19:10:28 +01:00
Sanne Raymaekers
1abd887c23 templates/packer: rename executor log group
In app-interface the output resource names need to be unique, and the
log group name is already shared with the role.
2024-02-16 11:46:03 +01:00
Sanne Raymaekers
3400c5c20d templates/packer: setup vector in osbuild-executor 2024-02-15 12:05:17 +01:00
Sanne Raymaekers
a9010479e3 templates/packer: set -builder-path to /var/cache/osbuild-builder
The builder uses `/run/osbuild` as a default path for this argument. Yet
this directory doesn't exist when the builder writes the manifest. But
osbuild should own this directory, not the builder.

Furthermore `/run` is a tmpfs, so the executor might run into memory
issues if we use `/run` as the store and output directory (on the "host"
workers these are in `/var/cache`).

While `/tmp` might seem like a good candidate on RHEL, it's a tmpfs on
Fedora, so it's also to be avoided.
2024-02-15 01:02:10 +01:00
Sanne Raymaekers
04ce056b6a templates/packer: let the executor listen on all interfaces 2024-02-14 22:55:54 +01:00
Sanne Raymaekers
f03dac5834 templates/packer: don't subscribe fedora executor 2024-02-14 13:45:52 +01:00
Sanne Raymaekers
d5fd1bbbf0 templates/packer: add worker-executor service to image 2024-02-14 13:45:52 +01:00
Sanne Raymaekers
a9066ee793 templates/packer: remove worker_builder script from init service
This script doesn't actually exist.
2024-02-14 13:45:52 +01:00
Sanne Raymaekers
d0caac9d69 templates/packer: use aws.ec2 osbuild executor 2024-02-14 09:54:11 +01:00
Sanne Raymaekers
4111ca4e0e templates/packer: add worker-executor service 2024-02-14 09:54:11 +01:00
Sanne Raymaekers
e289b763e7 templates/packer: deal with unbound variables
Don't allow unbound variables, but for the variables that are used to
determine whether or not that part of the setup should continue, default
to empty/undefined.
2024-01-30 21:41:31 +01:00
Gianluca Zuccarelli
3fe36d0012 templates/packer: configure pulp creds on startup 2023-11-07 10:48:00 +01:00
Sanne Raymaekers
9d7159dab3 templates/packer: retry subscribtion 2023-09-25 11:56:42 +02:00
Sanne Raymaekers
0dc1a01077 templates/packer: configure oracle cloud credentials on startup 2023-09-22 09:55:48 +02:00
Sanne Raymaekers
170feba87b templates/packer: use RH SSO for the default token endpoint
MAS SSO (identity.api.openshift.com) was deprecated, RH SSO should be
the default.
2023-06-29 11:32:44 +02:00
Sanne Raymaekers
a08eb69b2e templates/packer/ansible: fix enabling cdn repos on aarch64 2023-03-03 17:58:49 +01:00
Sanne Raymaekers
c1032f31e4 templates/packer/ansible: fix unregister
The community redhat_subscription module calls `subscription-manager
unsusbscribe`, which doesn't exist. Use shell for now.
2023-03-03 17:58:49 +01:00
Sanne Raymaekers
ca8a05bd3a templates/packer: subscribe packer machines
To avoid a mismatch between the RPMs (which are build using CDN content)
and the packer instances (RHUI, which might be older).
2023-03-03 13:00:05 +01:00
Sanne Raymaekers
0096ff3689 Revert "Packer: workaround missing authselect-compat-1.2.5-2.el9_1 in RHUI repos"
This reverts commit 0a4a75e19e.
2023-03-01 20:05:38 +01:00
Tomáš Hozza
0a4a75e19e Packer: workaround missing authselect-compat-1.2.5-2.el9_1 in RHUI repos
`authselect-compat-1.2.5-2.el9_1` package is currently missing in AWS
RHUI el9 AppStream repositories, which makes `dnf upgrade` fail on
RHEL-9.1. This is a RHUI-specific issue, since the package is available
in CDN repos.

In order to workaround the issue for now, `authselect-compat` needs to
be removed as part of the upgrade in order for it to succeed. Use
`--allowerasing` instead of just removing the issue, because this will
ensure that `authselect-compat` will be upgraded just fine, once the
issue is resolved.

Fix the issue in the CI script that builds the image using Packer, as
well as the Ansible playbook used by Packer to build the image.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-24 15:40:02 +01:00
Tomáš Hozza
6ae8904f5a templates/packer: add comment to get_aws_creds.sh
Add a comment explaining why it is important to set the AWS bucket in
the worker configuration, even if the `AWS_ACCOUNT_IMAGE_BUILDER_ARN` is
empty.
2022-10-11 13:23:18 +02:00