Commit graph

74 commits

Author SHA1 Message Date
Jakub Rusz
db0e6c9643 Packer: change fedora-38 aarch64 ami
This ami is currently broken, switch to a slightly older one.
2024-01-31 10:11:50 +01:00
Sanne Raymaekers
e289b763e7 templates/packer: deal with unbound variables
Don't allow unbound variables, but for the variables that are used to
determine whether or not that part of the setup should continue, default
to empty/undefined.
2024-01-30 21:41:31 +01:00
Gianluca Zuccarelli
3fe36d0012 templates/packer: configure pulp creds on startup 2023-11-07 10:48:00 +01:00
Sanne Raymaekers
9d7159dab3 templates/packer: retry subscribtion 2023-09-25 11:56:42 +02:00
Sanne Raymaekers
0dc1a01077 templates/packer: configure oracle cloud credentials on startup 2023-09-22 09:55:48 +02:00
Ondřej Budai
ba417dbf3d packer: use gp3 volumes
GP3 is cheaper than GP2, let's switch to it for storing our images:
https://fedoraproject.org/wiki/Changes/CloudEC2gp3

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-07-21 12:20:47 +02:00
Ondřej Budai
b461e403ef packer: move Fedora to 38
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-07-21 12:20:47 +02:00
Sanne Raymaekers
170feba87b templates/packer: use RH SSO for the default token endpoint
MAS SSO (identity.api.openshift.com) was deprecated, RH SSO should be
the default.
2023-06-29 11:32:44 +02:00
Ondřej Budai
dce2ced50b packer: bump the amazon plugin to 1.2.3
Since the previous commit removed the associate_public_ip_address, we should
not be hitting the new behaviour introduced in 1.2.3, thus everything will
hopefully work as before.
2023-05-05 11:07:05 +02:00
Ondřej Budai
a2a5618149 packer: remove associate_public_ip_address
The documentation for this option says the following:

> If using a non-default VPC, public IP addresses are not provided by default.
> If this is true, your new instance will get a Public IP. default: unset

We don't specify a VPC in the packer build, thus we are using the default
one. Therefore, I don't think we actually need this option as it's useful
only for non-default VPCs.

See
https://developer.hashicorp.com/packer/plugins/builders/amazon/ebs#run-configuration

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-05-05 11:07:05 +02:00
Ondřej Budai
edf4f7e879 packer: pin the version of the amazon plugin to 1.2.2
Version 1.2.3 made changes to how the plugin handles auto-selection of a
subnet when it's not specified, see

f1ec287c77

Sadly, the new algorithm selects us-east-1e for us that doesn't support
the machine types we use (c6*.large) which causes the build to fail.
I reported it here:
https://github.com/hashicorp/packer-plugin-amazon/issues/368

One workaround might be to pin a working subnet, but that's apparently also
broken in 1.2.3, see
https://github.com/hashicorp/packer-plugin-amazon/issues/367

Therefore, I decided to pin the plugin to 1.2.2 for now, and see what's
the recommended approach from terraform guys.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-04-20 13:02:34 +02:00
Sanne Raymaekers
a08eb69b2e templates/packer/ansible: fix enabling cdn repos on aarch64 2023-03-03 17:58:49 +01:00
Sanne Raymaekers
c1032f31e4 templates/packer/ansible: fix unregister
The community redhat_subscription module calls `subscription-manager
unsusbscribe`, which doesn't exist. Use shell for now.
2023-03-03 17:58:49 +01:00
Sanne Raymaekers
ca8a05bd3a templates/packer: subscribe packer machines
To avoid a mismatch between the RPMs (which are build using CDN content)
and the packer instances (RHUI, which might be older).
2023-03-03 13:00:05 +01:00
Sanne Raymaekers
0096ff3689 Revert "Packer: workaround missing authselect-compat-1.2.5-2.el9_1 in RHUI repos"
This reverts commit 0a4a75e19e.
2023-03-01 20:05:38 +01:00
Tomáš Hozza
0a4a75e19e Packer: workaround missing authselect-compat-1.2.5-2.el9_1 in RHUI repos
`authselect-compat-1.2.5-2.el9_1` package is currently missing in AWS
RHUI el9 AppStream repositories, which makes `dnf upgrade` fail on
RHEL-9.1. This is a RHUI-specific issue, since the package is available
in CDN repos.

In order to workaround the issue for now, `authselect-compat` needs to
be removed as part of the upgrade in order for it to succeed. Use
`--allowerasing` instead of just removing the issue, because this will
ensure that `authselect-compat` will be upgraded just fine, once the
issue is resolved.

Fix the issue in the CI script that builds the image using Packer, as
well as the Ansible playbook used by Packer to build the image.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-01-24 15:40:02 +01:00
Sanne Raymaekers
81a5ff1bf6 templates/packer: triple aws polling attempts
AMIs can take a long time to get ready.
2022-12-14 17:10:13 +01:00
Sanne Raymaekers
86c3036fe3 templates/packer: increase polling delay
A packer build failed due to being rate limited by the aws api.
2022-12-13 13:55:53 +01:00
Tomáš Hozza
6ae8904f5a templates/packer: add comment to get_aws_creds.sh
Add a comment explaining why it is important to set the AWS bucket in
the worker configuration, even if the `AWS_ACCOUNT_IMAGE_BUILDER_ARN` is
empty.
2022-10-11 13:23:18 +02:00
Tomáš Hozza
09daa75adf templates/packer: set the GCP bucket in the worker configuration
Similar to AWS, set the GCP bucket in the worker configuration.
2022-10-11 13:23:18 +02:00
Diaa Sami
5ffb9e693e tools/appsre: remove monit setup code & scripts
Since it doesn't not work since we moved workers to app-sre
2022-10-04 16:26:08 +02:00
Ondřej Budai
f25dca793d packer: remove Fedora 35
Our workers already run on Fedora 36 so there's no need to build F35 anymore.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-30 14:52:24 +02:00
Diaa Sami
98eda72499 templates/packer: update amazon plugin 2022-09-27 10:47:32 +02:00
Diaa Sami
06fbd926ae app-sre: Update AMIs to rhel-9.0 2022-09-27 10:47:32 +02:00
Sanne Raymaekers
5c12076b4f templates/packer: Allow token url to be set by cloud-init vars
Hardcoding the token url renders the image useless if it ever needs to
be changed.
2022-09-22 14:15:26 +02:00
Ondřej Budai
8f97c4788c packer: add fedora 36
F35 is going EOL soon, so let's update. I want to ditch F35 as soon as possible
after this is merged, but I want to have some overlap just to be sure.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-22 11:22:46 +02:00
Sanne Raymaekers
183e10e466 templates/packer: append distro and arch to the ami name
Because the rhel-8 images share the same name, and `force_deregister` is
true, packer will always deregister one of them.
2022-09-15 20:27:59 +02:00
Diaa Sami
819a63e50e templates/packer: reasonable aws_polling limits for rhel AWS builds 2022-09-09 12:08:29 +02:00
Diaa Sami
46d36a0e73 Revert "appsre: disable aarch64 AMI creation until issue is resolved"
This reverts commit 84f46eebdb.
2022-09-09 12:08:29 +02:00
Diaa Sami
84f46eebdb appsre: disable aarch64 AMI creation until issue is resolved
after merging of PR #2718, generation of AMIs has been failing with 'ResourceNotReady: exceeded wait attempts'.
issue tracked in #2961
2022-09-07 12:28:40 +02:00
Sanne Raymaekers
ab3bd7d94f templates/packer: Increase aws timeouts for rhel-8-aarch64
This job is failing with "ResourceNotReady: exceeded wait attempts".

https://www.packer.io/plugins/builders/amazon#resourcenotready-error
2022-09-05 14:39:12 +02:00
Diaa Sami
ec0a1944b4 appsre-ansible: support aarch64
make ansible playbooks arch-agnostic
extract embedded bash script into separate file with parameters
update packer template to support aarch64
Convert parts of bash script to python code that can start multi-arch instances to build RPMS
2022-09-05 12:08:57 +02:00
Ondřej Budai
767283b2d9 packer: use 8.6 as a base for RHEL images
Let's stay updated!

Also, let's remove 8.4 and 8.5 from Schutzfile, I strongly believe that it's
not used anywhere.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-07-05 11:54:12 +02:00
Ondřej Budai
5315264f2e packer: pin the vector version
See the comment inline.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-06-07 09:08:22 +02:00
Sanne Raymaekers
1ded72b4dc templates/packer: Set region in vector config
Vector 0.21 needs region set otherwise the healthcheck will
fail.
2022-04-19 13:24:33 +02:00
Sanne Raymaekers
2023f7731d worker: Support client_credentials grant type in client
This will allow us to use the service accounts which work against
identity.api.openshift.com. These are much easier to manage, especially
with the new multi-tenancy, as there's a single page to create/expire
them across an account.

They also have the added benefit of not expiring automatically when
they're not used like offline tokens, and immediate expiration when
desired.
2022-03-21 09:43:43 +01:00
Ondřej Budai
9ca74694a7 packer: use unique name tag for Fedora workers
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-16 12:58:05 +01:00
Ondřej Budai
418ae32cf8 packer: fix the secret ID variable in get_koji_creds.sh
Oops, we should probably start testing this.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-14 10:27:28 +01:00
Ondřej Budai
424a741de6 packer: make subscribing optional
We don't want to subscribe Fedora.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-13 22:31:40 +01:00
Ondřej Budai
c46376aea2 packer: add support for koji credentials
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-13 09:08:11 +01:00
Ondřej Budai
2dd5ae7bca packer: skip retrieving of creds if their ARN is not specified
So we can have workers without public cloud creds.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-13 09:08:11 +01:00
Ondřej Budai
4c0ba50ea1 packer: remove config tinkering from worker_service.sh
Let's set each cloud section of the config in the respective cloud script.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-13 09:08:11 +01:00
Ondřej Budai
2813507ac9 packer: split worker_external_creds.sh into one script per cloud
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-13 09:08:11 +01:00
Ondřej Budai
2e7815bf53 packer: move worker-config creation to ansible
I think it untangles the initialization a bit and allows me to do some more
refactorings.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-13 09:08:11 +01:00
Ondřej Budai
72de1b3bbe packer: don't save the AMIs on PRs
This should save us a ton of resources as we don't use AMIs from PRs.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-11 09:06:43 +01:00
Ondřej Budai
ad15179faf packer: build Fedora images
The decision logic which jobs to run is quite confusing but that's how we
roll for now:

Jenkins builds RHEL images only on main
Schutzbot builds RHEL images only in PRs
Schutzbot builds Fedora images on both PRs and on main

To achieve this, the commit re-enables running Packer on main on Schutzbot.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-11 09:06:43 +01:00
Ondřej Budai
ec070612ff packer: remove RHEL and x86_64-specific bits
Arch was easy.

For passing the repository distribution and osbuild_commit (it can be
different for each distro), I decided to go in the way of ansible
inventory directories. It adds a bit of structure but I think it's
the most clean solution.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-11 09:06:43 +01:00
Ondřej Budai
cd394bf67d packer: add default to aws auth variables
So you don't have to pass these if packer is supposed to find them
on its own (instance profile, local profile).

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-11 09:06:43 +01:00
Ondřej Budai
4ae71d3f3d packer: move all RHEL-specific options to a source block
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-11 09:06:43 +01:00
Ondřej Budai
22ec89f956 packer: add more tags identifying the image
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-11 09:06:43 +01:00