Commit graph

82 commits

Author SHA1 Message Date
Ondřej Budai
c46376aea2 packer: add support for koji credentials
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-13 09:08:11 +01:00
Ondřej Budai
2dd5ae7bca packer: skip retrieving of creds if their ARN is not specified
So we can have workers without public cloud creds.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-13 09:08:11 +01:00
Ondřej Budai
4c0ba50ea1 packer: remove config tinkering from worker_service.sh
Let's set each cloud section of the config in the respective cloud script.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-13 09:08:11 +01:00
Ondřej Budai
2813507ac9 packer: split worker_external_creds.sh into one script per cloud
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-13 09:08:11 +01:00
Ondřej Budai
2e7815bf53 packer: move worker-config creation to ansible
I think it untangles the initialization a bit and allows me to do some more
refactorings.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-13 09:08:11 +01:00
Ondřej Budai
72de1b3bbe packer: don't save the AMIs on PRs
This should save us a ton of resources as we don't use AMIs from PRs.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-11 09:06:43 +01:00
Ondřej Budai
ad15179faf packer: build Fedora images
The decision logic which jobs to run is quite confusing but that's how we
roll for now:

Jenkins builds RHEL images only on main
Schutzbot builds RHEL images only in PRs
Schutzbot builds Fedora images on both PRs and on main

To achieve this, the commit re-enables running Packer on main on Schutzbot.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-11 09:06:43 +01:00
Ondřej Budai
ec070612ff packer: remove RHEL and x86_64-specific bits
Arch was easy.

For passing the repository distribution and osbuild_commit (it can be
different for each distro), I decided to go in the way of ansible
inventory directories. It adds a bit of structure but I think it's
the most clean solution.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-11 09:06:43 +01:00
Ondřej Budai
cd394bf67d packer: add default to aws auth variables
So you don't have to pass these if packer is supposed to find them
on its own (instance profile, local profile).

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-11 09:06:43 +01:00
Ondřej Budai
4ae71d3f3d packer: move all RHEL-specific options to a source block
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-11 09:06:43 +01:00
Ondřej Budai
22ec89f956 packer: add more tags identifying the image
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-11 09:06:43 +01:00
Ondřej Budai
7301ea6b9d packer: use newer (=faster) instances
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-11 09:06:43 +01:00
Ondřej Budai
8664c1449a packer: reuse the build user for the ansible provisioner
We want to build multiple images at once and some of them use a different user.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-11 09:06:43 +01:00
Ondřej Budai
e45578d3b0 packer: remove the ami_id variable
We want to build multiple images at once so they have to be defined elsewhere.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-11 09:06:43 +01:00
Ondřej Budai
5ecbfbad9e packer: rename composer.pkr.hcl to worker.pkr.hcl
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-11 09:06:43 +01:00
Sanne Raymaekers
63a0bbc1f2 cmd/osbuild-worker: Configure s3 bucket on the worker itself
Parameterize aws section of worker config. If credentials is empty,
the iam role will be used.
2022-03-08 21:58:11 +01:00
Ondřej Budai
2ea2e9be09 templates/composer: give access to Fedora org
We will be using both offline tokens (account_id) and service accounts
(rh-org-id) for now.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-08 13:06:35 +01:00
Ondřej Budai
37181eb995 templates/composer: add tenant_provider_fields
account_id is for https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token

rh-org-id is for https://identity.api.openshift.com/auth/realms/rhoas/protocol/openid-connect/token

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-03-08 12:07:00 +01:00
Sanne Raymaekers
55b78bbd36 templates/packer: Remove -u flag from creds mapping script
We test if specific variables are set, and -u interferes with that.
2022-03-07 17:11:54 +01:00
Sanne Raymaekers
413a013b91 templates/composer: Parametrize bucket name 2022-03-02 09:56:32 +01:00
Sanne Raymaekers
e56248d3c8 templates: Add production worker account to acl 2022-02-25 16:57:13 +01:00
Sanne Raymaekers
b05723a37e templates/composer: Verify against mass sso and rh sso 2022-02-24 09:48:12 +01:00
Gianluca Zuccarelli
8e8d99336f templates/worker: fix depsolve error rate
The depsolve error rate had the incorrect query
and was returning the error rate for the build
jobs. This has now been fixed.
2022-02-22 19:55:14 +00:00
Ondřej Budai
5d304d2957 packer: make the worker image smaller
This should save us some money. 10 GB is the size of the underlying
RHEL 8.5 AMI so this should be the minimum.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-18 09:24:07 +01:00
Sanne Raymaekers
a173a3513d tools/appsre-build-worker-packer: Run on subscribed 8.5 machine 2022-02-09 16:54:22 +01:00
Gianluca Zuccarelli
e8d7519c7d templates/dashboard: worker metric queries
The prometheus queries have been updated with
the correct namepsace for the job metrics
Additionally, this commit fixes some of the
queries to add fallback values when the
query results are returned empty.
2022-02-09 14:09:50 +01:00
Sanne Raymaekers
a739151c71 Revert "templates: Add dnf-json template"
This reverts commit 8cb3900dd6.
2022-02-08 14:05:48 +01:00
Sanne Raymaekers
4956e48a0b service-maintenance: Skip db cleanup
Let's enable the cloud cleanup first, and then move on to the db.
2022-02-07 20:42:45 +01:00
Gianluca Zuccarelli
dbf396db2b templates/dashboards: worker error metrics
Update the grafana dashboard for the workers
to show information on the success rate for
osbuild and depsolve jobs.
2022-02-07 20:40:37 +01:00
Sanne Raymaekers
8cb3900dd6 templates: Add dnf-json template 2022-02-06 14:48:32 +00:00
sanne
8a8ed14319 templates/dashboards: Fixed grafana uids
This way we get a nice URL `.../d/image-builder-(composer|worker)`.
2022-01-19 12:27:33 +01:00
sanne
ef6c5df9fa templates/packer: Make cdn host check less sensitive 2022-01-18 17:00:17 +01:00
sanne
68e98244b9 templates/packer: Correct priority for worker rpms
Lower priority means higher, currently the images built through AppSRE's
infra install the worker from epel.
2022-01-17 14:30:11 +01:00
sanne
3c729be3c5 tools/appsre-build-worker-packer: Add image_users variable
packer will share the ami with those users.
2022-01-11 14:30:19 +01:00
sanne
d08147864a osbuild-service-maintenace: Map AWS secrets 2022-01-11 12:57:02 +01:00
sanne
4797ac281a osbuild-service-maintenance: Rework GCP credentials mapping
Because of the way the gcp secrets are stored for the workers, and how
the mapping from vault to openshift works (unable to map a multiple key
secret into a single json file), there's a bit of juggling required to
get the gcp credentials in the right format.
2022-01-11 12:57:02 +01:00
sanne
71da979c81 tools: AppSRE packer build 2022-01-05 22:13:55 +01:00
Ondřej Budai
8d81da7d7b packer: remove /var/lib/osbuild-composer check
This directory is not used on worker instances. It was a left-over from the
times when this AMI was also used for running composer.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-01-04 16:17:59 +01:00
Ondřej Budai
9d0ae3bc1f packer: add initialization scripts
The worker needs quite a lot of configuration involving secrets. Baking them
in the AMI is just awful so we need to fetch them during the instance startup.

Previously, this was all done using cloud-init. This makes the cloud-init
config huge and it is also very hard to test.

This commit moves all the configuration scripts into the image itself.
Cloud-init still needs to be used to push the secret variables into the
instance. The configuration scripts are run after cloud-init. They pick up
yhe secrets and initialize the worker correctly.

These scripts were adopted from
75b752a1c0
(private repository).

During the adoption, some changes has to be applied to make shellcheck happy.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-01-04 16:17:59 +01:00
Ondřej Budai
5697b43ad6 packer: update to RHEL 8.5
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-01-04 16:17:59 +01:00
sanne
60d4f5a751 composer: Disable artifacts for the service
When backed by a DB, composer has no need of a queue directory.

This also addresses "Error moving artifacts for job" logging noise.

Signed-off-by: sanne <sanne.raymaekers@gmail.com>
2021-12-16 17:04:08 +00:00
Gianluca Zuccarelli
10f34de88b templates: add worker dashboard
Add an initial dashboard for the job metrics.
For now, the dashboard includes graphs and
burn rates for osbuild job duration and depsolve
job duration
2021-12-15 08:52:52 +00:00
sanne
98abdf1902 templates: Max concurrent requests is required for the maintenance job 2021-12-08 10:31:33 +01:00
sanne
4224b2231b templates: CronJob is part of the batch/v1 api 2021-12-07 11:52:49 +01:00
sanne
0379cb5796 templates: Add maintenance cronjob 2021-12-06 22:51:24 +01:00
Alex Njaastad
0731857d6c fix uid 2021-12-03 18:38:50 +00:00
Alex Njaastad
595a6fea70 fix version, error-budget interval 2021-12-03 18:38:50 +00:00
Alex Njaastad
a389dae79d fix slo numbers 2021-12-03 18:38:50 +00:00
Alex Njaastad
72109bb775 more dashboard fixes 2021-12-03 18:38:50 +00:00
Alex Njaastad
79caf7b536 add more panels 2021-12-03 18:38:50 +00:00