particle-os/debian-forge-composer
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
6259 commits 1 branch 0 tags 688 MiB
Commit graph

198 commits

Author SHA1 Message Date
Diaa Sami
631133eabb templates/composer: give access to private quay repos 2022-05-12 10:30:54 +02:00
Diaa Sami
ca83eccc47 templates/composer: add fluentd sidecar 
The sidecar receives logs from the service and forwards them to Splunk
HEC
 2022-05-12 10:30:54 +02:00
Sanne Raymaekers
02debc0cda templates/composer: Parametrize tenants in acl 
This will allow us to specify tenants in the acl per namespace.
 2022-05-10 15:40:38 +02:00
Sanne Raymaekers
1ded72b4dc templates/packer: Set region in vector config 
Vector 0.21 needs region set otherwise the healthcheck will
fail.
 2022-04-19 13:24:33 +02:00
Sanne Raymaekers
11890682b7 templates/composer: Drop unused variables 2022-03-28 12:02:37 +02:00
Sanne Raymaekers
eba355bb60 templates/composer: Remove unused acl claims 
This leaves fedora and consoledot tenants.
 2022-03-28 11:38:48 +02:00
Ondřej Budai
fc86ffd968 container: fix liveness probe 
We don't have permissions to write to /run when running on OpenShift so let's
just use /tmp and change the filename to prevent any conflicts.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-25 14:02:12 +01:00
Sanne Raymaekers
9368b60401 templates/composer: Add prod service accounts owner 2022-03-23 16:43:10 +01:00
Tom Gundersen
d3cd3197c0 container: make liveness probe independent of webserver 
Currently liveness and readiness was treated the same. However, their
behaviour at shutdown is meant to be different. When a service is not read
no new connections are made to it, and when a service is not live it can be
cleaned up.

By considering our service live if and only if it listens to HTTP requests we
don't have the opportunity to clean up after we stop listening to new requests.

Leave readiness probes as they are, and instead use a file in the filesystem to
indicate when the service is live. It is created before composer is spawned and
deleted once composer exits.
 2022-03-22 14:17:37 +01:00
Sanne Raymaekers
f0a17d19f0 templates/composer: Add stage service accounts owner 2022-03-21 12:57:32 +01:00
Sanne Raymaekers
2023f7731d worker: Support client_credentials grant type in client 
This will allow us to use the service accounts which work against
identity.api.openshift.com. These are much easier to manage, especially
with the new multi-tenancy, as there's a single page to create/expire
them across an account.

They also have the added benefit of not expiring automatically when
they're not used like offline tokens, and immediate expiration when
desired.
 2022-03-21 09:43:43 +01:00
Ondřej Budai
9ca74694a7 packer: use unique name tag for Fedora workers 
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-16 12:58:05 +01:00
Gianluca Zuccarelli
19e2fb7fb5 template: composer dashboard queries 
Tidy up the queries for the composer dashboard
and making them more readable in grafana. Additionally
add some fallback values for when empty query results
are returned from prometheus.
 2022-03-14 16:11:05 +01:00
Gianluca Zuccarelli
1f2fd8cb76 templates: worker depsolve error display 
Fix the display of the depsolve error rate
panel. The panel had an incorrect min value of
3 (or 300%).
 2022-03-14 16:11:05 +01:00
Ondřej Budai
418ae32cf8 packer: fix the secret ID variable in get_koji_creds.sh 
Oops, we should probably start testing this.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-14 10:27:28 +01:00
Ondřej Budai
424a741de6 packer: make subscribing optional 
We don't want to subscribe Fedora.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-13 22:31:40 +01:00
Ondřej Budai
c46376aea2 packer: add support for koji credentials 
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-13 09:08:11 +01:00
Ondřej Budai
2dd5ae7bca packer: skip retrieving of creds if their ARN is not specified 
So we can have workers without public cloud creds.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-13 09:08:11 +01:00
Ondřej Budai
4c0ba50ea1 packer: remove config tinkering from worker_service.sh 
Let's set each cloud section of the config in the respective cloud script.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-13 09:08:11 +01:00
Ondřej Budai
2813507ac9 packer: split worker_external_creds.sh into one script per cloud 
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-13 09:08:11 +01:00
Ondřej Budai
2e7815bf53 packer: move worker-config creation to ansible 
I think it untangles the initialization a bit and allows me to do some more
refactorings.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-13 09:08:11 +01:00
Ondřej Budai
72de1b3bbe packer: don't save the AMIs on PRs 
This should save us a ton of resources as we don't use AMIs from PRs.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-11 09:06:43 +01:00
Ondřej Budai
ad15179faf packer: build Fedora images 
The decision logic which jobs to run is quite confusing but that's how we
roll for now:

Jenkins builds RHEL images only on main
Schutzbot builds RHEL images only in PRs
Schutzbot builds Fedora images on both PRs and on main

To achieve this, the commit re-enables running Packer on main on Schutzbot.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-11 09:06:43 +01:00
Ondřej Budai
ec070612ff packer: remove RHEL and x86_64-specific bits 
Arch was easy.

For passing the repository distribution and osbuild_commit (it can be
different for each distro), I decided to go in the way of ansible
inventory directories. It adds a bit of structure but I think it's
the most clean solution.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-11 09:06:43 +01:00
Ondřej Budai
cd394bf67d packer: add default to aws auth variables 
So you don't have to pass these if packer is supposed to find them
on its own (instance profile, local profile).

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-11 09:06:43 +01:00
Ondřej Budai
4ae71d3f3d packer: move all RHEL-specific options to a source block 
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-11 09:06:43 +01:00
Ondřej Budai
22ec89f956 packer: add more tags identifying the image 
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-11 09:06:43 +01:00
Ondřej Budai
7301ea6b9d packer: use newer (=faster) instances 
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-11 09:06:43 +01:00
Ondřej Budai
8664c1449a packer: reuse the build user for the ansible provisioner 
We want to build multiple images at once and some of them use a different user.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-11 09:06:43 +01:00
Ondřej Budai
e45578d3b0 packer: remove the ami_id variable 
We want to build multiple images at once so they have to be defined elsewhere.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-11 09:06:43 +01:00
Ondřej Budai
5ecbfbad9e packer: rename composer.pkr.hcl to worker.pkr.hcl 
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-11 09:06:43 +01:00
Sanne Raymaekers
63a0bbc1f2 cmd/osbuild-worker: Configure s3 bucket on the worker itself 
Parameterize aws section of worker config. If credentials is empty,
the iam role will be used.
 2022-03-08 21:58:11 +01:00
Ondřej Budai
2ea2e9be09 templates/composer: give access to Fedora org 
We will be using both offline tokens (account_id) and service accounts
(rh-org-id) for now.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-08 13:06:35 +01:00
Ondřej Budai
37181eb995 templates/composer: add tenant_provider_fields 
account_id is for https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token

rh-org-id is for https://identity.api.openshift.com/auth/realms/rhoas/protocol/openid-connect/token

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-03-08 12:07:00 +01:00
Sanne Raymaekers
55b78bbd36 templates/packer: Remove -u flag from creds mapping script 
We test if specific variables are set, and -u interferes with that.
 2022-03-07 17:11:54 +01:00
Sanne Raymaekers
413a013b91 templates/composer: Parametrize bucket name 2022-03-02 09:56:32 +01:00
Sanne Raymaekers
e56248d3c8 templates: Add production worker account to acl 2022-02-25 16:57:13 +01:00
Sanne Raymaekers
b05723a37e templates/composer: Verify against mass sso and rh sso 2022-02-24 09:48:12 +01:00
Gianluca Zuccarelli
8e8d99336f templates/worker: fix depsolve error rate 
The depsolve error rate had the incorrect query
and was returning the error rate for the build
jobs. This has now been fixed.
 2022-02-22 19:55:14 +00:00
Ondřej Budai
5d304d2957 packer: make the worker image smaller 
This should save us some money. 10 GB is the size of the underlying
RHEL 8.5 AMI so this should be the minimum.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2022-02-18 09:24:07 +01:00
Sanne Raymaekers
a173a3513d tools/appsre-build-worker-packer: Run on subscribed 8.5 machine 2022-02-09 16:54:22 +01:00
Gianluca Zuccarelli
e8d7519c7d templates/dashboard: worker metric queries 
The prometheus queries have been updated with
the correct namepsace for the job metrics
Additionally, this commit fixes some of the
queries to add fallback values when the
query results are returned empty.
 2022-02-09 14:09:50 +01:00
Sanne Raymaekers
a739151c71 Revert "templates: Add dnf-json template" 
This reverts commit 8cb3900dd6.
 2022-02-08 14:05:48 +01:00
Sanne Raymaekers
4956e48a0b service-maintenance: Skip db cleanup 
Let's enable the cloud cleanup first, and then move on to the db.
 2022-02-07 20:42:45 +01:00
Gianluca Zuccarelli
dbf396db2b templates/dashboards: worker error metrics 
Update the grafana dashboard for the workers
to show information on the success rate for
osbuild and depsolve jobs.
 2022-02-07 20:40:37 +01:00
Sanne Raymaekers
8cb3900dd6 templates: Add dnf-json template 2022-02-06 14:48:32 +00:00
sanne
8a8ed14319 templates/dashboards: Fixed grafana uids 
This way we get a nice URL `.../d/image-builder-(composer|worker)`.
 2022-01-19 12:27:33 +01:00
sanne
ef6c5df9fa templates/packer: Make cdn host check less sensitive 2022-01-18 17:00:17 +01:00
sanne
68e98244b9 templates/packer: Correct priority for worker rpms 
Lower priority means higher, currently the images built through AppSRE's
infra install the worker from epel.
 2022-01-17 14:30:11 +01:00
sanne
3c729be3c5 tools/appsre-build-worker-packer: Add image_users variable 
packer will share the ami with those users.
 2022-01-11 14:30:19 +01:00