particle-os/debian-forge-composer
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
6494 commits 1 branch 0 tags 688 MiB
Commit graph

893 commits

Author SHA1 Message Date
Michael Vogt
95b4a9e250 osbuild-worker-executor: make test output silent again 
Do not use the global logger but pass instead the locally created
logger. This means the test output is silent again.

Sadly using the global logger is difficult because it is a global
resource so replacing it in tests means all tests (that are
potentially run in parallel) will write to it which makes testing
specific log output hard.
 2024-06-06 16:16:33 +02:00
Michael Vogt
e34728b466 osbuild-worker-executor: appease errcheck 2024-06-05 18:26:08 +02:00
Michael Vogt
138bc73e37 osbuild-worker-executor: appease gosec 
Note that gosec IMHO is a bit silly here, the heuristics used are
note very good, i.e. the code is already validating the external
inputs and it's not clear to me that "filepath.Clean()" will help
but it seems to supress the error. I hope gosec provides value
in other places, here it seems to be adding work :/

I also excluded "gosec" from any _test.go files, I do not see
why we should gosec tests?
 2024-06-05 18:26:08 +02:00
Sanne Raymaekers
b0543e89f4 osbuild-worker-executor: fix lint warnings/errors 
The osbuild-composer linting found a bunch of issues that this
commit fixes.
 2024-06-05 18:26:08 +02:00
Michael Vogt
cbb8d79baf c/osbuild-worker-executor: update to match new name 
Update the imports/names to match the new name
"osbuild-worker-executor".
 2024-06-05 18:26:08 +02:00
Michael Vogt
372d9f07dd cmd/osbuild-worker-executor: import verbatim from mvo5/oaas 
This commit imports the repository https://github.com/mvo5/oaas
without keeping the history. The history is largely unimportant
and can be looked up in https://github.com/mvo5/oaas/commits/main
if needed.
 2024-06-05 18:26:08 +02:00
Tomáš Hozza
b01b080565 Update 'rhel-9' distro alias to 'rhel-9.5' 
New releases are landing in 9.5 already, so the on-prem version should
reflect that. The service can and does override this using a
configuration.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2024-06-04 13:03:37 +02:00
Tomáš Hozza
695febf39c Define distro alias for 'rhel-10' 
Set it to RHEL-10.0.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2024-06-04 13:03:37 +02:00
Sanne Raymaekers
5a776c5b79 templates/openshift: split worker from composer maintenance 2024-04-25 17:32:21 +02:00
Sanne Raymaekers
0b277937dd worker/server: make worker timeout configurable 2024-04-19 19:56:25 +02:00
Sanne Raymaekers
c9eb60d674 osbuild-worker: fix nil pointer in depsolve job 2024-04-18 12:21:07 +02:00
Sanne Raymaekers
7684dbeedd osbuild-worker: fix mtls credentials injection in depsolve job 2024-04-17 20:27:06 +02:00
Sanne Raymaekers
05a1e8f054 worker: support a proxy for repository mtls configuration 
Allows setting a proxy for the mtls settings for a specific
repository. This is useful when consuming content from the content
service in consoledot.
 2024-04-17 16:17:57 +02:00
Sanne Raymaekers
d6cfd04704 cmd/osbuild-dnf-json-tests: check repo configs 2024-04-17 10:27:08 +02:00
Sanne Raymaekers
98e3892e23 cmd/osbuild-store-dump: pass repoconfigs when serializing manifests 2024-04-17 10:27:08 +02:00
Sanne Raymaekers
3094eb474d cmd/gen-manifests: pass repoconfigs when serializing manifests 2024-04-17 10:27:08 +02:00
Sanne Raymaekers
e0759e01af worker: save repoconfigs in depsolve result 2024-04-17 10:27:08 +02:00
Achilleas Koutsou
809f2544ad deps: update images to v0.54.0 
Update images dependency and adjust import paths for distro package
changes.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
 2024-04-11 16:43:40 +02:00
Sanne Raymaekers
53f77368fd osbuild-worker: add support for mtls dnf repo secrets 2024-03-29 14:46:54 +01:00
Sanne Raymaekers
f311adf35d osbuild-jobsite-builder: disable http.Server timeouts 
Let's just rely on the timeouts surrounding the whole step.
 2024-03-21 17:08:07 +01:00
Sanne Raymaekers
d3074fc265 osbuild-jobsite-manager: clean up store archive after use 2024-03-21 15:20:52 +01:00
Sanne Raymaekers
bc17204001 osbuild-jobsite-manager: reseek when necessary 2024-03-21 15:20:52 +01:00
Sanne Raymaekers
c19e9fb3b9 osbuild-jobsite-manager: close writer before sending the store 2024-03-21 15:20:52 +01:00
Sanne Raymaekers
dabe63cb78 osbuild-jobsite: increase populate timeout 
30 seconds can be a bit too little for the entire store.
 2024-03-20 21:58:17 +01:00
Sanne Raymaekers
b4e361e093 osbuild-jobsite-builder: unpack source tarball in populate 2024-03-19 17:07:30 +01:00
Sanne Raymaekers
07598ea65c osbuild-jobsite-manager: pass store to builder 2024-03-19 17:07:30 +01:00
Brian C. Lane
01ba674cac cloudapi: Pass the RepoRegistry to the cloudapi Server 2024-03-11 03:04:54 -07:00
Brian C. Lane
b8967d53bc Move RepoRegistry setup into Composer 
This is so that both the weldr and cloud api's can use it as the source
of their repositories.
 2024-03-11 03:04:54 -07:00
Tomáš Hozza
e7743f17ec Worker: allow configuring executor CloudWatch group 
We need the ability to use different CloudWatch group for the
osbuild-executor on Fedora workers in staging and production
environment.

Extend the worker confguration to allow configuring the CloudWatch group
name used by the osbuild-executor. Extend the secure instance code to
instruct cloud-init via user data to create /tmp/cloud_init_vars file
with the CloudWatch group name in the osbuild-executor instance, to make
it possible for the executor to configure its logging differently based
on the value.

Cover new changes by unit tests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2024-03-08 13:13:44 +01:00
Brian C. Lane
ceddabc395 osbuild-worker: Convert osbuild raw json error to a string for logging 
Without this you get an array of bytes in the logs. Convert the raw json
output from osbuild to a string so there is some hope that it is
readable.
 2024-03-08 01:17:00 +01:00
Sanne Raymaekers
040eec4089 osbuild-worker: allow adding key to aws.ec2 executor 
This is useful during testing to set up the executor machine.
 2024-03-01 19:20:51 +01:00
Gianluca Zuccarelli
f6b76cce31 Update osbuild/images to v0.41.0 
Multiple blueprint fixes:

- Extend the blueprint service customizations to accept services to be
  masked.

- The `storage-path` and `container-transport` fields were removed in
  imagees 41.0 in order to simplify the way local storage containers are
  handled.
 2024-02-29 20:57:39 +01:00
Tomáš Hozza
f57e88d034 osbuild-composer: support setting distro_aliases via ENV variable 
This will allow us to configure the distro aliases in the service by
setting ENV variable.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2024-02-21 12:06:33 +01:00
Diaa Sami
76e686df10 switch to images/pkg/dnfjson and remove internal copy 
COMPOSER-2068
 2024-02-20 15:55:47 +01:00
Sanne Raymaekers
bb43f330b8 cmd/osbuild-jobsite-builder: shutdown http server gracefully 2024-02-16 15:21:20 +01:00
Simon de Vlieger
7533f58fb2 jobsite/manager: turn off compression 
We're seeing some unexpected EOFs in staging deployment; going on a
hunch I've seen these before when gzip gets involved in transfering
large files so let's disable that.
 2024-02-16 11:45:51 +01:00
Sanne Raymaekers
ab6fd9e53e cmd/osbuild-jobsite-builder: actually assign the stdout buffer 
The buffer needs to be assigned to the process' stdout for the buffer to
fill up.
 2024-02-15 18:28:07 +01:00
Simon de Vlieger
b9584099ab jobsite/manager: create export directory 
Be a bit more friendly and actually create the export directory instead
of assuming it exists.
 2024-02-15 16:23:55 +01:00
Sanne Raymaekers
1150f0f27e cmd/osbuild-jobsite: capture osbuild's stdout 
Write osbuild's stdout in the progress step. The manager can just copy
it to stdout and the executor will be able to parse the output into an
osbuild result.
 2024-02-15 14:31:24 +01:00
Diaa Sami
f08d1f6068 composer: send error log messages to sentry 2024-02-14 10:38:42 +01:00
Sanne Raymaekers
e7cadb16af osbuildexecutor: add aws.ec2 executor 
This executor spins up an instance which can only contact the host, and
uses the osbuild-jobsite manager & builder to invoke osbuild.
 2024-02-14 09:54:11 +01:00
Sanne Raymaekers
3db88960c2 cloud/awscloud: add ability to run a secure instance to awscloud 
This instance can only contact the host, and requires this host to be
running on AWS itself with the appropriate IAM role.
 2024-02-14 09:54:11 +01:00
Sanne Raymaekers
05a45ed233 cloud/awscloud: add ec2metadata client 2024-02-14 09:54:11 +01:00
Sanne Raymaekers
e10424de2f osbuildexecutor: introduce osbuildexecutor.Executor interface 
Wrap the current osbuildexecutor.Executor in an interface so it's easier
to add different executors, which for instance can run osbuild in a VM.
 2024-02-14 09:54:11 +01:00
Sanne Raymaekers
9e85050633 osbuild-worker: add osbuild_executor config option 2024-02-14 09:54:11 +01:00
Diaa Sami
c9c51613a4 composer: glitchtip integration 2024-02-13 14:57:57 +01:00
Simon de Vlieger
95b4979d88 jobsite/builder: move process into builder 
Moves the background process into the builder struct in preparation for
using `sync.Once`.
 2024-02-12 12:03:38 +01:00
Simon de Vlieger
3cf2531ae2 jobsite: go-staticcheck appeasement 
Log and errors should not end with punctuation.
 2024-02-12 12:03:38 +01:00
Simon de Vlieger
7b7de50e23 jobsite/builder: enable --json 
Use `--json`; this causes `osbuild` to buffer all output and then print
a huge blob of JSON at the end of the command.
 2024-02-12 12:03:38 +01:00
Simon de Vlieger
748db5410e jobsite/manager: always retry on some errors 
On network layer errors we will retry the request until the timeout
expires.
 2024-02-12 12:03:38 +01:00