particle-os/debian-forge-composer
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
6462 commits 1 branch 0 tags 688 MiB
Commit graph

885 commits

Author SHA1 Message Date
Sanne Raymaekers
5a776c5b79 templates/openshift: split worker from composer maintenance 2024-04-25 17:32:21 +02:00
Sanne Raymaekers
0b277937dd worker/server: make worker timeout configurable 2024-04-19 19:56:25 +02:00
Sanne Raymaekers
c9eb60d674 osbuild-worker: fix nil pointer in depsolve job 2024-04-18 12:21:07 +02:00
Sanne Raymaekers
7684dbeedd osbuild-worker: fix mtls credentials injection in depsolve job 2024-04-17 20:27:06 +02:00
Sanne Raymaekers
05a1e8f054 worker: support a proxy for repository mtls configuration 
Allows setting a proxy for the mtls settings for a specific
repository. This is useful when consuming content from the content
service in consoledot.
 2024-04-17 16:17:57 +02:00
Sanne Raymaekers
d6cfd04704 cmd/osbuild-dnf-json-tests: check repo configs 2024-04-17 10:27:08 +02:00
Sanne Raymaekers
98e3892e23 cmd/osbuild-store-dump: pass repoconfigs when serializing manifests 2024-04-17 10:27:08 +02:00
Sanne Raymaekers
3094eb474d cmd/gen-manifests: pass repoconfigs when serializing manifests 2024-04-17 10:27:08 +02:00
Sanne Raymaekers
e0759e01af worker: save repoconfigs in depsolve result 2024-04-17 10:27:08 +02:00
Achilleas Koutsou
809f2544ad deps: update images to v0.54.0 
Update images dependency and adjust import paths for distro package
changes.

Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
 2024-04-11 16:43:40 +02:00
Sanne Raymaekers
53f77368fd osbuild-worker: add support for mtls dnf repo secrets 2024-03-29 14:46:54 +01:00
Sanne Raymaekers
f311adf35d osbuild-jobsite-builder: disable http.Server timeouts 
Let's just rely on the timeouts surrounding the whole step.
 2024-03-21 17:08:07 +01:00
Sanne Raymaekers
d3074fc265 osbuild-jobsite-manager: clean up store archive after use 2024-03-21 15:20:52 +01:00
Sanne Raymaekers
bc17204001 osbuild-jobsite-manager: reseek when necessary 2024-03-21 15:20:52 +01:00
Sanne Raymaekers
c19e9fb3b9 osbuild-jobsite-manager: close writer before sending the store 2024-03-21 15:20:52 +01:00
Sanne Raymaekers
dabe63cb78 osbuild-jobsite: increase populate timeout 
30 seconds can be a bit too little for the entire store.
 2024-03-20 21:58:17 +01:00
Sanne Raymaekers
b4e361e093 osbuild-jobsite-builder: unpack source tarball in populate 2024-03-19 17:07:30 +01:00
Sanne Raymaekers
07598ea65c osbuild-jobsite-manager: pass store to builder 2024-03-19 17:07:30 +01:00
Brian C. Lane
01ba674cac cloudapi: Pass the RepoRegistry to the cloudapi Server 2024-03-11 03:04:54 -07:00
Brian C. Lane
b8967d53bc Move RepoRegistry setup into Composer 
This is so that both the weldr and cloud api's can use it as the source
of their repositories.
 2024-03-11 03:04:54 -07:00
Tomáš Hozza
e7743f17ec Worker: allow configuring executor CloudWatch group 
We need the ability to use different CloudWatch group for the
osbuild-executor on Fedora workers in staging and production
environment.

Extend the worker confguration to allow configuring the CloudWatch group
name used by the osbuild-executor. Extend the secure instance code to
instruct cloud-init via user data to create /tmp/cloud_init_vars file
with the CloudWatch group name in the osbuild-executor instance, to make
it possible for the executor to configure its logging differently based
on the value.

Cover new changes by unit tests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2024-03-08 13:13:44 +01:00
Brian C. Lane
ceddabc395 osbuild-worker: Convert osbuild raw json error to a string for logging 
Without this you get an array of bytes in the logs. Convert the raw json
output from osbuild to a string so there is some hope that it is
readable.
 2024-03-08 01:17:00 +01:00
Sanne Raymaekers
040eec4089 osbuild-worker: allow adding key to aws.ec2 executor 
This is useful during testing to set up the executor machine.
 2024-03-01 19:20:51 +01:00
Gianluca Zuccarelli
f6b76cce31 Update osbuild/images to v0.41.0 
Multiple blueprint fixes:

- Extend the blueprint service customizations to accept services to be
  masked.

- The `storage-path` and `container-transport` fields were removed in
  imagees 41.0 in order to simplify the way local storage containers are
  handled.
 2024-02-29 20:57:39 +01:00
Tomáš Hozza
f57e88d034 osbuild-composer: support setting distro_aliases via ENV variable 
This will allow us to configure the distro aliases in the service by
setting ENV variable.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2024-02-21 12:06:33 +01:00
Diaa Sami
76e686df10 switch to images/pkg/dnfjson and remove internal copy 
COMPOSER-2068
 2024-02-20 15:55:47 +01:00
Sanne Raymaekers
bb43f330b8 cmd/osbuild-jobsite-builder: shutdown http server gracefully 2024-02-16 15:21:20 +01:00
Simon de Vlieger
7533f58fb2 jobsite/manager: turn off compression 
We're seeing some unexpected EOFs in staging deployment; going on a
hunch I've seen these before when gzip gets involved in transfering
large files so let's disable that.
 2024-02-16 11:45:51 +01:00
Sanne Raymaekers
ab6fd9e53e cmd/osbuild-jobsite-builder: actually assign the stdout buffer 
The buffer needs to be assigned to the process' stdout for the buffer to
fill up.
 2024-02-15 18:28:07 +01:00
Simon de Vlieger
b9584099ab jobsite/manager: create export directory 
Be a bit more friendly and actually create the export directory instead
of assuming it exists.
 2024-02-15 16:23:55 +01:00
Sanne Raymaekers
1150f0f27e cmd/osbuild-jobsite: capture osbuild's stdout 
Write osbuild's stdout in the progress step. The manager can just copy
it to stdout and the executor will be able to parse the output into an
osbuild result.
 2024-02-15 14:31:24 +01:00
Diaa Sami
f08d1f6068 composer: send error log messages to sentry 2024-02-14 10:38:42 +01:00
Sanne Raymaekers
e7cadb16af osbuildexecutor: add aws.ec2 executor 
This executor spins up an instance which can only contact the host, and
uses the osbuild-jobsite manager & builder to invoke osbuild.
 2024-02-14 09:54:11 +01:00
Sanne Raymaekers
3db88960c2 cloud/awscloud: add ability to run a secure instance to awscloud 
This instance can only contact the host, and requires this host to be
running on AWS itself with the appropriate IAM role.
 2024-02-14 09:54:11 +01:00
Sanne Raymaekers
05a45ed233 cloud/awscloud: add ec2metadata client 2024-02-14 09:54:11 +01:00
Sanne Raymaekers
e10424de2f osbuildexecutor: introduce osbuildexecutor.Executor interface 
Wrap the current osbuildexecutor.Executor in an interface so it's easier
to add different executors, which for instance can run osbuild in a VM.
 2024-02-14 09:54:11 +01:00
Sanne Raymaekers
9e85050633 osbuild-worker: add osbuild_executor config option 2024-02-14 09:54:11 +01:00
Diaa Sami
c9c51613a4 composer: glitchtip integration 2024-02-13 14:57:57 +01:00
Simon de Vlieger
95b4979d88 jobsite/builder: move process into builder 
Moves the background process into the builder struct in preparation for
using `sync.Once`.
 2024-02-12 12:03:38 +01:00
Simon de Vlieger
3cf2531ae2 jobsite: go-staticcheck appeasement 
Log and errors should not end with punctuation.
 2024-02-12 12:03:38 +01:00
Simon de Vlieger
7b7de50e23 jobsite/builder: enable --json 
Use `--json`; this causes `osbuild` to buffer all output and then print
a huge blob of JSON at the end of the command.
 2024-02-12 12:03:38 +01:00
Simon de Vlieger
748db5410e jobsite/manager: always retry on some errors 
On network layer errors we will retry the request until the timeout
expires.
 2024-02-12 12:03:38 +01:00
Simon de Vlieger
46ace22007 jobsite/manager: rename pipeline and export 
A better descriptive name externally is to use `export` and
`export-file`.
 2024-02-12 12:03:38 +01:00
Simon de Vlieger
42892754fd jobsite/builder: no caching for osbuild. 
As builders are not re-used caching has no use and can likely only
lead to disk-size issues. This can be re-enabled when we have an
outside-of-the-builder cache.
 2024-02-12 12:03:38 +01:00
Simon de Vlieger
b9b476c430 osbuild/builder: check deferred serve. 
Make sure to keep track of the deferred error for serve.
 2024-02-12 12:03:38 +01:00
Simon de Vlieger
323d3a1f9c jobsite/builder: rename. 
Consistency, the agent is now called builder.
 2024-02-12 12:03:38 +01:00
Simon de Vlieger
7548870e91 jobsite/builder: check deferred close. 
Check the deffered close by providing a name return value.
 2024-02-12 12:03:38 +01:00
Simon de Vlieger
c7788798c6 jobsite/builder: wrap HTTP handlers for errors. 
Instead of going directly fatal inside the handlers this commit provides
a wrapper function allowing handlers to instead return an `error`; this
will allow for checking deferred errors.
 2024-02-12 12:03:38 +01:00
Simon de Vlieger
401cb01c8d jobsite/manager: PathEscape URL parameter. 2024-02-12 12:03:38 +01:00
Simon de Vlieger
0bd66cf8e8 jobsite: manager and builder 
WIP
 2024-02-12 12:03:38 +01:00