Commit graph

979 commits

Author SHA1 Message Date
Lukas Zapletal
f2bf47814f deps: upgrade go-systemd from v1 to v22 2025-06-16 17:46:45 +02:00
Michael Vogt
acec29dea2 osbuild-store-dump: use distrofactory instead of importing fedora
This commit removes the direct use of `pkg/distro/fedora` and
uses `distrofactory` instead. This is the more generic interface,
it also helps with PR#1563 as `pkg/distro/fedora` goes away.

Note that I did not test this but @bcl was kind enough to
mention how this is used:
```
The output from this is used in Test_upgrade in
internal/store/json_test.go, the files there are manually
created when we change the format of the store and add a new
parser to internal/store/json.go -- which hasn't happened in
a long time :)
```
I updated the cmd/README with a snippet of the above.
2025-06-04 12:41:39 +02:00
Lukas Zapletal
0e5d8a94f2 cmd: use log in service-maintenance 2025-05-12 15:20:16 +02:00
Lukas Zapletal
46f0a71053 cmd: use log in jobsite 2025-05-12 15:20:16 +02:00
Lukas Zapletal
71117f3ccc cmd: use log in koji 2025-05-12 15:20:16 +02:00
Lukas Zapletal
650e7b64c2 cmd: use log in upload-gcp 2025-05-12 15:20:16 +02:00
Lukas Zapletal
b87d3d78ec docs: add README to cmd/ 2025-05-12 15:20:16 +02:00
Achilleas Koutsou
cf956ff5a6 Delete internal/blueprint/ and import from osbuild/blueprint
Import osbuild/blueprint v1.6.0
2025-04-17 11:17:58 +02:00
Sanne Raymaekers
536b7d95c5 go.mod: update osbuild/images to v0.123.0
Includes modularity support.
2025-03-10 17:56:19 +01:00
Tomáš Hozza
985e582953 osbuild-composer/config: update distro aliases for RHEL 9 and 10
Bump the `rhel-9` and `rhel-10` distro aliases to the next
in-development releases, because we don't plan to land any new releases
in 9.6 or 10.0.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-28 09:35:23 +01:00
Brian C. Lane
0256e09031 lint: Clean up golangci-lint v1.60 complaints
This cleans up the linting results by adding checks for
integer underflow/overflow in several places, suppressing the error in
places where it has been checked, or fixing the types when possible.
2025-02-24 04:20:42 -08:00
Michael Vogt
35167ce83f many: update for new reporegistry.New() api (c.f. pr#1179)
This commit updates osbuild-composer for the new API in images
for the `reporegistry.New()`. The main incompatible change is
that the `/repositories` part is not longer automatically added
inside the library so we need to add it on the call-site.

This needs https://github.com/osbuild/images/pull/1179
2025-02-19 19:19:42 +01:00
Michael Vogt
90f069c2eb many: take repositories from the "images" library
This commit changes the spec file so that most of the repositories
are taken from the `images` library. See images PR#1112 for details.

Note that we still need the -no-auth-keys and the centos-stream
symlinks.
2025-02-13 14:00:45 +01:00
Tomáš Hozza
bb43578d92 osbuild-composer: bump rhel-9 distro alias to rhel-9.6
The alias in the built-in config is meant for on-prem and needs to be
set to the current in-development minor RHEL version.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2025-02-12 12:53:36 +01:00
Achilleas Koutsou
083d84f70c test: update all calls to image-info
Call the script without a path since it's now in the PATH.

Drop the SELinux labelling workaround (chcon) from filesystem.sh and
image_tests.sh.  The packaged tool is already labelled correctly.

Install osbuild-tools instead of osbuild-composer-tests in
regression-old-worker-new-composer.sh to get osbuild-image-info.
2025-01-31 08:22:29 +01:00
Brian C. Lane
3c2b415e53 jobimpl-depsolve: Always return result
Previously it would not return a result if there was an error. This adds
a deferred function that always returns the current contents of result,
and if there is an error it logs it.

Related: RHEL-60125
2025-01-30 08:00:12 +01:00
Brian C. Lane
d8df7e7cd4 worker: Add search job implementation to worker client
This is similar to the depsolve job, and it shares the solver (which
supports locking, as does DNF itself). This will allow searching for
specific package names, names with globs, or names as substrings of
other names using * as the wildcard.

Related: RHEL-60136
2025-01-30 08:00:12 +01:00
Brian C. Lane
84c0f79286 osbuild-worker: Add helper to setup Repo SSL
This same operation will be done in a few places, turn it into a
function.

Related: RHEL-60136
2025-01-30 08:00:12 +01:00
Michael Vogt
145f8e9f8a osbuild-store-dump: port to new images API 2025-01-29 18:03:11 +01:00
Michael Vogt
e91f4ba961 gen-manifests: port to new images API 2025-01-29 18:03:11 +01:00
Michael Vogt
40011e35a2 composer: use reporegistry.New() again (see PR#4378)
This commit changes cmd/composer to use the `reporegistry.New`
call again. This is a preparation for
https://github.com/osbuild/images/pull/1179

And undoes parts of https://github.com/osbuild/osbuild-composer/pull/4378
but that is no longer necessary because in
https://github.com/osbuild/images/pull/946 the error is now passed
on from `reporegistry.New()` in the same way as from `LoadRepositories()`.
2025-01-29 13:27:32 +01:00
Sanne Raymaekers
a44a499553 worker: support worker server target results 2025-01-24 15:26:15 +01:00
Brian C. Lane
73101d2ff2 Fix non-constant log strings
Newer versions of the go compiler (1.24 in this case) fail when running
go test during a mock rebuild of the srpm created by 'make srpm' on
Fedora 42.

Even though we currently don't support go1.24, fix these so they don't
become an issue when we do.
2025-01-21 16:51:20 -08:00
Michael Vogt
af0543d27c many: update images Manifest() API for PR#1107
This updates composer to use the updated API in images around the
seed handling for manifests, see images PR#1107 for details.

Note that this has no semantic changes yet. We could now simplfy
some things because images will auto-seed but that is for a followup.
2025-01-20 09:50:49 +01:00
Florian Schüller
153bcadf2d osbuild-service-maintenance/aws: merge errors
Collect and merge errors, instead of
nesting errors.
In this case we want to continue execution if only one
cleanup fails.
2024-12-10 11:43:51 +01:00
Florian Schüller
87ef1527fc osbuild-service-maintenance/aws: avoid error on empty list
Passing an empty list to `TerminateInstances` causes an
error message, which is not necessary, as there is
nothing to terminate.
2024-12-10 11:43:51 +01:00
Florian Schüller
24e256c225 osbuild-service-maintenance: add test for allTerminated
HMS-3632
2024-12-10 11:43:51 +01:00
Florian Schüller
65b7ee65b2 osbuild-service-maintenance: implement removal of launch templates
Launch templates of instances that are terminated should be removed.
HMS-3632
2024-12-10 11:43:51 +01:00
Florian Schüller
a96ea533c0 osbuild-service-maintenance: implement removal of security groups
Security groups of instances that are terminated should be removed.
HMS-3632
2024-12-10 11:43:51 +01:00
Florian Schüller
7ebe266d3c osbuild-service-maintenance: implement removal on invalid parent
Add a safeguard to ensure secure instances without valid
parent instances are terminated, as they are unnecessary to retain.
Typically, the parent does not exist if the secure instance is
older than 2 hours, but this check provides additional validation.
HMS-3632
2024-12-10 11:43:51 +01:00
Florian Schüller
a7119a4d0f osbuild-service-maintenance/aws: support aws credential file
Support running the maintenance locally with a valid
`~/aws/credentials` file. HMS-3632
2024-12-10 11:43:51 +01:00
Tomáš Hozza
580366d1f3 osbuild-dnf-json-tests: don't set OSTree options for non-OSTree images
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-12-09 09:46:54 +01:00
Lukas Zapletal
e7a7cda3bc cmd: extra env logging for osbuild worker 2024-11-29 11:57:11 +01:00
Michael Vogt
bc7b8355bf worker: report cashes directly to logrus
This is a bit of an RFC commit, I noticed that when we discussed
a crash from the worker we looked at individual message from
syslog/journald for the stacktrace deatils. I was wondering if
having a more direct crash report would be more useful? We can
of course also add more logrus features to flag those with tags
like "crash" or something (I did not do that in this PR, I don't
know much about the operational side, sorry).
2024-11-25 12:02:05 +01:00
Sanne Raymaekers
f672610509 cmd/osbuild-worker: specify hyper v gen for azure images 2024-11-21 11:22:20 +01:00
Lukas Zapletal
03e74e77b2 worker: log proxy setting 2024-11-18 19:33:19 +01:00
Lukas Zapletal
86f903339a worker: parse ostree MTLS proxy early 2024-11-15 10:16:26 +01:00
Lukas Zapletal
2a5d25d9c0 worker: check MTLS config for ostree 2024-11-12 12:12:52 +01:00
Sanne Raymaekers
056b3c5ea6 jobqueue: return if a job was requeued or not 2024-11-07 17:18:48 +01:00
Lukas Zapletal
64f479092d osbuild-worker: use the new ostree resolver API 2024-11-07 16:17:56 +01:00
Florian Schüller
00d3f07d08 Makefile: implement make db-tests
enables the option to run the DB tests locally
that are executed in the github actions
2024-11-06 15:16:42 +01:00
Achilleas Koutsou
af48971981 osbuild-composer: fail weldr init when repos are nil
If weldr tries to initialise when there are no repositories set and
ignore_missing_repos is enabled, return with an error.
2024-11-05 08:21:42 +01:00
Achilleas Koutsou
51287ea57e osbuild-composer/config: new option: ignore_missing_repos
osbuild/images added an error type that's returned when the reporegistry
loader doesn't find any repository configurations to load [1].  This
lets callers decide whether to stop or continue execution based on
whether repository configurations are required.

A new top-level configuration option is added for osbuild-composer that
makes it possible to start the service without having static rpm
repositories configured.  This is useful in certain (SaaS) modes where
build requests specify their own repository configurations.
2024-11-05 08:21:42 +01:00
Sanne Raymaekers
c1b67440c4 cmd/osbuild-service-maintenance: respect dry run
Respect dry run when terminating leftover SIs.
2024-10-28 10:59:25 +01:00
Lukas Zapletal
350ad58c31 worker: use the new resolver API 2024-10-24 11:53:04 +02:00
Sanne Raymaekers
661f39cbb9 cmd/osbuild-service-maintenance: add test for filtering SIs 2024-10-23 10:32:57 +02:00
Sanne Raymaekers
04a5ca6965 cmd/osbuild-service-maintenance: clean up secure instances
Now and then there are leftover secure instances, probably when worker
instances get terminated during builds, this is possible in ASGs. 2
hours as a cutoff should be enough, since the build times out after 60
minutes, and fetching the output archive after 30 minutes, so that
leaves 30 minutes for booting and connection.
2024-10-23 10:32:57 +02:00
Tomáš Hozza
7437770352 composer: don't create RepoRegistry using reporegistry.New()
The `reporegistry.New()` has been enhanced to return an error, in case
there were no repositories loaded. This was to fix the situation in many
unit tests, which were previously not loading any repositories and
silently not running any tests.

This however broke our SaaS deployment, where we actually do not
configure any repositories on the filesystem. As a result,
osbuild-composer started to fail on it.

Workaround this situation in osbuild-composer by reverting to the old
behavior by loading the repo configs separately and then using the
loaded repos (which could be empty map) to initialize the RepoRegistry.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-23 18:51:39 +02:00
Tomáš Hozza
71a12742d4 Worker/osbuild/koji: upload SBOM documents
Extend the Koji target handling in the osbuild job, to also upload SBOM
documents attached to the related depsolve job result.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00
Tomáš Hozza
1c7462b275 Worker/koji-finalize: import uploaded SBOM documents
If the Koji target result contains information about any uploaded SBOM
documents, import them to Koji as part of the finalize task.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2024-09-20 17:02:09 +02:00