particle-os/debian-forge-composer
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
6544 commits 1 branch 0 tags 688 MiB
Commit graph

6544 commits

This branch
This branch
All branches
Author SHA1 Message Date
Sanne Raymaekers
2a621521a8 osbuildexecutor/aws.ec2: set hostname of executor via cloud-init 
This way much more of the journal will be captured under the new
hostname.
 2024-06-25 10:58:10 +02:00
Sanne Raymaekers
14052e25db Revert "osbuildexecutor: write the job-id intro control.json" 
This reverts commit c13d37c3c0.
 2024-06-25 10:58:10 +02:00
Sanne Raymaekers
4853bf3ec0 Revert "osbuild-worker-executor: job-id in control.json as hostname" 
This reverts commit fc1d1c3b8f.
 2024-06-25 10:58:10 +02:00
Sanne Raymaekers
ae4467ab0d internal/awscloud: retry CreateFleet 
When receiving the "UnfillableCapacity" error from CreateFleet, retry
the request with an OnDemand instance.
 2024-06-24 12:50:37 +02:00
Florian Schüller
55c5602f91 osbuild-worker-executor/main_test: use random port for tests 
this for sure is racy but better than colliding with other tests
with a fixed port for sure
 2024-06-24 09:18:44 +02:00
Alexander Todorov
06d0bb63f6 Use latest 9.5 for RHEL upgrade tests 
Related: RHELMISC-5615
 2024-06-19 13:53:33 +02:00
Florian Schüller
6093492c33 golangci: fix integration tests for images 
images also calls this linter but with path-prefix
 2024-06-19 09:12:53 +02:00
Alexander Todorov
0361013b66 Remove rhel-8.10 jobs from test matrix 
Related: COMPOSER-2226
 2024-06-17 13:14:57 +02:00
dependabot[bot]
b8a2592719 build(deps): bump the go-deps group across 1 directory with 15 updates 
Bumps the go-deps group with 12 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) | `1.41.0` | `1.42.0` |
| [github.com/Azure/azure-sdk-for-go/sdk/azidentity](https://github.com/Azure/azure-sdk-for-go) | `1.5.2` | `1.6.0` |
| [github.com/Azure/go-autorest/autorest/azure/auth](https://github.com/Azure/go-autorest) | `0.5.12` | `0.5.13` |
| [github.com/BurntSushi/toml](https://github.com/BurntSushi/toml) | `1.3.2` | `1.4.0` |
| [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) | `1.53.6` | `1.54.2` |
| [github.com/getsentry/sentry-go](https://github.com/getsentry/sentry-go) | `0.27.0` | `0.28.1` |
| [github.com/gophercloud/gophercloud](https://github.com/gophercloud/gophercloud) | `1.11.0` | `1.12.0` |
| [github.com/hashicorp/go-retryablehttp](https://github.com/hashicorp/go-retryablehttp) | `0.7.6` | `0.7.7` |
| [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go) | `0.1.420` | `0.1.425` |
| [github.com/osbuild/images](https://github.com/osbuild/images) | `0.65.0` | `0.66.0` |
| [github.com/spf13/cobra](https://github.com/spf13/cobra) | `1.8.0` | `1.8.1` |
| [github.com/vmware/govmomi](https://github.com/vmware/govmomi) | `0.37.2` | `0.37.3` |



Updates `cloud.google.com/go/storage` from 1.41.0 to 1.42.0
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.41.0...spanner/v1.42.0)

Updates `github.com/Azure/azure-sdk-for-go/sdk/azidentity` from 1.5.2 to 1.6.0
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/internal/v1.5.2...sdk/azcore/v1.6.0)

Updates `github.com/Azure/go-autorest/autorest/azure/auth` from 0.5.12 to 0.5.13
- [Release notes](https://github.com/Azure/go-autorest/releases)
- [Changelog](https://github.com/Azure/go-autorest/blob/main/CHANGELOG.md)
- [Commits](https://github.com/Azure/go-autorest/compare/autorest/azure/auth/v0.5.12...autorest/azure/auth/v0.5.13)

Updates `github.com/BurntSushi/toml` from 1.3.2 to 1.4.0
- [Release notes](https://github.com/BurntSushi/toml/releases)
- [Commits](https://github.com/BurntSushi/toml/compare/v1.3.2...v1.4.0)

Updates `github.com/aws/aws-sdk-go` from 1.53.6 to 1.54.2
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.53.6...v1.54.2)

Updates `github.com/getsentry/sentry-go` from 0.27.0 to 0.28.1
- [Release notes](https://github.com/getsentry/sentry-go/releases)
- [Changelog](https://github.com/getsentry/sentry-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-go/compare/v0.27.0...v0.28.1)

Updates `github.com/gophercloud/gophercloud` from 1.11.0 to 1.12.0
- [Release notes](https://github.com/gophercloud/gophercloud/releases)
- [Changelog](https://github.com/gophercloud/gophercloud/blob/master/CHANGELOG.md)
- [Commits](https://github.com/gophercloud/gophercloud/compare/v1.11.0...v1.12.0)

Updates `github.com/hashicorp/go-retryablehttp` from 0.7.6 to 0.7.7
- [Changelog](https://github.com/hashicorp/go-retryablehttp/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/go-retryablehttp/compare/v0.7.6...v0.7.7)

Updates `github.com/openshift-online/ocm-sdk-go` from 0.1.420 to 0.1.425
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.420...v0.1.425)

Updates `github.com/osbuild/images` from 0.65.0 to 0.66.0
- [Release notes](https://github.com/osbuild/images/releases)
- [Commits](https://github.com/osbuild/images/compare/v0.65.0...v0.66.0)

Updates `github.com/spf13/cobra` from 1.8.0 to 1.8.1
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.8.0...v1.8.1)

Updates `github.com/vmware/govmomi` from 0.37.2 to 0.37.3
- [Release notes](https://github.com/vmware/govmomi/releases)
- [Changelog](https://github.com/vmware/govmomi/blob/main/CHANGELOG.md)
- [Commits](https://github.com/vmware/govmomi/compare/v0.37.2...v0.37.3)

Updates `golang.org/x/oauth2` from 0.20.0 to 0.21.0
- [Commits](https://github.com/golang/oauth2/compare/v0.20.0...v0.21.0)

Updates `golang.org/x/sys` from 0.20.0 to 0.21.0
- [Commits](https://github.com/golang/sys/compare/v0.20.0...v0.21.0)

Updates `google.golang.org/api` from 0.181.0 to 0.183.0
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.181.0...v0.183.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azidentity
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/Azure/go-autorest/autorest/azure/auth
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/BurntSushi/toml
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/getsentry/sentry-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/gophercloud/gophercloud
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/hashicorp/go-retryablehttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/osbuild/images
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/spf13/cobra
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/vmware/govmomi
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-17 09:54:49 +02:00
Michael Vogt
c13d37c3c0 osbuildexecutor: write the job-id intro control.json 
Set the jobID into control.json so that the osbuild-worker-executor
can set the hostname based on the jobID.
 2024-06-14 15:02:08 +02:00
Michael Vogt
aa3d70a429 osbuildexecutor: tweak RunOSBuild() signature and use opts 
Introduce a new OsbuildOpts struct to make the API a bit easier
to extend and use in the packages.

Also add a new `JobID` field in the `OsbuildOpts`.
 2024-06-14 15:02:08 +02:00
Michael Vogt
fc1d1c3b8f osbuild-worker-executor: job-id in control.json as hostname 
This commit adds support to set the hostname to the job-id that
is part of the control.json.
 2024-06-14 15:02:08 +02:00
Sanne Raymaekers
2e31ea50aa cloud/awscloud: use instance requirements when creating secure instance 2024-06-14 10:59:58 +02:00
dependabot[bot]
7ce03144da build(deps): bump golangci/golangci-lint-action from 5 to 6 
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 5 to 6.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v5...v6)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-13 15:44:16 +02:00
Sanne Raymaekers
4bb61da37e Revert "prometheus: active worker gauge" 
This reverts commit 68bc8e0c88.
 2024-06-12 17:20:01 +02:00
Sanne Raymaekers
55439fc6d3 templates/dashboards: remove active worker count 
It's misleading since it counts the amount of workers that have
registered to the current composer pods, it doesn't actually keep track
of the active workers.

Remove it and keep the worker-api stats as a proxy for active workers.
 2024-06-12 17:20:01 +02:00
Michael Vogt
fedbd72d57 osbuildexecutor: allow file type tar.TypeGNUSparse too 
We need to allow files of type `tar.TypeGNUSparse` in the result
that we get from the osbuild-worker-executor too.
 2024-06-12 16:31:44 +02:00
Sanne Raymaekers
971e1df148 internal/osbuildexecutor: verify expected input from handleBuild 2024-06-12 13:10:52 +02:00
Sanne Raymaekers
8a5f486715 osbuildexecutor: wait until worker-executor closes the connection 
Otherwise the client will try to fetch the output archive before the
build output is archived on the worker-executor side.
 2024-06-12 11:36:30 +02:00
Sanne Raymaekers
db43ec8e60 osbuildexecutor: move tests to separate _test package 2024-06-12 11:36:30 +02:00
Michael Vogt
984f51feb8 osbuildexecutor: add validateOutputArchive() and run before extract 
The tar file from the `osbuild-worker-executor` is potentially
tainted. Ensure we validate and only extract if it harmless.
 2024-06-12 11:36:30 +02:00
Michael Vogt
22769305d8 osbuild-worker-executor: fix tar warning and log unexpected output 
This commit fixes a warning from tar that the archive cannot contain
itself. It also makes any tar output a warning (maybe even an error?)
as we do not expect anything from the tar command. The test is updated
to also check this.
 2024-06-12 11:36:30 +02:00
Sanne Raymaekers
1f52150ff1 internal/osbulid-executor/aws-ec2: return more verbose errors 
In RunOsbuild return more verbose errors where the error doesn't
originate from one of the local helper functions.
 2024-06-12 11:36:30 +02:00
Sanne Raymaekers
840f46b07f osbuild-worker-executor: expect osbuild-store as store directory 2024-06-12 11:36:30 +02:00
Sanne Raymaekers
7d7bce76c0 templates/packer: use osbuild-worker-executor 2024-06-12 11:36:30 +02:00
Sanne Raymaekers
a769d73745 osbuild-worker: print stacktrace in case osbuild job panics 2024-06-12 11:36:30 +02:00
Sanne Raymaekers
4df04643ab internal/osbuildexecutor/aws-ec2: use osbuild-worker-executor 
Adds some unit tests as well.
 2024-06-12 11:36:30 +02:00
Sanne Raymaekers
9d62f01c7d test/worker-executor: switch to oaas or osbuild-worker-executor 2024-06-12 11:36:30 +02:00
Sanne Raymaekers
f18293c2f1 osbuild-worker-executor: adapt worker-executor to osbuild-composer 2024-06-12 11:36:30 +02:00
Tomáš Hozza
591ff8c46c Ship el8 repos without aux key on el10 
The RHEL-8 auxiliary key uses SHA-1 in its signature. To enable RHEL-8
cross-builds on RHEL-10, we can't include the RHEL-8 auxiliary key in
the repo definitions.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2024-06-12 11:29:56 +02:00
Tomáš Hozza
8c54226793 Test manifests: update RH release key and regenerate manifests 
Regenerate all manifests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2024-06-12 11:29:56 +02:00
Tomáš Hozza
ab4dd4995c Remove F37 and F38 test manifest, because they are EOL 
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2024-06-12 11:29:56 +02:00
Tomáš Hozza
ff433563c0 Remove c8s test manifest, because it is EOL 
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2024-06-12 11:29:56 +02:00
Tomáš Hozza
30f221401a Repositories: use RH GPG release key signed using SHA256 
The RPM GPG release key used by Red Hat to sign its content used to be
signed using SHA-1. SHA-1 is no longer accepted on RHEL-10 / c10s and as
a result, such key can't be imported during image build. The RH GPG
release key has been resigned using SHA256 some time ago. Let's use this
version of the key for all RHEL repositories.

The key is taken from:
https://access.redhat.com/security/team/key

Specifically:
https://access.redhat.com/security/data/fd431d51.txt

The second key (auxiliary key 3) was not changed.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2024-06-12 11:29:56 +02:00
schutzbot
08fe4ef4fb Post release version bump 
[skip ci]
 2024-06-12 08:15:23 +00:00
Sanne Raymaekers
7e89085808 templates/openshift/composer: remove maintenance cronjob 
This is now deployed from a separate tempate.
 2024-06-12 09:42:27 +02:00
Michael Vogt
2704b18663 obuild-worker: extract workerClientErrorFrom() helper and add tests 
Tiny commit to extract a helper from DepsolveJobImpl.Run() that
can then be unit tested.

This should help with https://github.com/osbuild/images/issues/727
 2024-06-11 10:42:00 +02:00
Achilleas Koutsou
7abcd279eb go.mod: bump osbuild/images to v0.65.0 2024-06-07 17:11:16 +02:00
Alexander Todorov
c80ca0a64d Temporary disable 2 regression tests on 9.5 
Related: COMPOSER-2264
 2024-06-07 10:31:54 +02:00
Alexander Todorov
ee1f70ec2e Add self-signed CA files to system trust store 
in the hope that dnf in 9.5 will actually trust them and not fail
 2024-06-07 10:31:54 +02:00
Alexander Todorov
3860f03d68 Update distro aliases in test 2024-06-07 10:31:54 +02:00
Alexander Todorov
e792bd3c13 Update 9.5 repositories in Schutzfile 2024-06-07 10:31:54 +02:00
Alexander Todorov
74421fa647 Generate manifests for RHEL 9.5 2024-06-07 10:31:54 +02:00
Alexander Todorov
537fe1a033 Define 9.5 repositories for gen-manifests 2024-06-07 10:31:54 +02:00
Alexander Todorov
8d0eb8b0cb Replace 9.4 conditions with 9.5 conditions inside test scripts 2024-06-07 10:31:54 +02:00
Alexander Todorov
4bee7cffb9 Replace 9.4 with 9.5 runners 2024-06-07 10:31:54 +02:00
Alexander Todorov
39370abce4 Define rhel-9.5 repos 
Related: COMPOSER-2227
 2024-06-07 10:31:54 +02:00
Michael Vogt
a691df2353 osbuild-worker-executor: fix order of assert.Equal() in tests 
The `assert.Equal()` expects that the "expected" value is put
first. Which is not what I'm used to. It's also slightly inconsistent
because `assert.EqualError()` expects the "actual" err first and
then the expected string. But this commit is not about ranting :)

This commit fixes the order in the tests assert.Equal() so that
mismatches actually are displayed correctly.
 2024-06-07 08:17:32 +02:00
Michael Vogt
61bf0c3235 osbuild-worker: do not use error in clienterror.Error.Details 
This is an alternative/complementary fix for PR#4137. It is very
simple so should be uncontroverisal.

It fixes an issue that @schuellerf discovered, i.e. that when an error
interface is passed into clienterrors.Error.Details the details get
lost because the json.Marshaler will not know how to handler an
error interface.

To find the problematic uses of `error` a custom vet checker was
build in https://github.com/mvo5/osbuild-cvet. With that the
result is:
```
$ go run github.com/mvo5/osbuild-cvet@latest ./...
/home/mvogt/devel/osbuild/osbuild-composer/cmd/osbuild-worker/jobimpl-depsolve.go:93:26: do not pass 'error' to WorkerClientError() details, use error.Error() instead
/home/mvogt/devel/osbuild/osbuild-composer/cmd/osbuild-worker/jobimpl-osbuild.go:404:31: do not pass 'error' to WorkerClientError() details, use error.Error() instead
/home/mvogt/devel/osbuild/osbuild-composer/cmd/osbuild-worker/jobimpl-osbuild.go:519:31: do not pass 'error' to WorkerClientError() details, use error.Error() instead
/home/mvogt/devel/osbuild/osbuild-composer/cmd/osbuild-worker/jobimpl-osbuild.go:556:31: do not pass '[]error' to WorkerClientError() details, use []string instead
```
and once this commit is in no more errors.

Just like PR#4137 this is not perfect because it will not do a
recursive check for the passed argument.
 2024-06-07 01:19:11 +02:00
Michael Vogt
8ebefbdbc9 main: rework the way the mock logger is passed 
Pass the mock logger directly to `run()` instead of mocking
`logrus.New`. Doing the later leads to a data race when multiple
parallel tests modify the (global) `var logrusNew logrus.New`.

Thanks to Tomas Hozza for reporting.
 2024-06-06 21:14:31 +02:00