particle-os/debian-forge-composer
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
6572 commits 1 branch 0 tags 688 MiB
Commit graph

6572 commits

This branch
This branch
All branches
Author SHA1 Message Date
Florian Schüller
c36367d4cc makefile: implement make lint 2024-07-04 17:52:44 +02:00
Sanne Raymaekers
786f44e7e7 templates/dashboards: human readable job duration targets 
Also makes the default 40m, which is the new slo target for osbuild
jobs.
 2024-07-04 12:46:19 +02:00
Sanne Raymaekers
3b3ffe0d08 internal/prometheus: more human-readable time buckets 
Let's make them slightly easier to query and reason about. From 100 ms
to 24 hours.
 2024-07-04 11:19:25 +02:00
Nicolás M
7c7fd5e741 Added spot_instance = true for CIV deployments in AWS. 2024-07-04 17:57:08 +09:00
Sanne Raymaekers
8bbbcce2e4 Schutzfile: update osbuild 
Includes https://github.com/osbuild/osbuild/pull/1809.
 2024-07-03 20:13:04 +02:00
Sanne Raymaekers
5347c82dc9 tools/koji-compose: deal with rhsm repositories 2024-07-03 16:52:02 +02:00
Sanne Raymaekers
b48b88005c .gitlab-ci.yml: remove rhel-8.4 
8.4 EUS is now EOL, just test on 8.10 from now on.
 2024-07-03 16:52:02 +02:00
Sanne Raymaekers
3ec4e225ab test/data/repositories: switch to cdn for rhel 8.10 and 9.4 2024-07-03 16:52:02 +02:00
Sanne Raymaekers
6483a75971 .gitlab-ci.yml: switch to rhel 8.10 and 9.4 GA 2024-07-03 16:52:02 +02:00
Sanne Raymaekers
6b09c59365 .gitlab-ci: run filesystem test on f40 2024-07-01 14:09:08 +02:00
Sanne Raymaekers
b8cebc17ae tools/image-info: fix _read_inifile_to_dict 
`RawConfigParser.readfp` was deprecated in favour of `.read_file`.
 2024-07-01 14:09:08 +02:00
Sanne Raymaekers
547f74e7db awscloud: try to terminate previous secure instance 
In case the previous executor SI belonging to the worker did not get
shut down properly, attempt to do it again when starting a new one,
otherwise replacing the SG or LT will not work.
 2024-06-28 15:33:08 +02:00
Sanne Raymaekers
899360f200 test/cases/api: fix nginx url 
This snapshot got deleted as well.
 2024-06-28 12:02:24 +02:00
Sanne Raymaekers
3530f2686f test/data/manifests: generate manifests 2024-06-28 12:02:24 +02:00
Sanne Raymaekers
a83c794a95 tools/test-case-generators: update rhel8.10 repos 
Only the newer snapshots have `rhui-azure-rhel8-base-sap-ha`.
 2024-06-28 12:02:24 +02:00
Sanne Raymaekers
ae5f82dc33 tools/test-case-generators: use newer google repo snapshots 2024-06-28 12:02:24 +02:00
schutzbot
b0f25047e5 schutzfile: Update snapshots to 20240626 
Don't upgrade centos-9-stream repos in the Schutzfile, the newer ones
are broken. Considering CI is broken on main right now, that takes
priority, fixing centos-9 can be done in a followup.
 2024-06-28 12:02:24 +02:00
Sanne Raymaekers
2e4793907d .gitlab-ci: remove fedora-38 
Move to fedora 39 or 40 where appropriate.
 2024-06-26 10:56:57 +02:00
Sanne Raymaekers
af73f2eccf templates/packer: make set_executor_hostname executable 
Prevents `worker-executor.service: Failed at step EXEC spawning
/usr/local/libexec/worker-initialization-scripts/set_executor_hostname.sh:
Permission denied`.
 2024-06-26 10:56:57 +02:00
Sanne Raymaekers
791ec07bc2 internal/awscloud: fix cloud-init userdata for secure instance 
The conditional only checked if the cloudwatch group was set, and if it
wasn't, the hostname variable wouldn't be set either. So the executor
would try to look for a hostname but not find any.
 2024-06-26 10:56:57 +02:00
schutzbot
beebf3cec8 Post release version bump 
[skip ci]
 2024-06-26 08:18:04 +00:00
Sanne Raymaekers
ca19a5b5ed cloudapi/v2: add test for errors 2024-06-25 17:27:28 +02:00
Sanne Raymaekers
86fd957c9d cloudapi/v2: add details for non-service errors 
In case Echo throws an error, or in case even echo doesn't manage to
catch an underlying error and wrap it, return as many details as
possible.
 2024-06-25 17:27:28 +02:00
Sanne Raymaekers
39d2068d31 cloudapi/v2: detach HTTPErrorHandler from Server struct 2024-06-25 17:27:28 +02:00
Sanne Raymaekers
135dd94de5 cloudapi/v2: include details in case internal error is set 
In case details aren't explicitly given, and the internal error is set,
include the internal error message in the details.
 2024-06-25 17:27:28 +02:00
Lukas Zapletal
5ce8f65a58 cloudapi: propagate operation/external id 
Signed-off-by: Lukas Zapletal <lzap+git@redhat.com>
 2024-06-25 13:58:53 +02:00
Lukas Zapletal
f3c0daebbf cmd/osbuild-composer: journald support 2024-06-25 13:58:53 +02:00
Lukas Zapletal
e68449404f DEPLOYING/HACKING.md: improve onboarding build info 2024-06-25 13:58:53 +02:00
Sanne Raymaekers
2a621521a8 osbuildexecutor/aws.ec2: set hostname of executor via cloud-init 
This way much more of the journal will be captured under the new
hostname.
 2024-06-25 10:58:10 +02:00
Sanne Raymaekers
14052e25db Revert "osbuildexecutor: write the job-id intro control.json" 
This reverts commit c13d37c3c0.
 2024-06-25 10:58:10 +02:00
Sanne Raymaekers
4853bf3ec0 Revert "osbuild-worker-executor: job-id in control.json as hostname" 
This reverts commit fc1d1c3b8f.
 2024-06-25 10:58:10 +02:00
Sanne Raymaekers
ae4467ab0d internal/awscloud: retry CreateFleet 
When receiving the "UnfillableCapacity" error from CreateFleet, retry
the request with an OnDemand instance.
 2024-06-24 12:50:37 +02:00
Florian Schüller
55c5602f91 osbuild-worker-executor/main_test: use random port for tests 
this for sure is racy but better than colliding with other tests
with a fixed port for sure
 2024-06-24 09:18:44 +02:00
Alexander Todorov
06d0bb63f6 Use latest 9.5 for RHEL upgrade tests 
Related: RHELMISC-5615
 2024-06-19 13:53:33 +02:00
Florian Schüller
6093492c33 golangci: fix integration tests for images 
images also calls this linter but with path-prefix
 2024-06-19 09:12:53 +02:00
Alexander Todorov
0361013b66 Remove rhel-8.10 jobs from test matrix 
Related: COMPOSER-2226
 2024-06-17 13:14:57 +02:00
dependabot[bot]
b8a2592719 build(deps): bump the go-deps group across 1 directory with 15 updates 
Bumps the go-deps group with 12 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) | `1.41.0` | `1.42.0` |
| [github.com/Azure/azure-sdk-for-go/sdk/azidentity](https://github.com/Azure/azure-sdk-for-go) | `1.5.2` | `1.6.0` |
| [github.com/Azure/go-autorest/autorest/azure/auth](https://github.com/Azure/go-autorest) | `0.5.12` | `0.5.13` |
| [github.com/BurntSushi/toml](https://github.com/BurntSushi/toml) | `1.3.2` | `1.4.0` |
| [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) | `1.53.6` | `1.54.2` |
| [github.com/getsentry/sentry-go](https://github.com/getsentry/sentry-go) | `0.27.0` | `0.28.1` |
| [github.com/gophercloud/gophercloud](https://github.com/gophercloud/gophercloud) | `1.11.0` | `1.12.0` |
| [github.com/hashicorp/go-retryablehttp](https://github.com/hashicorp/go-retryablehttp) | `0.7.6` | `0.7.7` |
| [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go) | `0.1.420` | `0.1.425` |
| [github.com/osbuild/images](https://github.com/osbuild/images) | `0.65.0` | `0.66.0` |
| [github.com/spf13/cobra](https://github.com/spf13/cobra) | `1.8.0` | `1.8.1` |
| [github.com/vmware/govmomi](https://github.com/vmware/govmomi) | `0.37.2` | `0.37.3` |



Updates `cloud.google.com/go/storage` from 1.41.0 to 1.42.0
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.41.0...spanner/v1.42.0)

Updates `github.com/Azure/azure-sdk-for-go/sdk/azidentity` from 1.5.2 to 1.6.0
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/internal/v1.5.2...sdk/azcore/v1.6.0)

Updates `github.com/Azure/go-autorest/autorest/azure/auth` from 0.5.12 to 0.5.13
- [Release notes](https://github.com/Azure/go-autorest/releases)
- [Changelog](https://github.com/Azure/go-autorest/blob/main/CHANGELOG.md)
- [Commits](https://github.com/Azure/go-autorest/compare/autorest/azure/auth/v0.5.12...autorest/azure/auth/v0.5.13)

Updates `github.com/BurntSushi/toml` from 1.3.2 to 1.4.0
- [Release notes](https://github.com/BurntSushi/toml/releases)
- [Commits](https://github.com/BurntSushi/toml/compare/v1.3.2...v1.4.0)

Updates `github.com/aws/aws-sdk-go` from 1.53.6 to 1.54.2
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.53.6...v1.54.2)

Updates `github.com/getsentry/sentry-go` from 0.27.0 to 0.28.1
- [Release notes](https://github.com/getsentry/sentry-go/releases)
- [Changelog](https://github.com/getsentry/sentry-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-go/compare/v0.27.0...v0.28.1)

Updates `github.com/gophercloud/gophercloud` from 1.11.0 to 1.12.0
- [Release notes](https://github.com/gophercloud/gophercloud/releases)
- [Changelog](https://github.com/gophercloud/gophercloud/blob/master/CHANGELOG.md)
- [Commits](https://github.com/gophercloud/gophercloud/compare/v1.11.0...v1.12.0)

Updates `github.com/hashicorp/go-retryablehttp` from 0.7.6 to 0.7.7
- [Changelog](https://github.com/hashicorp/go-retryablehttp/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/go-retryablehttp/compare/v0.7.6...v0.7.7)

Updates `github.com/openshift-online/ocm-sdk-go` from 0.1.420 to 0.1.425
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.420...v0.1.425)

Updates `github.com/osbuild/images` from 0.65.0 to 0.66.0
- [Release notes](https://github.com/osbuild/images/releases)
- [Commits](https://github.com/osbuild/images/compare/v0.65.0...v0.66.0)

Updates `github.com/spf13/cobra` from 1.8.0 to 1.8.1
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.8.0...v1.8.1)

Updates `github.com/vmware/govmomi` from 0.37.2 to 0.37.3
- [Release notes](https://github.com/vmware/govmomi/releases)
- [Changelog](https://github.com/vmware/govmomi/blob/main/CHANGELOG.md)
- [Commits](https://github.com/vmware/govmomi/compare/v0.37.2...v0.37.3)

Updates `golang.org/x/oauth2` from 0.20.0 to 0.21.0
- [Commits](https://github.com/golang/oauth2/compare/v0.20.0...v0.21.0)

Updates `golang.org/x/sys` from 0.20.0 to 0.21.0
- [Commits](https://github.com/golang/sys/compare/v0.20.0...v0.21.0)

Updates `google.golang.org/api` from 0.181.0 to 0.183.0
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.181.0...v0.183.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azidentity
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/Azure/go-autorest/autorest/azure/auth
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/BurntSushi/toml
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/getsentry/sentry-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/gophercloud/gophercloud
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/hashicorp/go-retryablehttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/osbuild/images
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/spf13/cobra
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/vmware/govmomi
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-17 09:54:49 +02:00
Michael Vogt
c13d37c3c0 osbuildexecutor: write the job-id intro control.json 
Set the jobID into control.json so that the osbuild-worker-executor
can set the hostname based on the jobID.
 2024-06-14 15:02:08 +02:00
Michael Vogt
aa3d70a429 osbuildexecutor: tweak RunOSBuild() signature and use opts 
Introduce a new OsbuildOpts struct to make the API a bit easier
to extend and use in the packages.

Also add a new `JobID` field in the `OsbuildOpts`.
 2024-06-14 15:02:08 +02:00
Michael Vogt
fc1d1c3b8f osbuild-worker-executor: job-id in control.json as hostname 
This commit adds support to set the hostname to the job-id that
is part of the control.json.
 2024-06-14 15:02:08 +02:00
Sanne Raymaekers
2e31ea50aa cloud/awscloud: use instance requirements when creating secure instance 2024-06-14 10:59:58 +02:00
dependabot[bot]
7ce03144da build(deps): bump golangci/golangci-lint-action from 5 to 6 
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action) from 5 to 6.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases)
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v5...v6)

---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-06-13 15:44:16 +02:00
Sanne Raymaekers
4bb61da37e Revert "prometheus: active worker gauge" 
This reverts commit 68bc8e0c88.
 2024-06-12 17:20:01 +02:00
Sanne Raymaekers
55439fc6d3 templates/dashboards: remove active worker count 
It's misleading since it counts the amount of workers that have
registered to the current composer pods, it doesn't actually keep track
of the active workers.

Remove it and keep the worker-api stats as a proxy for active workers.
 2024-06-12 17:20:01 +02:00
Michael Vogt
fedbd72d57 osbuildexecutor: allow file type tar.TypeGNUSparse too 
We need to allow files of type `tar.TypeGNUSparse` in the result
that we get from the osbuild-worker-executor too.
 2024-06-12 16:31:44 +02:00
Sanne Raymaekers
971e1df148 internal/osbuildexecutor: verify expected input from handleBuild 2024-06-12 13:10:52 +02:00
Sanne Raymaekers
8a5f486715 osbuildexecutor: wait until worker-executor closes the connection 
Otherwise the client will try to fetch the output archive before the
build output is archived on the worker-executor side.
 2024-06-12 11:36:30 +02:00
Sanne Raymaekers
db43ec8e60 osbuildexecutor: move tests to separate _test package 2024-06-12 11:36:30 +02:00
Michael Vogt
984f51feb8 osbuildexecutor: add validateOutputArchive() and run before extract 
The tar file from the `osbuild-worker-executor` is potentially
tainted. Ensure we validate and only extract if it harmless.
 2024-06-12 11:36:30 +02:00
Michael Vogt
22769305d8 osbuild-worker-executor: fix tar warning and log unexpected output 
This commit fixes a warning from tar that the archive cannot contain
itself. It also makes any tar output a warning (maybe even an error?)
as we do not expect anything from the tar command. The test is updated
to also check this.
 2024-06-12 11:36:30 +02:00