particle-os/debian-forge-composer
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
6258 commits 1 branch 0 tags 688 MiB
Commit graph

6258 commits

This branch
This branch
All branches
Author SHA1 Message Date
Michael Vogt
614593084c .gitlab-ci.yml: upgade neetle early to workaround RHEL-17890 2023-12-12 14:10:59 +04:00
Sanne Raymaekers
6cdd95a072 README: remove IRC in favour of matrix channel 2023-12-11 18:37:07 +01:00
Xiaofeng Wang
15a94899a8 test: remove persistent log checking on minimal raw test 
Persistent log is not related with minimal raw image itself and
it is not stable. After discuss with team, remove this test
 2023-12-11 15:12:04 +01:00
Sanne Raymaekers
bfad6d50e1 cloudapi/v2: tweak customizations 
Fix verbiage of groups customization, fields which accept an array
should be plural.

Remove the sshkey customization, sshkeys are merged into user
customizations anyway, so users should use the "users" customization
instead.

Since these customizations aren't in use yet, this edit should be fine.

See #3716
 2023-12-08 15:31:56 +01:00
Tomáš Hozza
1afe7d20c8 Adjust code to new version of osbuild/images 
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-12-06 17:32:18 +01:00
dependabot[bot]
016051a4b8 build(deps): bump the go-deps group with 5 updates 
Bumps the go-deps group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) | `1.48.1` | `1.48.13` |
| [github.com/gophercloud/gophercloud](https://github.com/gophercloud/gophercloud) | `1.7.0` | `1.8.0` |
| [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go) | `0.1.385` | `0.1.388` |
| [github.com/osbuild/images](https://github.com/osbuild/images) | `0.18.0` | `0.21.0` |
| [google.golang.org/api](https://github.com/googleapis/google-api-go-client) | `0.152.0` | `0.153.0` |


Updates `github.com/aws/aws-sdk-go` from 1.48.1 to 1.48.13
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.48.1...v1.48.13)

Updates `github.com/gophercloud/gophercloud` from 1.7.0 to 1.8.0
- [Release notes](https://github.com/gophercloud/gophercloud/releases)
- [Changelog](https://github.com/gophercloud/gophercloud/blob/v1.8.0/CHANGELOG.md)
- [Commits](https://github.com/gophercloud/gophercloud/compare/v1.7.0...v1.8.0)

Updates `github.com/openshift-online/ocm-sdk-go` from 0.1.385 to 0.1.388
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.385...v0.1.388)

Updates `github.com/osbuild/images` from 0.18.0 to 0.21.0
- [Release notes](https://github.com/osbuild/images/releases)
- [Commits](https://github.com/osbuild/images/compare/v0.18.0...v0.21.0)

Updates `google.golang.org/api` from 0.152.0 to 0.153.0
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.152.0...v0.153.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/gophercloud/gophercloud
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/osbuild/images
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-12-06 17:32:18 +01:00
Sanne Raymaekers
d3dd83aceb internal/test: return in TestRouteWithReply on empty body 
Check if the response is empty, and return instead of continuing.
 2023-12-06 17:22:36 +01:00
Sanne Raymaekers
fd4a3a941a worker: let client register itself with the worker server 
Sends a status update to the worker server every 5 minutes.

Also fixes a bug where the body the worker client sent would be empty if
it had to refresh the JWT token. Instead of io.Reader use io.ReadSeeker
so the body can be reread to create the second request (after the token
refresh).
 2023-12-06 17:22:36 +01:00
Sanne Raymaekers
794acd8e34 worker: add ability to track workers serverside 
Unresponsive workers (>=1 hour of no status update) are cleaned up.

Several things are enabled by keeping track of workers, in future the
worker server could:
- keep track of how many workers are active
- see if a worker for a specific architecture is available
 2023-12-06 17:22:36 +01:00
Sanne Raymaekers
d784075d31 jobqueue: add ability to track workers 2023-12-06 17:22:36 +01:00
Jakub Rusz
2410b00eb9 rpmbuild: add fedora-40 2023-12-06 16:52:40 +01:00
Paweł Poławski
c1fae1ef7a Makefile: Fix typo in printed project name 2023-12-06 11:38:01 +01:00
dependabot[bot]
5e46230182 build(deps): bump actions/github-script from 6 to 7 
Bumps [actions/github-script](https://github.com/actions/github-script) from 6 to 7.
- [Release notes](https://github.com/actions/github-script/releases)
- [Commits](https://github.com/actions/github-script/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/github-script
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-12-06 09:11:10 +01:00
Paweł Poławski
7deaa293d2 Readme: Fix broken link 
After documentation refactoring "developer guide" link was pointing
to non existing page. This commit updates the link to point to the
latest upstream developer guide page.
 2023-12-04 12:56:59 +01:00
schutzbot
2a6bad7283 Post release version bump 
[skip ci]
 2023-11-29 08:24:21 +00:00
Diaa Sami
6cfa26399f composer: use logrus hook instead of k8s sidecar 
for splunk log forwarding
Fixes COMPOSER-2051
 2023-11-28 12:42:00 +01:00
Diaa Sami
6b6af41e1d Revert "containers/osbuild-composer: wait for fluentd in entrypoint" 
This reverts commit b4cf032239.
No longer needed after removing sidecar COMPOSER-2051
 2023-11-28 12:41:46 +01:00
Jakub Rusz
4050aa92fe test/cases: Additional migration updates 
Mostly addressing comments from review.
 2023-11-27 10:04:47 +01:00
Jakub Rusz
5cbc87eade test/cases: clean up older distros in ostree.sh 2023-11-27 10:04:47 +01:00
Jakub Rusz
0bdc94ea94 test/cases: run upgrade test using new nightlies 2023-11-27 10:04:47 +01:00
Xiaofeng Wang
1cbcba92e1 test: fix repo file for 9.4 and 8.10 and fix check-minimal 2023-11-27 10:04:47 +01:00
Jakub Rusz
8960a51d2f Fix filesystems test 
The new partition rules are in osbuild-composer v94 and higher.
 2023-11-27 10:04:47 +01:00
Jakub Rusz
04ba4b81d8 Switch testing to 8.10 and 9.4 nightly 2023-11-27 10:04:47 +01:00
Jakub Rusz
c77b73a888 test/cases: Add workaround for oscap firewalld rules 
The oscap remediation is failing due to an issue with remediating the
firewall loopback traffic. The remediation fails and bails out early
resulting in a lower score and causing a regression in the test. This
will need to be fixed on the OpenSCAP side, so as a workaround, we can
remove this rule from the remediation.
 2023-11-24 16:29:32 +01:00
Achilleas Koutsou
5b19bd6809 deps: update images to v0.18.0 
Update the images dependency to v0.18.0
This includes a change in the Fedora IoT remote configuration which is
now installed through an RPM instead of being hard-coded in the image
definitions.
 2023-11-22 09:56:08 +01:00
Jakub Rusz
bb76ddb2b1 tests/repositories: Update snapshtos for 9.4 and 8.10 
We need this to udpate CIV testing.
 2023-11-21 13:43:16 +01:00
Brian C. Lane
aca748bc14 Don't Panic in getComposeStatus and skip invalid jobs in fsjobqueue New 
This handles corrupt job json files by skipping them. They still exist,
and errors are logged, but the system keeps working.

If one or more of the json files in /var/lib/osbuild-composer/jobs/
becomes corrupt they can stop the osbuild-composer service from
starting, or stop commands like 'composer-cli compose status' from
working because they quit on the first error and miss any job that
aren't broken.
 2023-11-20 13:34:40 +01:00
Diaa Sami
e969a9dc3c pkg/splunk_logger: make it a module that can be imported seprately 
Fixes COMPOSER-2051
 2023-11-17 18:48:45 +01:00
Achilleas Koutsou
901393d791 test/api: add some greenprints 2023-11-17 16:48:16 +01:00
Achilleas Koutsou
894c7046d9 test: compare upload_statuses[0] with upload_status 
Check that the first element of the upload_statuses array matches the
top-level upload_status.
We only test one upload target for now.
 2023-11-17 16:48:16 +01:00
Achilleas Koutsou
d3921dcc31 test: use upload_targets for edge in api s3.sh 
When making the upload request for edge commit image types, use the new
upload_targets array to define the aws.s3 upload options.
Leave other upload target definitions as is for now to test the old
options.
 2023-11-17 16:48:16 +01:00
Achilleas Koutsou
fbf63f6a1f cloudapi: add upload targets to route handler tests 2023-11-17 16:48:16 +01:00
Achilleas Koutsou
38664d1b64 cloudapi: add GetTarget() test case for pulp.ostree 2023-11-17 16:48:16 +01:00
Gianluca Zuccarelli
b711e302ba cloudapi: add pulp upload target 
Add the pulp.ostree upload target to the cloud API and enable it for
edge/iot commits.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
 2023-11-17 16:48:16 +01:00
Achilleas Koutsou
5a93166f6b cloudapi: support multiple upload statuses in ComposeStatus 
Add the new upload_statuses under the image_status in the result of the
ComposeStatus object.  The first status is also included in the old
top-level 'upload_status' property for backwards compatibility.

Tests are updated to match the new results.
 2023-11-17 16:48:16 +01:00
Achilleas Koutsou
ba2240b578 cloudapi: test GetTargets() 
Test some valid and invalid combinations for the GetTargets() upload
target selection.
Includes tests with and without the upload options for the default
target.
 2023-11-17 16:48:16 +01:00
Achilleas Koutsou
9fc4551fdf cloudapi: check upload targets in request validation 
It is now valid for UploadOptions to be nil but only if there is at
least one UploadTarget defined.
 2023-11-17 16:48:16 +01:00
Achilleas Koutsou
5c95eb565c cloudapi: support multiple upload targets 
Read the upload target types and options in the UploadTargets array of
the ImageRequest and initialise the Target array.  If the top-level
(old) UploadOptions are also specified, prepend them to the array using
the image type's default target type.

Each upload target type is checked against a support map for
compatibility.
 2023-11-17 16:48:16 +01:00
Achilleas Koutsou
d7ab1f2112 cloudapi: multiple upload targets in request 
Add an array of targets in the imageRequest and return an array from
ImageRequest.GetTargets() (renamed from GetTarget()).  Currently, the
function still only returns one target, the default for the image type
with the top level upload options.
 2023-11-17 16:48:16 +01:00
Achilleas Koutsou
9d990ea5da cloudapi: separate target selection from initialisation 
Separate the target selection in GetTarget() into two steps.  First
determine the default target name for the image type and then use the
name to initialise the target object.  This is a bit more work (and
double switching) but will be needed to support selecting targets
externally.
 2023-11-17 16:48:16 +01:00
Achilleas Koutsou
8a6e0e3862 cloudapi: new error type ErrorInvalidUploadTarget 2023-11-17 16:48:16 +01:00
Achilleas Koutsou
b011b9845a cloudapi: split GetTarget() into functions 
Separate the handling of each individual target type into its own
function called by GetTarget()'s case switch.  This makes the function
more readable and the target object creation reusable.

Added an empty line after each creation of irTarget to make it easier to
visually distinguish the cases that fall through.
 2023-11-17 16:48:16 +01:00
Achilleas Koutsou
27beb73365 cloudapi: add multiple upload_statuses to ImageStatus 
To report on the multiple upload requests, we add an upload_statuses
property to the ImageStatus response.
 2023-11-17 16:48:16 +01:00
Achilleas Koutsou
24b4647a14 cloudapi: add multiple upload_targets to ImageRequest 
Add an upload_targets field to the image request.  This lets the API
caller specify multiple upload targets and upload options to be used.
If the upload target type does not match the upload options, the request
is invalid.

For backwards compatibility, the upload targets field is optional.  If
it is not specified, the default upload target and upload options for
the image type are assumed, which is the same as the old behaviour.

Adding an explicit selection to the request makes it possible to support
multiple upload targets for the same image type.  We plan to support
ostree commits being uploaded to both aws.s3 and pulp.

To report on the multiple upload requests, we add an upload_statuses
field to the ImageStatus response.
 2023-11-17 16:48:16 +01:00
Sanne Raymaekers
952c7a483b schutzbot/terraform: aws instance types rework 2023-11-17 16:31:53 +01:00
Tomáš Hozza
6649e7b36e Test/filesystem.sh: update the test case to the new policy 
The mountpoint policy changed as a result of COMPOSER-2030 [1]. Modify
the test case accordingly, to comprehensively test the updated
mountpoint policy.

[1] https://issues.redhat.com/browse/COMPOSER-2030

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-11-15 15:51:14 +01:00
Tomáš Hozza
5f04544af7 Test/filesystem.sh: remove mentions of RHEL-8.5 
There is really nothing specific to RHEL-8.5 in the test case.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-11-15 15:51:14 +01:00
Sanne Raymaekers
7d339b2989 test/cases/ubi-wsl: query Windows VM ip address via vm show 
`vm list-ip-addresses` has proven to be quite unreliable.
 2023-11-15 15:12:08 +01:00
schutzbot
12d35fedcc Post release version bump 
[skip ci]
 2023-11-15 08:22:11 +00:00
dependabot[bot]
a1e428fc53 build(deps): bump the go-deps group with 10 updates 
Bumps the go-deps group with 10 updates:

| Package | From | To |
| --- | --- | --- |
| [cloud.google.com/go/compute](https://github.com/googleapis/google-cloud-go) | `1.23.1` | `1.23.3` |
| [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) | `1.33.0` | `1.35.1` |
| [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) | `1.45.27` | `1.47.9` |
| [github.com/hashicorp/go-retryablehttp](https://github.com/hashicorp/go-retryablehttp) | `0.7.4` | `0.7.5` |
| [github.com/labstack/echo/v4](https://github.com/labstack/echo) | `4.11.2` | `4.11.3` |
| [github.com/labstack/gommon](https://github.com/labstack/gommon) | `0.4.0` | `0.4.1` |
| [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go) | `0.1.374` | `0.1.385` |
| [github.com/osbuild/images](https://github.com/osbuild/images) | `0.12.0` | `0.15.0` |
| [github.com/spf13/cobra](https://github.com/spf13/cobra) | `1.7.0` | `1.8.0` |
| [golang.org/x/oauth2](https://github.com/golang/oauth2) | `0.13.0` | `0.14.0` |


Updates `cloud.google.com/go/compute` from 1.23.1 to 1.23.3
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/documentai/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.23.1...compute/v1.23.3)

Updates `cloud.google.com/go/storage` from 1.33.0 to 1.35.1
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.33.0...storage/v1.35.1)

Updates `github.com/aws/aws-sdk-go` from 1.45.27 to 1.47.9
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.45.27...v1.47.9)

Updates `github.com/hashicorp/go-retryablehttp` from 0.7.4 to 0.7.5
- [Changelog](https://github.com/hashicorp/go-retryablehttp/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/go-retryablehttp/compare/v0.7.4...v0.7.5)

Updates `github.com/labstack/echo/v4` from 4.11.2 to 4.11.3
- [Release notes](https://github.com/labstack/echo/releases)
- [Changelog](https://github.com/labstack/echo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/labstack/echo/compare/v4.11.2...v4.11.3)

Updates `github.com/labstack/gommon` from 0.4.0 to 0.4.1
- [Release notes](https://github.com/labstack/gommon/releases)
- [Commits](https://github.com/labstack/gommon/compare/v0.4.0...v0.4.1)

Updates `github.com/openshift-online/ocm-sdk-go` from 0.1.374 to 0.1.385
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.374...v0.1.385)

Updates `github.com/osbuild/images` from 0.12.0 to 0.15.0
- [Release notes](https://github.com/osbuild/images/releases)
- [Commits](https://github.com/osbuild/images/compare/v0.12.0...v0.15.0)

Updates `github.com/spf13/cobra` from 1.7.0 to 1.8.0
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.7.0...v1.8.0)

Updates `golang.org/x/oauth2` from 0.13.0 to 0.14.0
- [Commits](https://github.com/golang/oauth2/compare/v0.13.0...v0.14.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/compute
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/hashicorp/go-retryablehttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/labstack/echo/v4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/labstack/gommon
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/osbuild/images
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/spf13/cobra
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-11-13 14:26:11 +01:00