particle-os/debian-forge-composer
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
6021 commits 1 branch 0 tags 688 MiB
Commit graph

6021 commits

This branch
This branch
All branches
Author SHA1 Message Date
Achilleas Koutsou
9a2f6fc33e cmd: update the final message in osbuild-upload-pulp-ostree 
Since the UploadAndDistributeCommit() function now waits for all tasks
to finish, update the wording of the final message to indicate that the
commit is available.
 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
15dbb90eda upload/pulp: wait for tasks in UploadAndDistributeCommit 
When uploading and distributing a commit, wait for any async tasks to
finish before returning.  There are two tasks that can block this
function:
- Creating a distribution: this only happens when a new repository is
  created.
- Import commit: this will always happen in this function.
 2023-10-18 21:14:46 +02:00
yih
6a98136cc0 add pulp test case 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
cfddd448c3 osbuild-worker: add pulp configuration 
Add support for pulp client configuration in the worker config.
Add test values to worker config test.
 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
fd0cae4366 cmd: add a osbuild-upload-pulp-ostree utility 
Simple command line wrapper around the UploadAndDistributeCommit()
function.
 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
3b8e595351 weldr: define the pulp.ostree upload settings 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
4edeab80e3 osbuild-worker: add the pulp.ostree target to the worker 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
503fbfb312 target: add the pulp.ostree target to the marshal functions 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
ab659c013c upload/pulp: helper function to upload and distribute 
Helper function that performs the whole upload and distribute procedure.
Two more helper functions are added for retrieving the href for a
repository based on its name, and for retrieving the base URL for a
repository's distribution.
 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
752040d10f target: add pulp ostree target and options 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
22e4169649 upload/pulp: task state functions 
Define the task state enum based on the available values defined in the
API.

Add a helper function that returns true if a task is running or waiting
and ignores errors.
 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
50367e6f14 upload/pulp: distribute ostree repo 
Function for distributing an ostree repository, making it available for
consumption.
 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
57f1bb8bf4 upload/pulp: import commit into repository 
Function for importing a commit artifact (that's already been uploaded)
into a given repository.  Note that the "repo" argument to the
NewOstreeImportAll() function refers to the name of the repository
directory inside the archive, which for the commits we produce in
osbuild is always "repo".
 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
4c002f3d54 upload/pulp: file upload method 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
1365347382 upload/pulp: list and create repositories 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
bc6c2fb6af upload: define new pulp uploader 
Define a basic client struct to pull in the pulp-client library.
 2023-10-18 21:14:46 +02:00
schutzbot
6663e69872 Post release version bump 
[skip ci]
 2023-10-18 08:22:48 +00:00
dependabot[bot]
fedfab3daf build(deps): bump the go-deps group with 2 updates 
Bumps the go-deps group with 2 updates: [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) and [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go).


Updates `github.com/aws/aws-sdk-go` from 1.45.25 to 1.45.27
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.45.25...v1.45.27)

Updates `github.com/openshift-online/ocm-sdk-go` from 0.1.373 to 0.1.374
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.373...v0.1.374)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-10-18 09:17:38 +02:00
Tomáš Hozza
6267ecf12c Add RHEL-8.10 and RHEL-9.4 (test and prod) repositories 
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-10-17 09:05:03 +02:00
Tomáš Hozza
8e8d11e6db Update Fedora repositories 
- Add ppc64le and s390x repo URLs to Fedora repositories.
- Add Fedora 40 (rawhide) repositories.
- Update Fedora testing repositories to latest snapshots, adding new
  arches and F40 repos. Basically took what is in osbuild/images repo.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-10-17 09:05:03 +02:00
Brian C. Lane
fa9f2f86b1 osbuild-playground: Update for NewPartitionTable change 
Uses raw partitioning mode.
 2023-10-17 09:05:03 +02:00
Tomáš Hozza
1903556198 Bump osbuild/images dependency to v0.12.0 
This brings in the support for:
 - RHEL-8.10
 - RHEL-9.4
 - ppc64le and s390x on Fedora (qcow2, container)

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-10-17 09:05:03 +02:00
schutzbot
73edc381d8 schutzfile: Update snapshots to 20231012 2023-10-13 17:40:01 +02:00
Brian C. Lane
e3f162cd56 cloudapi: Move Services to an actual struct 
This removes the need to make anonymous structs in the middle of the
Customizations request.
 2023-10-13 16:02:40 +02:00
Brian C. Lane
ab56a625c4 cloudapi: Simplify the bp.Customizations code 
There is a lot of repeated checks for bp.Customization != nil, this
simplifies that by creating an empty blueprint.Customizations at the
top, and checking to see if it is still empty at the bottom and setting
it back to nil.

Includes a new test for calling with an empty (not nil)
v2.Customizations set on the request.
 2023-10-12 16:53:35 +02:00
dependabot[bot]
d4af58c9f5 build(deps): bump the go-deps group with 7 updates 
Bumps the go-deps group with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go) | `1.1.0` | `1.2.0` |
| [github.com/google/go-cmp](https://github.com/google/go-cmp) | `0.5.9` | `0.6.0` |
| [github.com/labstack/echo/v4](https://github.com/labstack/echo) | `4.11.1` | `4.11.2` |
| [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go) | `0.1.371` | `0.1.373` |
| [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) | `1.16.0` | `1.17.0` |
| [golang.org/x/sync](https://github.com/golang/sync) | `0.3.0` | `0.4.0` |
| [google.golang.org/api](https://github.com/googleapis/google-api-go-client) | `0.145.0` | `0.146.0` |


Updates `github.com/Azure/azure-sdk-for-go/sdk/storage/azblob` from 1.1.0 to 1.2.0
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/v1.1...v1.2)

Updates `github.com/google/go-cmp` from 0.5.9 to 0.6.0
- [Release notes](https://github.com/google/go-cmp/releases)
- [Commits](https://github.com/google/go-cmp/compare/v0.5.9...v0.6.0)

Updates `github.com/labstack/echo/v4` from 4.11.1 to 4.11.2
- [Release notes](https://github.com/labstack/echo/releases)
- [Changelog](https://github.com/labstack/echo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/labstack/echo/compare/v4.11.1...v4.11.2)

Updates `github.com/openshift-online/ocm-sdk-go` from 0.1.371 to 0.1.373
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.371...v0.1.373)

Updates `github.com/prometheus/client_golang` from 1.16.0 to 1.17.0
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.16.0...v1.17.0)

Updates `golang.org/x/sync` from 0.3.0 to 0.4.0
- [Commits](https://github.com/golang/sync/compare/v0.3.0...v0.4.0)

Updates `google.golang.org/api` from 0.145.0 to 0.146.0
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.145.0...v0.146.0)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/google/go-cmp
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/labstack/echo/v4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: golang.org/x/sync
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-10-12 15:22:57 +02:00
Sanne Raymaekers
0a255df1ca cloudapi/v2: add all existing customizations to openapi spec 2023-10-11 17:29:56 +02:00
Tomáš Hozza
d7e960f0c1 Internal: delete unused common.VersionLessThan() 
The function is a leftover from the image definitions split and it is
not used. Moreover, the `images` copy of it is being reimplemented by
[1]. It is better to remove this copy to prevent any unintended use of
it or confusion.

[1] https://github.com/osbuild/images/pull/195

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-10-10 14:30:58 +02:00
Ondřej Budai
b228886fe1 cmd/gen-manifests: fix compilation with the latest images 
The latest images introduced a new blueprint option: minimal. Since we use
direct type-conversion, we need to add it also in gen-manifests. A warning
is also added, since this feature is considered experimental.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2023-10-10 08:17:49 +02:00
Ondřej Budai
19edaca01a go.mod: update images to 0.11 
Mainly to include:

- distro/rhel9: Make /boot 600 MiB big on RHEL 9.3+
- fedora: exclude sdubby
- Minimal image builds

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2023-10-10 08:17:49 +02:00
Alexander Todorov
edc45fde72 Enable Azure testing on HyperV Gen2 
previously added in 1eb51b7bab but was
later removed for some reason. See COMPOSER-1796.
 2023-10-09 19:31:42 +03:00
Tomáš Hozza
ebc256b073 Container: set the composer git revision during the build 
When the container with osbuild-composer gets built in our CI or by
AppSRE, we do not set the composer version to any value (as we do when
we built RPMs). As a result, the version reported by composer is always
"devel". This is not useful for debugging and determining the used
version of composer. In addition, this information now gets exposed in
Koji builds, therefore it makes sense to make it useful.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-10-09 14:06:50 +02:00
Tomáš Hozza
21e09c75bc Worker/osbuild: fix typo in error message 
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-10-09 13:49:53 +02:00
Tomáš Hozza
cf0571b908 Worker/osbuild: indent manifest before uploading to Koji 
Indent the osbuild manifest before uploading it to Koji. This will make
it much nicer for reading by humans.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-10-09 13:49:53 +02:00
yih
98c50e3f6e fix ostree vmdk test timeout 2023-10-08 00:08:08 +08:00
schutzbot
3e5a01baf9 Post release version bump 
[skip ci]
 2023-10-04 08:25:19 +00:00
Xiaofeng Wang
7f2414caf4 test: remove fdo package workaround from PR#3690 2023-10-03 15:53:40 -07:00
Sanne Raymaekers
bf3ff40a65 dashboards: drop interval from composer dashboard and fix slo 
The latency budget remaining used $__range instead of the 28d constant.
 2023-10-03 11:48:37 +02:00
Sanne Raymaekers
f05a5b59f3 dashboards: drop API section from worker job stats dashboard 
Renames the worker dashboard to worker job stats dashboard.

Drops the interval variable and relies solely on $__range and
$__rate_interval.
 2023-10-03 11:48:37 +02:00
Sanne Raymaekers
1475e216d2 dashboards: add worker api dashboard 
Also this one is made without a separate interval variable, instead
relying on $__rate_interval and $__interval.
 2023-10-03 11:48:37 +02:00
Sanne Raymaekers
64e9f1a2c7 worker: don't log job not pending dequeue errors 
This happens a lot when requesting a job by ID, which happens for the
manifest jobs.
 2023-10-02 23:37:26 +01:00
Sanne Raymaekers
6e4cade608 cloudapi/v2: log manifest generation errors as a warning 
Sometimes invalid customisations slip through, and this will make it
easier to read the logs looking for them.
 2023-10-02 23:37:26 +01:00
Brian C. Lane
8ec783a4dd tests: Update to use golangci-lint v1.54.2 2023-10-02 23:59:12 +02:00
Brian C. Lane
b1f021c35e lint: Fix implicit memory aliasing in for loops in osbuild-worker 2023-10-02 23:59:12 +02:00
Sanne Raymaekers
33f9a6726e dashboards: fix composer dash request rate errors 2023-10-02 18:50:37 +02:00
Alexander Todorov
164df6ffcb Revert "Add softlockup_all_cpu_backtrace=1 boot argument" 
This reverts commit 8aca402dff.
 2023-10-02 16:50:32 +02:00
Tomáš Hozza
f6e0e99391 Worker/koji-finalize: include osbuild version in image metadata 
Include the osbuild version used to build the image, in the image extra
metadata attached to the Koji build and image output.

Extend `koji.sh` to verify that the version is set and of the expected
value.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-09-27 12:39:28 +02:00
Tomáš Hozza
dcb66db5e7 Worker/koji-finalize: set the actual osbuild version in buildroot info 
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-09-27 12:39:28 +02:00
Tomáš Hozza
cca362678b Worker/osbuild: include osbuild version in OSBuildJobResult 
This will make it easier to reproduce image builds using the same
tooling which was used by the worker.

This change will enable to include the osbuild version in Koji build
metadata.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-09-27 12:39:28 +02:00
Tomáš Hozza
2b6570c36c test/koji.sh: test manifest extra metadata in outputs and build 
Test the content of manifest outputs extra metadata and that they are
the same as the metadata attached to the build extra metadata.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-09-27 12:39:28 +02:00