particle-os/debian-forge-composer
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
5959 commits 1 branch 0 tags 688 MiB
Commit graph

5959 commits

This branch
This branch
All branches
Author SHA1 Message Date
Tomáš Hozza
efbaa93eef Target/Koji: rework result options to contain more than just image info 
The `KojiTargetResultOptions` previously contained information only
about the uploaded image file. And even then, some information, such as
the filename, were scattered in other structures such as
`KojiFinalizeJob` struct.

Since the plan is to start uploading also osbuild manifest and osbuild
build log to Koji, we need to extend the result options structure to
hold more information and also make it specific to which file is the
information related.

Rework the `KojiTargetResultOptions` to contain information about:
 - the built image
 - build log
 - osbuild manifest

Information about each file contains:
 - filename
 - checksum type
 - file checksum
 - file size

For now, only the built image information is set and consumed by the
worker.

Add custom JSON (un)marshaler for `KojiTargetResultOptions` to handle
backward compatibility when old version of worker or composer server
interact with each other. Cover them with unit tests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-09-27 12:39:28 +02:00
Tomáš Hozza
27f98ec313 Target/koji: mark optional fields in BuildOutput as omitempty 
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-09-27 12:39:28 +02:00
Tomáš Hozza
a610bc3843 test/koji.sh: refactor and extend testing of build metadata 
Refactor koji.sh:
- Store Koji hub URL in a variable and use it instead of a literal.
- Do not inspect builinfo output and don't parse it using grep and sed,
  but instead directly call Koji XMLRPC functions (using `koji call`)
  with JSON output.

In addition, test that the image archive extra metadata are the same as
those stored in the build extra metadata, stored under the image
filename.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-09-27 12:39:28 +02:00
dependabot[bot]
347239838c build(deps): bump the go-deps group with 1 update 
Bumps the go-deps group with 1 update: [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go).

- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.45.16...v1.45.17)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-09-27 10:13:26 +02:00
Tomáš Hozza
5903aad07b tools/provision.sh: fix unbound variable bug if no OCI secrets are set 
This was hit in c9s CI:
https://gitlab.com/redhat/centos-stream/rpms/osbuild-composer/-/merge_requests/88
https://artifacts.dev.testing-farm.io/658332f8-8705-4727-84d3-f27b57775037/

```
/usr/libexec/osbuild-composer-test/provision.sh: line 141: OCI_SECRETS: unbound variable
```

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-09-26 14:12:42 +02:00
dependabot[bot]
41aacd8817 build(deps): bump the go-deps group with 6 updates 
Bumps the go-deps group with 6 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) | `1.45.10` | `1.45.16` |
| [github.com/gophercloud/gophercloud](https://github.com/gophercloud/gophercloud) | `1.6.0` | `1.7.0` |
| [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go) | `0.1.364` | `0.1.371` |
| [github.com/osbuild/images](https://github.com/osbuild/images) | `0.5.1-0.20230915095808-dd48a38be218` | `0.7.0` |
| [github.com/vmware/govmomi](https://github.com/vmware/govmomi) | `0.30.7` | `0.31.0` |
| [google.golang.org/api](https://github.com/googleapis/google-api-go-client) | `0.142.0` | `0.143.0` |


Updates `github.com/aws/aws-sdk-go` from 1.45.10 to 1.45.16
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.45.10...v1.45.16)

Updates `github.com/gophercloud/gophercloud` from 1.6.0 to 1.7.0
- [Release notes](https://github.com/gophercloud/gophercloud/releases)
- [Changelog](https://github.com/gophercloud/gophercloud/blob/v1.7.0/CHANGELOG.md)
- [Commits](https://github.com/gophercloud/gophercloud/compare/v1.6.0...v1.7.0)

Updates `github.com/openshift-online/ocm-sdk-go` from 0.1.364 to 0.1.371
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.364...v0.1.371)

Updates `github.com/osbuild/images` from 0.5.1-0.20230915095808-dd48a38be218 to 0.7.0
- [Release notes](https://github.com/osbuild/images/releases)
- [Commits](https://github.com/osbuild/images/commits/v0.7.0)

Updates `github.com/vmware/govmomi` from 0.30.7 to 0.31.0
- [Release notes](https://github.com/vmware/govmomi/releases)
- [Changelog](https://github.com/vmware/govmomi/blob/main/CHANGELOG.md)
- [Commits](https://github.com/vmware/govmomi/compare/v0.30.7...v0.31.0)

Updates `google.golang.org/api` from 0.142.0 to 0.143.0
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.142.0...v0.143.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/gophercloud/gophercloud
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/osbuild/images
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/vmware/govmomi
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-09-26 13:05:54 +02:00
Sanne Raymaekers
9d7159dab3 templates/packer: retry subscribtion 2023-09-25 11:56:42 +02:00
Sanne Raymaekers
e81183daa8 tools/appsre-ansible/rpmbuild: retry all tasks 2023-09-25 11:56:42 +02:00
dependabot[bot]
9e3602c088 build(deps): bump actions/checkout from 3 to 4 
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-09-22 10:19:39 +02:00
Sanne Raymaekers
0dc1a01077 templates/packer: configure oracle cloud credentials on startup 2023-09-22 09:55:48 +02:00
Brian C. Lane
9e39080d00 lint: Fix implicit memory aliasing in for loops 
Fix all instances of gosec G601: Implicit memory aliasing in for loop.
 2023-09-21 16:56:54 +02:00
Brian C. Lane
1b65f15449 vendor: Update osbuild/images to commit dd48a38be218 
This is needed for the test_distro.NewTestDistro change.
 2023-09-21 16:56:54 +02:00
Brian C. Lane
eab16830aa test: Add checksum to cloudapi tests 
These use 'pkg1' when depsolving, so they need an entry in the manifest
with the mocked checksum:

sha256:e50ddb78a37f5851d1a5c37a4c77d59123153c156e628e064b9daa378f45a2fe
 2023-09-21 16:56:54 +02:00
Brian C. Lane
428e2c0ff2 cloudapi: Catch and log Serialize errors 2023-09-21 16:56:54 +02:00
Brian C. Lane
61c9a66b7e test: Use test_distro.NewTestDistro from images 
and move the ResolveContent function from the images test_distro
package. api_test is the only place using it, so it belongs there, not
in images.
 2023-09-21 16:56:54 +02:00
Brian C. Lane
0db61e0c21 dnfjson: Add mock checksums to the packages used in testing 
Commit 312d87c6d5fab7ffd085a303e27b8db41111c86e adds validation of the
checksums to the ImageType.Manifest call, so it requires a valid looking
checksum, otherwise it will fail with a 'ManifestCreationFailed' error
when running the unit tests.
 2023-09-21 16:56:54 +02:00
Brian C. Lane
c675e919fd api_test: Log the path being tested in TestModulesInfo 2023-09-21 16:56:54 +02:00
Sanne Raymaekers
3a9bcded32 templates/composer: fix cpu request/limits 
The fluentd sidecar had the same request/limit as the service container,
and the migrate init-container had the fluentd request/limit. It should
be the other way round.
 2023-09-21 12:41:06 +02:00
Sanne Raymaekers
5bb9d414a2 templates/compose: add startingDeadlineSeconds to maintenance job 
The job won't run if it doesn't get scheduled within 30 minutes. This
prevents the job running multiple times in a row if it didn't get
scheduled, for instance due to resource limits.
 2023-09-21 12:41:06 +02:00
Sanne Raymaekers
e0b2455acf templates/composer: parameterise maintenance job cpu req/limit 2023-09-21 11:11:35 +02:00
schutzbot
3287290484 Post release version bump 
[skip ci]
 2023-09-20 08:22:18 +00:00
yih
2c65ee228b add test case for iot-qcow2-image 2023-09-20 13:43:24 +08:00
Sanne Raymaekers
1e7e1308ab test/cases/api: don't print command traces 
Causes some secrets to be dumped into the logs.
 2023-09-19 22:57:06 +02:00
Sanne Raymaekers
522a54188a test/api: add oci test 2023-09-19 22:57:06 +02:00
Sanne Raymaekers
a632848c59 cmd/osbuild-worker: add default OCI configuration 
Useful for hosted deployments, where target options are often empty or
incomplete.
 2023-09-19 22:57:06 +02:00
Sanne Raymaekers
a7794dc634 cloudapi/v2: add oci image type to cloudapi 2023-09-19 22:57:06 +02:00
Sanne Raymaekers
067366ed6a internal/target: add OCI object storage target 
Uploads an OCI image to OCI object storage, and generates a
pre-authenticated request for the object, which can be used to import it
into custom images.
 2023-09-19 22:57:06 +02:00
Xiaofeng Wang
7259deea3a test: add workaround for bug https://issues.redhat.com/browse/RHEL-4992 2023-09-19 11:34:48 +02:00
Xiaofeng Wang
44e6278c9d test: remove workaround for bug BZ#2234390 2023-09-18 18:14:58 -07:00
Gianluca Zuccarelli
0311ef449d test/cases: OpenSCAP cloudapi tests 
Run a basic check to see that the tailoring file has been created when
adding tailoring rules to the OpenSCAP config. Since the OpenSCAP scan
requires more memory and breaks some of the API tests, a full scan is
not run in the cloudapi tests.
 2023-09-18 20:46:49 +02:00
Gianluca Zuccarelli
9af8579eb5 test/cases: add tailoring to oscap test 
Add relevant checks to test for OpenSCAP tailoring features. The test
removes (unselects) a single rule.
 2023-09-18 20:46:49 +02:00
Gianluca Zuccarelli
0ce4ec7fc4 internal/cloudapi: add OpenSCAP tailoring 
Add support to the cloudapi for generating the tailoring file used
to customize the OpenSCAP remediation. This allows users to select and
unselect rules for the remediation and the `autotailor` stage generates
the tailoring file.
 2023-09-18 20:46:49 +02:00
Sanne Raymaekers
360df12548 test/cases/ubi-wsl: set --os-type when creating vm 
Solves `invalid usage for storage profile: attach existing managed OS disk`.
 2023-09-15 15:03:48 +02:00
Tomáš Hozza
eb8b63f3ca Actions: add workflow for marking and closing stale issues and PRs 
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
 2023-09-07 13:10:48 +02:00
schutzbot
554c9681a5 Post release version bump 
[skip ci]
 2023-09-06 08:22:33 +00:00
Sanne Raymaekers
e698e104d9 test/cases/ubi-wsl: fix waiting for a valid ipv4 
The counter never made it past 10.
 2023-09-01 14:04:37 +02:00
Ondřej Budai
23718dc707 upload/oci: fix a crash when the uploader returns an empty response 
resp.IsResumable crashes if resp.MultipartUploadResponse == nil. This happens
for instance when authentication fails. Fix this by also checking the
MultipartUploadResponse field.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2023-09-01 13:23:25 +02:00
Ondřej Budai
d6eacad494 jobimpl-osbuild: report a failed job on panic 
Previously, the worker would happily report success if osbuild succeeded,
there was no JobError, but the job actually panicked in the meantime.
Let's fix this by adding a recovery mechanism.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2023-09-01 13:23:25 +02:00
Ondřej Budai
19e45b528f jobimpl-osbuild: split temp dir cleanup into its separate defer 
It doesn't make sense to have them together.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2023-09-01 13:23:25 +02:00
Alexander Todorov
f2f1c8d0f0 Remove deprecated --os-type cli argument 
test logs print:

[2023-08-22T10:18:14-04:00] 💿 Install image via installer(ISO) on VM
WARNING  --os-type is deprecated and does nothing. Please stop using it.
 2023-09-01 13:22:30 +02:00
Xiaofeng Wang
f0c1fe2d98 test: fix "Waiter SnapshotImported failed: Max attempts exceeded" 2023-08-31 11:54:18 +02:00
Xiaofeng Wang
f0eb8e9738 test: add workaround for bug BZ#2234390 2023-08-30 09:29:50 +02:00
Sanne Raymaekers
e5c4640923 cloudapi/v2: expose wsl image type 2023-08-28 14:51:01 +02:00
dependabot[bot]
f39792d927 build(deps): bump the go-deps group with 2 updates 
Bumps the go-deps group with 2 updates: [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) and [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go).


Updates `github.com/aws/aws-sdk-go` from 1.44.329 to 1.44.332
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.329...v1.44.332)

Updates `github.com/openshift-online/ocm-sdk-go` from 0.1.362 to 0.1.364
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.362...v0.1.364)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-08-28 11:52:45 +02:00
Sanne Raymaekers
b26458f1f1 oci/upload: create image capability schema 
After creating the custom image, make sure the capabilities default to
paravirtualized.
 2023-08-28 10:29:14 +02:00
Sanne Raymaekers
adf2d5b22a test/cases/oci: add oci integration test 2023-08-28 10:29:14 +02:00
Sanne Raymaekers
93fc55f98a test/cases/ubi-wsl: public ip fixes 
Use the "Standard" public ip sku, as "Basic" produces warnings.

Spin until the vm gets assigned a valid ipv4 address.
 2023-08-28 08:44:12 +02:00
Alexander Todorov
8aca402dff Add softlockup_all_cpu_backtrace=1 boot argument 
should trigger a kernel trace instead of only warnings
 2023-08-28 08:36:07 +03:00
schutzbot
6f7af9e2ef Post release version bump 
[skip ci]
 2023-08-24 19:47:57 +00:00
Ondřej Budai
567fb680d4 dependabot: group go package updates 
Imho, this is much saner than having so many PRs for all individual
dependencies. Taken from osbuild/images.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
 2023-08-24 21:40:26 +02:00