Commit graph

6036 commits

Author SHA1 Message Date
Tomáš Hozza
3df67e9adb Makefile: add unit-tests target
Add Makefile target to run all unit tests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
b41705adfb Delete unused internal/platform package
This is a leftover from the `osbuild/images` split`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
e27cb2d397 Delete unused internal/environment package
This is a leftover from the `osbuild/images` split`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
0d9a8df0f8 Delete unused internal/users package
This is a leftover from the `osbuild/images` split`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
ec6cff5223 Delete unused internal/workload package
This is a leftover from the `osbuild/images` split`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
4eb70326ee Delete unused internal/shell package
This is a leftover from the `osbuild/images` split`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
3da602873d Delete unused internal/oscap package
This is a leftover from the `osbuild/images` split`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
dfb55ebeb4 Delete unused internal/ignition package
This is a leftover from the `osbuild/images` split`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
2b54f5bb14 Delete unused internal/fdo package
This is a leftover from the `osbuild/images` split`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
2cc55065b0 Delete unused function converting repository BP customization
These functions are leftover from the `osbuild/images` split and are not
used by any code in osbuild composer. Instead, the version in
`osbuild/images` is used by distro definitions.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
d4e3173234 Delete unused blueprint.ValidateDirFileCustomizations() function
This function is no longer used by any code. Instead, its copy in the
`osbuild/images` repository is used by distro definitions to validate
the customization.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
d57f2e5bb5 Delete unused internal/pathpolicy package
This copy of the package in osbuild-composer is no longer used by any
code. Instead, the copy which is in the `osbuild/images` repository (and
module) is the implementation used by distro definitions.

Delete this code to reduce potential confusion and code duplication.

Also delete code in `internal/blueprint` package, which uses the
pathpolicy package. This code is no longer used, but instead the version
from `osbuild/images` is being used.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
yih
3d7b01b18d update pulp test case 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
5a337cd274 mockbuild: add a swapfile to el8 aarch64 builds
The pulp client is very large and defines a lot of symbols in on
package, which causes very large memory usage on el8 aarch64 (presumably
because of 64k page sizes).
Adding a 1 GiB swapfile fixes the issue in our CI runners.
2023-10-18 21:14:46 +02:00
yih
f4afef2890 update pulp test case 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
9a2f6fc33e cmd: update the final message in osbuild-upload-pulp-ostree
Since the UploadAndDistributeCommit() function now waits for all tasks
to finish, update the wording of the final message to indicate that the
commit is available.
2023-10-18 21:14:46 +02:00
Achilleas Koutsou
15dbb90eda upload/pulp: wait for tasks in UploadAndDistributeCommit
When uploading and distributing a commit, wait for any async tasks to
finish before returning.  There are two tasks that can block this
function:
- Creating a distribution: this only happens when a new repository is
  created.
- Import commit: this will always happen in this function.
2023-10-18 21:14:46 +02:00
yih
6a98136cc0 add pulp test case 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
cfddd448c3 osbuild-worker: add pulp configuration
Add support for pulp client configuration in the worker config.
Add test values to worker config test.
2023-10-18 21:14:46 +02:00
Achilleas Koutsou
fd0cae4366 cmd: add a osbuild-upload-pulp-ostree utility
Simple command line wrapper around the UploadAndDistributeCommit()
function.
2023-10-18 21:14:46 +02:00
Achilleas Koutsou
3b8e595351 weldr: define the pulp.ostree upload settings 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
4edeab80e3 osbuild-worker: add the pulp.ostree target to the worker 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
503fbfb312 target: add the pulp.ostree target to the marshal functions 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
ab659c013c upload/pulp: helper function to upload and distribute
Helper function that performs the whole upload and distribute procedure.
Two more helper functions are added for retrieving the href for a
repository based on its name, and for retrieving the base URL for a
repository's distribution.
2023-10-18 21:14:46 +02:00
Achilleas Koutsou
752040d10f target: add pulp ostree target and options 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
22e4169649 upload/pulp: task state functions
Define the task state enum based on the available values defined in the
API.

Add a helper function that returns true if a task is running or waiting
and ignores errors.
2023-10-18 21:14:46 +02:00
Achilleas Koutsou
50367e6f14 upload/pulp: distribute ostree repo
Function for distributing an ostree repository, making it available for
consumption.
2023-10-18 21:14:46 +02:00
Achilleas Koutsou
57f1bb8bf4 upload/pulp: import commit into repository
Function for importing a commit artifact (that's already been uploaded)
into a given repository.  Note that the "repo" argument to the
NewOstreeImportAll() function refers to the name of the repository
directory inside the archive, which for the commits we produce in
osbuild is always "repo".
2023-10-18 21:14:46 +02:00
Achilleas Koutsou
4c002f3d54 upload/pulp: file upload method 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
1365347382 upload/pulp: list and create repositories 2023-10-18 21:14:46 +02:00
Achilleas Koutsou
bc6c2fb6af upload: define new pulp uploader
Define a basic client struct to pull in the pulp-client library.
2023-10-18 21:14:46 +02:00
schutzbot
6663e69872 Post release version bump
[skip ci]
2023-10-18 08:22:48 +00:00
dependabot[bot]
fedfab3daf build(deps): bump the go-deps group with 2 updates
Bumps the go-deps group with 2 updates: [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) and [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go).


Updates `github.com/aws/aws-sdk-go` from 1.45.25 to 1.45.27
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.45.25...v1.45.27)

Updates `github.com/openshift-online/ocm-sdk-go` from 0.1.373 to 0.1.374
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.373...v0.1.374)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-18 09:17:38 +02:00
Tomáš Hozza
6267ecf12c Add RHEL-8.10 and RHEL-9.4 (test and prod) repositories
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-17 09:05:03 +02:00
Tomáš Hozza
8e8d11e6db Update Fedora repositories
- Add ppc64le and s390x repo URLs to Fedora repositories.
- Add Fedora 40 (rawhide) repositories.
- Update Fedora testing repositories to latest snapshots, adding new
  arches and F40 repos. Basically took what is in osbuild/images repo.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-17 09:05:03 +02:00
Brian C. Lane
fa9f2f86b1 osbuild-playground: Update for NewPartitionTable change
Uses raw partitioning mode.
2023-10-17 09:05:03 +02:00
Tomáš Hozza
1903556198 Bump osbuild/images dependency to v0.12.0
This brings in the support for:
 - RHEL-8.10
 - RHEL-9.4
 - ppc64le and s390x on Fedora (qcow2, container)

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-17 09:05:03 +02:00
schutzbot
73edc381d8 schutzfile: Update snapshots to 20231012 2023-10-13 17:40:01 +02:00
Brian C. Lane
e3f162cd56 cloudapi: Move Services to an actual struct
This removes the need to make anonymous structs in the middle of the
Customizations request.
2023-10-13 16:02:40 +02:00
Brian C. Lane
ab56a625c4 cloudapi: Simplify the bp.Customizations code
There is a lot of repeated checks for bp.Customization != nil, this
simplifies that by creating an empty blueprint.Customizations at the
top, and checking to see if it is still empty at the bottom and setting
it back to nil.

Includes a new test for calling with an empty (not nil)
v2.Customizations set on the request.
2023-10-12 16:53:35 +02:00
dependabot[bot]
d4af58c9f5 build(deps): bump the go-deps group with 7 updates
Bumps the go-deps group with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go) | `1.1.0` | `1.2.0` |
| [github.com/google/go-cmp](https://github.com/google/go-cmp) | `0.5.9` | `0.6.0` |
| [github.com/labstack/echo/v4](https://github.com/labstack/echo) | `4.11.1` | `4.11.2` |
| [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go) | `0.1.371` | `0.1.373` |
| [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) | `1.16.0` | `1.17.0` |
| [golang.org/x/sync](https://github.com/golang/sync) | `0.3.0` | `0.4.0` |
| [google.golang.org/api](https://github.com/googleapis/google-api-go-client) | `0.145.0` | `0.146.0` |


Updates `github.com/Azure/azure-sdk-for-go/sdk/storage/azblob` from 1.1.0 to 1.2.0
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/v1.1...v1.2)

Updates `github.com/google/go-cmp` from 0.5.9 to 0.6.0
- [Release notes](https://github.com/google/go-cmp/releases)
- [Commits](https://github.com/google/go-cmp/compare/v0.5.9...v0.6.0)

Updates `github.com/labstack/echo/v4` from 4.11.1 to 4.11.2
- [Release notes](https://github.com/labstack/echo/releases)
- [Changelog](https://github.com/labstack/echo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/labstack/echo/compare/v4.11.1...v4.11.2)

Updates `github.com/openshift-online/ocm-sdk-go` from 0.1.371 to 0.1.373
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.371...v0.1.373)

Updates `github.com/prometheus/client_golang` from 1.16.0 to 1.17.0
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.16.0...v1.17.0)

Updates `golang.org/x/sync` from 0.3.0 to 0.4.0
- [Commits](https://github.com/golang/sync/compare/v0.3.0...v0.4.0)

Updates `google.golang.org/api` from 0.145.0 to 0.146.0
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.145.0...v0.146.0)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/google/go-cmp
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/labstack/echo/v4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: golang.org/x/sync
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-12 15:22:57 +02:00
Sanne Raymaekers
0a255df1ca cloudapi/v2: add all existing customizations to openapi spec 2023-10-11 17:29:56 +02:00
Tomáš Hozza
d7e960f0c1 Internal: delete unused common.VersionLessThan()
The function is a leftover from the image definitions split and it is
not used. Moreover, the `images` copy of it is being reimplemented by
[1]. It is better to remove this copy to prevent any unintended use of
it or confusion.

[1] https://github.com/osbuild/images/pull/195

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-10 14:30:58 +02:00
Ondřej Budai
b228886fe1 cmd/gen-manifests: fix compilation with the latest images
The latest images introduced a new blueprint option: minimal. Since we use
direct type-conversion, we need to add it also in gen-manifests. A warning
is also added, since this feature is considered experimental.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-10-10 08:17:49 +02:00
Ondřej Budai
19edaca01a go.mod: update images to 0.11
Mainly to include:

- distro/rhel9: Make /boot 600 MiB big on RHEL 9.3+
- fedora: exclude sdubby
- Minimal image builds

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-10-10 08:17:49 +02:00
Alexander Todorov
edc45fde72 Enable Azure testing on HyperV Gen2
previously added in 1eb51b7bab but was
later removed for some reason. See COMPOSER-1796.
2023-10-09 19:31:42 +03:00
Tomáš Hozza
ebc256b073 Container: set the composer git revision during the build
When the container with osbuild-composer gets built in our CI or by
AppSRE, we do not set the composer version to any value (as we do when
we built RPMs). As a result, the version reported by composer is always
"devel". This is not useful for debugging and determining the used
version of composer. In addition, this information now gets exposed in
Koji builds, therefore it makes sense to make it useful.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-09 14:06:50 +02:00
Tomáš Hozza
21e09c75bc Worker/osbuild: fix typo in error message
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-09 13:49:53 +02:00
Tomáš Hozza
cf0571b908 Worker/osbuild: indent manifest before uploading to Koji
Indent the osbuild manifest before uploading it to Koji. This will make
it much nicer for reading by humans.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-09 13:49:53 +02:00
yih
98c50e3f6e fix ostree vmdk test timeout 2023-10-08 00:08:08 +08:00