Commit graph

6277 commits

Author SHA1 Message Date
Achilleas Koutsou
fbf63f6a1f cloudapi: add upload targets to route handler tests 2023-11-17 16:48:16 +01:00
Achilleas Koutsou
38664d1b64 cloudapi: add GetTarget() test case for pulp.ostree 2023-11-17 16:48:16 +01:00
Gianluca Zuccarelli
b711e302ba cloudapi: add pulp upload target
Add the pulp.ostree upload target to the cloud API and enable it for
edge/iot commits.

Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2023-11-17 16:48:16 +01:00
Achilleas Koutsou
5a93166f6b cloudapi: support multiple upload statuses in ComposeStatus
Add the new upload_statuses under the image_status in the result of the
ComposeStatus object.  The first status is also included in the old
top-level 'upload_status' property for backwards compatibility.

Tests are updated to match the new results.
2023-11-17 16:48:16 +01:00
Achilleas Koutsou
ba2240b578 cloudapi: test GetTargets()
Test some valid and invalid combinations for the GetTargets() upload
target selection.
Includes tests with and without the upload options for the default
target.
2023-11-17 16:48:16 +01:00
Achilleas Koutsou
9fc4551fdf cloudapi: check upload targets in request validation
It is now valid for UploadOptions to be nil but only if there is at
least one UploadTarget defined.
2023-11-17 16:48:16 +01:00
Achilleas Koutsou
5c95eb565c cloudapi: support multiple upload targets
Read the upload target types and options in the UploadTargets array of
the ImageRequest and initialise the Target array.  If the top-level
(old) UploadOptions are also specified, prepend them to the array using
the image type's default target type.

Each upload target type is checked against a support map for
compatibility.
2023-11-17 16:48:16 +01:00
Achilleas Koutsou
d7ab1f2112 cloudapi: multiple upload targets in request
Add an array of targets in the imageRequest and return an array from
ImageRequest.GetTargets() (renamed from GetTarget()).  Currently, the
function still only returns one target, the default for the image type
with the top level upload options.
2023-11-17 16:48:16 +01:00
Achilleas Koutsou
9d990ea5da cloudapi: separate target selection from initialisation
Separate the target selection in GetTarget() into two steps.  First
determine the default target name for the image type and then use the
name to initialise the target object.  This is a bit more work (and
double switching) but will be needed to support selecting targets
externally.
2023-11-17 16:48:16 +01:00
Achilleas Koutsou
8a6e0e3862 cloudapi: new error type ErrorInvalidUploadTarget 2023-11-17 16:48:16 +01:00
Achilleas Koutsou
b011b9845a cloudapi: split GetTarget() into functions
Separate the handling of each individual target type into its own
function called by GetTarget()'s case switch.  This makes the function
more readable and the target object creation reusable.

Added an empty line after each creation of irTarget to make it easier to
visually distinguish the cases that fall through.
2023-11-17 16:48:16 +01:00
Achilleas Koutsou
27beb73365 cloudapi: add multiple upload_statuses to ImageStatus
To report on the multiple upload requests, we add an upload_statuses
property to the ImageStatus response.
2023-11-17 16:48:16 +01:00
Achilleas Koutsou
24b4647a14 cloudapi: add multiple upload_targets to ImageRequest
Add an upload_targets field to the image request.  This lets the API
caller specify multiple upload targets and upload options to be used.
If the upload target type does not match the upload options, the request
is invalid.

For backwards compatibility, the upload targets field is optional.  If
it is not specified, the default upload target and upload options for
the image type are assumed, which is the same as the old behaviour.

Adding an explicit selection to the request makes it possible to support
multiple upload targets for the same image type.  We plan to support
ostree commits being uploaded to both aws.s3 and pulp.

To report on the multiple upload requests, we add an upload_statuses
field to the ImageStatus response.
2023-11-17 16:48:16 +01:00
Sanne Raymaekers
952c7a483b schutzbot/terraform: aws instance types rework 2023-11-17 16:31:53 +01:00
Tomáš Hozza
6649e7b36e Test/filesystem.sh: update the test case to the new policy
The mountpoint policy changed as a result of COMPOSER-2030 [1]. Modify
the test case accordingly, to comprehensively test the updated
mountpoint policy.

[1] https://issues.redhat.com/browse/COMPOSER-2030

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-11-15 15:51:14 +01:00
Tomáš Hozza
5f04544af7 Test/filesystem.sh: remove mentions of RHEL-8.5
There is really nothing specific to RHEL-8.5 in the test case.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-11-15 15:51:14 +01:00
Sanne Raymaekers
7d339b2989 test/cases/ubi-wsl: query Windows VM ip address via vm show
`vm list-ip-addresses` has proven to be quite unreliable.
2023-11-15 15:12:08 +01:00
schutzbot
12d35fedcc Post release version bump
[skip ci]
2023-11-15 08:22:11 +00:00
dependabot[bot]
a1e428fc53 build(deps): bump the go-deps group with 10 updates
Bumps the go-deps group with 10 updates:

| Package | From | To |
| --- | --- | --- |
| [cloud.google.com/go/compute](https://github.com/googleapis/google-cloud-go) | `1.23.1` | `1.23.3` |
| [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) | `1.33.0` | `1.35.1` |
| [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) | `1.45.27` | `1.47.9` |
| [github.com/hashicorp/go-retryablehttp](https://github.com/hashicorp/go-retryablehttp) | `0.7.4` | `0.7.5` |
| [github.com/labstack/echo/v4](https://github.com/labstack/echo) | `4.11.2` | `4.11.3` |
| [github.com/labstack/gommon](https://github.com/labstack/gommon) | `0.4.0` | `0.4.1` |
| [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go) | `0.1.374` | `0.1.385` |
| [github.com/osbuild/images](https://github.com/osbuild/images) | `0.12.0` | `0.15.0` |
| [github.com/spf13/cobra](https://github.com/spf13/cobra) | `1.7.0` | `1.8.0` |
| [golang.org/x/oauth2](https://github.com/golang/oauth2) | `0.13.0` | `0.14.0` |


Updates `cloud.google.com/go/compute` from 1.23.1 to 1.23.3
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/documentai/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.23.1...compute/v1.23.3)

Updates `cloud.google.com/go/storage` from 1.33.0 to 1.35.1
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.33.0...storage/v1.35.1)

Updates `github.com/aws/aws-sdk-go` from 1.45.27 to 1.47.9
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.45.27...v1.47.9)

Updates `github.com/hashicorp/go-retryablehttp` from 0.7.4 to 0.7.5
- [Changelog](https://github.com/hashicorp/go-retryablehttp/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/go-retryablehttp/compare/v0.7.4...v0.7.5)

Updates `github.com/labstack/echo/v4` from 4.11.2 to 4.11.3
- [Release notes](https://github.com/labstack/echo/releases)
- [Changelog](https://github.com/labstack/echo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/labstack/echo/compare/v4.11.2...v4.11.3)

Updates `github.com/labstack/gommon` from 0.4.0 to 0.4.1
- [Release notes](https://github.com/labstack/gommon/releases)
- [Commits](https://github.com/labstack/gommon/compare/v0.4.0...v0.4.1)

Updates `github.com/openshift-online/ocm-sdk-go` from 0.1.374 to 0.1.385
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.374...v0.1.385)

Updates `github.com/osbuild/images` from 0.12.0 to 0.15.0
- [Release notes](https://github.com/osbuild/images/releases)
- [Commits](https://github.com/osbuild/images/compare/v0.12.0...v0.15.0)

Updates `github.com/spf13/cobra` from 1.7.0 to 1.8.0
- [Release notes](https://github.com/spf13/cobra/releases)
- [Commits](https://github.com/spf13/cobra/compare/v1.7.0...v1.8.0)

Updates `golang.org/x/oauth2` from 0.13.0 to 0.14.0
- [Commits](https://github.com/golang/oauth2/compare/v0.13.0...v0.14.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/compute
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/hashicorp/go-retryablehttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/labstack/echo/v4
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/labstack/gommon
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: go-deps
- dependency-name: github.com/osbuild/images
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: github.com/spf13/cobra
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: go-deps
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-11-13 14:26:11 +01:00
Diaa Sami
13d642bb46 pkg/splunk_logger: handle potentially dangling goroutine 2023-11-10 17:35:25 +01:00
Diaa Sami
43f43c84f6 pkg: Add logger package from image-builder into pkg folder
rename and adapt it to be usable by composer
2023-11-10 17:35:25 +01:00
Gianluca Zuccarelli
fdf1364bb5 internal/blueprints: filesystem TOML tests
Move the filesystem customization tests into their own file.
Additionally add tests for unmarshalling filesystem customizations
from TOML, since we have added a new `minsize` tag to ensure
consistency with the json tag.

The new tests check the validation of the TOML input and ensures
that either one or both of the `minsize` and `size` inputs are
set. If both are set, the input is checked to ensure that both
match.
2023-11-09 15:47:07 +01:00
Gianluca Zuccarelli
4ddb2c300b internal/blueprints: fix mountpoint size keyword
Due to an oversight, the toml and json tags for the `MinSize`
field had different keywords. This commit fixes this by creating
a `minsize` toml tag and ensuring backwards compatability by
checking the old `size` tag.

If both `minsize` & `size` are set in the toml block, the
custom unmarshal function validates the input for inconsistencies.
2023-11-09 15:47:07 +01:00
Sanne Raymaekers
c749fb275e schutzbot/terraform: update rhel-9.2 aarch64 image 2023-11-09 12:05:11 +01:00
Sanne Raymaekers
cf10847d3e .github: update apt metadata before installing deps 2023-11-08 10:39:28 +01:00
Gianluca Zuccarelli
3fe36d0012 templates/packer: configure pulp creds on startup 2023-11-07 10:48:00 +01:00
Sanne Raymaekers
4ee00f7dab tools/provision: disable tracing before manipulating OCI secrets 2023-11-07 10:43:32 +01:00
Jakub Rusz
d68abf3bb5 Generate RHEL-94 and RHEL-810 manifests + update 2023-11-06 13:39:01 +04:00
Brian C. Lane
8e933f84f9 store: Fix test for CheckGPG
Forgot the #1 rule of map iteration. Don't depend on the order.
This uses a slice instead, which should be just as good for testing the
loop variable alias behavior.
2023-11-01 23:10:51 +01:00
Nicolás Muñoz
a693928b7a Added hyper_v_generation metadata to the instance used by CIV.
This will allow us to define which HyperV generation to use when
creating the image on the fly.
2023-11-01 11:40:55 +01:00
Brian C. Lane
b786178077 store: Fix SourceConfig.RepoConfig() function call
The SourceConfig pointer may be a loop variable that gets reused. This
results in unexpected behavior when the value pointed to is overwritten
by the loop calling this function.

Includes a test to make sure this is fixed.

So, DO NOT point to unsafe variables. Make a new pointer using
common.ToPtr where it is passed by value and returns a pointer to that
new value.

NOTE: This is NOT caught by golangci-lint. There may be other places
where this happens, but I have gone through the potential looking code
in osbuild-composer and images and not found any (other than a couple
places already noted with G601 tags as not a problem).
2023-11-01 11:38:41 +01:00
schutzbot
a65f17e73e Post release version bump
[skip ci]
2023-11-01 08:23:41 +00:00
Jakub Rusz
d71574976e rpmbuild: build rpms on RHEL-9.4 and RHEL-8.10 2023-10-31 16:44:22 +01:00
Tomáš Hozza
e1434746a2 Test: lint Packit configuration as part of CI
This will ensure that our Packit config stays valid.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-31 08:10:30 +01:00
Tomáš Hozza
b58a4b0009 Test: rename 'Lint' -> 'Golang Lint'
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-31 08:10:30 +01:00
Tomáš Hozza
5fcbce535c Packit: fix deprecated config option
`packit validate-config` reported:

```
common_package_config.py WARNING synced_files option is deprecated. Use files_to_sync option instead.
```

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-31 08:10:30 +01:00
Tomáš Hozza
fc7810e158 Packit: handle only tags without a 'dot'
Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-31 08:10:30 +01:00
Achilleas Koutsou
61aea1a06f test/README: run go tests when updating images
Add a small paragraph with the instruction to run the go tests when
pulling in a new version of images as a guide for making sure that
osbuild-composer builds and runs successfully, before moving on to
integration tests.
2023-10-30 20:27:36 +01:00
Brian C. Lane
81240f58ba weldr: Add support for PartitioningMode 2023-10-23 14:52:30 +02:00
Brian C. Lane
2729f6ac54 blueprint: Add partitioning_mode customization 2023-10-23 14:52:30 +02:00
Brian C. Lane
d0877e68dc cloudapi: Add partitioning_mode support to the API
This adds a 'partitioning_mode' field that can be set to 'auto-lvm',
'lvm' or 'raw'. It defaults to 'auto-lvm'.
2023-10-23 14:52:30 +02:00
Tomáš Hozza
3df67e9adb Makefile: add unit-tests target
Add Makefile target to run all unit tests.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
b41705adfb Delete unused internal/platform package
This is a leftover from the `osbuild/images` split`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
e27cb2d397 Delete unused internal/environment package
This is a leftover from the `osbuild/images` split`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
0d9a8df0f8 Delete unused internal/users package
This is a leftover from the `osbuild/images` split`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
ec6cff5223 Delete unused internal/workload package
This is a leftover from the `osbuild/images` split`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
4eb70326ee Delete unused internal/shell package
This is a leftover from the `osbuild/images` split`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
3da602873d Delete unused internal/oscap package
This is a leftover from the `osbuild/images` split`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
dfb55ebeb4 Delete unused internal/ignition package
This is a leftover from the `osbuild/images` split`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00
Tomáš Hozza
2b54f5bb14 Delete unused internal/fdo package
This is a leftover from the `osbuild/images` split`.

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-10-20 18:59:00 +02:00