Commit graph

4793 commits

Author SHA1 Message Date
Antonio Murdaca
7a4f6d2cbd fix tests for LVM
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-09-22 10:20:27 +02:00
Antonio Murdaca
c07574f87f regenerate manifests
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2022-09-22 10:20:27 +02:00
Irene Diez
0f8ed4e04b distro/rhel90: edge images default to LVM
Change partition tables on edgeBase images to use
'LVM partitioning'. We need to ensure that LVM
stages are done before LUKS stages (e.g. remove-key)
or the pipelines will break (we cannot open a device
when its password has changed).

Add relevant tests on device_test.go plus a new
test partition table on common_test.go
2022-09-22 10:20:27 +02:00
Ondřej Budai
f0e48d36a9 schutzfile: use the latest osbuild commit
It's best to test against the latest and greatest. I actually need this because I need
f699720dbd
in the Fedora worker images.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-21 18:03:13 +02:00
Ondřej Budai
83cb17b4ba spec: bump osbuild dep to >= 65
osbuild 65 got support for gpgkeypaths in the ostree.remotes stage that we
need for iot-raw-image implemented recently.

See
2bff83364b

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-21 18:03:13 +02:00
schutzbot
629c31d92f Post release version bump
[skip ci]
2022-09-21 08:37:47 +00:00
Achilleas Koutsou
632f371a08 test: add iot-commit to api tests by name 2022-09-20 18:29:28 +02:00
Achilleas Koutsou
60b338085f test/generic_s3: fix file indentation 2022-09-20 18:29:28 +02:00
Achilleas Koutsou
fc4450cfbf cloudapi: add fedora iot image types
All except installer, which needs to be fixed.
2022-09-20 18:29:28 +02:00
Achilleas Koutsou
1d121126b0 test: use new canonical image type names in tests 2022-09-20 18:29:28 +02:00
Achilleas Koutsou
cdfd305e0f test: update format request map and test manifests 2022-09-20 18:29:28 +02:00
Achilleas Koutsou
f9c37a167a fedora: make fedora-iot-... names the aliases
Remove the 'fedora' prefix from the canonical name for fedora-iot image
types.  Make the previous names aliases.
This has little functional change since we're simply swapping the
canonical name with an existing alias.
2022-09-20 18:29:28 +02:00
Xiaofeng Wang
434362e81e test: User in commit will not be support after osbuild-composer 64
After osbuild-composer 64, user should be configured in raw-image
blueprint
2022-09-20 12:54:55 +02:00
Sanne Raymaekers
bfd90cf191 test/api/aws: Fix cloned image cleanup
The ami id of the cloned image is stored in `AMI_ID_2`.

Also tag the cloned image with `gitlab-ci-test`
2022-09-20 11:11:19 +02:00
dependabot[bot]
e0ce32922c build(deps): bump github.com/labstack/echo/v4 from 4.8.0 to 4.9.0
Bumps [github.com/labstack/echo/v4](https://github.com/labstack/echo) from 4.8.0 to 4.9.0.
- [Release notes](https://github.com/labstack/echo/releases)
- [Changelog](https://github.com/labstack/echo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/labstack/echo/compare/v4.8.0...v4.9.0)

---
updated-dependencies:
- dependency-name: github.com/labstack/echo/v4
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-19 22:59:56 +02:00
Ondřej Budai
d78f1e6229 cloudapi: add option to upload S3 artifacts publicly
A small test is added to cover this new option. Also, a check for AWS region
in the URL was removed. The public URL doesn't actually contain it and it's
imho useless - S3 buckets are always tied to a certain region.

The elaborate grep command parsing a path from a URI was switched to being
a small python script. The python script can actually handle an URI without
a query (which was always present in the pre-signed URI, but it's not in the
ordinary one)

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-19 22:56:36 +02:00
Ondřej Budai
54e2c2304c worker: add an option to upload public objects to aws.s3 target
If the object is marked as public, its direct download URL will be returned
instead of the presigned one.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-19 22:56:36 +02:00
Ondřej Budai
0e6c132ee6 awscloud: add option to mark S3 object as public
By setting the object's ACL to "public-read", anyone can download the object
even without authenticating with AWS.

The osbuild-upload-generic-s3 command got a new -public argument that
uses this new feature.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-19 22:56:36 +02:00
Ondřej Budai
1c3fe82d1e osbuild-upload-generic-s3: exit(1) on error
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-19 22:56:36 +02:00
Ondřej Budai
c9f3777f2a osbuild-upload-generic-s3: switch to fmt.Println everything
The built-ins shouldn't really be used.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-19 22:56:36 +02:00
Ondřej Budai
381bce9ac0 awscloud: close the file after it's uploaded to S3
Oops, this was forgotten.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-09-19 22:56:36 +02:00
dependabot[bot]
70b6353918 build(deps): bump github.com/jackc/pgx/v4 from 4.17.1 to 4.17.2
Bumps [github.com/jackc/pgx/v4](https://github.com/jackc/pgx) from 4.17.1 to 4.17.2.
- [Release notes](https://github.com/jackc/pgx/releases)
- [Changelog](https://github.com/jackc/pgx/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jackc/pgx/compare/v4.17.1...v4.17.2)

---
updated-dependencies:
- dependency-name: github.com/jackc/pgx/v4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-19 15:44:34 +02:00
dependabot[bot]
e2cc53eb72 build(deps): bump google.golang.org/api from 0.94.0 to 0.96.0
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.94.0 to 0.96.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.94.0...v0.96.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-19 15:38:54 +02:00
dependabot[bot]
ba8de9a6a9 build(deps): bump github.com/openshift-online/ocm-sdk-go
Bumps [github.com/openshift-online/ocm-sdk-go](https://github.com/openshift-online/ocm-sdk-go) from 0.1.266 to 0.1.287.
- [Release notes](https://github.com/openshift-online/ocm-sdk-go/releases)
- [Changelog](https://github.com/openshift-online/ocm-sdk-go/blob/main/CHANGES.md)
- [Commits](https://github.com/openshift-online/ocm-sdk-go/compare/v0.1.266...v0.1.287)

---
updated-dependencies:
- dependency-name: github.com/openshift-online/ocm-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-19 11:55:50 +02:00
Tomáš Hozza
ff82dec37f test/api/common: add cleanupVSphere() and use it where appropriate
Clean up VSphere in all S3 test case implementations, which were
previously not cleaning up after themselves.
2022-09-19 10:37:32 +02:00
Tomáš Hozza
f1bf5b32f9 test/api/common: move all VSphere code from s3.sh to vsphere.sh 2022-09-19 10:37:32 +02:00
Tomáš Hozza
7e2d112783 test/api: add checkEnvVSphere() to common/vsphere.sh
Add a common function to check for all environment variables needed to
test VSphere image. Call the function from all S3 implementations of
`checkEnv()` if testing the VSphere image type.
2022-09-19 10:37:32 +02:00
Tomáš Hozza
9e9aec5b53 test/api: move govc installation to common/vsphere.sh
Move the installation of the VSphere client out of the
`api/common/aws.sh` `installClient()` function into a dedicated
`installClientVSphere()` function in `api/common/vsphere.sh`.

Rename the `installClient()` in `api/common/aws.sh` to
`installAWSClient()`. Introduce `installClient()` in both,
`api/aws.sh` and `api/aws.s3.sh`, which calls `installAWSClient()`.

Call `installClientVSphere()` conditionally when testing VMDK image type
in `api/aws.s3.sh` and `api/generic.s3.sh`.
2022-09-19 10:37:32 +02:00
Tomáš Hozza
2f6c238b7a test/cases/api.sh: fix indentation 2022-09-19 10:37:32 +02:00
Tomáš Hozza
9981a05190 test/cases/api/common/s3.sh: delete duplicate functions
All of the deleted functions were defined in the same file twice.
2022-09-19 10:37:32 +02:00
Tomáš Hozza
9285b7c9fe test/cases/api/aws.s3.sh: delete duplicate functions
The `verifyEdgeCommit()` and `verifyDisk()` functions are already
defined in the `test/cases/api/common/s3.sh` file sourced by
`test/cases/api/aws.s3.sh`.
2022-09-19 10:37:32 +02:00
Tomáš Hozza
ce8cc11739 test/cases/api/common/s3.sh: fix indentation 2022-09-19 10:37:32 +02:00
dependabot[bot]
c685a00694 build(deps): bump github.com/vmware/govmomi from 0.28.0 to 0.29.0
Bumps [github.com/vmware/govmomi](https://github.com/vmware/govmomi) from 0.28.0 to 0.29.0.
- [Release notes](https://github.com/vmware/govmomi/releases)
- [Changelog](https://github.com/vmware/govmomi/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vmware/govmomi/compare/v0.28.0...v0.29.0)

---
updated-dependencies:
- dependency-name: github.com/vmware/govmomi
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-16 18:02:30 +02:00
dependabot[bot]
b8cbe84fda build(deps): bump gopkg.in/ini.v1 from 1.66.6 to 1.67.0
Bumps [gopkg.in/ini.v1](https://github.com/go-ini/ini) from 1.66.6 to 1.67.0.
- [Release notes](https://github.com/go-ini/ini/releases)
- [Commits](https://github.com/go-ini/ini/compare/v1.66.6...v1.67.0)

---
updated-dependencies:
- dependency-name: gopkg.in/ini.v1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-16 13:52:38 +02:00
Jakub Rusz
1050f8a506 tests/upgrade: copy shared_lib.sh to the test vm
shared_lib.sh now contains some more helper functions and the
verification script uses them. Copy it over so that we can source it.
Also install all other dependencies before the actual upgrade.
2022-09-16 08:36:03 +02:00
dependabot[bot]
d40dbd6f0b build(deps): bump cloud.google.com/go/compute from 1.7.0 to 1.9.0
Bumps [cloud.google.com/go/compute](https://github.com/googleapis/google-cloud-go) from 1.7.0 to 1.9.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/video/v1.7.0...pubsub/v1.9.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/compute
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-16 01:42:14 +02:00
Tomáš Hozza
fd1850de40 distro: use storage capacity multiple constants in partition tables 2022-09-15 20:39:40 +02:00
Sanne Raymaekers
183e10e466 templates/packer: append distro and arch to the ami name
Because the rhel-8 images share the same name, and `force_deregister` is
true, packer will always deregister one of them.
2022-09-15 20:27:59 +02:00
Brian C. Lane
561bbbbdf3 azure: storageErr is already azblob.StorageError type 2022-09-15 03:57:40 -07:00
Brian C. Lane
7bae91eb9a gcp: Placeholder credential string is not a hardcoded credential 2022-09-15 03:57:40 -07:00
Brian C. Lane
0298dc85ab azure: Add note that azure-sdk-for-go module is deprecated 2022-09-15 03:57:40 -07:00
Brian C. Lane
cf18e2990a distro: Fix bug found by linting
Using new() to create a new struct assigns an empty struct to the
variable, meaning it can never be tested for nil. This means this code
would never detect a missing kernel package.
2022-09-15 03:57:40 -07:00
Brian C. Lane
826e9d8cc6 osbuild-composer: Set ReadHeaderTimeout to 5s
This satisfies the linter complaint about potential Slowloris attack
where headers are read slowly in an attempt to DoS the server.

The uses of ListenAndServe are only for testing purposes and are not run
in the production server so ignore the lint errors in
osbuild-mock-openid-provider.
2022-09-15 03:57:40 -07:00
Brian C. Lane
dfb69dc8e7 golangci-lint: The parameters passed to exec.Command are safe
None of these parameters are user controlled, they are either
constructed from paths or are constants.
2022-09-15 03:57:40 -07:00
Brian C. Lane
29fb97f3d6 workflow: Use latest golangci-lint workflow action 2022-09-15 03:57:40 -07:00
Brian C. Lane
9827126d30 dnfjson: Add dnf-json result cache to BaseSolver
This is used to cache the results of dump and search requests for 60s.
Once the timeout has passed the request is repeated and the timeout
reset. The timeout is *not* reset on every cache hit which prevents, for
example, a request every 59 seconds from keeping the cache from
updating.

When the existing CleanCache() function is called to check the on-disk
metadata cache it will also delete any expired entries from the
resultCache in order to keep it from eventually consuming all memory.
2022-09-15 11:34:39 +01:00
Brian C. Lane
e307a8174a dnfjson: Add Hash functions to repoConfig and Request
These will be used to generate a unique hash to be used with the cache
of dnf-json results.
2022-09-15 11:34:39 +01:00
Brian C. Lane
35059ca60e dnfjson: Add a cache of dnf-json results
This adds a cache structure with timeout handling and cache cleanup.
Also adds some testing of the new functions.
2022-09-15 11:34:39 +01:00
Tomas Hozza
f4aed3e6e2 distro/rhel8: use azure-rhui pkg set and image config for vhd
Extract the non-RHUI specific package set and image configuration into a
common definitions, which will be used by both image types.

Redefine the package sets and default image configuration used by both
image types to inherit from a common definition.

Regenerate image manifests for RHEL-8 / c8s `vhd` and `azure-rhui`
images.

There is no change in the resulting manifest for the `azure-rhui` image
type. However there are substantial changes to the `vhd` image
definition, which is now almost identical to the `azure-rhui` image
type, to provide consistent experience regardless if using RHUI or not.

The default partition table used by the `vhd` image type has been kept
as it was before, since there is yet no consensus on what size to
standardize for both image types.
2022-09-15 08:50:33 +02:00
Tomas Hozza
b4abacbaa2 distro/rhel9: use azure-rhui pkg set and image config for vhd
Extract the non-RHUI specific package set and image configuration into a
common definitions, which will be used by both image types.

Redefine the package sets and default image configuration used by both
image types to inherit from a common definition.

Regenerate image manifests for RHEL-9 / c9s `vhd` and `azure-rhui`
images.

There is no change in the resulting manifest for the `azure-rhui` image
type. However there are substantial changes to the `vhd` image
definition, which is now almost identical to the `azure-rhui` image
type, to provide consistent experience regardless if using RHUI or not.

The default partition table used by the `vhd` image type has been kept
as it was before, since there is yet no consensus on what size to
standardize for both image types.
2022-09-15 08:50:33 +02:00