Commit graph

5817 commits

Author SHA1 Message Date
Gianluca Zuccarelli
fa6f3ba197 cloudapi: openscap integration
Enable the openscap customizations in the cloudapi.
2023-06-29 13:42:49 +01:00
Gianluca Zuccarelli
12e7b806b6 internal/distro: default oscap datastreams
Set fallback datastreams for openscap if no datastream is provided by
the user. This will also simplify the cloudapi experience by not
exposing the `datastream`
2023-06-29 13:42:49 +01:00
Gianluca Zuccarelli
c32fac7169 distro/rhel8: enable oscap for centos8
Remediations for centos 8 images were not previously enabled.
This commit enables running oscap remediations for centos 8.
2023-06-29 13:42:49 +01:00
Diaa Sami
962ad9e582 containers/osbuild-composer: add prometheus port parameter 2023-06-29 14:34:15 +02:00
Sanne Raymaekers
170feba87b templates/packer: use RH SSO for the default token endpoint
MAS SSO (identity.api.openshift.com) was deprecated, RH SSO should be
the default.
2023-06-29 11:32:44 +02:00
Sanne Raymaekers
38093100e3 templates/composer: No longer accept MAS SSO 2023-06-29 11:32:44 +02:00
Sanne Raymaekers
ebe43bd680 test/cases/api: use the request_count metric
In future it will be possible to drop the total_compose_requests metric
in favour of the request_count metric.
2023-06-28 15:08:37 +02:00
Sanne Raymaekers
2837b2a3ad prometheus: split off request timing information into separate mw
Tracks the worker api in addition to the composer api.
2023-06-28 15:08:37 +02:00
Sanne Raymaekers
9594156baf internal/worker: use TenantChannelMiddleware 2023-06-28 15:08:37 +02:00
Sanne Raymaekers
06038b2af6 internal/prometheus: add tenant to http and status metrics 2023-06-28 15:08:37 +02:00
Sanne Raymaekers
0f946e1c9e internal/auth: add TenantChannelMiddleware
Extracts the tenant from the JWT and sets it in the request context.
2023-06-28 15:08:37 +02:00
Xiaofeng Wang
bec17b6d47 test: Remove rebase test shell script
BIOS rebase test and UEFI rebase test have been moved to
raw image and simplified installer test script by PR#3512 already
2023-06-28 13:20:26 +02:00
Tomáš Hozza
815f6c4f16
Pause scheduled releases
We need to hold-off the next release until https://github.com/osbuild/osbuild-composer/pull/3524 is merged
2023-06-28 09:43:56 +02:00
schutzbot
36158ab843 schutzfile: Update snapshots to 20230627 2023-06-27 16:17:46 +02:00
Ondřej Budai
7dd018a482 test: regen manifests
The two last commits touched the image definitions, so this has to be done.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-27 16:13:25 +02:00
Ondřej Budai
b5c9feeff9 distro/fedora: remove obsolete conditions for package set
osbuild-composer no longer supports Fedora 36, so we are safe to drop these
now.

iotCommitPackageSet would really benefit from sorting, but that would
create hard to handle conflicts with other PRs upstream, so I decided
not to do it in this commit.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-27 16:13:25 +02:00
Achilleas Koutsou
78931c0f05 test: update fedora repos
Update Fedora 37, 38, and 39 repositories for the test case generator.
Changes are only package versions and dependencies.

For Fedora 39, a package (lklug-fonts) had to be removed from the
Anaconda package set since it is no longer available.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-27 16:13:25 +02:00
Alexander Todorov
1757e5473d Sanity check if cloud-cleaner workflow is enabled
GitHub will automatically disable scheduled actions defined on
repositories which don't receive much activity. In particular such
scheduled jobs will be disabled after 60 days of repo inactivity.

This CI job sanity checks the current status and reports back to
every PR so we can manually re-enable cloud-cleaner if necessary.
2023-06-27 15:08:42 +03:00
Ondřej Budai
d108849a59 packit: remove epel builds
osbuild-composer doesn't need the epel buildroot, it should be totally
buildable in just a plain RHEL buildroot. Remove the EPEL targets to save
the copr team some resources.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-27 15:07:04 +03:00
Ondřej Budai
1d9ba597d9 packit: add aarch64 builds to RHEL 8
This makes our matrix for RHEL 8 and EPEL 8 the same.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-27 15:07:04 +03:00
Ondřej Budai
90e6596172 packit: sort copr build targets alphabetically
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-27 15:07:04 +03:00
Ondřej Budai
6dafd59eaa packit: use yaml anchors to dedup the copr build targets
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-27 15:07:04 +03:00
Simon de Vlieger
46f93614f2 fedora: live-installer image type
This image type produces the same artifact as the current workstation
installer live media.

During the implementation of this new installer some names have been
changed to make a bit more sense in the source tree. Installer images
now always mention which installer they are (anaconda, etc).
2023-06-27 13:18:56 +02:00
dependabot[bot]
32c3865507 build(deps): bump github.com/vmware/govmomi from 0.29.0 to 0.30.5
Bumps [github.com/vmware/govmomi](https://github.com/vmware/govmomi) from 0.29.0 to 0.30.5.
- [Release notes](https://github.com/vmware/govmomi/releases)
- [Changelog](https://github.com/vmware/govmomi/blob/main/CHANGELOG.md)
- [Commits](https://github.com/vmware/govmomi/compare/v0.29.0...v0.30.5)

---
updated-dependencies:
- dependency-name: github.com/vmware/govmomi
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-27 09:41:01 +02:00
Achilleas Koutsou
4f91e956aa osbuild: validate checksums for algo prefix
Make sure checksums used in the file input helper functions contain only
1 colon delimiter and it is not at the start of the string.

Adjusted tests to work with new restriction.
2023-06-27 08:57:06 +02:00
Achilleas Koutsou
464f91e35b osbuild: test NewRpmStageSourceFilesInputs
Unit test for the function where the original issue was located.

See rhbz#2215043.
2023-06-27 08:57:06 +02:00
Achilleas Koutsou
c6c0509780 osbuild: don't restrict checksum hashing algorithm
Helper functions that create stage input objects with references always
hard-coded `sha256:` as a prefix/algorithm for the checksum.  This
prevents the functions from being used in cases where other algorithms
are use, like sha1, which is possible with (perhaps older) RPM
repositories.  The inputs in osbuild a number of hashing algorithms and
we should be able to generate stages with other prefixes when necessary.

Remove the `sha256:` prefix in the helper functions and assume all
arguments to these functions provide the correct prefix.

Update tests to match.
2023-06-27 08:57:06 +02:00
Ondřej Budai
feaa093ef0 test/check_ostree: fix check of custom systemd unit override
Fedora 38 contains a global override for all units, thus checking for
equality no longer works. Instead, this commit just checks whether
the override is included.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
a4719c6b3b test/libvirt.sh: don't test the aliased openstack image on Fedora
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
2070c3206e .gitlab-ci.yml: enable libvirt tests on Fedora
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
cf41c15695 .gitlab-ci.yml: enable CI on Fedora 38
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
6e756eb76b tools/libvirt_test.sh: add fedora support
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
6b1f1848e4 tools/libvirt_test.sh: remove hardcoded os-variant
Using rhel8-unknown for all distros was really weird, let's use the proper
variants.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
2c4215a2d0 tools/libvirt_test.sh: fix weird indentation
There were 5 spaces instead of 4.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
89b71d339e tools/libvirt_test.sh: merge two similar switch statements
No need for more switching than just one.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
385dc4254e test/cases/libvirt.sh: enable uefi tests everywhere
All our images already support UEFI, let's simplify the code.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
3fb55413dc test/cases: make fedora conditions generic for all versions
To save us some work when updating the tests.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Xiaofeng Wang
dda50003cb test: Remove ostree-rebase-bios/uefi.sh and add rebase test into
ostree-raw-image.sh and ostree-simplified-installer.sh

Don't need dedicated rebase test workflow, move rebase test into
other test script. That will save runner and reduce running time

Add BIOS rebase test in ostree-raw-image.sh
Add UEFI rebase test in ostree-simplified-installer.sh
2023-06-26 12:33:36 +02:00
schutzbot
9ae8632933 schutzfile: Update snapshots to 20230623 2023-06-23 22:45:08 +03:00
Alexander Todorov
5660416a31 Use appropriate file name for 9.3 2023-06-23 10:00:03 +02:00
Alexander Todorov
7fca93c59a Remove redundant script in upgrade8to9 test
Upstream leapp no longer uses separate channels for obtaining its
data and instead packages them into the RPM. See
https://redhat-internal.slack.com/archives/C04JP91FB8X/p1687428000886329
2023-06-23 10:00:03 +02:00
Alexander Todorov
6fba5ef2f8 Upgrade 8to9 upgrade test for 9.3 2023-06-22 09:34:23 +02:00
Alexander Todorov
fdf4c89f0b Skip ova-boot Image Test on osbuild-composer < v84
note: don't fail if grep doesn't select anything
2023-06-20 18:23:09 +02:00
Tomáš Hozza
789838682c distro/rhel8: fix Azure EAP7 RHUI image definition
PR#3421 [1] unintentionally removed the `rhui-azure-rhel8` package from
the Azure EAP7 RHUI image base package set. As a result, the image
manifest can't be built successfully. The reason is that the removed
package installs a RPM GPG key, which is hard-coded in the image manifest
to be imported as part of the image build.

Add the package back to the image base package set and regenerate all
affected test manifests.

[1] https://github.com/osbuild/osbuild-composer/pull/3421

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-06-20 14:55:26 +02:00
Xiaofeng Wang
cf27954a0d test: Move ignition test out of simplified installer test
1. Remove ignition test from simplified installer and raw image
2. Use a new test job for ignition test
3. Add ignition test for raw image
2023-06-20 17:12:14 +08:00
Eng Zer Jun
537add3d70 jsondb: improve performance of list operation
Since we only need to retrieve the file names, we can use
`(*os.File).Readdirnames` to avoid reading the whole file info for
better performance.

Sample benchmark:

func Benchmark_Readdir(b *testing.B) {
	for i := 0; i < b.N; i++ {
		f, err := os.Open("/")
		if err != nil {
			b.Fatal(err)
		}

		_, err = f.Readdir(-1)
		if err != nil {
			f.Close()
			b.Fatal(err)
		}

		f.Close()
	}
}

func Benchmark_Readdirnames(b *testing.B) {
	for i := 0; i < b.N; i++ {
		f, err := os.Open("/")
		if err != nil {
			b.Fatal(err)
		}

		_, err = f.Readdirnames(-1)
		if err != nil {
			f.Close()
			b.Fatal(err)
		}

		f.Close()
	}
}

goos: linux
goarch: amd64
pkg: github.com/osbuild/osbuild-composer/internal/jsondb
cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics
Benchmark_Readdir-16         	   31304	     33551 ns/op	    5638 B/op	      70 allocs/op
Benchmark_Readdirnames-16    	  128443	     12124 ns/op	    1228 B/op	      30 allocs/op
PASS
ok  	github.com/osbuild/osbuild-composer/internal/jsondb	3.098s

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-06-20 10:45:22 +02:00
Alexander Todorov
47bbe8f0fe Bump spot instance time to 5hrs
b/c the timeout in GitLab is 5hrs and b/c the Image Tests job
on x86_64 RHEL 8.9 takes more than 4hr
2023-06-19 13:40:47 +02:00
Alexander Todorov
420f5af816 Update test condition for vmware ova image type
b/c osbuild-85 is now downstream but osbuild-composer-84 isn't
2023-06-15 20:15:19 +02:00
Alexander Todorov
b6cd24e3c0 Add temporary workaround for CLOUDX-488
https://redhat-internal.slack.com/archives/C0235DZB0DT/p1686812091343739
2023-06-15 20:15:19 +02:00
Alexander Todorov
f5965a98da Introduce test condition for AWS hybrid mode 2023-06-15 20:15:19 +02:00