Commit graph

5796 commits

Author SHA1 Message Date
Ondřej Budai
6dafd59eaa packit: use yaml anchors to dedup the copr build targets
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-27 15:07:04 +03:00
Simon de Vlieger
46f93614f2 fedora: live-installer image type
This image type produces the same artifact as the current workstation
installer live media.

During the implementation of this new installer some names have been
changed to make a bit more sense in the source tree. Installer images
now always mention which installer they are (anaconda, etc).
2023-06-27 13:18:56 +02:00
dependabot[bot]
32c3865507 build(deps): bump github.com/vmware/govmomi from 0.29.0 to 0.30.5
Bumps [github.com/vmware/govmomi](https://github.com/vmware/govmomi) from 0.29.0 to 0.30.5.
- [Release notes](https://github.com/vmware/govmomi/releases)
- [Changelog](https://github.com/vmware/govmomi/blob/main/CHANGELOG.md)
- [Commits](https://github.com/vmware/govmomi/compare/v0.29.0...v0.30.5)

---
updated-dependencies:
- dependency-name: github.com/vmware/govmomi
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-27 09:41:01 +02:00
Achilleas Koutsou
4f91e956aa osbuild: validate checksums for algo prefix
Make sure checksums used in the file input helper functions contain only
1 colon delimiter and it is not at the start of the string.

Adjusted tests to work with new restriction.
2023-06-27 08:57:06 +02:00
Achilleas Koutsou
464f91e35b osbuild: test NewRpmStageSourceFilesInputs
Unit test for the function where the original issue was located.

See rhbz#2215043.
2023-06-27 08:57:06 +02:00
Achilleas Koutsou
c6c0509780 osbuild: don't restrict checksum hashing algorithm
Helper functions that create stage input objects with references always
hard-coded `sha256:` as a prefix/algorithm for the checksum.  This
prevents the functions from being used in cases where other algorithms
are use, like sha1, which is possible with (perhaps older) RPM
repositories.  The inputs in osbuild a number of hashing algorithms and
we should be able to generate stages with other prefixes when necessary.

Remove the `sha256:` prefix in the helper functions and assume all
arguments to these functions provide the correct prefix.

Update tests to match.
2023-06-27 08:57:06 +02:00
Ondřej Budai
feaa093ef0 test/check_ostree: fix check of custom systemd unit override
Fedora 38 contains a global override for all units, thus checking for
equality no longer works. Instead, this commit just checks whether
the override is included.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
a4719c6b3b test/libvirt.sh: don't test the aliased openstack image on Fedora
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
2070c3206e .gitlab-ci.yml: enable libvirt tests on Fedora
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
cf41c15695 .gitlab-ci.yml: enable CI on Fedora 38
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
6e756eb76b tools/libvirt_test.sh: add fedora support
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
6b1f1848e4 tools/libvirt_test.sh: remove hardcoded os-variant
Using rhel8-unknown for all distros was really weird, let's use the proper
variants.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
2c4215a2d0 tools/libvirt_test.sh: fix weird indentation
There were 5 spaces instead of 4.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
89b71d339e tools/libvirt_test.sh: merge two similar switch statements
No need for more switching than just one.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
385dc4254e test/cases/libvirt.sh: enable uefi tests everywhere
All our images already support UEFI, let's simplify the code.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Ondřej Budai
3fb55413dc test/cases: make fedora conditions generic for all versions
To save us some work when updating the tests.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2023-06-26 17:56:20 +02:00
Xiaofeng Wang
dda50003cb test: Remove ostree-rebase-bios/uefi.sh and add rebase test into
ostree-raw-image.sh and ostree-simplified-installer.sh

Don't need dedicated rebase test workflow, move rebase test into
other test script. That will save runner and reduce running time

Add BIOS rebase test in ostree-raw-image.sh
Add UEFI rebase test in ostree-simplified-installer.sh
2023-06-26 12:33:36 +02:00
schutzbot
9ae8632933 schutzfile: Update snapshots to 20230623 2023-06-23 22:45:08 +03:00
Alexander Todorov
5660416a31 Use appropriate file name for 9.3 2023-06-23 10:00:03 +02:00
Alexander Todorov
7fca93c59a Remove redundant script in upgrade8to9 test
Upstream leapp no longer uses separate channels for obtaining its
data and instead packages them into the RPM. See
https://redhat-internal.slack.com/archives/C04JP91FB8X/p1687428000886329
2023-06-23 10:00:03 +02:00
Alexander Todorov
6fba5ef2f8 Upgrade 8to9 upgrade test for 9.3 2023-06-22 09:34:23 +02:00
Alexander Todorov
fdf4c89f0b Skip ova-boot Image Test on osbuild-composer < v84
note: don't fail if grep doesn't select anything
2023-06-20 18:23:09 +02:00
Tomáš Hozza
789838682c distro/rhel8: fix Azure EAP7 RHUI image definition
PR#3421 [1] unintentionally removed the `rhui-azure-rhel8` package from
the Azure EAP7 RHUI image base package set. As a result, the image
manifest can't be built successfully. The reason is that the removed
package installs a RPM GPG key, which is hard-coded in the image manifest
to be imported as part of the image build.

Add the package back to the image base package set and regenerate all
affected test manifests.

[1] https://github.com/osbuild/osbuild-composer/pull/3421

Signed-off-by: Tomáš Hozza <thozza@redhat.com>
2023-06-20 14:55:26 +02:00
Xiaofeng Wang
cf27954a0d test: Move ignition test out of simplified installer test
1. Remove ignition test from simplified installer and raw image
2. Use a new test job for ignition test
3. Add ignition test for raw image
2023-06-20 17:12:14 +08:00
Eng Zer Jun
537add3d70 jsondb: improve performance of list operation
Since we only need to retrieve the file names, we can use
`(*os.File).Readdirnames` to avoid reading the whole file info for
better performance.

Sample benchmark:

func Benchmark_Readdir(b *testing.B) {
	for i := 0; i < b.N; i++ {
		f, err := os.Open("/")
		if err != nil {
			b.Fatal(err)
		}

		_, err = f.Readdir(-1)
		if err != nil {
			f.Close()
			b.Fatal(err)
		}

		f.Close()
	}
}

func Benchmark_Readdirnames(b *testing.B) {
	for i := 0; i < b.N; i++ {
		f, err := os.Open("/")
		if err != nil {
			b.Fatal(err)
		}

		_, err = f.Readdirnames(-1)
		if err != nil {
			f.Close()
			b.Fatal(err)
		}

		f.Close()
	}
}

goos: linux
goarch: amd64
pkg: github.com/osbuild/osbuild-composer/internal/jsondb
cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics
Benchmark_Readdir-16         	   31304	     33551 ns/op	    5638 B/op	      70 allocs/op
Benchmark_Readdirnames-16    	  128443	     12124 ns/op	    1228 B/op	      30 allocs/op
PASS
ok  	github.com/osbuild/osbuild-composer/internal/jsondb	3.098s

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-06-20 10:45:22 +02:00
Alexander Todorov
47bbe8f0fe Bump spot instance time to 5hrs
b/c the timeout in GitLab is 5hrs and b/c the Image Tests job
on x86_64 RHEL 8.9 takes more than 4hr
2023-06-19 13:40:47 +02:00
Alexander Todorov
420f5af816 Update test condition for vmware ova image type
b/c osbuild-85 is now downstream but osbuild-composer-84 isn't
2023-06-15 20:15:19 +02:00
Alexander Todorov
b6cd24e3c0 Add temporary workaround for CLOUDX-488
https://redhat-internal.slack.com/archives/C0235DZB0DT/p1686812091343739
2023-06-15 20:15:19 +02:00
Alexander Todorov
f5965a98da Introduce test condition for AWS hybrid mode 2023-06-15 20:15:19 +02:00
Alexander Todorov
dabc51d252 Update manifests
go run ./cmd/gen-manifests/ -workers 50
2023-06-15 17:21:24 +02:00
Alexander Todorov
1c3bd128d0 Update rpmrepo snapshots 2023-06-15 17:21:24 +02:00
Xiaofeng Wang
ae04d807a3 test: Some enhancements to make test stable
1. Checking FDO user created before running ansible playbook
2. Wait 10 seconds before and after VM rebooting to make VM stable
2023-06-15 17:09:41 +02:00
Alexander Todorov
eb5dd8ae1b Conditionally skip test based on osbuild-compose.rpm version from distro 2023-06-15 08:54:57 +02:00
Alexander Todorov
81c1262969 Don't trigger Edge CI on nightly CI pipelines
because there is no PR for those pipelines and there is no place to
report the results to nor we can review these results!
2023-06-15 08:54:57 +02:00
Alexander Todorov
83cec26a5a Remove cloud-init/network-config file from tests
- not used in oscap.sh test at all
- causes VM in libvirt.sh test to acquire a different IP address instead
  of the hard-coded one inside the test, which causes SSH to fail

- not used anywhere else
2023-06-15 08:54:57 +02:00
Alexander Todorov
c0c4ecd937 Add my ssh key to the list of known keys 2023-06-15 08:54:57 +02:00
Alexander Todorov
2251e037c1 Use osbuild v86 for testing on 8.9 & 9.3 runners
fixes an issue with vmware_ova job
2023-06-15 08:54:57 +02:00
Alexander Todorov
1f28e50d21 Adjust tests for new distros
References:
- https://redhat-internal.slack.com/archives/C0235DZB0DT/p1685528139831589
- 1fbd9d975f
2023-06-15 08:54:57 +02:00
Alexander Todorov
4aff9076ea Update rpmrepos & manifest files via command
go run ./cmd/gen-manifests/ -workers 50
2023-06-15 08:54:57 +02:00
Alexander Todorov
fa5be10342 Update test runners: 8.8 -> 8.9; 9.2 -> 9.3 2023-06-15 08:54:57 +02:00
Alexander Todorov
22437f7784 Add RPM repository definitions for 8.9 and 9.3 2023-06-15 08:54:57 +02:00
Achilleas Koutsou
40719798d0 test: update manifests
Edge and IoT manifests are modified from the new option handling.  The
"parent" commit ID isn't specified in the options anymore, but it is
(fake) resolved by the manifest generator.

Of particular note is the iot-raw-image manifest that now properly uses
the commit ID in the copy stage for the firmware.

The resolved ostree commits are now stored in the content part of the
manifest metadata alongside package specs and containers.
2023-06-14 11:19:29 +02:00
Achilleas Koutsou
403fe74ce4 format-request-map: clean up ostree params
Don't specify ref ID for parent, instead only specify a ref and URL and
let the manifest generator resolve the ID using the sha256 hash.
2023-06-14 11:19:29 +02:00
Achilleas Koutsou
9ed61d021b distro/test: update test distro with new behaviour
Add a checksum as a hash of URL + Ref.
Use the parent ref instead of the image ref when it's set.  This makes
the test distro always behave like ostree commit and container types
(image types that can use an ostree parent) and not raw image or
installers (that use ostree commits as a payload).

Modify the weldr API test with the expected error message.
2023-06-14 11:19:29 +02:00
Achilleas Koutsou
96b7f05f6f osbuild-pipeline: resolve ostree commits
Properly resolve any ostree commit checksums.  Don't make the user
specify the checksum through the parent.
2023-06-14 11:19:29 +02:00
Achilleas Koutsou
0eb999d510 cloudapi: remove ostree from imageRequest
Use ostree.ImageOptions for the request parameters instead of a
SourceSpec on the imageRequest.

When preparing the image request, add the ostree values from the API's
compose request to the ostree options on the image options of the image
request.

It's not necessary to create a source spec and it's also not necessary
to add the default ref when it's not specified in the request for an
ostree-based image type.  Both of these will be handled by the Manifest
generation based on the ostree options (imageOptions.OSTree).  The image
functions will take care of setting any missing parameters or returning
errors if any required parameters are missing.
2023-06-14 11:19:29 +02:00
Achilleas Koutsou
8e5ac9790e worker: update ostree job structs
Change the OSTreeResolveSpec to match the ostree SourceSpec by removing
the Parent field.

Change OSTreeResolveResultSpec to match the CommitSpec by adding the
Secrets field.  The RHSM field is kept for backwards compatibility with
older workers.
2023-06-14 11:19:29 +02:00
Achilleas Koutsou
7e2855bbb3 ostree: rewrite Resolve() to take SourceSpec and return CommitSpec
The Resolve() function is now only responsible for resolving a
SourceSpec to a CommitSpec.  It only resolves a checksum if a URL is set
and sets the option for the RHSM secrets.

The Parent has been removed from the SourceSpec.  The SourceSpec is a
simple reference to a single ostree ref and has no connection with the
ostree options.
2023-06-14 11:19:29 +02:00
Achilleas Koutsou
3a42770548 distro/test: test ostree options
Test all combinations of ostree options and verify the ostree commit
source spec returned by the manifest.
2023-06-14 11:19:29 +02:00
Achilleas Koutsou
d34ad18aaf distro/test: remove test skip line
Oops!
2023-06-14 11:19:29 +02:00