Commit graph

3495 commits

Author SHA1 Message Date
Chloe Kaubisch
0619ffa588 cloudapi: expose filesystem customizations 2022-02-19 11:13:55 +00:00
ondrejbudai
9c1f552210 Post release version bump
[skip ci]
2022-02-18 18:36:49 +00:00
Ondřej Budai
10c0891fe0 mockbuild: temporarily pin RHEL 9 compose to an older one
A quick and dirty solution just to see the CI green before the release.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-18 16:03:20 +01:00
Achilleas Koutsou
2a504db9ed test: drop ANSIBLE_STDOUT_CALLBACK
Leftover from 1a6f6fdc4b
2022-02-18 10:39:08 +00:00
Achilleas Koutsou
12cbae3891 test: wait longer for reboot during ostree rollback test
Co-Authored-By: Xiaofeng Wang <henrywangxf@me.com>
2022-02-18 10:39:08 +00:00
Achilleas Koutsou
22f23ec38e test: update manifests and image info 2022-02-18 10:39:08 +00:00
Achilleas Koutsou
96e09e0322 Update cs9 test repos to latest snapshot 2022-02-18 10:39:08 +00:00
Achilleas Koutsou
f7d8535b16 tools/test-case-generators: add s390x for CS9 to test gen map 2022-02-18 10:39:08 +00:00
Achilleas Koutsou
d1ee43c9ec CI: run installers test on CS9 2022-02-18 10:39:08 +00:00
Achilleas Koutsou
85383ca950 test/cases: add centos OS_VARIANTS to installers test
Set conditional OS_VARIANT, copied from ostree-ng.sh.
Add CentOS Stream 9 variant.
2022-02-18 10:39:08 +00:00
Achilleas Koutsou
32fe534292 distro/rhel90: fix ISO Label for CS9
Should be CentOS-Stream-9-BaseOS-$arch (based on official CS9 ISOs)
instead of CentOS-Stream-9-$arch-dvd, which was the format that CS8
used.
2022-02-18 10:39:08 +00:00
Achilleas Koutsou
d271de346f test/ostree: use production compose for virt boot location 2022-02-18 10:39:08 +00:00
Achilleas Koutsou
a087fc5c86 test/cases: use correct OS_VARIANT for CentOS Stream 9
Correct name is 'centos-stream9'
See 'osinfo-query os'
2022-02-18 10:39:08 +00:00
Achilleas Koutsou
ff992e00f2 tests/libvirt: test uefi on cs9 2022-02-18 10:39:08 +00:00
Achilleas Koutsou
0ec9dcbc63 distro/rhel90: enable s390x for cs9
CentOS Stream 9 supports s390x.
2022-02-18 10:39:08 +00:00
Ondřej Budai
16f6864d58 mockbuild: fix CS9 mockbuild
- name mangling from centos to centos-stream was enabled also for
  centos 9
- CS9 mock configs from mock-core-configs-36.3-1.fc35.noarch were added

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-18 10:39:08 +00:00
Achilleas Koutsou
96f3647464 CI: run tests on CentOS Stream 9
Tests need to be on the internal network to fetch the cert for epel for
EL9 (see provision.sh).
Use the large runner for all ostree-based tests since the regular one
seems to be running out of space.
2022-02-18 10:39:08 +00:00
Achilleas Koutsou
6bc4afbcbd test,repositories: add cs9 repositories
Add rpmrepo repositories for testing.
Symlink centos-9 to centos-stream-9.
Add rpmrepo snapshot repositories to test and
test-case-generator repos.
Remove unused repositories from cs9 test repositories.
2022-02-18 10:39:08 +00:00
Achilleas Koutsou
0932fd700b distroregistry: enable CS9
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-18 10:39:08 +00:00
Brian C. Lane
52e9d45e58 client: Add Cancel function and tests
NOTE that the tests against the rpmmd fixure for running and waiting are
a bit of a kludge. It is impossible to mock up a running or waiting that
will allow an actual cancel, so the tests check for the internal server
error generated by using the fixure data.

Getting this error means that the API code did it's job and tried to
cancel the compose so that should be a good compromise.
2022-02-18 09:31:07 +01:00
Brian C. Lane
c657713181 weldr: Allow cancel on waiting and running composes
Fix the cancel API to allow a waiting compose to be canceled.
This also fixes the cancel return code to be 400, the lorax-composer
behavior was a bug, and using 400 allows composer-cli to properly
display the error.
2022-02-18 09:31:07 +01:00
Alexander Todorov
c877f14379 tests: Use human readable size for mountpoint
Follow up to
https://github.com/osbuild/osbuild-composer/pull/1921 and
https://bugzilla.redhat.com/show_bug.cgi?id=2023635
2022-02-18 09:28:01 +01:00
Simon Steinbeiss
cdddc3f81c worker: Properly log successful image builds
This will make counting successful image builds in Splunk and Cloudwatch
much simpler and robust.
2022-02-18 09:27:35 +01:00
Tomas Hozza
657b7a5833 Move ostreePullStageInputs() from distros to osbuild2
Move the `ostreePullStageInputs()` function duplicated in all
distro definitions to the `osbuild2` package as
`NewOstreePullStageInputs()`.

Delete `stage_inputs.go` from all distro definitions.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-18 09:26:35 +01:00
Tomas Hozza
d946199441 Move bootISOMonoStageInputs() from distros to osbuild2
Move the `bootISOMonoStageInputs()` function duplicated in all
distro definitions to the `osbuild2` package as
`NewBootISOMonoStagePipelineTreeInputs()`.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-18 09:26:35 +01:00
Tomas Hozza
069423ea67 Move rpmStageInputs() from distros to osbuild2
Move the `rpmStageInputs()` function duplicated in all
distro definitions to the `osbuild2` package as
`NewRpmStageSourceFilesInputs()`.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-18 09:26:35 +01:00
Tomas Hozza
8e6826e743 Move OSBuildMetadataToRPMs and PackageMetadataToSignature to osbuild2
Move `OSBuildMetadataToRPMs()` and `PackageMetadataToSignature()`
functions from the `rpmmd` package to `osbuild2` package to prevent
import cycles while de-duplicating `rpmStageInputs()` function from
`stage_inputs.go` of distro definitions.

Rename `PackageMetadataToSignature()` to
`RPMPackageMetadataToSignature()`, since it takes specifically
`RPMPackageMetadata` type as an argument.

Adjust affected parts of code (unit tests, cloudapi, worker).

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-18 09:26:35 +01:00
Tomas Hozza
0bd0f57231 Move qemuStageInputs() from distros to osbuild2
Move the `qemuStageInputs()` function duplicated in most
distro definitions to the `osbuild2` package as
`NewQemuStagePipelineFilesInputs()`.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-18 09:26:35 +01:00
Tomas Hozza
71ae94b40f Move xorrisofsStageInputs() from distros to osbuild2
Move the `xorrisofsStageInputs()` function duplicated in all
distro definitions to the `osbuild2` package as
`NewXorrisofsStagePipelineTreeInputs()`.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-18 09:26:35 +01:00
Tomas Hozza
c9b72033ee Move copyPipelineTreeInputs() from distros to osbuild2
Move the `copyPipelineTreeInputs()` function duplicated in many
distro definitions to the `osbuild2` package as
`NewCopyStagePipelineTreeInputs()`.

This will prevent creating another copy of the code in rhel-84 for
the `gce` image.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-18 09:26:35 +01:00
Tomas Hozza
0efbe0c55e RHEL-90 beta: remove duplicate import in pipelines.go
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-18 09:26:35 +01:00
Tomas Hozza
8713b8002a Move kernelVerStr() from distros to rpmmd
Move the `kernelVerStr()` function duplicated in many
distro definitions to the `rpmmd` package as
`GetVerStrFromPackageSpecListPanic()`.

I could not come up with a better name, sorry.

This will prevent creating another copy of the code in rhel-84 for
the `gce` image.

This change initially exposed a bug in the original implementation of
`kernelVerStr()`. Since on the first line, we allocate an empty structure
into `kernelPkg` variable, it can never be `nil` and the function never
panicked even if there was no `kernel` package in the PackageSpec list.

Fix all unit tests to provide valid arguments when calling `Manifest()`
method of image types.

Signed-off-by: Tomas Hozza <thozza@redhat.com>

kernelVerStr fixup

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-18 09:26:35 +01:00
Tomas Hozza
a392d71da6 Move kernelCmdlineStageOptions() from distros to osbuild2
Move the `kernelCmdlineStageOptions()` function duplicated in many
distro definitions to the `osbuild2` package as
`NewKernelCmdlineStageOptions()`.

This will prevent creating another copy of the code in rhel-84 for the
`gce` image.

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-18 09:26:35 +01:00
Jakub Rusz
43bfcc5389 tests/upgrade: gather more logs
Also disable firwalld AllowZoneDrifting because it's not supported in
RHEL-9.
2022-02-18 09:25:01 +01:00
Ondřej Budai
5d304d2957 packer: make the worker image smaller
This should save us some money. 10 GB is the size of the underlying
RHEL 8.5 AMI so this should be the minimum.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-18 09:24:07 +01:00
Jakub Rusz
8331af3ffd ci/nightly: report composer NVR in slack
Storing composer NVR in a file after it's installed and reporting on
Slack. Also reducing duplications of artifacts in .gitlab-ci.yml
2022-02-17 11:14:31 +01:00
Ondřej Budai
9c80a17ee5 fsjobqueue: refactor to allow dequeuing by multiple criteria
Previous implementation of fsjobqueue is amazing but it has its drawbacks:
- dequeueing can be done only based on a job type
- it's limited to 100 jobs per a job type

As we soon want to be able to dequeue also by another criteria (job channel),
we need to refactor the queue.

The new implementation is more naive but also more flexible. It basically
works like the dbjobqueue - dequeueing goroutines listen for newly added
jobs. When that happens, a signal is sent to all of them and they all inspect
all pending jobs and dequeue ones that match their needs. Ones that don't find
a suitable job, are waiting for the next signal.

This is certainly slower implementation as every time a new job is added into
the queue, all dequeueing goroutines will have to iterate over all
pending jobs. I think that's fine because fsjobqueue is not recommended
to use for composer instances with heavy load.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-16 17:14:36 +01:00
Ondřej Budai
b34571c1ec github: fetch more PRs when triggering gitlab
By default, the API will fetch only 30 open PRs. This is enough currently
as we have 35 open ones. Bump the limit to the maximum. 100 should be enough
for some time. 🤞

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-16 11:01:43 +01:00
Xiaofeng Wang
0bfc6cb4dd CI: Run ostree.sh on large RHEL 8.6 VM to have more memory
Fix "qemu-kvm: cannot set up guest memory 'pc.ram': Cannot allocate
memory" error
2022-02-16 10:49:55 +01:00
Xiaofeng Wang
b378fee2e5 test: conditional greenboot package checks in ansible playbooks
Co-Authored-By: Achilleas Koutsou <achilleas@koutsou.net>
2022-02-16 10:49:55 +01:00
Achilleas Koutsou
1b0b54f853 test: regenerate RHEL 8.6 and 9.0 manifests
Repositories were update for the two distros so we've regenerated all
the manifests.
2022-02-16 10:49:55 +01:00
Jakub Rusz
14ee64321b Schutzfile: update rhel-9 rpmrepo snapshot 2022-02-16 10:49:55 +01:00
Achilleas Koutsou
59dcab700e distro/rhel86: conditional greenboot packages for centos
CS8 and RHEL 8.6 share package sets but the greenboot package names were
only changed in RHEL 8.6.
2022-02-16 10:49:55 +01:00
Achilleas Koutsou
e0822bff44 repos: update repos for test case generators
test/data/repositories:
    new snapshots for RHEL 8.6 (20220201) and 9.0 (20220208)

Switching RHEL 8.6 and RHEL 9.0 to the latest rpmrepo snapshots:
    We need updated packages for greenboot, used by edge image types, so
    we technically only need to change the architectures that are
    supported by edge, but let's update all of them for consistency.

test/data/repositories:
    Remove rhel-90 symlink to beta

Keep the rhel-90-beta.json repository definitions, but make rhel-90.json
a file that contains the previously named rhel-90-ga.json definitions.
2022-02-16 10:49:55 +01:00
Peter Robinson
35b5b2b042 Update greenboot packaging names
In the greenboot 0.13 release we updated packaging due to the increase
in new tests and it not making sense to have packaging so granular.

Signed-off-by: Peter Robinson <pbrobinson@gmail.com>
2022-02-16 10:49:55 +01:00
Ondřej Budai
11893485cd github: split checks into 3 jobs
To speed it up a bit.

Signed-off-by: Ondřej Budai <ondrej@budai.cz>
2022-02-16 10:00:25 +01:00
Sanne Raymaekers
d492e8f702 cmd/osbuild-service-maintenance: GCP deletes by image name 2022-02-15 18:22:39 +01:00
Tomas Hozza
1273ef7b35 Regenerate all image test cases with the enhanced image-info
Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-15 15:56:56 +01:00
Tomas Hozza
2182996aae tools/image-info: extend the gathered information
Extend the information gathered by `image-info`. This is needed to
properly inspect GCE images.

Enhancements:
 - read all ssh client configuration files
 - read all sshd configuration files
 - read all YUM / DNF repos
 - read DNF Automatic configuration
 - fix reading of DNF configuration

Signed-off-by: Tomas Hozza <thozza@redhat.com>
2022-02-15 15:56:56 +01:00
Brian C. Lane
9f29f8eb25 store: Filter out blank blueprint name
Filter out listing blueprints with no name. This prevents old state.json
data from continuing to cause problems with the UI.

Resolves: rhbz#1922845
2022-02-15 10:10:21 +01:00