Add `--build-rpms` option which will make the script build osbuild-composer
RPMs on the remote runner from the sources which were copied over. These
RPMs are then installed on the system, before any image test cases are
generated.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Script is run with the image type to build as the argument.
The target / cloud service is selected based on the image type
specified. This is how the API actually works now: Only an image type
can be specified.
The script now supports all the blobby image types for testing:
- edge-commit
- edge-container
- edge-installer
- image-installer
- guest-image (qcow2)
- vsphere (vmdk)
These are image types that are uploaded to S3 and provided to the user
as an object to download rather than a VM image on a cloud provider.
To verify the cloud api compose request options for the qcow2 and vmdk
image types, download the object and inspect it using image-info.
Checks if postgresql is installed and that user1 and user2 exist in the
passwd file.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Temporarily switch to the nightly devel composes for RHEL 9.0.
We're testing new packages and changes right now (like coreos-installer)
and it takes time to make new snapshots every time.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Don't use the `@core` package group in image definitions, because it is
not intended as the minimal package set for virtual / cloud images. In
addition, its content is changing without us knowing, which has
consequences such as the recent discovery of the fact that TuneD is no
longer installed by default on RHEL images, while it definitely should be.
Replace the `@core` package group with the `coreOsCommonPackageSet`
package set. The content of it is based on the latest `@core` group
definition with a few modifications, so that image package sets
never end up having the same package listed in the `Include` and `Exclude`
package set at the same time. All additions have been accompanied with a
comment and all removals have been kept commented out with a comment.
The fact that the change does not have any effect on image package sets
was verified by regenerating all RHEL-9.0 image test cases. There is
however one change in the VMDK image. Specifically the
`python3-libselinux` package have been added. The reason is that the
latest `@core` group definition was used when defining the content of
`coreOsCommonPackageSet`, however the `@core` group definition in the
RPMRepo snapshot used for the image test case didn't include the package
yet.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
The `@core` package group used to include TuneD package by default on
RHEL-8. It has been removed from the group in Fedora as part of [1] and
inherited into RHEL-9. As a result, TuneD is no longer installed by
default on RHEL images.
After a discussion on rhel-devel there seems to be an agreement, that
TuneD should be installed by default on all RHEL virtual images. At
least we should keep the consistency in this regard with RHEL-8.
Regenerate all RHEL-9.0 image test cases.
Related to https://bugzilla.redhat.com/show_bug.cgi?id=2026709
[1] https://pagure.io/fork/adelton/fedora-comps/c/a5d4f1b6c9fcbe20cb0c38eac5048d7d45d1dd17
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Add TuneD package to the base package set for all EC2 image types,
including the `ami` image type. In addition to installing the package,
also enable the service by default. TuneD will by default auto-detect
the environment in which the image is running and set the most
appropriate TuneD profile, with exception of the `ec2-sap` image, which
explicitly sets a specific TuneD profile.
This change affects the `ami`, `ec2`, and `ec2-ha` image types on all
supported architectures.
Regenerate affected image test cases.
Related to RHELPLAN-102615
Fix#1972
Signed-off-by: Tomas Hozza <thozza@redhat.com>
Generated image test case manifests for all supported distros, arches and
image-types are being tested as part of distro unit tests. However due
to time constrains, the unit test does not depsolve the image's default
package sets and thus does not check if they changed in the internal
osbuild-composer's representation, compared to the generated image test
case.
Extend the `TestDistro_Manifest()` function used by the unit test to
allow depsolving image's package sets.
Introduce a new test case binary `osbuild-composer-manifest-tests`
allowing to check the manifests generated by composer for all supported
combinations of images against generated manifests, including depsolving
image's default package sets.
Introduce a new CI test case `manifest_tests.sh` executing the
`osbuild-composer-manifest-tests` binary and testing all existing image
test cases. Run it in CI on RHEL-9 runner.
Modify SPEC file to ship the newly added test case.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
By default, pgxpool.Pool has 4 connections (or number of cpus if higher).
Currently, we have 3 replicas, that means max 3*4=12 DB connections.
The dequeue operation is actually blocking - when a worker is waiting for
a job, one connection is blocked. My theory is that with 16 workers, we just
don't have enough connections that causes all sorts of weird slowdowns.
This commit bumps the number of connection from one replica to 10, therefore
we should be at 30 connections in total.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
The CDN repositories require client side SSL certificates, it would have
to be specified in httpd configuration which is technically possible but
in practise it is more troubles than benefits.
It is similar case for the RH internal repositories, there is a new
issue every time we enable the test with them.
Stop wasting dev&qe time and disable the test for anything but rpmrepo
snapshots. It is better to have it running at least for few distros than
to have it disabled globally.
Also display logs for the HTTP proxy to see if there were any errors.
Use newly added RHUI-4 repo snapshots for all RHEL-9 EC2* image test
cases. This includes RHEL-9.0 and RHEL-9.0-Beta images. The removal
of installed repos on EC2-HA and EC2-SAP images is expected, as RHUI
client RPMs for these variants are empty for 9.0 Beta. This should
change for GA once there are updated RHUI clients RPMs available.
Signed-off-by: Tomas Hozza <thozza@redhat.com>
RHEL 9.0 AWS API test is failing with
Host key verification failed.
This is probably due to a recent change in openssh deprecating rsa host
keys (or likely rsa keys in general).
- turn off StrictHostKeyChecking when checking groups
- use 'ed25519' type for user ssh keys
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Changes from previous (RHEL 9.0 Beta):
- Edge container: Switched to nginx from apache. Change in pipeline and
package sets.
- All bootable x86_64: Packages for hybrid boot added (efibootmgr).
- Edge commit and edge container no longer require explicitly enabling
greenboot services.
All three "distros" share the same code for now. Adding manifests to
track changes before updating the RHEL 9.0 GA definitions.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
- Updated repos to use latest nightly (20211105)
- Added rt repository
- Both -ga and -beta are identical
- rhel-90.json repo file is a symlink to rhel-90-beta.json
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
Workers now depsolve in parallel to image builds, so we can
again move depsolivng to the workers. This will help us deal
with increases in traffic as we currently only have one
depsolve handler per pod. It would also avoid any issues with
composer running out of disk space due to dnf metadata caches.
This reverts commit c65b1e9b26.
It no longer makes sense because:
- we don't make any changes to 8.5
- we don't regenerate test manifests for 8.5
- osbuild-composer for 8.5 is in the rhel-8.5.0 branch
Also, the latest-8.5.0 symlink was removed, which broke the CI.
Signed-off-by: Ondřej Budai <ondrej@budai.cz>
The commonly used 'greenprint' function now adds a date + timestamp to
each message for debugging and tracking the duration of segments of each
scripts.
Installer test manifests don't contain image info but are useful for
detecting changes to the manifest.
Signed-off-by: Achilleas Koutsou <achilleas@koutsou.net>
